HomeBenchmarksMethodology
Behind the numbers · methodology v1.0 proposal

Repeatable tests.
Inspectable evidence.

A benchmark should help you understand a workload, a machine, and the conditions behind a result.

The directory shows reviewed measurements from individual deployments. Pilot reports cover their recorded test windows and do not establish 24-hour consistency. The broader controlled sampling programme below is a proposal, not a claim about every published report.
01 /

Start with the deployment.

A specific instance is the unit of evidence.

For a controlled plan summary, provision three fresh instances independently across different days. Observe each for approximately 24 hours and repeat workloads in defined windows. Fresh instances do not guarantee different physical hosts.

Record the provider, plan, region, CPU, RAM, storage, OS, kernel, virtualization, agent release, and workload configuration. Unknown fields stay unknown. Operator-supplied values remain distinct from detected facts.

02 /

Every workload. Together.

AreaWorkloadWhat the report shows
CPUSingle-thread and all-vCPU sysbench; compressionThroughput, runtime, variability
MemoryFixed working set exceeding CPU cacheRead and write bandwidth; copy remains unimplemented in the pilot
Diskfio direct I/O: separate read, write, and mixed jobsMB/s, IOPS, p50/p95/p99 latency
NetworkLibreSpeed CLI 1.0.14 to named public HTTPS destinationsDownload and upload per destination, HTTP latency and jitter; packet loss is not measured
DatabasePostgreSQL pgbench; in-memory and storage-boundTransactions/sec, average latency and durability; pilot latency percentiles are unavailable
ConsistencyRepeated measurements across the test windowMedian, range, degradation, CPU steal

Published internet speed tests use IPv4 over HTTPS, three concurrent streams and one 10-second download and upload per named destination, with no configured rate limit. These path observations do not establish maximum link capacity. A network-only report identifies whether it came from the same deployment or a separate VM. When a full report shows a later separate-VM test, its source, date and report link are displayed; its original JSON remains the historical snapshot.

The original pilot used controlled iperf3 endpoints with a 100 Mbit/s rate setting and ICMP latency probes. Those capped path checks remain in historical report evidence; they are not presented as internet speed tests. Missing matching published speed tests stay unmeasured.

Run workloads sequentially so they do not compete with one another. Pin tool versions and publish exact job settings. Storage tests use isolated files, never arbitrary raw devices. Database tests keep durability enabled.

03 /

Compare like with like.

Keep shared, dedicated, burstable, and ARM instances in separate cohorts. Different CPU and memory allocations are visibly labelled. A regional comparison does not imply identical physical locations.

Publish raw measurements and workload-specific comparisons. Missing data is missing, never zero. There is no opaque overall score.

Real plan prices must include native currency, regular monthly cost, mandatory extras, transfer terms, tax treatment, and verification date. Test bills and retail monthly pricing are separate facts. Prices are not inferred from workload measurements.

04 /

Review precedes publication.

The agent saves the full local report and submits to private intake. Automated validation checks structure, checksums, versions, completeness, and duplicates. Passing those checks does not grant approval.

A reviewer inspects every category, detection evidence, failures and artifacts. Review may be AI-assisted under user-delegated authorization; each published report identifies its reviewer and discloses its measurement limits. Publication binds to that exact approved revision. Any changed content needs another review. Only sanitized, approved snapshots and artifacts can become public.

Explore the benchmark agent
05 /

Read the boundaries.

Three deployments are a small sample. They cannot establish provider-wide performance, long-term uptime, or the experience of every customer. Twenty-four hours describes short-term consistency only.

Network values apply to the named endpoints and directions. Tool settings, datasets, operating systems, time of day, and hardware placement can affect results. Price checks and benchmark dates are recorded independently.

External submissions retain their source classification. Client checksums establish integrity, not the truth of provider identity or execution. Evidence review, including AI-assisted review, does not establish hardware attestation or independent human verification.

06 /

Make the evidence visible.

Provider marketing specifications and observed measurements are separate. Valid slow runs remain in the evidence. Corrections need a visible reason and a new reviewed revision; historical reports are not silently overwritten.

Any future commercial relationships or referral links must be disclosed alongside the relevant recommendations. A provider’s commercial participation must not determine test results.

Read the publication changelog ↗