Every operation provisioned its own tape, ran the full size ladder, and read the payload back through each cache tier. Reads are compared byte-for-byte against what was written, so a row that says ok means the bytes matched, not that the request returned. A fifth leg, on a pathological host, is charted alongside the four as the outlier case rather than as a peer.
Median time in each phase, across all four regions. Below 50 MiB the chain steps register and certify are the largest cost, which is why a 512 B write and a 1 MiB write take the same wall time. From 512 MiB up, store overtakes them by more than an order of magnitude and the write becomes payload-bound.
Phases are shown side by side, not stacked, because they overlap and do not sum to the write total: slice uploads inside store run concurrently, so its summed time exceeds wall clock at the large sizes. Hover any bar for the write's actual wall time.
The same fixed cost seen from the other side. Throughput climbs as the constant per-write overhead amortizes over more payload, and only becomes meaningful above about 50 MiB. Jakarta writes fastest despite being furthest from most of the fleet, because a write returns on quorum and nearby nodes are enough to satisfy it. The Hetzner box is shown in grey as the pathological case, an order of magnitude below every other client at the same sizes. 512 B is omitted: a payload that small over a multi-second write rounds to zero MiB/s, so latency is the only meaningful measure there.
R1 decodes slices in-process with no gateway involved, so it is the one read path measured identically in every region and the only fair cross-region read comparison here. Geography inverts between writing and reading: Jakarta writes fastest and reads slowest, because a read must gather specific slices wherever they live and most of the fleet sits in North America.
Phase timings for R1, the only read path the SDK instruments. Reads are transfer-bound at every size: download dominates while decode stays two orders of magnitude below it. Erasure decoding is not a constraint anywhere on this ladder.
Local, EU and Asia only: those are the regions that read through a CDN-fronted gateway and recorded a cache header, so the tier that served each read is measured rather than assumed. Each step down is a real cache, the gateway assembling from slices once, then its own object cache, then the edge.
At 1 GiB the CDN paths return BYPASS rather than a hit: Cloudflare will not cache an object that large. That is a property of the CDN, not a cache that failed to warm, so those rungs are reported as not-applicable rather than as a slow hit.
Failed slice uploads are routine: nearly every write at every size on every host logs some, and the recovery worker absorbs them. So the failure count explains nothing. What separates a healthy client from a pathological one is what each failure costs before the write gives up on that peer.
The Hetzner box logged fewer failed slices than Frankfurt and was still many times slower, because each failure cost tens of seconds instead of a few. That points at the write path having no fast-fail for a peer that hangs rather than refuses, which is a protocol finding rather than a property of one provider. It is the same behaviour that hung four writes outright during the campaign. The two boxes also ran different kernels, so this is provider-plus-kernel, not provider alone.
Median wall time and throughput at every rung, per client. Latency is measured from the first client call to certification on a write, and from discovery through reconstruction and verification on a read. Throughput is payload bytes over that wall time, so it is omitted at 512 B where it rounds to nothing.
Cache-tier rows pool the clients that read through a CDN-fronted gateway. A dash means no client measured that rung on that path. The 1 GiB CDN figures are not edge hits: Cloudflare returns BYPASS above 512 MiB, so those requests travelled through the CDN but were answered by the origin, which is why the 512 MiB to 1 GiB step loses an order of magnitude.
Every record, sortable by any column and filterable by region, size or operation. Click a row to open the per-phase timings the SDK recorded for that single operation. This is the raw data every chart above is derived from.
The regions were not all measured under identical conditions. Write and R1 are comparable everywhere; the cache tiers are not. Stated plainly so nothing here gets read as more than it is.
| region | host | epoch | cache headers | reads verified | use for |
|---|---|---|---|---|---|
| US (Iowa) | GCP t2d-standard-8 | 10 | none recorded | 72 of 72 | write, R1, R2, R3 |
| EU (Frankfurt) | GCP t2d-standard-8 | 11-12 | yes | 64 of 70 | all paths |
| Asia (Jakarta) | GCP t2d-standard-8 | 10 | yes, R4 only | 48 of 84 | write, R1, R2, R3; R4 timing only |
| Local (laptop) | consumer uplink | 10 | yes | 60 of 60 | all paths |
| EU Hetzner | Hetzner ccx33 | 10 | none recorded | 60 of 60 | the outlier case only |
Four things to carry forward. Several legs pointed the CDN template at a hostname that resolved straight to the origin rather than through Cloudflare, so their R4 fetched the origin twice and recorded no cache header; those rows are an origin measurement mislabelled as an edge one, which is why R4 is only trusted where a header proves it. Asia's R4 column was repaired by re-reading the same objects afterwards, so it carries correct cache headers but was never compared against the source bytes. The EU leg spans epochs 11 and 12 while the others sit at epoch 10, so the committee was not identical across the campaign. And EU was measured twice, on Hetzner and on GCP, with only the GCP leg counted among the four.
Every file below is stored on Tapedrive itself and served through the gateway, so this page and its evidence are hosted on the system being measured. One record per operation, newline-delimited JSON.
Harness output as written, with a region and run_id on every row. The merged file is what this page reads and every table and chart above regenerates from it; the per-leg files are the same rows split by client.