Cloudflare announced K2 in public beta today, a durable event streaming primitive designed to decouple producers and consumers of data. The service addresses a core challenge in traditional Remote Procedure Call architectures: when producers send data faster than consumers can handle, or when consumers become unavailable, events are dropped.
K2 solves this by inserting a buffering service between producers and consumers. Events are sent to a K2 stream, which stores them as an ordered log. Multiple consumers can then independently read from the same stream at their own pace, enabling scenarios like an ecommerce backend emitting transaction events that need to be processed by both an analytics system and a fraud detection service simultaneously.
According to Cloudflare, K2 is fully serverless, scales to large data volumes, and supports long-term retention so that consumer downtime does not result in data loss. The service implements a partitioned, durable log on top of R2 object storage, allowing it to handle vast quantities of storage.
Cloudflare built K2 initially to serve as the ingestion layer for Basin Pipelines, which operates on a pull-based model requiring durable event storage. The company’s edge infrastructure, which spans over 335 cities, cannot run traditional distributed systems software like Apache Kafka due to its architecture of small, ephemeral machine slices with internet-based networking.
To work around these constraints, K2 relies on R2 object storage as a powerful state primitive, offloading replication and consensus to the storage layer. This approach keeps the application layer simpler and cheaper while enabling independent scaling of compute and storage. Since R2 does not support appends, K2 accumulates writes in-memory on an edge service, then writes complete segment files after a brief waiting period. The service achieves ordering using R2’s atomic operations without requiring a separate coordination service.
One tradeoff is produce latency: writing to object storage is slower than local disk, and batching delays add approximately 1 second of latency at the 99th percentile. Cloudflare plans to share additional technical details in an upcoming deep dive.
K2 differs from Cloudflare’s existing asynchronous delivery products. Queues focus on individual expensive or time-consuming work items with features like retries and dead-letter queues. Basin Pipelines is designed for ingestion with transformation and writing to object storage or Iceberg tables. K2 targets high-scale data movement with long-term retention and fan-out consumption, processing messages in batches rather than individually.
Key facts
- K2 stores events as an ordered log accessible to multiple independent consumers
- The service is built on R2 object storage and operates across Cloudflare’s edge infrastructure spanning over 335 cities
- Produce latency is approximately 1 second at the 99th percentile due to batching and object storage write speeds
- K2 differs from Queues (designed for individual work items) and Basin Pipelines (designed for storage ingestion)
- The service is available in public beta and can be created via command-line, Wrangler, dashboard, or API