The Spectrum Dispatch News

technology

PlanetScale releases Neki, a sharded Postgres database system

Neki lets teams scale Postgres across multiple machines while keeping real Postgres on every shard, with online workflows for schema changes and resharding.

PlanetScale releases Neki, a sharded Postgres database system

PlanetScale has announced Neki, a sharded Postgres solution now available in platform preview. The product is designed to help organizations that have outgrown single-machine Postgres deployments without requiring them to abandon the database system or accept significant complexity.

PlanetScale releases Neki, a sharded Postgres database system

According to PlanetScale, Neki is built on lessons learned from eight years of operating large-scale sharded MySQL clusters. The company noted that since releasing PlanetScale Postgres 18 months ago, it has onboarded thousands of customers, with some rivaling the size of their largest MySQL deployments. PlanetScale observed that teams repeatedly hit the limits of single-machine Postgres capacity, and existing solutions forced compromises.

Neki addresses this by keeping real Postgres on every shard rather than using a custom storage engine. Applications connect to a Neki router using the standard Postgres wire protocol, meaning existing drivers, ORMs, and connection strings continue to work without modification. Each shard operates as a full Postgres cluster with one primary and at least two replicas distributed across three availability zones.

The system uses a JSON-based data topology configuration that maps logical tables to physical shards. Users define shard indexes specifying the column Neki routes on and shard groups controlling how tables distribute across shards. The architecture includes four main components: Neki routers that parse queries and plan distributed execution; sharding and shard groups organizing data; sidecars providing connection pooling alongside each Postgres instance; and a control plane managing node health and failovers.

Neki enables online workflows for operations typically requiring maintenance windows. According to PlanetScale, schema changes, version upgrades, planned and unplanned failovers, imports, and resharding all run as built-in online operations. The control plane provisions new nodes, catches them up via replication, switches traffic, and retires old nodes through the same psql connection applications use.

Users can run Neki initially as a single primary with replicas, deferring sharding until needed. When scaling becomes necessary, resharding operates as a workflow against the existing cluster. Neki also includes PlanetScale features such as Insights, schema recommendations, branching, and managed connection pooling.

PlanetScale recommends against running production workloads on Neki during platform preview, as the product is still changing and some changes may be breaking. The company is accepting feedback and offering private demos for teams with large Postgres clusters.

Key facts

  • Neki is a sharded Postgres system from PlanetScale now in platform preview
  • Each shard is a full Postgres cluster with one primary and at least two replicas across three availability zones
  • Applications connect via standard Postgres wire protocol with no changes to existing drivers or ORMs
  • Users define shard keys through JSON data topology configuration
  • Schema changes, upgrades, failovers, and resharding operate as online workflows without maintenance windows
  • Neki includes PlanetScale features like Insights, schema recommendations, and branching

Sources

← All posts