Jepsen Finds 96 Issues in ParticleDB Versions 1.1.4 to 1.1.18

Jepsen tested ParticleDB versions 1.1.4 through 1.1.18.11.rc-22 and found 96 issues in the database, ranging from crashes and deadlocks to data loss and anomalous query results. Most issues occurred on healthy single-node instances with a single client and no faults. By version 1.1.18.11.rc-22, all but ten of those issues appeared resolved, but ParticleDB still lost committed writes during normal operation with the default occ transaction mode and allowed internal anomalies and Lost Update with serializable, its strongest setting.

Safety claims and documentation

ParticleDB is a multi-model database for AI workloads that offers transactional SQL, a key-value store, vector search, live analytics, and user-defined functions called Particles. It supports Postgres and Redis protocols plus gRPC and HTTP2 APIs, and was written in Rust. The home page advertised Full ACID and a TLA+ proof, but during collaboration no TLA+ specification was available. Documentation also disagreed about WAL sync modes and transaction modes, and ignored or rejected SET TRANSACTION ISOLATION LEVEL statements.

Test scope

Tests ran on Debian 13 nodes in containers and EC2 VMs, using Jepsen workloads and new ParticleDB-specific workloads. The harness could simulate process crashes, loss of writes not yet fsynced using LazyFS, and disk truncation or corruption. All three WAL sync modes and each transaction mode were tested. Replication was not covered because provided builds did not support it.

Findings

Issues included abysmal table-2pl performance, spontaneous mystery crashes, deadlocks, missing Postgres features, malformed prepared statement responses, authentication gaps, and predicate bugs such as SELECT ignoring WHERE, WHERE 4 = 5 returning a row, and nonsense SELECT results inside transactions. Orkes has since released 1.1.19-rc.13 and claims it resolves all remaining issues discussed in the report. This research was funded by Orkes, Inc.

Sources: Lobsters