Provider / transactional key-value store
ExperimentalFoundationDB
FoundationDB stores the split filesystem in a volume-scoped keyspace. It offers transactional serialization, but writer expiry still needs a shared provider-time authority and an explicit deployment durability assertion.
Metadata is a manifest at meta/manifest, sharded namespace values under meta/chunk/, and lease/fence/oracle records under the same volume prefix.
Immutable blocks use keys of the formprefix\0block/sha256:<hex-digest> and values containing the bytes. The provider validates block IDs and chunk configuration before opening or publishing.
Storage anatomy
Follow one file through both planes.
A mount-rs namespace stores names, attributes, directory entries, layouts, and block references. The bytes stay in the independent block plane. The example below is illustrative; the provider-specific schema, keys, and prefixes are the inspectable contract.
inode 42 · size 8,192NodeMetadata → NodeData::File(FileLayout)BlockId → raw bytesNamespace {
nodes[42] = NodeMetadata {
stats: { ino: 42, size: 8192, mode: regular },
data: File(FileLayout {
chunker: { algorithm: "fixed-size", version: 1, chunk_size: 4096 },
extents: [
{ file_offset: 0, block: "b012...", block_offset: 0, length: 4096 },
{ file_offset: 4096, block: "bfed...", block_offset: 0, length: 4096 }
]
})
}
}
BlockId "b012..." -> [4 KiB raw bytes]
BlockId "bfed..." -> [4 KiB raw bytes]Key/value storage
FoundationDB keyspace keyspace
Every record is a FoundationDB key/value under one caller-owned prefix. Metadata is sharded and described by a binary manifest; blocks are content-addressed.
demo-volume\0meta/manifestMRM1 | revision:u64 | chunk_count:u32 | payload_len:u64metadata manifest and shard count
demo-volume\0meta/chunk/<u32-be-index>serialized Namespace bytes for one shardmetadata payload
demo-volume\0meta/leaseMRL1 | fence:u64 | expires:u64 | owner_len:u32 | ownercurrent writer lease
demo-volume\0meta/fenceMRF1 | last_fence:u64fencing floor that survives lease replacement
demo-volume\0meta/lease-oracleMRO1 | provider_time_ms:u64persisted shared time authority when configured
demo-volume\0flushnot a data record; read as an acknowledgement barrierflush round trip
demo-volume\0block/sha256:<64-hex>raw immutable block bytescontent-addressed block
The metadata chunk keys form one range: <prefix>\0meta/chunk/ followed by a big-endian u32 index. Clearing a whole volume prefix is destructive; reconcile ownership first.
Real 7.4.7 provider and RustFS composition checkpoints, protected shared lease-authority code, shared-provider consumer selection, and terminal hosted Linux plus macOS feature-compile qualification now exist; live macOS service/mount, production identity, recovery, capacity, and release evidence remain open. The real pinned Linux ARM64 provider and RustFS composition checks cover blocks, metadata, CAS, fencing, reopen, and the latest exactowned-prefix cleanup with sibling/parent sentinel preservation. The real cluster authority test covers two independent readers, an absent- authority fail-closed result, backward-sample clamping, forward recovery, and stale-writer fencing. The RustFS composition now publishes the authority, opens consumers through the read-only oracle, republishes after the owned FoundationDB restart, and reopens from a fresh client. A hosted FoundationDB lane now builds the feature-enabled Node addon and can run the configuration-driven Linux FUSE CLI lifecycle when /dev/fuse is present; the live Node gate and native CLI use a separately published shared-provider authority prefix. The macOS job compiles the FoundationDB-enabled CLI NFS test, but live macOS service/cluster acceptance and hosted results remain open. The latest arm64 durable Ozone composition created three fixed-address FoundationDB 7.4.7 servers with three coordinators, separate persistent volumes, anddouble/SSD configuration. Transactional readiness passed before and after restarting replicated node 2; first and fresh-client Ozone compositions emittedFOUNDATIONDB_RUSTFS_CHUNKED_PASS andFOUNDATIONDB_RUSTFS_SERVICE_RESTART_PASS, followed byFOUNDATIONDB_TEST_PASS topology=durable and owned cleanup. This is real multi-node restart evidence, not production auth/TLS or power-loss proof. Hosted run 35636591071 at revision97b63aed then passed a bounded five-round Linux FoundationDB/RustFS qualification with live Node/N-API, native Linux CLI/FUSE mount and reopen, service restart, and schema-2/provenance evidence; its artifact isfoundationdb-production-qualification-35636591071-1(SHA-256123c2c5ece757fda141342d2b8e415e34269fff4246f5c3a8c147902415c137d). The exact-current-main run 35638932960 at revision3817efc9 passed the same bounded packet withp50_us=9751, p95_us=28824,p99_us=28824, andthroughput_ops_per_sec=99.31; its artifact isfoundationdb-production-qualification-35638932960-1(SHA-256c474b5ef9275ef88daf73e7fe90e36bd25eba8d149ac6849edfc672217ef4bd0). The latest mainline run 35652638242 at revisionc6f0039 also passed the explicit lease-TTL policy fixtures, durable three-server composition, five isolated soak rounds, live Node/N-API, native Linux CLI/FUSE mount and reopen, service restart, and schema-2 provenance validation. Its base marker recordedp50_us=7712, p95_us=228723,p99_us=228723, andthroughput_ops_per_sec=42.41; the five-round throughput range was 77.45–95.05 ops/s. The retained artifact isfoundationdb-production-qualification-35652638242-1with SHA-256366cb78d19bc5182a438a8459ebfe54efc510232e9dab667a815aba5eacfbee7. The newer mainline run 35657842924 at revision87a500b1 also passed the explicit lease-publication marker policy, five isolated soak rounds, live Node/N-API, native Linux CLI/FUSE mount and reopen, service restart, and schema-2 provenance validation. Its base marker recorded p50_us=9426,p95_us=29053, p99_us=29053, andthroughput_ops_per_sec=95.22; soak p95/p99 ranged from 27,345µs to 29,175µs and throughput from 85.77 to 94.49 ops/s. The retained artifact isfoundationdb-production-qualification-35657842924-1with SHA-256dc37b10f82ecd1c4c4513f4cf3ecc9a95e2ef37ccdadf0f4d69e56d30620846b. The latest requalified mainline run 35663914822 at revision 147c53a8 passed the same bounded policy, authority/consumer, restart, native Linux CLI/FUSE, and RustFS recovery packet after a lockfile repair. Its base marker recordedp50_us=9564, p95_us=39516,p99_us=39516, andthroughput_ops_per_sec=91.71; soak p95/p99 ranged from 21,487µs to 24,210µs and throughput from 91.97 to 106.58 ops/s. The retained artifact isfoundationdb-production-qualification-35663914822-1with SHA-256098ab3a4bb1fd6ae84971cebbb67baf2e50d9cb271accdaa9351ec28e49c3042. The newer current-tip run 35666991514 at revision5b9af323 also passed the same bounded policy, durable three-server composition, five isolated soak rounds, live Node/N-API, native Linux CLI/FUSE mount and reopen, service restart, and RustFS recovery packet. Its base marker recorded p50_us=8101,p95_us=35120, p99_us=35120, andthroughput_ops_per_sec=105.73; soak p95/p99 ranged from 20,100µs to 21,587µs and throughput from 107.93 to 119.84 ops/s. The retained artifact isfoundationdb-production-qualification-35666991514-1 with SHA-2567313f7ce6f7fb188d84d79a5c5d98df01a1319f071208f1058c5ab87a72acb12. The newer hosted run 35671492720 at revision9460a62 passed the same guarded-authority packet with a base marker of p50_us=3846, p95_us=38409,p99_us=38409, andthroughput_ops_per_sec=155.90; five-round soak throughput ranged from 199.39 to 214.47 ops/s. Its retained artifact isfoundationdb-production-qualification-35671492720-1 with SHA-256815dfecadab366a9fca9a7501c4a36d771324d8f71b521e0e8082261cbe431c3. The latest extended hosted qualification 35692674674 at exact revision 5a6d6507 passed the guarded-authority packet, durable three-server composition, live Node/N-API, Linux CLI/FUSE mount and reopen, service restart, and RustFS recovery checks across ten isolated soak rounds. Its bounded workload recorded 1,200 successful lifecycle operations at 110.18 IOPS with zero timeouts and cleanup failures; the base p95/p99 was439603 microseconds at 21.41 ops/s, while the ten-round soak ranged from 70204 to 236130microseconds and 18.19 to 96.22 ops/s. The retained artifact isfoundationdb-production-qualification-35692674674-1 with SHA-25644beef451832c307a53571bb86f92293b70c302d0add5104152059ed4c41d2ec. The fixed-profile floor is structural rather than a production capacity target; identity/ACL/TLS, backup/restore, production capacity, multi-day operation, failover, macOS coverage, and release approval remain open. The terminal cross-platform qualification packet at exact sourcea19d37b6 passed hosted run 35698630720: Linux, macOS FoundationDB feature compilation, and the fail-closed aggregate all passed, includingW07_PLATFORM_QUALIFICATION_PASS. Linux retained the durable Node/N-API, CLI/FUSE, restart, fresh-client, RustFS, heartbeat, and ten-round soak packet; the bounded workload recorded 1,200 successful lifecycle operations at 467.15 IOPS with zero timeouts or cleanup failures. Base p95/p99 was 25063microseconds at 208.39 ops/s, with ten-round soak p95/p99 from 13399 to 13906 microseconds. Retained Linux, macOS, and aggregate artifact digests are0b8ead4a8cf548322f71cf2a391b036f65ebf8412e52fc7720042487f61fcb11,19a86b0d050cdcb792e5c83e925ba0e79f9c3c72f073c21329600ed637c5b241, anda6666fe8afd143d6b525b95f917332dff6af54c1d65f913bfab44c3cc05632f8. This is terminal cross-platform qualification, not a live macOS service/cluster/mount, clean-install, signing/package, production capacity, identity/ACL, backup/restore, failover, observability, or release approval. These hosted results do not establish production identity/ACL/TLS, backup/restore, production capacity, multi-day operation, failover, macOS acceptance, or release approval. The newer W07 candidate at exact source87f3cdf0 passed hosted run 35702352395 across the durable FoundationDB/RustFS/Node/CLI/restart job, the macOS FoundationDB feature-compile job, and the evidence assembly job. Its bounded workload recorded 466.18 lifecycle IOPS over 400 iterations with 64-way concurrency and the explicitminimum_iops=1 floor; the aggregate emittedW07_PLATFORM_QUALIFICATION_PASS withmacos=feature-compile-only. Retained Linux and macOS evidence digests are2229862b90ca978f9a9d205ab59bb8414a4396e2d381f35380ad9df22543d5fdand84185d564272f3a360f9d20606b0c99af31162d685a9fb26b103843c4b13ceeb; the aggregate upload digest is082db52c7760bf8cad5420b0539dbf7f6658c42a6b0f572fbf3181690c9ac8ab. This supersedes the older terminal packet for current candidate tracking, but remains feature-compile-only on macOS and does not prove live macOS service/mount, production capacity, identity/ACL, backup/restore, failover, observability, signing, or release approval. The current-public-main W07 qualification at sourceff5cc581 passed hosted run 35712676265 across the durable provider job, macOS feature compilation, and aggregate evidence assembly. Linux passed the policy/ledger/workload preflight, durable FoundationDB/RustFS metadata, Node/N-API, Linux CLI/FUSE, restart/reopen, fresh-client, authority heartbeat/stats, and ten-round soak; macOS passed the FoundationDB provider/test, CLInative_lifecycle, N-API feature compilation, and bound provenance; the aggregate emittedW07_PLATFORM_QUALIFICATION_PASS provenance=bound. The bounded workload recorded 716.30 lifecycle IOPS over 1,200 successful operations with zero timeouts or cleanup failures. Base telemetry was p50=2311us,p95/p99=37919us, and 164.77 ops/s; soak telemetry ranged from 269.41 to 330.32 ops/s. This supersedes the previous terminal packet for current-main tracking but remains qualification-only and NO-GO for live macOS service/mount, production capacity, identity/ACL, backup/restore, failover, observability, signing, and release approval. The current implementation at source fba61979 overlaps the independent lease, manifest, and authority-time reads inside the existing FoundationDB publication transaction while preserving one read version, lease/fence validation, revision CAS, and fail-closed ambiguous-commit handling. Feature compile/test-target checks and strict Clippy pass, but the focused native tests remain blocked by the missing fdb_c linker library. The immutable W05 candidate also completed W07 run35714147247 with bound provenance: Linux was terminal while macOS was feature-compile-only and the assembly artifact was retained. This is a candidate sub-gate rather than a current-public-main refresh, and it does not close live macOS service/mount, production capacity, recovery, identity, or release approval. Exact-head run35715790619 at workflow head af7e73df includes this source as an ancestor but remains queued, so no new performance or provider qualification is promoted from this optimization yet.
Layout and chunking
Defaults keep blocks at 64 KiB, metadata shards at 8 KiB, and one namespace publication at 512 KiB. The hard 100,000-byte value and 10,000,000-byte affected-data limits are part of the composition preflight, not a late upload error.
Durability and consistency
Writer leases use a transactional persisted logical clock by default; without an oracle, lease operations fail closed withENOTSUP. Maybe-committed non-idempotent metadata operations return an error for reopen/reconciliation rather than being replayed.with_durable(true) remains a caller assertion about the cluster. The guarded with_production_lease_oracle entry point rejects unverified, development, and single-authority clocks unless LeaseOracle::authority_kind declaresSharedProvider; that declaration is a trust boundary, not proof of distributed safety. The integration now exposes a write-sideFoundationDbLeaseAuthority and a read-onlyFoundationDbSharedLeaseOracle. The authority publishes monotonic provider time under its protected keyspace; workers fail closed when the record is absent or unavailable and never fall back to a local clock. Consumer configuration selectslease_authority: "shared-provider" with an explicitauthority_prefix; the persisted single-authority mode is reserved for an owned test cluster. Production deployment must enforce one write-capable authority identity per prefix, read-only consumer credentials, a monitored clock-skew bound, and publication more often than the smallest lease TTL; authority time must be republished after authority restart. These controls are deployment contracts, not permissions created by the library. The authority prefix is also checked against FoundationDB's key-size limit at construction, so an oversized prefix fails before its first transaction.
Across every split-store composition, new blocks are flushed before metadata references are published. A provider acknowledgement is not automatically a power-loss or distributed-locking guarantee.
Inspect a volume-scoped key range with fdbcli
# Use a dedicated cluster file and an owned prefix.
fdbcli -C /path/to/fdb.cluster
# Keys contain a NUL separator; inspect the metadata and block ranges
# for <prefix> with the FoundationDB CLI/key-value tooling.
getrange <prefix>\x00meta/ <prefix>\x00meta0
getrange <prefix>\x00block/ <prefix>\x00block0Credentials, cleanup, and safe operations
Keep the cluster file and native client outside the repository. Grant the storage authority only its keyspace, and clear/delete only a volume prefix after exclusive ownership is established. A broadclear_range is irreversible and is not normal test cleanup.
Current limits
The feature is opt-in and requires a matching FoundationDB 7.4 native client and cluster. The production path now fails closed for unverified, development, and single-authority clocks. Hosted runtime, deployment-enforced read-only authority credentials, multi-host clock-skew/recovery controls beyond the tested topology, production identity/ACL/TLS, backup/restore, capacity, multi-day soak, failover rehearsal, hosted root integration, and broader platform coverage remain open; commit versions are not wall-clock expiry. The shared-provider Rust SDK/CLI and Node selection paths are opt-in native features, not portable-default support, and the authority service must enforce the read-only worker boundary outside the library API. The durable Ozone composition is loopback/non-secure test deployment evidence; it does not establish production authentication, TLS, or power-loss durability.
Sources: FoundationDB integration README · FoundationDB keyspace implementation · FoundationDB workstream evidence · Durable composition harness · Ozone durability progress ledger · Previous hosted FoundationDB qualification · Previous hosted W07 FoundationDB qualification · Previous hosted W07 FoundationDB qualification · Current-public-main W07 FoundationDB qualification · FoundationDB publication read-overlap source · Queued exact-head read-overlap qualification · Candidate W07 assembly qualification