h5i-db is an embedded time-series database and event-driven backtesting engine for quantitative research. Built for AI agents, with native ASOF joins, OHLCV rollups, and high-throughput event replay.
cargo install h5i-db-cli
pip install h5i-db
A storage kernel and query engine tuned for the time axis: data is time-sorted on disk, pruned by manifest statistics before I/O, and queried through DataFusion SQL with native time-series operators. Version control, time travel, and previewable mutations come built in, so the speed never costs you safety.
ASOF JOIN, time_bucket, vwap, wavg, and
ewma. Storage is time-sorted, so bucketed aggregations stream instead of sorting.--stats.version_conflict, never last-writer-wins.h5i('trades', 42) reads version 42 in O(1) and an RFC3339 timestamp
reads as of a commit time in O(log V); named snapshots pin exact versions across tables, keeping
a backtest reproducible forever.--plan runs the full write path but stops before publish:
exact affected rows, segments rewritten vs reused, before/after samples. plan apply
is a metadata-only swap, and a mutation policy can force agents through review.verify --deep re-checks the whole store.A replay kernel that reads the same versioned tables everything else queries. No export step: the pinned dataset, the run, and its results are one object, so a result stays attributable to the exact data it was produced from.
book_deltas, trades, bars,
instruments, resolutions, funding. The kernel never
learns which vendor a row came from.bt_orders,
bt_fills, bt_positions, and bt_equity there. Forks share
data instead of copying it, so a wide sweep costs close to nothing and discards cleanly.alphalens and
empyrical parity, plus deflated Sharpe, probability of backtest overfitting, and
purged cross-validation, so a number found by searching is discounted for the search.Two workloads, each measured against the tools it would replace, with storage and I/O included on every side.
20M trades and 5M quotes: Polars, DuckDB, pandas, and PyArrow over the identical h5i-db Parquet segment files, ArcticDB over its own LMDB store loaded with the same data.
| workload | h5i-db | polars 1.35 | duckdb 1.3 | pandas 2.3 | pyarrow 24 | arcticdb 6.19 |
|---|---|---|---|---|---|---|
| time-range scan 0.01% (48/50 segments pruned) | 10.0 ms | 28.1 ms | 45.5 ms | 23.9 ms | 22.8 ms | 4.2 ms² |
| time-range scan 1% | 12.8 ms | 29.9 ms | 40.9 ms | 25.6 ms | 24.0 ms | 5.6 ms² |
| 1-minute OHLCV + VWAP rollup | 1 558 ms | 7 309 ms | 7 237 ms | 5 115 ms | 7 121 ms | 3 504 ms |
| ASOF join trades × quotes (by symbol) | 1 548 ms | 1 485 ms | 11 566 ms | 6 624 ms | n/a¹ | 7 008 ms |
| full aggregation (group by symbol) | 353 ms | 370 ms | 228 ms | 2 578 ms | 553 ms | 886 ms |
| ingest (50 durable versioned commits) | 1.9 M rows/s | n/a | n/a | n/a | n/a | n/a |
| cold read of an old version | 1.1 ms | n/a | n/a | n/a | n/a | n/a |
200k top-of-book events and 200 orders through each engine. Medians of three fresh-process runs after one warm-up, with every adapter verifying it saw all 200k events and submitted all 200 orders.
| engine | measured boundary | median | throughput |
|---|---|---|---|
| h5i-db | decoded records through the replay kernel | 65.7 ms | 3.05 M events/s |
| h5i-db | full persisted run: scan, decode, fork, replay, write | 331 ms | 605 k events/s |
| NautilusTrader 1.230.0 | in-memory objects through BacktestEngine.run() | 767 ms | 261 k events/s |
LEAN 11ba019f6 | first Slice callback to OnEndOfAlgorithm, disk-fed | 2 033 ms | 98.4 k events/s |
h5i-db ui market.db serves a loopback-only review surface:
pending plans with exact previews, apply and reject, a version timeline with audit badges,
version diffs, and an SQL scratchpad that shows pruning on every query. Read-only unless started
with --allow-mutations.
One embedded database for backtests, live research, and the agents that run them. Rust core, DataFusion SQL, versioned Parquet storage. Apache 2.0, no server, no lock-in.