How to Scale Agent Memory Horizontally Without Two DBs

Your agent memory worked fine with five test users. Then you shipped, and now two hundred agents are hitting it at once, each one doing a semantic lookup, walking a few hops of relationships, and pulling recent events before every single LLM call. Latency climbs. The vector DB is fine. The graph DB is fine. The thing that is not fine is the sync job between them.

This guide takes a stance: agent memory is a read-heavy workload, so scaling it is mostly a read-scaling problem. You don't need to shard the world. You need to know your read/write ratio, put replicas behind the reads, cache the subgraphs agents keep revisiting, and partition by tenant. Those steps work on any stack.

By the end you'll have a concrete plan to scale agent memory horizontally, a staleness budget per read type, and a clear view of why running a vector DB plus a graph DB doubles every step. You'll also see where HelixDB's open-source core and HelixDB Cloud fit.

Prerequisites: an agent memory layer already in a prototype, basic metrics on your query path, and a rough idea of how many concurrent agents or users you need to serve.

Step 1: Measure your agent memory read/write ratio first

Don't add a single replica until you have numbers. Most teams guess that memory is write-heavy because agents "remember things". It almost never is. An agent turn typically does one write (store the new event or fact) and several reads (semantic recall, neighbour expansion, recent history, entity lookup). Count them.

Instrument three things per agent turn:

• Reads per turn, split by type: pure vector lookup, graph traversal, full-text, and time-range scans.

• Writes per turn, and whether they are single-entity inserts or multi-entity batches (an event plus its edges plus an embedding).

• Read latency at p50 and p99 under real concurrency, not a single-user benchmark.

If you have a ratio of, say, 8 reads to 1 write per turn, your scaling problem is a read problem, and a single serialized writer is fine. If you're seeing 1 to 1, look at your write batching before you look at replicas.

Also tag every read with the tenant (user, org, or agent fleet) that issued it. You'll need that in Step 4, and retrofitting it is miserable.

The contrarian bit: p99 matters more than throughput here. An agent that waits 900ms on a memory read blocks an LLM call that costs real money per second of held context. Track tail latency per read type, and write down the number you're trying to beat. If you want a method for doing this properly, the guide on how to benchmark a graph database for AI retrieval covers the traps.

Step 2: Add read replicas, and set a staleness budget per read

Once you know reads dominate, the standard move is read replicas: one writer, many readers. The part people skip is deciding how stale a read is allowed to be.

Not every read needs the latest write. Sort your memory reads into buckets:

• Must be fresh: the agent just wrote a fact and the very next step depends on it. Route these to the writer, or to a reader you know has caught up.

• Seconds of staleness is fine: semantic recall of older conversations, entity profiles, relationship lookups across prior sessions.

• Minutes is fine: analytics-style scans, "what has this user talked about this month".

Write those budgets down as a number per read type, for example 0s, 5s, 60s. Then your router has a rule instead of a vibe. A read-your-own-writes path can use a session token or a "last write version" the client carries, and the router sends the read to any reader at or past that version.

The failure mode to design against is stale recall that looks like a hallucination. An agent stores "user moved to Berlin", the next turn reads from a lagging replica, and the agent confidently says the user lives in London. The model gets blamed. The replica was the culprit.

Keep the replica count boring. Start with two or three readers, watch p99 and replica lag, and add more only when the numbers say so. Horizontal read scaling is cheap when the read path is stateless and the data is durable somewhere shared. It's painful when each replica owns a full copy it has to sync itself.

Step 3: Cache hot subgraphs, not just hot vectors

Most caching advice for agent memory stops at "cache the embedding" or "cache top-k results for repeated queries". That helps, but agents don't just ask similar questions. They revisit the same neighbourhoods: the same user entity, their recent threads, the projects those threads touch, the people attached to those projects.

That neighbourhood is a subgraph, and it's the unit worth caching.

Here is how to find your hot subgraphs:

1. Log the root entity ID and the hop pattern for every traversal-based read.

2. Group by root entity and count. A power-law distribution shows up fast: a small share of entities drive most of the traversals.

3. Cache the expanded result (root plus N hops plus the properties you actually return), keyed by root and pattern.

4. Invalidate on write to any node or edge inside the cached region, or accept a TTL that matches the staleness budget from Step 2.

Caching only vectors misses the expensive part. A vector lookup is one index probe. A three-hop expansion with property fetches touches many records, and that's where tail latency lives.

You can also narrow the work up front instead of caching after the fact. When vector search is scoped to entities related to each other, similarity runs over a graph-filtered candidate set instead of the whole index. HelixDB supports this as vector pre-filtering. Fewer candidates means less to rank, and less to cache.

Keep the cache layer thin and keyed to your read types. If it grows its own query logic, you've built a third database.

Step 4: Partition by tenant before you partition by anything else

When teams hear "scale horizontally" they reach for sharding by hash of node ID. Don't. Agent memory already has a natural partition key: the tenant. A user's memory, an organisation's company brain, one customer's agent fleet. Queries almost never cross that boundary.

Partitioning by tenant buys you three things at once:

• Smaller working sets. A vector index or graph region scoped to one tenant is far smaller than the global one, so searches touch less data and caches stay warm.

• Noisy-neighbour control. You can see which tenant is hot (you tagged reads in Step 1) and move or throttle it.

• Clean deletion and export. Removing a customer's data is a partition drop, not a scavenger hunt across edges.

Generic hash sharding gives you none of that, and it makes multi-hop traversals cross shard boundaries, which is exactly the thing graph reads are bad at.

One honest caveat: tenant partitioning is data isolation on shared infrastructure. It isn't an access-control system. If you need per-user permissions inside a tenant, enforce that in your application layer.

In HelixDB, a vector index can be global or tenant-partitioned, so the partition choice lives at the index level. For a deeper walkthrough of isolation patterns, see multi-tenant agent memory: how to isolate per tenant.

Order matters. Tenant first, then replicas inside the hot tenants, and only then anything exotic.

Step 5: Stop running two clusters for one memory (why vector DB + graph DB doubles the scaling problem)

Now count what Steps 2 through 4 cost when your memory is split across a vector DB and a graph DB.

Replicas: two clusters, two replication topologies, two lag numbers. Your staleness budget from Step 2 is now a budget across both, and the worse of the two wins. A fact can land in the graph store and not yet in the vector store, so a semantic recall misses something a traversal would find.

Caching: the hot subgraph from Step 3 is half in one system and half in the other. To cache an expanded neighbourhood with similarity scores attached, you join across databases in your application, then cache the join.

Partitioning: you partition tenants in the vector DB, and again in the graph DB, and keep the two partition maps consistent. A tenant migration is two migrations that have to agree.

And underneath all of it sits the sync job. Every write becomes a dual write or a change-data pipeline, with its own failure modes. When it breaks at 2am, memory is inconsistent and nobody knows which side is right.

That's the two-database tax, and it grows with your read fleet. Every scaling step you take, you take twice and then reconcile.

The alternative is one engine where graph traversal, vector search, and BM25 full-text run in the same query over the same data. That's what HelixDB is: vectors are a property on nodes and edges, and a single query can combine semantic similarity, keyword search, and multi-hop traversal. One replica topology, one cache, one partition map, one transaction boundary. Graph and vector writes commit or roll back together, so there's no window where the two views disagree.

Step 6: Scale reads in HelixDB: open-source core vs HelixDB Cloud

Be precise about which HelixDB you're scaling, because the two run modes behave differently.

The open-source core (Apache-2.0, github.com/HelixDB/helix-db) runs embedded in your application, or as a self-hosted Docker server, in memory, on disk, or against S3-compatible object storage. It's fast and simple, and a single node bounds it. For a lot of agent workloads that's plenty. The TypeScript client provides Client.embedded for embedded use and Client.embeddedReader for read-only access to an existing disk or object-storage database.

import { Client } from "@helix-db/helix-db";

const writer = await Client.embedded({ kind: "disk", database: "./memory" });
const reader = await Client.embeddedReader({ kind: "disk", database: "./memory" });

HelixDB Cloud is the managed, object-storage-backed service. A gateway routes traffic, a single writer serializes mutations, and reader nodes auto-scale horizontally, with tiered caching and high availability. Transactions are serializable snapshot-isolation ACID. That maps straight onto the steps above: the single writer matches the read-heavy shape you measured in Step 1, and the readers are your replica tier from Step 2. Because the data lives durably in object storage, adding a reader doesn't mean copying a full dataset to it.

The queries stay the same either way. You write JSON queries or use the native SDKs (TypeScript, Python, Go, Rust), sent to POST /v2/query. Same engine, same SDKs, same endpoint across embedded, self-hosted, and Cloud, so moving up is a configuration choice.

For pricing, check helix-db.com/pricing directly, since plans change. For the object-storage design, read graph database on object storage.

Step 7: Verify it, then troubleshoot (stale recall, hot tenants, writer pressure)

Don't trust the architecture diagram. Verify with load that looks like your agents.

The verification run:

1. Replay a captured trace of real agent turns at your target concurrency, with the read/write mix you measured in Step 1.

2. Record p50 and p99 per read type, and compare against the number you wrote down.

3. Inject a write and read it back through the staleness path for each budget bucket. Confirm the "must be fresh" reads never see old data.

4. Kill a reader mid-run and confirm traffic reroutes without errors.

Troubleshooting the usual three:

• Stale recall. The agent "forgets" something it just stored. Check replica lag first, then whether the read was routed through the freshness path. Fix the routing rule, not the prompt.

• Hot tenants. One tenant's p99 is bad while everyone else is fine. Your Step 1 tags will show it. Give that tenant its own reader capacity or cache budget, and check whether its subgraph cache is being invalidated too often by writes.

• Writer pressure. If the single writer saturates, you probably have chatty single-entity writes. Batch the event, its edges, and its embedding into one transaction. Reads scale out, so don't solve a writer problem with more readers.

One limit worth knowing when scoping searches: an unrestricted vector search caps at 800 effective results, and a traversal-scoped search rejects rather than silently clamping if it exceeds the ceiling. If recall looks truncated, check whether you hit a documented limit before you suspect the index.

Conclusion

Scaling agent memory is a read problem with a tenant-shaped partition key. Measure the ratio, set a staleness budget per read, cache subgraphs, partition by tenant, and refuse to run those steps twice across two clusters.

If your memory layer is a vector DB plus a graph DB held together by a sync job, make that the first thing you remove, before you add a single replica. Clone the open-source core from github.com/HelixDB/helix-db, embed it, and run your Step 1 trace against one engine that does graph, vector, and full-text in a single query. Star the repo if the p99 numbers hold up, and join the Discord to show us what you're building. When one node is no longer enough, the same queries move to HelixDB Cloud's auto-scaling readers.