Knowledge Graph RAG: How to Retrieve Over Entities and Edges

Knowledge graph RAG is retrieval-augmented generation where the retriever finds seed entities by vector and keyword search, then walks the edges between entities to pull in connected context that top-k similarity would miss. This guide builds it step by step: schema, embeddings, hybrid seed retrieval, scoped expansion, and context assembly. You first see it as a two-store stack (a vector DB plus a graph DB), then collapsed into one engine with HelixDB.

Prerequisites: you know embeddings, ANN search and BM25, and you already have a RAG pipeline that returns orphaned chunks. You don't need a definition of a knowledge graph. You need a retrieval design that works.

Step 1: Decide what knowledge graph RAG retrieves (entities, edges, chunks) and when top-k is enough

Knowledge graph RAG retrieves three things: entities (the nodes your domain cares about), edges (the typed relationships between them), and chunks (the source text that backs each entity or edge). Plain RAG retrieves only chunks, ranked by similarity to the query.

Top-k is enough when the answer lives inside one passage: a policy lookup, a definition, a single API reference. Add a graph when the answer spans passages that share no vocabulary with the query. "Which contracts are affected by the vendor that failed last quarter?" has a vector-friendly half (the vendor failure) and a relationship half (the contracts), and no single chunk contains both. Top-k returns the failure report and stops.

The benefits of knowledge graph RAG follow from that gap. Edges connect facts that embeddings treat as unrelated, so you get multi-hop answers. Every hop is an edge you can cite, so answers are traceable. Expansion follows relationships instead of widening k, so you get less irrelevant context. Research on knowledge-graph-guided RAG reports gains from using graph structure to guide retrieval. If you're unsure which side of the line you're on, read when to use a knowledge graph instead of embeddings first.

Step 2: Define the entity schema: node types, edge types, and where chunks attach

A knowledge graph is simple: nodes are things, edges are typed relationships between them, and both carry properties. Building one starts with a schema small enough to extract reliably.

Pick 3 to 6 node types from your domain. For a support knowledge base that might be Product, Component, Issue, Resolution, Document. Pick edge types that answer questions you actually get: AFFECTS, RESOLVED_BY, DEPENDS_ON, MENTIONS. Resist the urge to model every noun.

Then decide where chunks attach. The pattern that works: each Chunk node holds the text, its embedding, and a PART_OF edge to its Document, plus MENTIONS edges to each entity extracted from it. Entities keep a short description with its own embedding. Every entity can reach its evidence, and every chunk can reach the entities it talks about.

Two rules keep this tractable. Give entities canonical keys (a normalized name plus type) so extraction runs don't create duplicates. And store the source chunk id on every edge you derive, so Step 6 can cite it. A paper on scalable GraphRAG for enterprise text found that a lighter extraction path can be much cheaper than LLM-based extraction. Start simple and upgrade extraction only where recall suffers.

Step 3: Embed entities and chunks and index them for vector and BM25 search

Embed two things: chunk text and entity descriptions. Use one embedding model for both so a single query vector can score either. Store the vector as a property on the node.

Index each for both retrieval modes. For vector search, build an ANN index per node type with an explicit dimension and distance metric (cosine is the usual default). For keyword search, build a BM25 index over chunk text and entity names. BM25 matters more than people expect in graph retrieval. Entity names, product codes and error strings are exact tokens that embeddings blur.

In the two-store version of this stack, the vectors go to a vector database and the nodes and edges go to a graph database, keyed by a shared id. That works, but you now own a sync job. Every re-embed, deletion or schema change has to land in both stores, and a lagging sync produces seed ids that the graph can't find. Keep that in mind for Step 7.

HelixDB indexes vector, BM25 and graph data in one engine. Vector indexes need a declared dimension and metric. The HelixDB docs cover index definition.

Step 4: Seed retrieval with vector plus BM25 and fuse the results

Seed retrieval finds the entry points into the graph. Run vector search and BM25 in parallel over the same query, then fuse the two ranked lists.

Use reciprocal rank fusion (RRF). For each result, sum 1 / (60 + rank) across the lists it appears in, then sort. RRF needs no score normalization, which matters because cosine similarity and BM25 scores live on different scales. Take the top 5 to 10 fused hits as seeds.

Two details improve seeds. Search both chunks and entities, then map chunk hits to the entities they mention, so a seed is always an entity. And keep the query's own entity mentions: exact-match lookup on entity names, as used in enterprise GraphRAG work, is cheap and catches what both rankers miss.

Here is the seed step against HelixDB with the TypeScript SDK (npm install @helix-db/helix-db). The same query shape runs embedded or against a server:

import { g, VectorDistanceMetric } from "@helix-db/helix-db";

const vectorQuery = g()
  .nodes("Chunk")
  .vectorSearch("embedding", queryVector, {
    metric: VectorDistanceMetric.Cosine,
    limit: 10,
  });

const bm25Query = g()
  .nodes("Chunk")
  .search("content", queryText)
  .limit(10);

const [vectorHits, textHits] = await Promise.all([
  client.query(vectorQuery.toQueryRequest()).send(),
  client.query(bm25Query.toQueryRequest()).send(),
]);
// fuse with RRF in application code

Fusion stays in your code because it's a few lines and you will tune it.

Step 5: Expand from the seeds over edges, with scoped (pre-filtered) vector search

Expansion walks outward from the seed entities along edges to collect connected context, and scoped vector search keeps that context relevant to the query.

Start with a bounded traversal: 1 to 2 hops, a whitelist of edge types, and a cap on fan-out per node. Unbounded expansion is the most common way to wreck a graph RAG pipeline, because a hub node (a popular product, a generic concept) drags in thousands of neighbors. The cheapest precision gain available is whitelisting edge types per question class: dependency questions follow DEPENDS_ON, incident questions follow AFFECTS and RESOLVED_BY.

The second tool is scoped search. After traversal you hold a candidate neighborhood of a few hundred nodes. Instead of ranking them by hop distance alone, run vector search restricted to that neighborhood: similarity to the query, over only the graph-connected candidates. In the two-store stack you do this by sending candidate ids to the vector DB as a filter. That means a network round trip, an id list that grows with fan-out, and often a post-filter that quietly returns fewer than k results.

HelixDB runs this in one query. Vector search can be scoped to related nodes, so similarity runs over the graph-filtered candidate set, not the whole index. Traverse first, then search:

import { g, VectorDistanceMetric } from "@helix-db/helix-db";

const scoped = g()
  .nodes("Entity") // seed entity ids bound here
  .out("DEPENDS_ON")
  .out("MENTIONED_IN")
  .vectorSearch("embedding", queryVector, {
    metric: VectorDistanceMetric.Cosine,
    limit: 10,
  });

Know the documented limits. An unrestricted vector search caps at 800 effective results. A traversal-scoped search applies that ceiling after candidate intersection and errors instead of clamping silently. A candidate stream over 1,000,000 unique entities is a query error. Bound your traversal and you stay far inside them. Check the exact traversal step names against the current query guide in the HelixDB docs before you copy this.

Step 6: Assemble the context: rank, dedupe, cite edges and sources

Context assembly turns the retrieved subgraph into a prompt-sized, citable package: rank the candidates, dedupe, then serialize entities, edges and chunks together.

Rank by a blend: fused seed score, scoped vector similarity, and a small penalty per hop. Dedupe on chunk id first (the same chunk arrives via several paths), then on near-duplicate text. Cap by token budget, not by count. Drop the lowest-ranked chunks before you drop any edge, because edges are short and carry the relational facts the model can't infer.

Serialize in a form the model can read. A reliable layout is a short list of relationship triples (Component A --DEPENDS_ON--> Component B [source: chunk_41]) followed by the supporting chunk texts grouped by document. The triples give the model the structure, and the chunks give it the evidence. Because every edge stores its source chunk id (Step 2), each triple cites its origin, and your answer can cite sources instead of asserting them.

This is also where graph RAG earns its "explainable" reputation. When a user disputes an answer, you can show the path: seed, edge, edge, chunk. Plain top-k can't show a path because there was none.

Step 7: Collapse the two-store stack into one engine with HelixDB

HelixDB is an open-source graph-vector database written in Rust that runs graph traversal, vector search and BM25 full-text search in one engine, so the vector DB plus graph DB split disappears.

Here is what the two-store version costs you, mapped to the steps above:

Concern	Vector DB + graph DB	HelixDB
Storage	Embeddings in one store, nodes and edges in another, joined by id	Vectors are a property on nodes and edges, same graph
Keeping data in sync	Dual writes or a sync job; lag yields dangling seed ids	One write path; graph and vector writes commit or roll back together
Scoped vector search	Send candidate ids across the network as a filter	Traverse, then vector search over that candidate set in one query
Keyword search	Third system or one store's add-on	BM25 text indexes in the same engine
Operations	Two clusters, two bills, two failure modes	One engine

The id-list round trip in Step 5 and the sync job in Step 3 both vanish. HelixDB's open-source core is Apache-2.0 at github.com/HelixDB/helix-db, so you can embed it in your application or run it in Docker. HelixDB Cloud is the managed option, backed by S3-compatible object storage, with serializable snapshot-isolation ACID transactions.

To try it locally, run helix init local then helix start dev (needs Docker or Podman; it serves on port 6969). HelixDB provides native SDKs for TypeScript, Python, Go, and Rust that build JSON query requests, so there is no separate query language to learn. If you want the wider build around this retrieval step, see the guide on the best graph database for a RAG pipeline.

Step 8: Check retrieval quality and troubleshoot common failures

Check retrieval quality before generation. Build 30 to 50 real questions, mark the chunks and edges that should be retrieved, and measure recall of those at your token budget. Compare three configurations on the same set: top-k only, seeds only, and seeds plus expansion. If expansion doesn't beat seeds on the multi-hop questions, your graph is the problem, not your retriever.

The usual failures:

• Expansion floods the context. A hub node pulled in its whole neighborhood. Whitelist edge types, cap fan-out, and run scoped vector search so only query-relevant neighbors survive.

• Seeds miss the right entity. Duplicate entities from inconsistent extraction split the evidence. Canonicalize keys and merge before indexing.

• Good seeds, empty expansion. Edges are sparse because extraction only linked entities within one chunk. Add document-level MENTIONS edges and co-occurrence edges.

• Answers cite nothing. Edges lack source chunk ids. Add them at write time, because backfilling later is painful.

• Scoped search errors. Your candidate set exceeded the documented ceiling. Tighten the traversal instead of raising k.

For a broader evaluation method, see how to evaluate agent memory. The same discipline of fixed questions and recall measurement applies.

Start with seeds and one hop, then measure

Knowledge graph RAG isn't a bigger index. It's a retrieval design: hybrid seeds, bounded edge expansion, scoped vector search, cited assembly. Build it with one hop and a whitelist of edge types, and let your recall numbers tell you whether to go deeper. To skip the two-store plumbing, run helix init local and helix start dev, load a slice of your entities, and star HelixDB on GitHub if the single-query version holds up for your data.