An engineer asks your internal AI assistant a straightforward question:
"If we retire this authentication service, which customer-facing applications could break?"
The assistant retrieves the service documentation, an architecture overview, and a migration guide. Its answer is clear, well written, and supported by citations.
It still misses an application that depends on the service through a shared gateway.
Nothing in the answer looks obviously broken. The retrieved documents are relevant. The missing piece is the connection between them.
This is a useful place to examine your retrieval architecture. Finding text about a system and tracing its dependencies are different tasks.
In a basic vector RAG pipeline, a query is embedded, similar chunks are retrieved, and those chunks become context for the model.
That works well when the evidence needed for an answer appears in the retrieved passages.
Our dependency question is harder. One document might say the checkout application uses a gateway. Another says the gateway delegates authentication to the service being retired. Neither passage needs to describe the complete chain.
A single similarity search may retrieve the authentication document while missing the application document. Increasing the number of chunks can help, but it can also bring in unrelated material without supplying the missing link.
Before changing the prompt, ask: did the model receive enough evidence to answer the question?
Figure 1 is the three-way comparison: direct LLM, vector RAG, and GraphRAG answering the same question about 19th-century art movements.
Figure 1: Direct LLM, vector RAG, and GraphRAG answer the same question. Retrieved text gives passages; retrieved triplets give connected facts.
A knowledge graph represents entities and the relationships between them explicitly.
For our example, you could record two facts:
Checkout application → depends on → API gateway
API gateway → authenticates through → Authentication service
Now the system has a path it can inspect when answering the retirement question.
In a graph-augmented retrieval design, the retriever can locate the service, follow relevant dependency relationships, and provide the resulting evidence to the LLM. The model then has a connected explanation to work with.
The direction of traversal matters. To find what could be affected by retiring a service, you need to follow its dependants, including indirect ones.
This does not mean every RAG system needs a graph. Query decomposition and iterative document retrieval can also uncover connected evidence. A graph becomes worth investigating when relationships are central to the questions you repeatedly need to answer.
Finding the right entity is only the beginning.
Retrieving the authentication service node might tell you its owner and status. It will not necessarily tell you which applications depend on it.
Chapter 7 distinguishes several levels of retrieval: nodes, triplets, paths, and subgraphs.
Figure 2 is the retrieval-level diagram: node, triplet, path, and subgraph retrieval on the same graph, with the retrieved portions highlighted.
Figure 2: The four retrieval levels. Node retrieval returns one entity; triplet retrieval returns direct relationships; path retrieval returns a connected chain; subgraph retrieval returns a neighborhood.
For "Who owns this service?", a node property or a single relationship may be enough. For "Which applications depend on it indirectly?", you need connected relationships.
Retrieving every nearby node is not automatically useful. As you expand the search, the context can fill with relationships that do not help answer the question.
The engineering decision is how much connected evidence the task requires, and where to stop.
Explicit relationships are useful only when they reflect the system accurately.
If an extraction pipeline confuses two services with similar names, the graph can connect the wrong applications. If a dependency changed last month, an outdated edge can produce a convincing but incorrect explanation.
For the retirement example, I would retain the source and update time for each dependency. I would also distinguish "no dependency found" from "confirmed independent." An incomplete graph cannot establish that an application is unaffected.
Graph retrieval changes how you find evidence. You still need to check whether that evidence is trustworthy.
Choose a question whose answer requires connecting facts from two or three sources. Write down the expected evidence chain before running it through your assistant.
Then inspect the retrieved context. Did it contain every required relationship? Did the answer stay within what those sources support?
That small exercise can reveal whether the next improvement belongs in retrieval, data quality, or answer generation.
Building AI Agents with LLMs, RAG, and Knowledge Graphs, by Salvatore Raieli and Gabriele Iuculano, explores how these components fit together. Chapter 7 moves from constructing a knowledge graph to retrieving connected information and using it as context for an LLM.
Figure 3 is the book cover
Figure 3: Building AI Agents with LLMs, RAG, and Knowledge Graphs, by Salvatore Raieli and Gabriele Iuculano (Packt, 2025).
If you want to turn this concept into a working system, explore the book's practical examples and build alongside them. The book is available in paperback and eBook on Amazon.
Figure 4 is the call-to-action button at the end of the original piece.
Figure 4: The call to action from the original piece.
The paid section below includes example code, suggested best practices, and the builder’s playbook.
