GraphRAGProv: GraphRAG Meets How-Provenance: what it means for business leaders

A demo system answers multi-hop questions by running SPARQL over an LLM-built graph and returns, with each answer, a provenance polynomial that resolves to the exact source text, exposing extraction and query errors behind correct answers.

1. What the method is

GraphRAGProv is a question-answering architecture in which the answer is computed by a database query, not written by a language model. An LLM converts source paragraphs into RDF statements, and every statement keeps a link to the exact characters in the text that support its subject, predicate and object. A second LLM step translates the user's question into a SPARQL query, which the GraphDB engine executes; the returned bindings are the final answer, with no text retrieval or generative fallback. The query is rewritten with NPCS so that each binding also carries a how-provenance polynomial over statement-occurrence identifiers: multiplication marks statements used together, addition marks alternative derivations. Following any factor of the polynomial leads to the extracted statement and from there to the passage it came from. The authors position querying itself as the way to inspect and evaluate the text-to-graph transformation.

2. Why the method was developed

Most GraphRAG systems, including Microsoft GraphRAG and HippoRAG, use the graph to decide which evidence reaches a language model, but the model still composes the answer. A retrieved path shows what was available, not how the value was computed, which is a weak basis for verification in settings such as medical diagnosis or legal decision support. Answering by querying a knowledge graph directly is well established when the graph already exists, but document collections require building the graph first, as Neo4j's LLM Graph Builder and KAG do. KAG reports answering from retrieved subgraphs in only 33, 34 and 18 percent of cases on HotpotQA, 2WikiMultiHopQA and MuSiQue, and falls back to text retrieval otherwise. A derivation alone names the statements used without showing whether they are faithful to the source. The authors built one chain from question to query to statement to exact text so that extraction errors, entity mismatches and bad queries can each be located.

3. Who should care

This touches any team that is building or buying retrieval over our own documents where an answer has to be defended, not just produced. Platform and data engineering leads who maintain knowledge-graph or RAG pipelines get a concrete pattern for grounding every extracted triple in character offsets and for exposing the computation behind an answer. Compliance, legal, audit and clinical-safety functions get something they can review: a derivation that resolves to quoted text rather than a model's paraphrase. Product owners of internal search and analyst tooling should note that this design returns database bindings, which are precise but fail silently with an empty result when entities are not reconciled. Procurement and vendor-management staff evaluating GraphRAG products can use this as a benchmark question: does the product show how an answer was computed and against which source text, or only which passages were retrieved. Research and data-quality teams gain a diagnostic workbench for extraction pipelines.

4. How the method works

Graph construction starts with an LLM, DeepSeek-V4-Flash here, that reads a paragraph and emits RDF triples together with quotations supporting each component; a relation such as nationality may be inferred from an adjective like Lebanese rather than copied verbatim. A deterministic localizer resolves each quotation to exact character spans. Quotations it cannot find trigger a correction request to the model, and anything still unresolved is rejected as evidence. Each grounded statement, which the authors call a symbolic chunk, receives an occurrence identifier and is reified with RDF-star so the triple, its paragraph and its spans stay connected; identical triples from different passages keep separate identifiers. At query time the same model writes SPARQL from the question, a Rust implementation of NPCS rewrites it to return bindings and provenance polynomials together, and a React and FastAPI workbench lets users click from a polynomial factor to its statement and highlighted text, or edit the question or query and watch the derivation change.

5. How it was evaluated

This is a demonstration paper, and the evaluation measures whether the trace exists rather than whether the answers are right. The authors took 200 compositional questions from 2WikiMultiHopQA and 200 answerable questions from MuSiQue, both multi-hop benchmarks. For each case the graph was built only from the benchmark's designated supporting passages, so document retrieval is outside the experiment. Graph construction and SPARQL generation used DeepSeek-V4-Flash with access to the question, the expected answer and the reasoning evidence; the authors call this a supervised setting and state that it does not establish generalization to unseen questions. The first generated query that passed mechanical validation and executed was kept, with no retry based on answer correctness. They then counted how many cases reached each stage: query executed, answer returned, provenance decoded, and a complete trace in which every factor resolves to a statement occurrence with matching source spans. No accuracy figures and no comparison against other GraphRAG systems are reported.

6. How it performed

Queries executed in 398 of 400 cases and returned an answer in 327, 193 on 2Wiki and 134 on MuSiQue. Complete text traces existed for 193 2Wiki and 118 MuSiQue cases, 311 in total; all passed the occurrence and span checks and matched ordinary execution. Sixteen MuSiQue cases answered without decodable provenance and 66 returned no answer. The inspected cases carry the lesson: a correct answer of India whose cited word described the film rather than its director, a correct year of 1986 from a query that never linked the game to its publisher, and an empty result because two IRIs stood for the same father. The authors stress that a correct answer does not imply a correct computation, and that the supervised setup, missing accuracy figures and absent baselines limit the claims. Our call is to watch: copy the provenance pattern when we next audit a text-to-graph pipeline, but deploy nothing from it yet. (Source: arXiv 2610.37237, 2026)