The answer provenance graph in the KB
Every block in the knowledge base tracks source, author, approver, and last-used-in. The provenance graph isn't bookkeeping — it's a product surface. Here's what it stores and what it powers.
Every answer block in PursuitAgent’s knowledge base carries a provenance graph. The graph is not a single field on the block; it is a structured set of relationships that track where the block came from, who has touched it, who approved it, where it has been used, and what it currently inherits from. Provenance is a product surface, not engineering bookkeeping. This post explains what the graph stores and what it powers.
Why provenance matters
The Stanford HAI study on grounded legal RAG measured hallucination rates of 17–33% on commercial systems that wired citations into their output. Those systems had pointers. What they often lacked was the rest of the provenance chain: which human approved this content, which document is the source of record, what version of the source the block was distilled from, and when the block was last reviewed. Without that chain, a citation footer is a UI element. With it, the citation is auditable.
A proposal evaluator on the buyer side does not click a citation to admire the engineering. They click it to verify the claim. The provenance graph is the structure that makes verification cheap.
What the graph stores
For every approved KB block, the graph stores six relationships, each with timestamps, actor IDs, and version pointers.
Source documents. The block’s content was distilled from one or more source documents — a security policy, a past proposal section, a product spec, a compliance attestation. The graph records the document IDs and the page-range or section spans that the block was derived from. A block without a source-document edge cannot be approved into the production library.
Author chain. Who created the block, who edited it, in what order, with what diffs. The author chain is append-only. An edit to a block does not overwrite the author history; it adds a new entry that points at the previous version’s author.
Approver edge. The senior reviewer who promoted the block from draft state to approved state. Approval is not edit; the approver may not have written any words. The approver edge encodes accountability — when a buyer asks “who is responsible for the accuracy of this answer,” the answer is a name.
Inherited-from edges. A block that was created by editing another block (for a buyer-specific tone shift, for example) keeps an inherited-from pointer to the parent. Updates to the parent surface in the child as inheritance signals — see the recent shipped post on cross-project inheritance.
Last-used-in edges. Each time a proposal references the block, an edge is recorded with the proposal ID, the section it was used in, and the timestamp. The recent-usage subgraph is the input to freshness signals: a block last used six months ago is more likely to have drifted from current reality than one used last week.
Approval-as-of-version edge. The approver doesn’t approve “the block in general” — they approve a specific version. If the block is later edited, the edit is in draft state until re-approved. Production retrieval pulls only blocks at their currently-approved version.
How retrieval uses the graph
Retrieval over the KB does not stop at semantic similarity. The graph layer applies four filters and one re-ranker.
Filter: approved-only. Blocks whose latest version is in draft or pending-approval state are not returned to the drafting pipeline. They are visible in the KB UI to authors and reviewers; they are invisible to the generator.
Filter: scope. Some blocks are scoped to a specific buyer, vertical, or product line. The graph encodes scope edges, and the retrieval layer filters on the active proposal’s context. A block scoped “federal civilian” is not returned for a state-and-local query.
Filter: freshness threshold. Blocks that have not been reviewed within a configurable freshness window (default 90 days for compliance content, 180 days for capability content, never for boilerplate corporate-info) are flagged or, on strict-mode pursuits, suppressed. The freshness threshold is the operator’s lever, not a hard product setting.
Filter: deprecated-edge. When a block is replaced — typically because a compliance attestation was superseded by a new audit, or a product capability changed — the old block has a deprecated-by edge to the new one. Deprecated blocks are invisible to retrieval but visible in the audit trail.
Re-ranker: usage-recency boost. Among blocks that pass all filters, the re-ranker biases toward blocks that have been used in proposals the team won. The signal is weak (any single proposal’s win/loss is high-variance), but at the population level it surfaces the answers that consistently land well with evaluators. We monitor for over-fitting; if a single block dominates the recency boost, we flag it for review.
What the graph powers
Three product surfaces run directly on the provenance graph.
The citation footer. Every drafted sentence in a PursuitAgent proposal carries a citation pointer. Behind the pointer is the full provenance chain — source document, page span, author, approver, last-used-in, version timestamp. The reviewer sees a citation; if they click it, they see the chain. If they need to verify a number with the SME who wrote it, the SME’s name is in the chain.
The freshness alerter. When a KB block is referenced by an in-flight proposal and the block crosses the freshness threshold, the proposal pane surfaces an alert. Not a block on submission — a heads-up to the reviewer that the cited content may have drifted. The reviewer’s call whether to ask the SME to refresh.
The post-mortem feed. When a proposal closes (won or lost), the post-mortem step traverses the proposal’s last-used-in edges in reverse — every block referenced, with its current version compared to the version-at-time-of-use. Blocks that were updated upstream during the bid window get flagged. Blocks that scored well in winning proposals (per buyer feedback, where we have it) get promoted to the recency-boost list. The feed runs automatically; the post-mortem reviewer sees a structured input rather than a blank page.
What we don’t store on the graph
Two things, both deliberate.
No bid-content lineage across customers. Block A was used in customer X’s proposal, then re-edited for customer Y. The graph stores the edit chain, but the proposal-to-proposal lineage is not surfaced as a “this block was previously sent to X.” We don’t want a content reuse pattern that exposes one customer’s proposal language to another customer’s review pane. Cross-customer leakage is a class of failure we engineer against; the graph is a place where the absence of an edge matters.
No model-prompt history. Some experimental drafting systems track the LLM prompt that produced each generated sentence. We don’t store that on the production block graph. The block is the unit of truth; the prompt that produced it is debugging metadata, not provenance. Model-prompt history lives in the engineering observability stack with shorter retention.
Why this is product, not infrastructure
A proposal team using PursuitAgent sees the provenance graph in three places — the citation footer, the freshness alert, the post-mortem feed. None of those surfaces are framed as “the provenance graph”; they are framed as “where did this answer come from,” “is this still current,” “what did we learn from the last bid.” The graph underneath is the structure that makes those questions cheap to answer.
The grounded-retrieval pillar makes the architectural case: pointer, provenance, entailment. This post is what the second word of that triple looks like when you implement it concretely. A pointer without provenance is a hash. Provenance is the difference between a citation footer and a verifiable claim.
The next post in the engineering build-log series covers retrieval over diagrams — how the provenance graph extends beyond text to D2 figures and diagram descriptions, and what the citation chain looks like when a “source” is a rendered image. That ships Thursday.