Reviewer feedback routing: comment to block to KB

A reviewer leaves an inline comment on paragraph 12 of a red-team draft: “this claim is stale — we retired that SOC-2 control last quarter.” Three weeks later, a different proposal in a different region uses the same sentence. Because the comment lived on the draft, not on the source.

This post is how we route reviewer feedback from a draft comment into an edit on the KB block the draft was generated from, so the next draft starts from a corrected source instead of inheriting the stale answer.

The shape of the problem

A draft paragraph is not a first-class object. It is a rendering of one or more KB blocks, interpolated with RFP-specific context, produced by a model. If the reviewer corrects the rendering, the correction disappears the moment the next draft runs. If the reviewer corrects the underlying block, every future draft inherits the correction.

Most review tools fix the rendering. Google Docs comments, Word track-changes, the proposal-tool-of-the-week comment sidebar — all of them attach feedback to the document, not to the source. The correction rides the lifecycle of that single document and dies when the document does.

The data model

We attach three identifiers to every rendered paragraph:

type RenderedParagraph = {
  paragraphId: string;          // unique per draft paragraph
  draftId: string;              // the proposal draft
  sourceBlocks: {
    blockId: string;            // KB block
    blockVersion: number;       // version at render time
    spanOffset: [number, number]; // character range within block
  }[];
};

When a reviewer leaves a comment on a paragraph, we store the comment with a pointer to both the paragraph and the source blocks. The routing logic decides which block the comment is about — usually the one whose span contributed the most text to the paragraph, but sometimes the reviewer has explicitly highlighted a span and the routing is obvious.

The routing is a function:

function routeComment(
  comment: Comment,
  para: RenderedParagraph
): BlockEditRequest {
  const targetBlock = comment.selectedSpan
    ? findBlockBySpan(para.sourceBlocks, comment.selectedSpan)
    : para.sourceBlocks[0]; // largest contributor

  return {
    blockId: targetBlock.blockId,
    fromVersion: targetBlock.blockVersion,
    comment: comment.body,
    suggestedEdit: comment.suggestion ?? null,
    raisedBy: comment.authorId,
    context: {
      draftId: para.draftId,
      paragraphId: para.paragraphId,
    },
  };
}

The BlockEditRequest lands in a KB owner’s queue. The KB owner is the person who owns that block — usually an SME for technical blocks, a proposal manager for company-wide boilerplate, a legal reviewer for compliance blocks. Ownership is metadata on the block.

The KB owner’s queue

Each block has an owner. Each owner has a queue of incoming edit requests sourced from reviewer comments across all active drafts. The queue interface is deliberately boring: a list of requests with the originating paragraph, the originating draft, the comment body, and three buttons — Apply as suggested, Apply with edits, Reject with reason.

When an owner applies the edit, the block version bumps. Every downstream artifact — active drafts that cite the old version, rendered proposals, retrieval indices — gets notified. Active drafts get a subtle badge: “the source for paragraph 12 has been updated since this draft was rendered.” The drafter can re-render the paragraph with the new source, or keep the old render if the RFP is days from submission.

Rejected requests close with a written reason that the original commenter sees. This is the detail that matters most. A rejection without a reason teaches the reviewer nothing; their next comment has the same flaw. A rejection with “this claim is correct as of Feb 2026 — you’re thinking of the ISO 27001 control we retired, not SOC-2” closes the loop.

What we measure

Three numbers, all weekly:

• Routing rate. Of all comments left on drafts, what percentage reach a block-edit request? Target: above 80%. The rest are typically format suggestions, tone nits, or comments on RFP-specific language that doesn’t live in a block. Those stay on the draft.
• Block-edit turnaround. Median time from comment to block edit applied. Target: under 48 hours during active proposal cycles. If this grows, owners are overloaded and the routing becomes a backlog that never drains.
• Inheritance rate. Of block edits applied, what percentage are referenced by a subsequent draft within 30 days? If the number is low, the edit was a one-time RFP artifact and didn’t belong in the KB. If the number is high, the routing is doing its job — reviewer attention is becoming corpus attention.

Where it breaks

Three failure modes we have found so far.

False consolidation. The reviewer leaves a comment on a paragraph that rendered from three blocks. The comment is about block A. The routing picks block B because block B contributed more text. The owner of block B looks at the request, can’t map it to their block, and rejects. The comment is lost. Fix: when the reviewer highlights a span, use the span; when they don’t, show the owner the full source context and an “actually this is about another block” button.

Owner churn. A block’s owner leaves the company. Edit requests queue against a vacant seat. We now route ownerless blocks to a proposal-ops default queue and alert the proposal ops lead. Not elegant. It works.

The reviewer was wrong. Sometimes the claim is correct and the reviewer’s correction is not. Rejection with reason closes the loop, but we lose the reviewer’s time and trust. We added a lightweight verification step: before an owner can apply an edit that contradicts an existing block with a recent source citation, the interface asks them to confirm the source is no longer canonical. This has caught three reversals in the last month.

Takeaway

Reviewers leave comments on the draft because that’s where the rendering is. The comments are worth keeping even after the draft ships. A boring routing layer — comment to paragraph to block to owner queue — is what turns reviewer time into corpus improvement. Without it, you re-learn the same lessons in every RFP cycle.

This is one of the stages in the eight-stage pipeline that most tools leave on the floor. The post-mortem stage gets the glory; the review-to-KB loop does the compounding. We wrote more about the review discipline in color-team reviews, modern teams. The citation plumbing that makes paragraph-to-block attribution possible is covered in citation rendering stack.

Posts bylined to “The PursuitAgent engineering team” are written by the people building the product. Views reflect PursuitAgent’s position; mechanisms described match what’s in the product at publication date.