The Quote That Ends Too Early

Everyone is watching for the hallucinated case. That is the wrong thing to watch for.

I have been running a citation gate over AI-assisted drafts for a while now, and I keep a tally of what it catches. Across one batch of articles it flagged roughly twenty-three instances of the same defect. In a later, cleaner batch, four of seven total findings were the same defect again. It is not close. It is the dominant failure mode by a wide margin.

The defect is this: a real quotation from a real source that stops one clause too soon, with no ellipsis.

Think about why that survives review. The citation is correct. The source exists and says what the cite says it says. The quoted words genuinely appear in the source, in that order, verbatim. Every check that a careful lawyer runs by hand comes back clean, because every check is looking for something that is wrong, and nothing here is wrong. Something is missing, and the missing part is invisible by construction — you cannot see the words that are not there.

This piece is about building a check that can.

All examples below are invented drafts written for this article. No client matter, filing, or document appears anywhere. The statutory text quoted was retrieved from the Revisor while writing and is used to demonstrate the technique.

Two examples of how much a clause is worth

Example one. Minn. Stat. § 322C.0410, subd. 5, ends:

Any restriction or condition imposed by the operating agreement or under subdivision 7 applies both to the agent or legal representative and the member or dissociated member.

Here is an invented draft sentence:

The statute is explicit. Restrictions in the operating agreement reach the representative only: any restriction “applies both to the agent or legal representative and the member.” A dissociated member therefore stands outside the restriction entirely.

Three words dropped — or dissociated member — and the conclusion built on top of them is the exact opposite of what the sentence says. In an article about dissociated members, that is not a rough edge. That is the whole point, reversed.

Example two. A sentence in Minn. Stat. § 604.02, subd. 3, ends:

…but not among the claimant or others at fault who are not in the chain of manufacture or distribution of the product.

Invented draft:

Reallocation is confined to the supply chain. The uncollectible amount is spread among persons in the chain, “but not among the claimant or others at fault who are not in the chain.”

The truncated version reads as a tidy, self-contained rule about a “chain.” The full sentence defines which chain, and the definition is the operative limit. Cut it and the sentence still parses, still sounds like law, and no longer says anything determinate.

Notice what both have in common, because it is the seam a detector can grip: the draft supplied a period that the source does not have at that point. The quotation asserts “the sentence ends here.” The source says it does not. That is a mechanical, checkable disagreement, and it does not require anybody to have an opinion about the law.

Why this defect is generated rather than merely committed

Worth a paragraph on mechanism, because it explains the frequency.

A model producing a quotation is producing tokens that continue a plausible sequence. Quotations end. Sentences end at periods. A grammatically complete, semantically satisfying stopping point is an extremely attractive place for a generator to stop — and the qualifying tail of a statutory sentence (“and the member or dissociated member,” “of manufacture or distribution of the product”) is precisely the part that adds no fluency and can be dropped without the sentence sounding wrong.

So the generator is biased toward stopping exactly where the meaning-bearing qualifier begins. That is not an occasional slip. It is a systematic pull toward the most dangerous available cut point, and it is why this shows up twenty-three times in a batch rather than once.

The detector

Five steps. All of it is string work. None of it requires a model, which is the entire reason to trust it.

Step 1 — Extract the quoted spans

Handle straight quotes, curly quotes, and markdown block quotes, and do not double-count a block quote that also contains inline quotation marks.

import re
from dataclasses import dataclass

CURLY = re.compile(r"“(.+?)”", re.S)
STRAIGHT = re.compile(r'"(.+?)"', re.S)
BLOCKQUOTE = re.compile(r"(?:^>[ \t]?.*(?:\n|$))+", re.M)

@dataclass
class Span:
    text: str
    start: int
    kind: str

def extract_quotes(draft, min_words=4):
    spans, claimed = [], []

    def overlaps(a, b):
        return any(not (b <= s or a >= e) for s, e in claimed)

    for m in BLOCKQUOTE.finditer(draft):
        body = "\n".join(re.sub(r"^>[ \t]?", "", ln)
                         for ln in m.group(0).splitlines())
        body = body.strip().strip("“”\"")
        if len(body.split()) >= min_words:
            spans.append(Span(body, m.start(), "block"))
        claimed.append((m.start(), m.end()))

    for pat, kind in ((CURLY, "curly"), (STRAIGHT, "straight")):
        for m in pat.finditer(draft):
            if overlaps(m.start(), m.end()):
                continue
            if len(m.group(1).split()) >= min_words:
                spans.append(Span(m.group(1), m.start(), kind))
                claimed.append((m.start(), m.end()))

    return sorted(spans, key=lambda s: s.start)

min_words=4 exists because two-word quotes are usually terms of art, and flagging every "good faith" in a brief is how a check gets turned off.

Step 2 — Normalize both sides identically

This is where most homemade quote checkers quietly die. The draft went through a word processor; the source came off a website. They will differ in curly quotes, dash width, non-breaking spaces, and line wrapping without differing in a single word.

import unicodedata

QUOTES = {0x2018: "'", 0x2019: "'", 0x201a: "'", 0x201b: "'",
          0x201c: '"', 0x201d: '"', 0x201e: '"', 0x2032: "'", 0x2033: '"'}
DASHES = {0x2010: "-", 0x2011: "-", 0x2012: "-", 0x2013: "-",
          0x2014: "-", 0x2015: "-", 0x2212: "-"}
SPACES = {0x00a0: " ", 0x2007: " ", 0x202f: " ", 0x2009: " ", 0x200a: " "}
TRANS = {**QUOTES, **DASHES, **SPACES}

FOOTNOTE = re.compile(r"[⁰-₟¹²³]+|\[\s*(?:fn|FN)?\d+\s*\]")
SINGLE_BRACKET = re.compile(r"\[(\w)\]")

def normalize(s):
    # Footnote markers FIRST: NFKC folds superscript ³ to an ordinary 3,
    # after which it is indistinguishable from text and cannot be stripped.
    s = FOOTNOTE.sub("", s)
    s = unicodedata.normalize("NFKC", s).translate(TRANS)
    s = SINGLE_BRACKET.sub(r"\1", s)      # "[t]he" -> "the"
    return re.sub(r"\s+", " ", s).strip()

That comment about ordering is a bug I shipped and then fixed. I had NFKC first, which is the natural place for it; NFKC turns ³ into 3, so by the time the footnote regex ran there was no superscript left to find and the stray digit stayed glued to the quotation. It made a correct quote fail to match. The lesson generalizes: normalization is order-dependent, and the wrong order fails as a false positive, which is the merciful direction but still erodes trust in the tool.

SINGLE_BRACKET handles the [t]he convention. Multi-character brackets — [sic], [emphasis added], [the defendant] — are additions by the drafter and must not be searched for in the source. Same for ellipses. So both become split points:

ELLIPSIS = re.compile(r"(?:…|\.\s*\.\s*\.)(?:\s*\.)?")
MULTI_BRACKET = re.compile(r"\[[^\]]{2,}\]")

def fragments(q):
    """Split a quotation at ellipses and multi-character bracketed insertions."""
    out = []
    for p in ELLIPSIS.split(q):
        out.extend(MULTI_BRACKET.split(p))
    return [f.strip() for f in out if len(f.strip().split()) >= 2]

Every fragment must independently appear in the source. That is what makes an ellipsis honest rather than a license.

Step 3 — Containment

idx = src.find(frag)

That is the whole thing, and its plainness is the point. FOUND or NOT FOUND is a mechanical answer with no judgment in it. No model, no similarity score, no threshold anybody has to defend. A string appears in the source or it does not.

This step alone catches the changed word, the shifted tense, the silently normalized punctuation, and the dropped “not.”

Step 4 — The continuation check

Here is the step almost nobody builds, and it is the one that catches truncation.

For a fragment that is found, look at what comes after it in the source, and ask whether the quotation ended where a sentence ended.

TERMINAL = ".?!"          # a claim about where the sentence ends
CONVENTION = ',;:"” )'    # punctuation placed inside quotes by typographic custom

def check_fragment(frag, src, lookahead=90):
    idx = src.find(frag)
    if idx >= 0:
        hit, added_stop = frag, False
    else:
        stripped = frag.rstrip(TERMINAL + CONVENTION)
        idx = src.find(stripped) if stripped else -1
        if idx < 0:
            return ("NOT FOUND", "no exact match in source after normalization", "")
        hit = stripped
        added_stop = frag.rstrip(CONVENTION).endswith(tuple(TERMINAL))

    after = src[idx + len(hit): idx + len(hit) + lookahead]
    continues = bool(after) and not after.lstrip().startswith(tuple(TERMINAL))

    if added_stop and continues:
        return ("TRUNCATED",
                "quotation closes a sentence the source continues "
                "(terminal punctuation supplied by the draft)",
                f"source continues: …{after.strip()}")
    return ("OK", "", "")

The distinction between TERMINAL and CONVENTION is doing real work. American typographic convention puts a comma inside the closing quotation mark whether or not the source has one there, so a trailing comma tells you nothing. A trailing period that the source does not have is a substantive claim about where the sentence stops — and when the source keeps going with a qualifier, that claim is false.

That single rule catches both examples above, and it has an extremely low false-positive rate, because there is almost no innocent reason to insert a full stop the source does not contain.

The looser sibling — a quotation that ends mid-sentence with no ellipsis and no added period — needs a severity tier, or it will fire on every legitimate phrase woven into an author’s own sentence. My rule:

presented_as_complete = span.kind == "block" or len(hit.split()) >= 12
severity = "TRUNCATED" if presented_as_complete else "REVIEW"

A block quote or a twelve-word run is being offered as a complete proposition, and ending it mid-clause is a defect. A five-word phrase inside your sentence is ordinary writing. REVIEW findings are suppressed by default and printable with a flag. Tune the threshold on your own corpus; the number matters less than having the tier.

Step 5 — Report position and context

The output has to let a human adjudicate in five seconds without opening the source.

[TRUNCATED] line 2 (straight quote)
  quoted : “applies both to the agent or legal representative and the member.”
  problem: quotation closes a sentence the source continues
           (terminal punctuation supplied by the draft)
  source continues: …or dissociated member.

That is real output from the code above, on the first invented draft. Compare:

[TRUNCATED] line 2 (straight quote)
  quoted : “but not among the claimant or others at fault who are not in the chain.”
  problem: quotation closes a sentence the source continues
           (terminal punctuation supplied by the draft)
  source continues: …of manufacture or distribution of the product.

source continues: is the field that makes the report actionable. It shows the words that were dropped, which is the one thing a reviewer reading the draft cannot see.

Test it by breaking it

A check you have never watched fail is not a check. Three tests, and the middle one is the whole reason the tool exists:

SRC = ("Any restriction or condition imposed by the operating agreement or under "
       "subdivision 7 applies both to the agent or legal representative and the "
       "member or dissociated member.")

CLEAN = ('The restriction "applies both to the agent or legal representative and '
         'the member or dissociated member."')
BROKEN = ('The restriction "applies both to the agent or legal representative and '
          'the member."')

def run(draft):
    return [f for s in extract_quotes(draft) for f in check_quote(s, SRC)]

def test_clean_draft_passes():
    assert run(CLEAN) == []

def test_truncation_is_caught():
    findings = run(BROKEN)
    assert [f.severity for f in findings] == ["TRUNCATED"]
    assert "or dissociated member" in findings[0].context

def test_altered_word_is_caught():
    bad = CLEAN.replace("agent or legal representative", "agent or legal successor")
    assert [f.severity for f in run(bad)] == ["NOT FOUND"]

Three tests, all passing. The first one matters more than it looks: a detector that flags everything is as useless as one that flags nothing, and it is much easier to build by accident.

Where this sits in a workflow

The detector needs the source text, and the source text has to be retrieved, not remembered. That is the pairing: a fetch layer that returns the actual bytes of the actual authority, and a checker that compares the draft against them character by character. Neither half works alone. I wrote up the retrieval half in Parsing Primary Law Without a Summarizer — the short version is that a summarizing fetch layer hands you a paraphrase, and running a truncation detector against a paraphrase produces confident nonsense.

Practically:

1. Drafting produces a manifest of every quotation and the URL or file it came from.
2. Retrieval pulls each source, raw, cached.
3. The detector runs. NOT FOUND and TRUNCATED block.
4. A human adjudicates every finding. Some TRUNCATED findings are fine — the dropped tail really was immaterial and an ellipsis fixes it. That is a five-second decision, made with the dropped words in front of you.
5. The tool exits non-zero if anything unresolved remains, so it can sit in a pre-commit hook or a build step and actually stop things.

Runtime on an article-length document is under a second once sources are cached, which means there is no scheduling argument against running it on every save.

What it does not catch

Being precise here matters, because a check that gets oversold becomes a check that gets over-relied on.

This tool verifies that quoted words appear in the source, and that quotations do not silently amputate the rest of a sentence. That is the form of the quotation.

It cannot catch a perfectly accurate quotation cited for a proposition it does not support. The words are all there. The sentence is complete. The source is real, current, and correctly cited. And the sentence does not stand for the thing the brief says it stands for — because it is dicta, or the procedural posture is wrong, or the jurisdiction does not govern, or the surrounding paragraph limits it in a way the quoted sentence does not disclose.

No amount of string comparison touches that. It requires somebody who has read the surrounding text and is actively trying to disbelieve the draft — a separate adversarial pass, by a human or by a model given the source and an instruction to argue against the proposition. I have written that pass up in Verifying Form Is Not Verifying Truth and will not re-derive it here.

The reason to build the mechanical half anyway is division of labor. Every defect the string check removes is a defect the expensive human pass does not have to hunt for, and truncation — twenty-three in a batch — is a lot of hunting. Give the machine the part that is mechanical so the reviewer’s attention lands on the part that is not.

Sources

Statutory text quoted above was retrieved directly from the Revisor of Statutes while writing this article, not cited from memory. Every code snippet was executed; the reports and test results shown are actual output.

• Minn. Stat. § 322C.0410, subd. 5
• Minn. Stat. § 604.02, subd. 3

General commentary on practice management and legal technology. Not legal advice and not a statement of Minnesota law — the statutory sentences above are used as strings to demonstrate a text-comparison technique, and anyone relying on either provision should read it in full at the source. All draft excerpts are invented for this article; no client information appears anywhere in it. Questions: Send us a message or 612-470-6529.