Parsing Primary Law Without a Summarizer

The tool I draft with can fetch web pages. Hand it a URL and it hands back the content. It’s convenient, it works on most of the internet, and for about two months it was how my drafting pipeline read Minnesota statutes.

It was getting quotations wrong. Not made up. Wrong. The cite was right, the section was right, every word was a real word from that section, and the quote stopped somewhere the statute didn’t.

That wasn’t bad luck. It was built in. The fetch layer doesn’t give you the page. It reads the page and gives you a model-processed rendering of it, and on a long section it quietly drops material. What comes back sounds like the statute, moves like the statute, and isn’t the statute. Nothing flags the gap, because as far as the tool is concerned nothing went wrong.

It’s the most important thing I’ve learned building legal tooling: a summarizing fetch layer is disqualifying for primary law. Not “use with care.” Disqualifying. Pull the raw bytes and parse them yourself.

So here’s the parser, the four bugs it turned up, and what it cost me.

Everything below ran against the live Revisor site while I wrote this. No client matter, file, or document appears anywhere in it. The statutes are here to show parsing technique, not to tell you what Minnesota law is today. Check that yourself, at the source.

Rule one: fetch bytes, parse yourself

import urllib.request

BASE = "https://www.revisor.mn.gov"
UA = "MadgettLaw-StatuteFetch/1.0 (internal research; 612-470-6529)"

def raw(url, timeout=20):
    req = urllib.request.Request(url, headers={"User-Agent": UA})
    with urllib.request.urlopen(req, timeout=timeout) as r:
        return r.status, r.read().decode("utf-8", "replace")

Six lines. Those six lines are the difference between a research tool and a machine that makes plausible text. Set a real User-Agent, one that says who you are and how to reach you. Set a timeout. It’s somebody else’s server, so behave like a guest.

Then comes the parsing, and that’s where it gets fun.

Bug 1: the heading that isn’t a heading

The first thing you want off a statute page is the official section heading. On revisor.mn.gov/statutes/cite/<cite> it lives in exactly one place:

<h1 class="shn">604.02 APPORTIONMENT OF DAMAGES.</h1>

shn is section heading, one per page. Anchor on it:

import re, html

H1_SHN = re.compile(r'<h1[^>]*class="[^"]*\bshn\b[^"]*"[^>]*>(.*?)</h1>', re.S)

def detag(fragment):
    fragment = re.sub(r"<[^>]+>", "", fragment)
    return html.unescape(re.sub(r"\s+", " ", fragment)).strip()

That holds up. What doesn’t hold up is the first thing everybody writes: strip the tags, find the cite number, grab what comes after it.

text = re.sub(r"<[^>]+>", " ", body)
text = re.sub(r"\s+", " ", text)
m = re.search(r"604\.02\s+(.{0,60}?\.)", text)
print(m.group(1))

I ran that against the live page while writing this paragraph. Here’s what it prints:

MN Statutes window.

That’s not a garbled heading. That’s the page’s <title> element (“Sec. 604.02 MN Statutes”) run straight into the first line of an analytics script, window.dataLayer = window.dataLayer || []. Stripping the tags mashed <title>, <script>, and the navigation menu into one stream of words with the statute, and the first match was page furniture.

Sit with how it failed. No error. It handed back a string, roughly the right shape, from roughly the right spot, and a caller would happily drop it into a section_title field and print it at the top of a memo.

Here’s the habit that catches this kind of bug: validate your extractor against a section whose heading you already know, and assert on the exact value. Not “does it return something.” Does it return this.

def test_heading():
    s = get_section("604.02")
    assert s["heading"] == "604.02 APPORTIONMENT OF DAMAGES.", s["heading"]

It’s cheap, and it fails loud the day the Revisor changes its markup. That day’s coming.

Bug 2: the section that doesn’t exist

Ask for a cite that doesn’t exist and you get HTTP 404, and the body carries a specific phrase:

<h1>Minnesota Statutes</h1>
<p>Statute could not be found.</p>

Check both. A site can serve a “not found” page with a 200 status, and this one doesn’t owe you a 404 forever. I check the status and the phrase and that class="shn" is missing.

I ran this over a batch of cites pulled from draft articles. It caught six section numbers that don’t exist. None of them looked wrong. Every one was a well-formed Minnesota cite in a believable chapter, sitting in a sentence that read just fine.

An existence screen is the cheapest high-value check in this whole category, and once you cache it’s close to free:

def screen(cites):
    return {c: get_section(c)["exists"] for c in cites}

Cold, six cites took 4.1 seconds. Warm, off the cache, 0.0015 seconds. That’s the whole case for caching. It turns “screen this article” into “screen the entire corpus, on every save, forever.”

The cache doubles as your manners

import hashlib, json, time, urllib.error
from pathlib import Path

CACHE = Path.home() / ".cache" / "mnstat"
CACHE.mkdir(parents=True, exist_ok=True)

_last_request = 0.0
MIN_INTERVAL = 0.5

def fetch(url, *, timeout=20, max_age=86400):
    """Return (status, body). Disk-cached. Caches 404s too."""
    key = hashlib.sha256(url.encode()).hexdigest()[:32]
    path = CACHE / f"{key}.json"
    if path.exists() and (time.time() - path.stat().st_mtime) < max_age:
        d = json.loads(path.read_text())
        return d["status"], d["body"]

    global _last_request
    wait = MIN_INTERVAL - (time.time() - _last_request)
    if wait > 0:
        time.sleep(wait)

    req = urllib.request.Request(url, headers={"User-Agent": UA})
    try:
        with urllib.request.urlopen(req, timeout=timeout) as r:
            status, body = r.status, r.read().decode("utf-8", "replace")
    except urllib.error.HTTPError as e:
        status, body = e.code, e.read().decode("utf-8", "replace")
    finally:
        _last_request = time.time()

    path.write_text(json.dumps({"url": url, "status": status, "body": body}))
    return status, body

Four calls in there, and I got every one of them wrong the first time.

Cache the raw body, not the parsed result. You’re going to fix parser bugs. When you do, you want to re-parse a thousand cached pages without touching the network.

Cache the 404s. A negative result is still a result. Cache only the hits and every re-run asks for every phantom cite again, which is slow and rude.

Rate-limit inside the fetch function, not at the call site. A limiter you have to remember to call won’t get called.

Keep max_age short. One day. Primary law moves, and a cache that never expires is a currency bug that happens to run fast.

On parallel fetches: I’ve had them fail against this source where sequential ones with a modest timeout went through. When I re-ran the test for this article, twelve cites at eight workers, it finished clean in one second with zero errors. I’m giving you both results because the inconsistency is the point. You don’t control how much load somebody else’s server will take, so don’t build on it. The cache makes the question moot, and that beats fiddling with a worker count.

Bug 3: currency, the one that bites

Everything so far is plumbing. This one’s a real correctness trap, and your tests won’t find it.

The Revisor shows one annual edition, and every section page says which:

<h1>2025 Minnesota Statutes</h1>

Parse it. Store it. Print it on anything your tool puts out.

Here’s why. The body text on the screen is that edition’s text, and a later session may have changed it. When that’s happened, the page carries a banner. I pulled this one live while writing:

<div id="warning" class="alert alert-warning" role="alert">
    This section has been affected by law enacted during the 2026 Regular Session.
    <a id="more_info_link" href="#">More info...</a>
    <div id="more_info">
        <ul>
            <li>169.444 subd. 1 has been amended by
                <a href="/laws/2026/0/41/laws.0.1.0">Chapter 41, Section 1</a></li>
        </ul>
    </div>
</div>

Read it twice. Everything under that banner is the 2025 text, before the amendment. A tool that grabs the section, pulls the paragraphs, and quits has just quoted superseded language under a correct cite. And it’ll pass every test you wrote, because that text really is on the page.

So treat the banner as a field of its own and make noise about it:

WARNING = re.compile(r'<div id="warning"[^>]*>(.*?)</div>\s*</div>', re.S)
AMENDED_BY = re.compile(
    r"([\d.]+[A-Z]?[\d.]*)\s+(subd\.\s*[\w.]+\s+)?has been amended by\s*"
    r'<a href="(/laws/[^"]+)">([^<]+)</a>', re.S)

Against § 169.444 that returns:

{'cite': '169.444', 'subd': 'subd. 1',
 'url': 'https://www.revisor.mn.gov/laws/2026/0/41/laws.0.1.0',
 'label': 'Chapter 41, Section 1'}

Now you can make anything downstream refuse. If pending isn’t empty and the caller didn’t explicitly acknowledge it, raise. Don’t warn. Raise. A warning in a log is a warning nobody read.

One more trap on the same block. Every section page has a sidebar panel headed “Recent History.” It’s abbreviated, and it isn’t a currency check. For § 604.02 that panel lists exactly one entry, a 2003 amendment. The real History: line in the body reads:

1978 c 738 s 8; 1986 c 444; 1986 c 455 s 85; 1988 c 503 s 3;
1989 c 209 art 1 s 44; 2003 c 71 s 1

Six acts in the history, one in the panel. The panel’s a convenience, and it does that job fine. Treat it as the amendment record and the bug is yours, not theirs. Parse the History: block, follow the version list, and know that none of this replaces actually shepardizing.

Bug 4: never strip tags off a session law

If another lawyer steals one trick from this piece, I hope it’s this one.

Minnesota publishes session laws in struck-and-inserted form: deleted language struck through, new language underlined. That markup carries the whole meaning of the document. Strip the tags, which is the obvious first move and the one every generic HTML-to-text utility makes, and you wipe out exactly what you came for.

Watch. Here’s 2003 c 71 s 1, tags stripped, run through a naive pipeline:

Subdivision 1. [JOINT LIABILITY.] When two or more persons are jointly severally liable, contributions to awards shall be in proportion to the percentage of fault attributable to each, except that each is the following persons are jointly and severally liable for the whole award. Except in cases where: (1) a person whose fault is greater than 50 percent; …

The doubled words (“jointly severally,” “each is the following persons are”) are obvious garbage, which is almost lucky. The real damage is further down. Several sentences the act deleted survive the strip whole and read like live law, with nothing to mark them. Hand that to a model and it’ll summarize the deleted rules as current. It can’t do anything else.

Keep the markup and parse it:

def render(fragment, version):
    """version: 'before' = pre-amendment law, 'after' = as amended."""
    if version == "before":
        fragment = re.sub(r"<u>.*?</u>", "", fragment, flags=re.S)  # drop insertions
        fragment = re.sub(r"</?s>", "", fragment)                    # keep deletions
    else:
        fragment = re.sub(r"<s>.*?</s>", "", fragment, flags=re.S)  # drop deletions
        fragment = re.sub(r"</?u>", "", fragment)                    # keep insertions
    return detag(fragment)

Same input, through render(..., "before"):

When two or more persons are jointly liable, contributions to awards shall be in proportion to the percentage of fault attributable to each, except that each is jointly and severally liable for the whole award. Except in cases where liability arises under chapters 18B … a person whose fault is 15 percent or less is liable for a percentage of the whole award no greater than four times the percentage of fault, including any amount reallocated to that person under subdivision 2.

And render(..., "after"):

When two or more persons are severally liable … except that the following persons are jointly and severally liable for the whole award: (1) a person whose fault is greater than 50 percent; (2) two or more persons who act in a common scheme or plan that results in injury; (3) a person who commits an intentional tort; or (4) a person whose liability arises under chapters 18B …

That’s a clean, mechanical before-and-after of a statutory amendment, straight from the primary source, in about fifteen lines. It’s how I confirmed that two Minnesota tort rules I’d seen repeated in secondary material were both struck by that 2003 act: the four-times-fault cap for a defendant at 15 percent or less, and the separate rule for a jointly liable state or municipality under 35 percent fault. Both show up in the deleted list. Neither shows up in the current section.

I’m not telling you what Minnesota comparative-fault law is today. Go read § 604.02 and its later history yourself. What I’m telling you is that a tool that strips tags first would’ve told me those rules were still on the books, smoothly, under a correct citation.

The trap inside the trap

The markup changes over the years. Chapters from 2005 and earlier use a <pre> block with <s> and <u>. Chapters from 2006 on use something else:

<span class="sr-only">deleted text begin </span>
<span style="text-decoration: line-through" class="del">an extended stop-signal arm and</span>
<span class="sr-only">deleted text end </span>
<span class="sr-only">new text begin </span>
<ins style="text-decoration: underline"> must</ins>
<span class="sr-only">new text end </span>

Sampling chapters on both sides put the switch between the 2005 and 2006 sessions. That’s the line I saw, not a promise about every chapter in every year, and it’s exactly why the parser sniffs the format instead of keying on the year:

OLD_DEL = re.compile(r"<s>(.*?)</s>", re.S)
OLD_INS = re.compile(r"<u>(.*?)</u>", re.S)
NEW_DEL = re.compile(r'<span[^>]*class="del"[^>]*>(.*?)</span>', re.S)
NEW_INS = re.compile(r"<ins[^>]*>(.*?)</ins>", re.S)

def session_law_changes(chapter_url):
    status, body = fetch(chapter_url)
    if status != 200:
        raise RuntimeError(f"{chapter_url}: HTTP {status}")
    body = re.sub(r'<p class="key">.*?</p>', "", body, flags=re.S)   # drop the legend
    body = re.sub(r'<span class="sr-only">.*?</span>', "", body, flags=re.S)
    if 'class="del"' in body or "<ins" in body:
        fmt, dels, inss = "modern", NEW_DEL, NEW_INS
    else:
        fmt, dels, inss = "legacy", OLD_DEL, OLD_INS
    return {"format": fmt,
            "deleted": [detag(x) for x in dels.findall(body) if detag(x)],
            "inserted": [detag(x) for x in inss.findall(body) if detag(x)]}

Two details cost me time. Both formats carry a legend paragraph, Key: (1) language to be deleted (2) new language, that uses the very markup you’re hunting for, so it shows up as a phantom edit in every result. Strip it first. And the modern format wraps each edit in screen-reader-only spans that say “deleted text begin” and “deleted text end” around it. Leave those in and your extracted text is littered with them.

Sure, it’s brittle. That’s the point.

All of this depends on the Revisor’s HTML staying more or less the way it is. It won’t stay that way forever. Some morning class="shn" turns into class="section-heading" and my parser quits.

Here’s what that morning looks like:

if not parsed:
    raise RuntimeError(
        f"{cite}: HTTP {status} but no <h1 class='shn'> found. "
        "The Revisor's markup changed; fix the parser before trusting output.")

It raises. Loud, right away, in my face, with the reason and the fix. Nothing downstream gets a wrong answer, because nothing downstream gets an answer.

Now look at how the thing I replaced fails. A summarizing fetch layer never breaks. It degrades: slowly, invisibly, differently every call, worse the longer the input. It produces something every single time, and that something is confident, tidy, and now and then two clauses short in the middle of the sentence you were about to quote to a court.

A scraper that breaks loudly is strictly safer than a summarizer that degrades quietly. An afternoon of maintenance is a known cost with a ceiling, and you can put it on the calendar. A quote that ends one clause early has no ceiling, and the tool will never tell you about it.

I’ll fix a regex twice a year and sleep fine.

Sources

Every claim below was verified by direct HTTP fetch against the live site on the date of this article, not taken from documentation or memory. Code snippets were executed; the outputs shown are actual outputs.

• Minn. Stat. § 604.02 — heading in <h1 class="shn">, 2025 Minnesota Statutes edition marker, History: block, “Recent History” sidebar panel showing one of six listed acts
• Minn. Stat. § 169.444 — live example of the <div id="warning"> pending-amendment banner over 2025 edition body text
• 2003 Minn. Laws ch. 71 — legacy <pre> / <s> / <u> struck-and-inserted format; source of the before/after rendering above
• 2026 Minn. Laws ch. 41 — modern class="del" / <ins> / sr-only format
• Nonexistent-cite behavior (HTTP 404 plus “Statute could not be found”) confirmed against four invented cites
• Session law markup format boundary observed between the 2005 and 2006 sessions by sampling chapters on both sides (legacy <s>/<u> through 2005; class="del"/<ins> from 2006 forward)

General commentary on legal technology, not legal advice and not a statement of current Minnesota law. Statutory text is reproduced to illustrate parsing technique; verify any rule at the source before relying on it. No client information appears in this article — all cites used as examples were chosen for their markup, not from any matter. Questions about anything here: Send us a message or 612-470-6529.