Cutting prior-auth review
time from days to minutes.
The problem.
Prior-authorization is one of the heaviest, slowest workflows in US healthcare. Our client, a Series B platform serving specialty clinics, had a queue of mixed-format insurance documents that were being read manually by clinical operations staff. The backlog was growing, and adding more people wasn't going to fix it.
The hypothesis we agreed on at the start of the engagement was simple: most cases follow predictable patterns and could be auto-routed; the rest need human judgment. Our job was to find that boundary.
The approach.
We started with a one-week discovery sprint reading 200 anonymized cases and writing the eval criteria by hand before any code was written. The criteria became our test set. Then a thin prototype using a small classifier plus an LLM-powered extractor against the actual document corpus.
From there: a production pipeline with structured output, confidence-banded routing, and a human-in-the-loop interface for cases the model wasn't sure about. Every decision is logged for audit and re-training.
The shape of the system.
Documents flow in from clinics through three channels: fax, secure email, and a portal upload. They're parsed into a canonical format, classified, and either auto-routed or sent to a clinical reviewer with the model's reasoning attached. Reviewers can correct the model, and corrections feed back into a continuous-evaluation loop that we re-run weekly.
We baked compliance into the architecture: end-to-end encryption, role-based access control, BAA-eligible infrastructure, and a complete audit log indexed by case ID. The pipeline runs in a HIPAA-compliant AWS account with isolated test/prod environments and reviewable infrastructure-as-code.
What we'd do again.
Building the eval harness in week one paid for itself five times over. Every time we tried a new model or a new prompt strategy, we knew within ten minutes whether it was an improvement. There was no debate. There was a number.
And we left the human-in-the-loop interface intentionally generous. Reviewers see the full document, the model's reasoning, and the prior decisions on similar cases. They can override anything in two clicks.
What's next.
The system is in steady-state operation. We co-own it with the client's internal team, with one of our engineers continuing to spend a day per week on tuning, eval review, and incident response. The eval set is now over 1,400 cases and growing.
Fourteen weeks, four phases.
From the first call to GA. No scope creep, no surprise invoices. The phases were locked at week one.
Eval-first discovery
Read 200 anonymized cases. Wrote the eval harness by hand. Locked the scoring rubric with clinical operations before any code was written.
Prototype & routing
Thin classifier + LLM extractor against the real corpus. Confidence-banded routing. First version of the human-in-the-loop UI for the gray-zone cases.
Production hardening
HIPAA-grade infrastructure, BAA-eligible AWS account, audit logging, role-based access. Soft launch to 5% of intake.
GA & handoff
Ramp to 100%, on-call playbook, weekly eval review cadence with client's clinical lead. One of our engineers stays at one day/week ongoing.
Why this stack, and not the other one.
Every architectural decision had an explicit trade-off written down at the time. Here are the ones that mattered most.
Anthropic Claude, not OpenAI
HIPAA BAA was the deciding factor. Claude's reasoning quality on long-form clinical documents also tested higher in our internal evals at the start of the engagement.
pgvector, not Pinecone
One database to back up, one to monitor, one to reason about. Pinecone's strengths weren't needed at this scale. Saved $1,400/month and one operational dependency.
FastAPI & Pydantic
Structured output from the model goes through Pydantic before it touches the routing logic. Schema violations are logged and routed to human review automatically.
Postgres as the eval store
Every model prediction, ground truth, and reviewer correction lives in the same database. The eval re-run is a SQL query plus a notebook, not a separate ML platform.
"They built the eval harness in the first week and refused to ship anything that didn't pass it. That discipline is why the system landed at 96% accuracy on day one of GA, not month six. We've never had a vendor hold themselves to a higher bar than we did."
Other engagements in regulated workflows.
Selected case studies from the same neighborhood. Different industries, same engineering rigor.
FinTech · Lending decisioning
Manual underwriting compressed from 4 days to 11 minutes. SOC 2 Type II in 7 months. See work index.
HealthTech · Clinical trial intake
Patient enrollment cut from 18 days to 4. HIPAA audit passed without findings. See work index.
FinTech · Real-time fraud scoring
96ms p99 latency, 4.2x catch rate at $0.0008 per check. See work index.
Have a regulated
workflow?
We've shipped HIPAA, SOC2, and PCI projects. Tell us what you're building.