← All projects
Automation Shipped

Invoice Pipeline Audit Harness

A second n8n workflow that audits the first: the Corpus Emailer replays a 150-invoice labeled corpus into the intake inbox, uid in every subject line, so extraction accuracy can be measured row-for-row against ground truth.

n8nGoogle DriveGmailGoogle Sheets
150
labeled invoices replayed
~4 s
throttle between sends
uid
joins truth to output

// the pipeline, end to end

  1. 01
    Load
    corpus CSV from Drive · parse rows
  2. 02
    Loop
    150 invoices, one at a time
  3. 03
    Send
    PDF → Gmail · uid in subject
  4. 04
    Throttle
    ~4 s so intake keeps pace

Why audit an automation

An automation you can’t measure is a demo. “It works on the invoices I tried” isn’t an accuracy figure, so this harness turns the intake pipeline into something you can put a number on: a labeled corpus goes in, an audit trail comes out, and the difference between them is the measurement.

How the harness works

A manual trigger downloads the corpus CSV from Google Drive (the ground-truth register of known-correct field values) and parses it into rows. The workflow then loops over all 150 invoices: for each row it downloads the matching invoice PDF from Drive and emails it to the intake inbox with the row’s uid in the subject line, throttling about four seconds between sends so the pipeline processes them the way real traffic arrives.

On the other side, intake v4 carries each uid through to its audit trail and dedupes on it. Joining the sent register to the audit trail on uid gives a per-field accuracy score (vendor, invoice number, totals) and shows exactly which document types the extraction struggles with.

Analysis, pointed at the automation

This is the same discipline as the churn diagnostic (ground truth, a controlled run, a measured gap), aimed at my own system instead of a customer base. The loop runs both directions: the diagnostic ends in moves an automation would execute, and this harness is analysis auditing an automation.