Skip to content

Messy Business Information → Structured Output

Can AI turn unstructured business messages into usable structured output with less manual effort — and where does it fail?

Published
29 Aug 2026
Type
Workflow comparison, human vs AI
Sample
10 cases
Author
FIELD

Summary

Ten synthetic packaging and printing inquiries were converted into a structured table by a human (about 30 minutes) and by an AI model. AI output was more systematic in extraction, completeness and flagging missing information. Human judgment remained stronger on commercial context. No time-saving figure was established.

Research question

Can AI transform messy, unstructured business information into useful structured output with less manual effort than doing it by hand?

Hypothesis

Small-business workflows containing messy, unstructured information may be strong candidates for AI-assisted processing when repeatable structure, identifiable fields and human verification exist. (Stated as a hypothesis, not a conclusion.)

Method

Ten synthetic inquiries were written to resemble messages a small packaging and printing supplier might receive. The human baseline was produced first: each message was read and summarized into a six-column table. The same messages were then processed by an AI model instructed to produce the same structure. The two outputs were compared against an evaluation list and a failure taxonomy defined before the comparison. The experiment was explicitly not testing software, product demand, client demand, AI tool ranking or autonomous AI.

Workflow

  1. 01Messy business information
  2. 02Extract relevant information
  3. 03Structure into fields
  4. 04Validate and identify uncertainty
  5. 05Usable output
  6. 06Human review

Dataset

Ten synthetic customer inquiries (C001–C010) covering custom boxes, paper bags, juice packaging, food containers, flyers, shopping bags and stickers. Synthetic: written for the experiment, not collected from real customers.

View the 10 cases
  1. C001Bella Cakes — 200 custom cupcake boxes for a wedding on October 18, with logo, in Buea; asks for price and production time.
  2. C002Takeaway restaurant — approximately 500 branded paper bags, next month, size undecided.
  3. C003New juice business — bottles or pouches, quantity undecided, asks for advice.
  4. C004Urgent: 1,000 food containers by Friday because the supplier cancelled that day.
  5. C005Flyer printing price inquiry with minimal information.
  6. C006Clothing brand launch December 5 — 300 shopping bags plus stickers, separate quotes requested.
  7. C007Previous supplier delivered 1,200 of 2,000 boxes with poor quality; wants a new supplier, samples and prices.
  8. C008Branded packaging inquiry asking for a catalogue.
  9. C009Bakery using plain plastic bags wants branded packaging; sells bread, pastries and cakes; unsure about size and material.
  10. C0105,000 waterproof bottle stickers, logo attached, needed before end of month; asks what information is required for a quote.

Baselines

Human baseline

A human processed all ten cases in approximately 30 minutes, producing a table with the columns ID, Business/Customer, Request/Information, Key Details, Missing Info and Action Needed. Some of the human-written actions were incomplete.

AI baseline

The AI model was given the same ten messages and the same target structure. Output was reviewed by a human for extraction errors, fabrication, missed information and formatting problems.

Evaluation criteria

Speed
How long the pass took (human measured approximately; AI not measured in a controlled way).
Extraction accuracy
Were the stated details captured correctly?
Fabrication / hallucination
Did the output invent details not present in the message?
Completeness
Were all relevant details and gaps captured?
Correction burden
How much human editing did the output need?
Usability
Could the output be acted on directly?

Results

The AI output was systematic across all ten cases in extraction, completeness, identification of missing information and consistency of format. The human table was produced in roughly 30 minutes and contained some incomplete actions. The human remained stronger in contextual and commercial judgment — reading what a customer probably needs beyond what they wrote. Because AI time was not measured in a controlled way, no time-saving claim is made.

Key findings

  • AI output was systematic in extraction, completeness, missing-information detection and consistency.
  • Human judgment remained stronger in contextual and commercial judgment.
  • AI's apparent advantage in this experiment was standardization rather than superior business judgment.
  • AI appeared useful for extraction, standardization, uncertainty detection and action preparation.
  • A workflow decomposition emerged: Extract → Interpret → Identify uncertainty → Determine next action → Execute.
  • No exact time-saving figure was established because time was not measured in a sufficiently controlled way.

Show the failure

Failure modes

  • F1Extraction

    Missing or misreading a detail that was present in the message.

  • F2Interpretation

    Correct extraction, wrong reading of what it means.

  • F3Fabrication

    Inventing details the message did not contain.

  • F4Overconfidence

    Presenting an uncertain reading as settled.

  • F5Formatting

    Output does not fit the required structure.

  • F6Prioritization

    Wrong emphasis on what matters in the case.

  • F7Context

    Missing situational context a human would apply.

  • F8Instruction

    Not following the task instructions.

Observed → Inferred → Unknown

Observed

  • The human pass took approximately 30 minutes for ten cases.
  • Some human-written actions were incomplete.
  • The AI produced the six-field structure consistently for every case.
  • The AI listed missing information for each case; the human did so less consistently.
  • Where messages implied a commercial situation (for example a customer burned by a previous supplier), the human's reading was richer than the AI's.

Inferred

Interpretations — our reading of the evidence, labelled as such.

  • The AI's advantage looked like consistency and coverage, not better judgment.
  • Standardizing the extraction step may be where the value is; the decision step still benefited from a human.
  • Messy inputs with repeatable fields seem like a good fit for AI-assisted processing — but only with human verification.

Unknown

What this experiment does not establish.

  • How much time AI actually saves on this task — not measured in a controlled way.
  • Whether accuracy holds on real, longer, less tidy messages.
  • Fabrication rate at scale; ten cases is too few to estimate it.
  • Whether a different model or prompt would change the result.

What we learned

The output structure from Experiment #001 was used as the target format. The failure taxonomy (F1–F8) was introduced here and is intended for reuse in later experiments.

Limitations

  • Ten synthetic cases; not real customer messages.
  • Single human baseline.
  • Time measured only approximately for the human and not in a controlled way for the AI.
  • Qualitative comparison; no scoring rubric was applied numerically.
  • Not a comparison between AI tools.

Practical implications

What operators might reasonably take from this evidence.

  • If a workflow starts with messy text and ends in a fixed set of fields, it is a candidate for AI-assisted extraction with human review.
  • Ask AI to list missing information explicitly; it did this more consistently than the human did.
  • Keep commercial interpretation with a human, especially where trust or history matter.
  • Do not assume a time saving until you have measured it in your own workflow.

Conclusion

On these ten synthetic cases, AI was reliably systematic at extracting, structuring and flagging what was missing, and a human was better at reading the commercial situation. The reasonable reading is that AI's value here is standardization of a messy step, with a human still doing interpretation and the final action. The hypothesis that messy, field-shaped small-business workflows are good candidates for AI assistance remains a hypothesis.

Findings from this experiment

Related experiments

Research updates

Follow the research

Get new FIELD experiments and findings when we publish them.

Email only. No lead magnet. Unsubscribe any time.