Automated FDA Label Compliance Using Machine Learning

Problem

Manually analyzing product labels against CFR 21 (Code of Federal Regulations Title 21), which governs food and drug products, presents a substantial obstacle for many businesses. This method is inherently slow and inconsistent, primarily due to its dependence on human review. Every single label demands examination to ensure that every detail adheres to a large set of regulations, which includes everything from the precise ingredient lists and allergen warnings to the comprehensive nutritional information. This not only consumes valuable operational time but also significantly elevates the risk of errors. Inconsistencies frequently arise from subjective interpretations of the rules among different human reviewers, which can lead to compliant labels being mistakenly flagged or even non-compliant ones slipping through undetected, potentially resulting in costly recalls or regulatory fines.

Specific Pain Points:

• Compliance staff manually compare every label against CFR Title 21, which results in hours of reading and cross‑checking for every product.
• Figuring out which rules apply is ad‑hoc and depends on individual experience.
• Layout requirements (font size, panel order, line length) are eyeballed which can lead to small mistakes slipping through.
• Each label revision forces a full re-check and prior work isn’t reusable or versioned.
• Evidence of compliance lives in PDFs, emails, and spreadsheets with no single audit trail.
• Updating for regulation changes or new product lines is painful and everything has to be re-read.

Solution

We built a knowledge‑graph + multi‑modal agent pipeline. CFR 21 is ingested into Neo4j; GraphRAG retrieves only relevant clauses. Vision agents parse the label image. A verifier agent matches extracted facts to the CFR‑derived checklist and outputs a traceable result.

Specific Steps Taken:

• Multi‑agent flow: LangGraph orchestrated specialized agents (text, vision, verifier).
• GraphRAG: Combined embeddings and graph traversals to pull precise CFR requirements.
• Regulation processing: Chunked CFR 21, extracted entities/definitions, modeled applicability in Neo4j.
• Checklist builder: Generated a product‑specific checklist with linked CFR references.
• Label analysis: OCR + layout checks (font size, line lengths, sections like Drug Facts, ingredients).
• Evaluation harness: Defined metrics, built golden datasets, tracked runs with Langfuse.

Unique Value Proposition:

• Every rule is backed by a CFR citation (audit ready).
• True multi‑modal reasoning (text + layout).
• Deterministic agent steps instead of opaque single prompts.
• Reusable pattern for other regulations.

Results

Key Metrics:

Completeness/Recall – achieved 90%

Smart Tip

Log every agent step with Langfuse and record which CFR sections and checklist items it referenced. One clear audit view makes debugging faster and QA reviews simple.

Smart Fact

CFR Title 21 is roughly nine million words. Structure matters more than a larger context window.

About the Clients

The startup focused on giving companies better tools for the tedious parts of compliance. They are still in the early stages of growth and SmartCat has worked as an engineering provider since the start.

Technologies Used

• Neo4j
• LangGraph
• LangChain
• Langfuse
• OpenCV
• OpenAI and Gemini
• AWS Bedrock