Stage 01–02
Schema Detective.
The agent reads your source schemas — ERP extracts, CSV dumps, SaaS APIs — and builds a complete structural map before you write a single transformation. No assumptions. No hand-coding. No surprises downstream.
What the agent does.
Reads source schemas
Connects to staged raw tables and profiles every column — data type, null rate, cardinality, value range, most common values. Identifies implicit patterns (phone numbers stored as text, dates as integers, IDs with mixed formatting).
Maps relationships
Detects implicit foreign key relationships by matching column name patterns and value overlap across tables. Builds a FK graph — no schema definition files required. Surfaces orphaned tables and columns with no downstream consumers.
Produces a structural fingerprint
Generates a schema report: every table, every column, every detected relationship, every anomaly. This report is the input to the Structural Normalisation stage and the audit anchor for every downstream decision.
Input
- Raw staged tables (any source — ERP extract, CSV, API)
- PyAirbyte connector output
- SQL dump files
Output
- Schema report (tables × columns × types × anomalies)
- FK relationship graph
- Structural fingerprint (versioned, committed to git)
Next stage
Normalise + Data Prep + MDM →Relevant solutions