ETL

ETL describes moving data from the systems that produce it into a system that analyses it: extract from the sources, transform into a consistent shape, load into the destination. The variant ELT loads first and transforms inside the destination, which suits modern warehouses that are fast enough to do the work themselves.

The transform step is where the real effort sits, and it is rarely technical. Two systems calling something a "customer" and meaning different things, dates in three formats, a country field with free text — reconciling those is the project.

ETL pipelines fail quietly, which is their characteristic danger. A source changes a field name, the load continues with nulls, and a dashboard shows a plausible wrong number for a month. Validation and alerting on the pipeline matter as much as the pipeline.