ELT approach for Data Pipelines
Introduction
While gathering data for Analytics, one often has to source data from multiple sources. Traditionally, the approach has been to do ETL (Extract-Transform-load) where,
- Extract - typically involves retrieving data from source. This could also be via streaming
- Transform - Apply transformation to the extracted data.
- Load - Loading the data in Operation Data store (ODS) or data warehouse Refer here for more details on ETL. ETL has been made easy by tools like Talend, SSIS and so on.
However, there has been shift from above approach due to,