At IBM, I design and develop end-to-end ETL/ELT pipelines on Microsoft Azure, including dynamic Azure Data Factory workflows for on-premises-to-cloud migration. I build full-load and incremental pipelines that bring data from multiple sources into Azure Data Lake Storage Gen2.
I also develop PySpark and Spark SQL transformations on Azure Databricks, tuning jobs and implementing data quality checks for production pipelines. In my projects, I built a telecom churn data pipeline and a retail analytics platform using layered data lakes.
