At Fujitsu, I developed reusable Azure Data Factory pipelines to orchestrate ETL processes, reducing development effort and increasing reusability across datasets. I also automated ETL pipelines using Azure Data Factory and SQL, cutting manual effort by 40%.
I built and managed Delta Lake tables in Azure Databricks and implemented incremental loads using watermarking and change data capture. My work supported data processing across the Silver and Gold layers.
I applied PySpark data quality checks and validation rules before records moved across the Bronze, Silver, and Gold layers. I also used Azure Monitor and Log Analytics to track pipeline performance and identify bottlenecks.
For the Sales Data Modernization project, I designed an Azure lakehouse using Azure Data Factory, ADLS Gen2, Databricks, and Delta Lake. I built Power BI reports on the Gold layer for sales reporting and analysis.
