At Business Data Solutions for BCI, I built data ingestions across Azure’s Bronze, Silver, and Platinum layers, including 40 ingestions for business domains. I integrated sources such as MongoDB, Azure Cosmos DB, and PostgreSQL, and worked with Data Governance to define domains and subdomains for cataloging.
As Technical Lead of the Black Box cell, I was responsible for ingestions from refinement through production continuity and led a team of three. I built close to 20 ingestions with Azure Data Factory and Azure Databricks, and implemented Spark SQL quality checks for completeness, source reconciliation, and duplicates.
I also migrated around 30 ESG data pipelines from Copy Data to parameterized Mapping Data Flows, making new ingestions configurable rather than requiring a new pipeline. Earlier, I developed REST APIs in C# and optimized database queries, and I adapted AutoQuad, a Python tool that generates PDF data quality reports.
