karan diwan

Designed and maintained scalable data pipelines using Microsoft Fabric, Azure Databricks, and Delta Lake. Improved data reliability and governance, reduced ETL processing time by 30%, and improved database performance by 35%.

Supported live-site operations by investigating production Informatica IICS workflow failures and triaging issues to restore SLA-critical pipelines. Debugged and resolved pipeline failures under SLA pressure, coordinating with upstream data owners.

Designed and built full and incremental ingestion pipelines to bring source data into the target platform within defined SLA windows. Implemented data quality and reconciliation checks at each ingestion stage to catch schema drift and record-count mismatches.

Led migration of a legacy Oracle database to PostgreSQL, covering schema conversion, data type mapping, and validation. Designed target data models using fact and dimension structures and implemented automated data quality checks, reducing post-migration errors by 25%.

Migrated existing ETL workflows from SSIS to Talend, re-implementing transformation logic while preserving business rules and output parity. Designed fact and dimension models and automated recurring data transformation and validation steps in Python.

Designed a Python and Data Science curriculum for 100+ students, structured around progressive, project-based learning. Built project-based learning modules and conducted technical knowledge-transfer sessions.

Databricks Certified Data Engineer Associate certification obtained in 2026.

SQL certification from LeetCode obtained in 2025.

OCI AI Foundation Associate certification from Oracle obtained in 2025.

Prompt Design in Vertex AI certification from Google obtained in 2025.

Bachelor of Technology in Computer Science and Engineering from Lovely Professional University, 2017 to 2021.

Python for Data Science certification from IBM obtained in 2020.