Open to opportunities

Dillep Ballem

@dillepballem

Message

Senior Data Engineer specializing in cloud-native, scalable data pipelines and AI-enabled analytics.

India

Message

What I'm looking for

I seek senior roles building cloud-native data platforms and ML/AI pipelines at companies valuing automation, observability, security, and measurable business impact.

I am a Senior Data Engineer with 7+ years designing and delivering scalable cloud-native data pipelines across AWS, Azure, and GCP, specializing in Spark, Databricks, and real-time streaming.

I have led end-to-end migrations, implemented Data Vault 2.0 architectures, and built ML/LLM pipelines and observability platforms that reduced transaction time and improved reliability.

I prioritize secure, optimized, business-aligned solutions, driving automation, CI/CD, and monitoring to lower MTTR and enable data-driven decision making.

Experience

Work history, roles, and key accomplishments

Sr. Data Engineer

Insight Software International Pvt. Ltd.

Apr 2023 - Sep 2024 (1 year 5 months)

Designed and delivered cloud-native, production-grade data and ML pipelines across AWS/Azure/GCP, modernized Spark/EMR workloads, and implemented Microsoft Fabric and DLT solutions that reduced transaction time by up to 60%.

Apache Airflow Microsoft Fabric Databricks Kafka Python

Data Engineer

Epiq Systems India Pvt Ltd

Mar 2020 - Jul 2022 (2 years 4 months)

Built and maintained scalable ETL/ELT pipelines across AWS, Azure and GCP using Glue, ADF and Dataflow, led large-scale migrations (Oracle→Postgres) and developed data models and Power BI reporting for analytics.

AWS Glue Azure Data Factory Dataflow Apache Spark Power BI DBT Snowflake S3

Data Engineer

Astoria Technologies Pvt Ltd

Nov 2017 - Jan 2020 (2 years 2 months)

Developed ETL/ELT solutions and Databricks-based data workflows on AWS/Azure, implemented data lakes and warehouses, and automated CI/CD and orchestration with Airflow and Azure DevOps.

Databricks Pyspark Azure Data Factory Delta Lake Azure DevOps Airflow AWS S3 SQL Docker

IT Engineer

ACT Pvt Ltd

Dec 2015 - May 2017 (1 year 5 months)

Implemented and optimized ETL/ELT workflows using Spark and Azure Data Factory, managed data lakes and monitoring, and delivered automated reporting and CI/CD pipelines for enterprise data operations.