Open to opportunities

Lalit Deore

@lalitdeore

Message

Senior Data Engineer building scalable AWS/Databricks pipelines for reliable analytics.

India

Message

What I'm looking for

I’m looking for a role where I can build scalable AWS/Databricks data pipelines, modernize legacy systems into modular architectures, and deliver data observability and anomaly detection through strong engineering discipline.

I’m a results-driven data engineer with 6.2+ years of experience designing and implementing scalable data pipelines, cloud-native architectures, and big data solutions. I specialize in AWS ecosystem services like S3, Glue, Lambda, Athena, Redshift, and EMR, alongside PySpark and Databricks Delta Lake. I focus on re-engineering legacy pipelines into modular, config-driven systems and migrating data to cloud platforms without sacrificing maintainability or scalability.

I’ve delivered high-impact solutions by enabling data observability (including Monte Carlo for anomaly detection and data quality insights) and standardizing validation logic inside dynamically generated Athena views. From building AWS S3-based data lakes and AWS Glue ETL pipelines to automating RDS snapshots/exports and migrating into Databricks-managed tables, I partner with cross-functional stakeholders to test, validate, and refine production-grade data workflows.

Experience

Work history, roles, and key accomplishments

Senior Data Engineer

Zensar Technologies

Jun 2025 - Sep 2025 (3 months)

Re-engineered an OML KPI pipeline by replacing complex Glue logic with a config-driven AWS Lambda engine that dynamically generates Athena KPI views. Integrated KPI scoring with Monte Carlo to enable automated data quality validation, anomaly detection, and standardized monitoring.

AWS Lambda Amazon Athena Monte Carlo Data Observability Amazon S3 AWS Glue Data Validation Monitoring

Data Engineer - Healthcare Platform

Emonics Pvt Ltd

Dec 2024 - May 2025 (5 months)

Unified three healthcare databases into a scalable AWS S3 data lake using AWS Glue and Databricks Delta Lake, with Monte Carlo ensuring data reliability and quality. Built automated RDS snapshot generation and exports, developed Glue ETL pipelines, and integrated observability for freshness, volume, and schema anomaly monitoring.

AWS Glue Databricks AWS S3 AWS Lambda Monte Carlo ETL Data Observability

Data Engineer - DL Migration

Future Focus Infotech Pvt Ltd

Dec 2023 - Dec 2024 (1 year)

Migrated and consolidated data from three databases into an AWS S3 data lake and then into Databricks-managed tables to create a centralized analytics platform. Automated RDS snapshot/export to raw S3, orchestrated Glue processing, and loaded historical data into Databricks using Delta table ingestion in notebooks.

Amazon S3 AWS Glue AWS Lambda Databricks Delta Lake Data Migration

Data Engineer - AWS Data Migration

ABWeb Infotech Pvt Ltd

Jul 2019 - Dec 2023 (4 years 5 months)

Built centralized data warehouse pipelines on AWS by consolidating multiple data sources into S3 and provisioning Redshift for unified querying. Implemented Glue-based migration with quality checks, used Lambda to trigger jobs on file upload, and added Step Functions/SNS for automated transformations and failure alerting.

Amazon S3 AWS Glue AWS Lambda Amazon Redshift AWS Step Functions Data Quality Data Pipelines