Open to opportunities

Harshal Bhadke

@harshalbhadke

Message

Data engineer specializing in AWS, PySpark, and scalable serverless data pipelines.

India

Message

What I'm looking for

I seek roles building scalable, serverless data pipelines on AWS where I can optimize ETL performance, drive cost efficiencies, and contribute within collaborative Agile teams.

I am a detail-driven data engineer with hands-on experience building serverless, scalable data pipelines using AWS services, PySpark, and SQL. At Tata Consultancy Services I implemented ETL processes and optimizations that reduced data processing time by 60% and storage costs by 20%.

My work spans data ingestion, transformation, orchestration and monitoring — ingesting CSVs to S3, transforming to Parquet with AWS Glue, loading to Redshift/PostgreSQL/MSSQL, and visualizing results with Power BI. I leverage Lambda and Step Functions for dynamic configuration and workflow orchestration, and CloudWatch for operational visibility.

I collaborate effectively in Agile teams, with prior experience improving SQL efficiency by 25% and data quality by 15%, and I ensure solutions are thoroughly unit tested and production-ready.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Tata Consultancy Services

Apr 2023 - Present (2 years 10 months)

Implemented serverless ETL pipelines using PySpark, AWS Glue, Lambda and Step Functions to standardise IFRS 17 financial reporting, reducing data processing time by 60% and storage costs by 20%. Monitored pipelines with CloudWatch and supported engineering environments for high availability.

Pyspark AWS Glue AWS Lambda AWS Step Functions AWS S3 AWS RedShift PostgreSQL ETL

Software Engineer

Atyeti Inc

Nov 2020 - Mar 2023 (2 years 4 months)

Designed and implemented complex SQL logic for major client projects, improving query efficiency by 25% and drove a 15% improvement in data quality through profiling and testing in an Agile environment.