Open to opportunities

SHILPA BANDI

@shilpabandi

Message

Data Engineer specializing in AWS Databricks, PySpark, Delta Lake pipelines.

India

Message

What I'm looking for

I seek a hands-on data engineering role focused on building scalable Databricks pipelines, cloud-native data platforms, and opportunities to lead data governance and performance optimization.

I am a Data Engineer with over four years of hands-on experience designing and delivering AWS Databricks data pipelines, specializing in Delta Live Tables, Structured Streaming, and scalable ingestion patterns. I have built one-time historical and incremental loads using PySpark, Python, SQL and S3, implemented checkpointed streaming with fault tolerance, and performed data validation, reconciliation, deployment, and unit testing.

At Infosys I progressed from System Engineer to Senior Systems Engineer, leading Unity Catalog migrations, developing advanced transformation logic (joins, aggregations, window functions, SCD handling), scheduling and monitoring via Databricks Jobs, and documenting test results. I hold relevant certifications including Databricks Certified Associate Data Engineer, SnowPro Core, and Google Associate Cloud Engineer.

Experience

Work history, roles, and key accomplishments

Current

Senior Systems Engineer

Current

Infosys Ltd

Jul 2023 - Present (2 years 6 months)

Designed and developed AWS Databricks pipelines using Delta Live Tables for batch and streaming ingestion, implemented Structured Streaming with checkpointing, and led Unity Catalog migration to centralize governance and access control.

Databricks Pyspark Python Delta Lake S3 Unity Catalog

System Engineer

Infosys Ltd

Nov 2021 - Jun 2023 (1 year 7 months)

Developed PySpark and SQL data extraction and integration modules, implemented incremental load logic with watermarking and partition-based processing, and performed data quality checks and performance tuning.

Pyspark SQL Watermarking Partitioning Data Quality Performance Tuning

Education

Degrees, certifications, and relevant coursework

Vidyanikethan Engineering College

Bachelor of Technology, Computer Science

2017 - 2021

Completed a Bachelor of Technology in Computer Science with coursework and projects focused on software development and data engineering fundamentals.