Open to opportunities

Sudheer Bandi

@sudheerbandi

Message

Senior Data Engineer optimizing Spark pipelines with Azure and Databricks for enterprise insight.

India

Message

What I'm looking for

I want to design and optimize scalable data pipelines using big data frameworks and distributed systems—focusing on high performance and reliability for enterprise applications while upscaling with emerging technologies.

I’m a Senior Data Engineer with around 5 years of experience building large-scale, production-grade data pipelines using Spark (Scala), Databricks, and Azure services. I’ve processed 200M+ records, optimized Spark jobs, and delivered cloud-native data platforms on Kubernetes with CI/CD automation.

On the PELE project for Lufthansa Airlines, I translate business requirements into reliable, high-performance data systems—designing Scala/Spark batch workflows, enforcing data quality with Delta Live Tables, and using Azure Key Vault for secure credential access. I also improve production reliability with event-driven orchestration (Azure Service Bus + ADF), automated monitoring and alerts (Azure Monitor + Kusto Query Language), and deployments to Kubernetes via Azure DevOps, cutting manual release effort by ~40%.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

LTIMindtree

Jun 2021 - Present (5 years)

Designed and implemented Scala/Spark batch pipelines for Lufthansa’s PELE project, processing 200M+ records per run and optimizing performance across multi-terabyte datasets. Built Azure Databricks (DLT/Auto Loader) and ADF workflows with secure credential handling, event-driven orchestration, monitoring/alerts, and Kubernetes-based CI/CD deployments to improve reliability and time-to-insight.