Venkatesh Madabathula

I’m a Data Engineer with 4 years of experience building AWS and Databricks-based ETL pipelines for analytics-ready healthcare datasets. I specialize in designing and optimizing pipelines for large-scale structured and semi-structured data, using PySpark, Spark SQL, and Delta Lake to support data warehousing and governance needs.

In my current role, I developed and optimized PySpark ETL on Databricks, implementing incremental loading, CDC, and SCD Type 2 to keep historical customer and account data accurate for regulated reporting. I applied the Medallion Architecture (Bronze, Silver, Gold), orchestrated reliable batch workflows with Databricks Jobs and AWS Step Functions, and tuned Spark/Delta workloads to achieve ~40% performance improvement—delivering optimized gold-layer datasets for Power BI risk analysis and regulatory reporting.