Open to opportunities

Kiran Pedakotla

@kiranpedakotla

Message

Senior Data Engineer specializing in cloud-native, scalable data platforms.

United States

Message

What I'm looking for

I seek senior data engineering roles building cloud-native, scalable pipelines and platforms that power AI/analytics, with strong DevOps, data governance, and collaborative teams focused on impact.

I am a Senior Data Engineer with around 9 years of experience building cloud-native data platforms, ETL pipelines, and big data solutions across Azure, AWS, GCP, OCI and Snowflake. I have led large migration projects (Hadoop-to-Snowflake), built real-time and batch pipelines with Databricks, ADF, PySpark, and optimized data platforms for AI/ML and analytics in healthcare and finance.

I combine hands-on engineering (Python, Scala, Spark, SQL), cloud infrastructure and DevOps (Terraform, Azure DevOps, CI/CD) with strong data modeling and governance practices to deliver reliable, HIPAA-compliant data products. I mentor teams, drive performance tuning, and automate workflows to improve operational efficiency and data accuracy.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Shell

Jul 2021 - Present (4 years 5 months)

Developed and maintained scalable end-to-end Azure and multi-cloud data pipelines supporting AI/ML and financial forecasting, ensuring 100% data accuracy and low-latency delivery for high-volume datasets. Built cloud-native infrastructure and CI/CD automation, led migrations to Snowflake and Databricks, and mentored data engineering teams to improve performance and reliability.

Azure Data Factory Databricks Pyspark Snowflake AWS S3 Lambda Terraform CI CD SQL Kafka

Senior Data Engineer

Chevron

Oct 2019 - Jul 2021 (1 year 9 months)

Designed data models and ETL pipelines using Azure Data Factory, Databricks, and Snowflake to migrate legacy financial reporting systems and optimize analytics performance. Implemented data quality monitoring, CI/CD for Databricks, and BigQuery/Dataform transformations to support scalable reporting.

Azure Data Factory Databricks Pyspark Snowflake BigQuery dataform CI CD SQL Airflow

Senior Data Engineer

Apple

Apr 2019 - Sep 2019 (5 months)

Designed and executed Spark ETL on Hadoop clusters and Databricks, using Hive and HBase integrations to build scalable reporting pipelines and improve transactional data processing for analytics. Led data fabric initiatives and built integrations across SaaS platforms to enable unified reporting.

Spark Pyspark Hive HBase Hadoop Databricks Python Sqoop

Software Engineer

U.S. Bank

Mar 2017 - Apr 2019 (2 years 1 month)

Implemented real-time streaming and batch data solutions using Spark, Kafka, and Hadoop; migrated legacy jobs to Databricks on AWS and operationalized ML models to improve maintainability and reduce infrastructure costs. Developed data processing pipelines and back-end services in Java and Spring Boot.

Spark Kafka Hadoop Databricks Scala Java SQL Shell Scripting Pl SQL

Technology Analyst

Walmart Health

Nov 2015 - Feb 2017 (1 year 3 months)

Built scalable healthcare ETL pipelines and GCP data solutions to integrate claims, enrollment, and clinical sources, improving star ratings metric processing times by over 30% and enabling near real-time member analytics. Implemented data quality checks and event-driven pipelines for care management.

GCP BigQuery Apache Spark Kafka AWS Glue Python Data Pipelines ETL Data Quality

Software Engineer

Apollo

Mar 2012 - Jul 2015 (3 years 4 months)

Developed web services and supported SDLC activities using Spring and REST; performed defect analysis, implemented business requirements, and provided maintenance and support for enterprise applications. Contributed to backend development and system integration tasks.