Looking for a job

srikarreddy maddi

@srikarreddymaddi

Message

I build reliable batch and streaming data platforms at scale.

United States

Message

What I'm looking for

I'm looking to build reliable batch and streaming data platforms where I can improve performance, cost efficiency, governance, and production resilience while partnering with analytics and ML teams.

At Wayfair, I own Kafka–Spark Structured Streaming pipelines processing more than 1 TB per day for near-real-time analytics and machine learning workloads.

I reduced micro-batch processing time by 25% and monthly compute costs by 28% through Spark and Kafka optimization. I also built reusable Python/PySpark ingestion components that deliver DynamoDB customer-product interaction data to S3 for recommendation model training.

Previously at Molina Healthcare and Wells Fargo, I built governed lakehouse and batch data pipelines for claims analytics, regulatory reporting, and risk monitoring using Databricks, Snowflake, Airflow, AWS Glue, Spark, and Kafka. I’ve improved batch compute costs, migrated legacy Hive/Sqoop pipelines to Spark, and implemented recovery patterns including checkpointing, retries, and DLQ handling.

I build production-ready platforms with data contracts, schema evolution, RBAC, Terraform, Docker, Kubernetes, and Jenkins. I also mentor junior engineers and enjoy making complex data systems reliable and maintainable.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Wayfair

Feb 2024 - Present (2 years 7 months)

Owned end-to-end design and operation of Kafka–Spark Structured Streaming pipelines processing 1+ TB/day. Optimized Spark and Kafka settings, reducing micro-batch time by 25% and compute costs by 28%.

Apache Spark Kafka PySpark Spark Structured Streaming DynamoDB S3 Datadog Docker Kubernetes Jenkins Terraform AWS IAM Data contracts Schema Evolution CI CD

Senior Data Engineer

Molina Healthcare

Nov 2022 - Jan 2024 (1 year 2 months)

Built curated Delta Lake tables on S3 and automated loading into Snowflake for claims analytics. Orchestrated 12+ production batch pipelines with Apache Airflow and implemented RBAC and column-level access controls.

Delta Lake Snowflake PySpark Airflow Databricks Unity Catalog RBAC PostgreSQL SQL Data Validation

Data Engineer

Wells Fargo

Feb 2021 - Jul 2022 (1 year 5 months)

Migrated 12 legacy Hive/Sqoop batch pipelines to Spark using PySpark and Scala. Orchestrated workflows with Airflow and implemented checkpointing, Kafka offset management, and DLQ handling for transaction streams.

Apache Spark PySpark Scala Airflow Kafka AWS Glue Hive Sqoop Data Validation Checkpointing DLQ

Junior Data Engineer

Anblicks

Dec 2019 - Jan 2021 (1 year 1 month)

Built batch ELT pipelines ingesting data from 10+ sources into Snowflake, implementing incremental loads to reduce source-to-report delays. Developed dbt models on Snowflake using star-schema designs for marketing analytics.

Snowflake DBT ELT Star Schemas Data Modeling SQL Incremental Load