Open to opportunities

Suhas varma Rudraraju

@suhasvarmarudraraju

Message

Senior data engineer building scalable, governed data pipelines on cloud platforms.

United States

Message

What I'm looking for

I’m looking to build reliable, observable data platforms on AWS, improving governance and pipeline performance with measurable impact. I want a collaborative team where I can mentor engineers and deliver incremental, high-quality improvements fast.

I’m a Senior Data Engineer with 6+ years of experience designing and scaling data pipelines, platforms, and infrastructure to support business objectives. I build reliable, observable, and scalable data architectures using AWS and open-source tools like Spark and Kafka to improve data accessibility and performance.

In my current role, I spearheaded a scalable AWS-based data platform, mentored junior engineers, and implemented ETL/ELT pipelines with Apache Spark and AWS Glue—reducing latency by 30%. I also established data governance protocols for compliance, used Apache Airflow to orchestrate complex workflows, and improved processing speed by 25% through performance tuning.

Previously, as a Cloud Data Engineer, I engineered real-time streaming pipelines with AWS and Apache Kafka, improved data quality and reduced processing times by 40%, and modernized architectures using Azure Data Factory and Amazon Redshift. Earlier big-data and data engineering experience with Hadoop, Hive, and Spark strengthened my focus on data quality, migration, and workflow optimization.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

National Institute Of Mental Health

Feb 2024 - Present (2 years 4 months)

Designed and built a scalable AWS-based data platform to improve data accessibility and reliability across departments. Reduced data processing latency by 30% using Apache Spark and AWS Glue, improved processing speed by 25%, and implemented data governance and Terraform-based infrastructure-as-code for compliant, reliable operations.

AWS Apache Spark AWS Glue Apache Airflow Data Governance Terraform ETL ELT Data Modeling Data Quality

Cloud Data Engineer

American Express

Jan 2022 - Aug 2023 (1 year 7 months)

Engineered AWS- and Kafka-based pipelines for real-time data streaming and processing for critical business applications. Improved data quality and reduced processing times by 40% using SQL and Spark, modernized analytics with Azure Data Factory and Amazon Redshift, and increased operational efficiency by 20% while reducing data storage costs.

AWS Apache Kafka SQL Apache Spark Azure Data Factory Amazon Redshift Data Quality Docker CI CD

Big Data Engineer

Goldman Sach

Jul 2020 - Dec 2021 (1 year 5 months)

Developed and maintained Hadoop and Hive solutions to enable efficient storage and retrieval of large datasets for analytics. Reduced data retrieval times by 35% through workflow optimization, implemented data quality checks, and improved scalability and performance via migrations to modern data architectures.

Hadoop Hive Apache Spark SQL Data Quality Data Warehousing Data Governance Data Modeling Performance Tuning

Data Engineer

Kroger

Jun 2019 - Jun 2020 (1 year)

Built ETL processes in Python and SQL to extract, transform, and load data into data warehouses for business intelligence. Improved reporting accuracy by implementing data quality controls and increased processing efficiency by 15% through workflow optimization, while supporting cloud migration and Tableau dashboard/report development.