Tarun Yellapu
@tarunyellapu
I build scalable AWS batch and streaming data pipelines with Python, PySpark, and Terraform.
IndiaWhat I'm looking for
I've built scalable batch and streaming data pipelines at NovusPlatform.io and Guardant Health, using Python, PySpark, AWS Glue, S3, DMS, Airflow, and Terraform. I translate business requirements into reliable ETL processes and data-platform solutions.
I migrated legacy Glue ETL jobs to Apache Hudi-based pipelines for performance, cost optimization, and hourly incremental merges, and built Oracle-to-S3 Parquet pipelines transformed with PySpark. I've also automated Tableau deployments, created an AWS CodeArtifact repository for Python packages, and developed a custom Terraform AWS provider fork for organization-specific infrastructure enhancements.
At Guardant Health, I've benchmarked Apache Iceberg, Apache Hudi, and native S3 tables to guide data lake architecture, built Athena query auditing with Python and SQLGlot, and am leading Unified SageMaker Studio onboarding. I also interview and train engineers while continuing to learn new technologies, including Rust.
Experience
Work history, roles, and key accomplishments
Conducted a comprehensive POC on Apache Iceberg, evaluating its suitability against Apache Hudi and native S3 tables. Designed and implemented an Athena query auditing solution using Python and SQLGlot. Currently leading the onboarding of Unified SageMaker Studio for collaborative ML development.
Data Engineer
NovusPlatform.io
Mar 2022 - Oct 2024 (2 years 7 months)
Built scalable batch and streaming data pipelines, designed complex Airflow pipelines, and collaborated with stakeholders for ETL requirements. Automated Tableau dashboard deployments, migrated Glue ETL jobs to Apache Hudi, developed a custom Terraform AWS provider fork, and built a data pipeline using AWS DMS and PySpark on AWS Glue.
Tested Transact-API microservice for Shopify, streamlining capture and settlement processes. Applied Selenium (Java) and Postman for testing functionalities like fetching product data, managing orders, updating inventory, and processing payments. Created a Proof of Concept for an automated widget placement system.
Education
Degrees, certifications, and relevant coursework
Chaitanya Engineering College
Bachelor of Technology, Computer Science
Bachelor of Technology in Computer Science with relevant coursework in data structures, algorithms, and operating systems.
Availability
Location
IndiaAuthorized to work in
Job categories
Skills
Interested in hiring Tarun?
You can contact Tarun and 90k+ other talented remote workers on Himalayas.
Message TarunGet matched with your dream remote job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
