Hemanth Kadiyala
@hemanthkadiyala
Senior data engineer building scalable ETL/ELT pipelines and analytics datasets across AWS and Azure.
United StatesWhat I'm looking for
I’m a Data Engineer with 6+ years of experience building and owning large-scale ETL/ELT pipelines and analytics datasets in AWS and Azure environments. I focus on delivering reliable, governed data products that help product, finance, and data science teams move faster.
At Meta, I designed and maintained ETL pipelines for multi-terabyte product analytics and operational reporting. I integrated Hive tables (ORC/Avro), built curated data marts with Python/Presto SQL/Hive/Spark, and enforced dimensional models, data lineage, and field-level access controls with audit trails for governance compliance.
I’m especially proud of performance and quality improvements: I optimized legacy PySpark pipelines to reduce compute consumption by 35% and cut cloud infrastructure costs by ~$3K/week. I also improved dataset reliability by resolving recurring data quality issues and raising dashboard acceptance scores from 30% to 98%, while implementing PII anonymization, privacy compliance, and centralized audit trails.
Previously at Hilton and Lululemon, I built end-to-end ELT and streaming ingestion workflows (Airflow orchestration, Glue ingestion, Kafka/Event Hubs/Structured Streaming), migrated sources into Snowflake, and automated data quality validation (including Great Expectations). I enjoy turning business questions into KPI definitions, operationalizing monitoring and SLA alerts, and standardizing delivery through data APIs using Swagger/OpenAPI contracts.
Experience
Work history, roles, and key accomplishments
Designed and maintained multi-terabyte ETL pipelines for product analytics, feedback analysis, and operational reporting. Improved dataset reliability and stakeholder acceptance from 30% to 98%, reduced compute costs 35% (≈$3K/week), and cut ETL resolution time by ~40% through monitoring, SLA alerts, and automated governance.
Built end-to-end ELT pipelines for web log and clickstream ingestion, transforming data into Redshift fact-dimension schemas for analytics. Migrated key datasets to Snowflake with role-based access, implemented Great Expectations to catch 95%+ issues pre-consumption, and replaced manual reporting with Tableau/Power BI dashboards saving ~15 hours/week.
Developed Azure-based ETL pipelines to ingest inventory and supply chain data into a centralized cloud warehouse. Built near real-time Kafka ingestion for store events, standardized retail datasets using Databricks, and implemented SCD Type 2 history tracking; migrated legacy workloads to Azure to reduce operational costs by ~35% and improve scalability.
Built ELT pipelines with Azure Data Factory and Databricks to ingest trading, risk, compliance, and portfolio data from Oracle and Teradata into Azure Synapse dimensional models. Implemented near-real-time ingestion using Event Hubs and Spark Structured Streaming, optimized Synapse SQL/data models, and added governance controls to support regulatory reporting and audit readiness.
Education
Degrees, certifications, and relevant coursework
University of North Carolina at Charlotte
Master of Science, Computer Science
2022 - 2023
Completed a Master of Science in Computer Science at the University of North Carolina at Charlotte from Jan 2022 to May 2023.
Jawaharlal Nehru Technological University
Bachelor of Technology, Computer Science and Engineering
Completed a Bachelor of Technology in Computer Science and Engineering from Jawaharlal Nehru Technological University in India.
Tech stack
Software and tools used professionally
Availability
Location
United StatesAuthorized to work in
Social media
Job categories
Skills
Interested in hiring Hemanth?
You can contact Hemanth and 90k+ other talented remote workers on Himalayas.
Message HemanthFind your dream job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
