Open to opportunities

HARSHAVARDHAN GUNDETI

@harshavardhangundeti

Message

Data engineer specializing in scalable lakehouse solutions and streaming pipelines.

India

Message

What I'm looking for

I am seeking a data/platform engineering role where I can build scalable lakehouse solutions, optimize OLAP performance, automate ETL, and enable data democratization in a collaborative, growth-oriented team.

I am an analytical and solution-driven Data Engineer with over 2 years of experience building large-scale data lakehouse platforms using open-source technologies. I have hands-on expertise across the full data pipeline lifecycle, from Kafka-based ingestion and Spark processing to storage optimization with Iceberg and OLAP enabling with ClickHouse.

At UIDAI I designed and migrated a multi-terabyte lakehouse (Hudi to Iceberg), reduced query latency by 90%, achieved 3000 events/sec streaming ingestion, and migrated 25 TB of MySQL data while establishing robust Airflow-driven ETL pipelines. I also implemented compaction jobs, deployed Metabase and Superset for dashboards, and enabled BI access for dozens of analysts.

I am passionate about solving business problems with data, improving reporting efficiency, and delivering performant analytical platforms that scale to billions of rows. I enjoy automating workflows, optimizing OLAP workloads, and empowering stakeholders with actionable insights.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Unique Identification Authority of India

Jul 2023 - Present (2 years 7 months)

Designed and deployed a scalable data lakehouse and migrated MySQL to Apache Iceberg, reducing query latency by 90% and enabling sub-second OLAP analytics on billions of rows. Built streaming ingestion (Kafka→Spark), automated ETL with Airflow, and delivered BI access via Metabase and Superset to accelerate reporting by up to 20x.