Open to opportunities

Hari vignesh User

@harivigneshuser

Message

Data engineer specializing in scalable ETL, Spark, Airflow, and AWS.

India

Message

What I'm looking for

I seek a data engineering role where I can build scalable ETL pipelines, optimize Spark workloads on cloud platforms, collaborate with stakeholders, and grow technical ownership in a data-driven, compliant environment.

I am a Data Engineer with over three years of hands-on experience building scalable ETL pipelines and optimizing large-scale data processing using Apache Spark, Airflow, and AWS.

I design and implement reliable data workflows on AWS EMR and S3, applying Spark DataFrame transformations, RDDs, and Hive schemas to process structured, semi-structured, and unstructured data.

I focus on performance tuning — partitioning, caching, and resource tuning — and have reduced EMR costs by 20% through optimizations while ensuring data quality and compliance for healthcare workloads.

I collaborate closely with business stakeholders to translate domain logic into production DAGs, automate batch workflows, and maintain accurate master data, bringing an agile, quality-driven approach to data engineering projects.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Claritrics India Private Limited

Jan 2023 - Present (3 years 2 months)

Built and optimized scalable ETL pipelines on AWS EMR using PySpark and Airflow, improving processing performance and reducing EMR costs by 20% while ensuring healthcare data quality and HIPAA-aligned traceability.

Pyspark Apache Airflow S3 Hive SQL Performance Tuning

Intern - Data Engineer

Claritrics India Private Limited

Oct 2022 - Dec 2023 (1 year 2 months)

Implemented Spark applications and Hive schemas to load and transform large semi-structured datasets (JSON, Avro, XML), and optimized Hive partitioning and bucketing for efficient access.

Spark Hive Avro SQL Hadoop Pyspark