Open to opportunities

Karan Jung Karki

@karanjungkarki

Message

Senior Data Engineer specializing in scalable ETL, cloud data platforms, and real-time analytics.

United States

Message

What I'm looking for

I seek senior roles building scalable, secure data platforms and automated ETL for real-time analytics, with strong cross-functional collaboration and growth opportunities.

I am a Senior Data Engineer with 7+ years designing and delivering scalable, high-performance data solutions across banking, e-commerce, regulatory compliance, and operational risk. I build automated, fault-tolerant ETL workflows using Spark, Python, and SQL, optimize performance for petabyte-scale workloads, and implement secure, governed data architectures on AWS, Azure, and GCP.

I lead cross-functional teams to deliver end-to-end data products, from ingestion through analytics and BI, and have reduced processing times by up to 40% through Spark optimization and pipeline automation. I prioritize data quality, security, and compliance while enabling real-time ML and analytics via containerization, orchestration, and CI/CD pipelines.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Pfizer

Sep 2022 - Present (3 years 1 month)

Architected and optimized petabyte-scale ETL pipelines using PySpark and Airflow, reducing batch processing time by 40% and improving financial query responsiveness for data science and reporting teams. Led cross-functional teams to deliver a HIPAA-aligned enterprise data lake and implemented Delta Lake with Snowflake for ACID-compliant real-time and batch processing.

Python Pyspark Apache Spark Airflow Snowflake Databricks Azure Service Bus Data Modeling ETL

Data Engineer II

Capital One

May 2020 - Aug 2022 (2 years 3 months)

Led migration of on-prem data warehouses to AWS data lake, reducing processing time and costs while implementing CI/CD pipelines to streamline deployments and reduce manual errors. Built scalable ETL pipelines with PySpark, EMR, Lambda, and Kafka to process millions of records daily and enabled real-time analytics via Spark Streaming.

Python Pyspark Lambda S3 Kafka Jenkins Terraform Snowflake

Data Engineer

Signify Health

Jan 2019 - Apr 2020 (1 year 3 months)

Modernized data warehouse by building HDFS/Hive archival and ingestion pipelines, improving storage efficiency and read performance, and implemented validation and lineage tracking to ensure data quality for healthcare reporting. Migrated legacy environments and developed PySpark apps and Sqoop-based ingestion to support patient analytics and operational dashboards.