Open to opportunities

Soham Katkar

@sohamkatkar

Message

Data Engineer focused on reliable, scalable analytics pipelines across cloud platforms and modern AI workflows.

United States

Message

What I'm looking for

I’m looking for a role where I can own scalable data pipelines on AWS/GCP/Azure, improve reliability and performance, and apply RAG/vector search to build useful, production-grade AI features for analytics teams.

I’m a Data Engineer with 4 years of experience building reliable batch and streaming pipelines that help analytics and reporting teams move faster with trustworthy data. I architect and optimize ETL/ELT workflows for structured datasets across data lakes, warehouses, and ERP sources.

My work centers on performance, uptime, and data quality—using patterns like incremental loading, automated data validation, and orchestration with Apache Airflow. I’ve improved pipeline performance by 30%, maintained 99.9% uptime for downstream BI, and reduced data corruption risks by proactively detecting schema drift.

I also bring practical generative AI experience, including retrieval-based systems and vector search for document-based question answering. From RAG assistants to AI-driven research workflows, I translate modern NLP needs into production-ready data and retrieval systems while staying focused on measurable impact.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Thaddeus Resource Center

Jul 2025 - Present (11 months)

Architected automated cloud workflows with Apache Airflow and GCP to ingest ERP transactional and customer records into BigQuery, maintaining 99.9% uptime for BI and analytics. Built incremental loading to improve performance by 30% and implemented automated data validation to detect schema drift and reduce data corruption risk.

Apache Airflow Google Cloud Platform (GCP)BigQuery ERP Data Integration Data Validation Schema Drift Detection Python SQL

Data Engineer

HSBC Technology

Aug 2021 - Dec 2023 (2 years 4 months)

Developed scalable PySpark ETL/ELT pipelines processing 5M+ daily transactions and identifying $2M+ in revenue opportunities. Improved data accuracy by 25% across 12+ sources using incremental loading and reduced setup time by 40% by automating troubleshooting with Shell scripts and Jenkins.

Pyspark ETL ELT SQL Shell Scripting Jenkins Transaction Processing

Junior Data Analyst

Incentius Solutions Pvt. Ltd.

Feb 2021 - Jul 2021 (5 months)

Designed enterprise star schemas and optimized SQL queries, improving performance by 40% for executive dashboards. Delivered clean datasets for BI developers using Tableau and Google Analytics, increasing stakeholder engagement by 50% across 6 European markets.