Open to opportunities

Yogesh Jagtap

@yogeshjagtap

Message

Dynamic Team Lead Data Engineer with over a decade of experience.

Zimbabwe

Message

What I'm looking for

I am looking for a role that fosters innovation and collaboration, where I can leverage my expertise in data engineering to drive impactful solutions and contribute to a forward-thinking team.

I am a dynamic Team Lead Data Engineer with over a decade of experience in delivering innovative data solutions. My expertise lies in leading cross-functional teams and developing cutting-edge data strategies that drive business growth through data-driven decision-making. I have a strong background in big data technologies, cloud services, and data pipeline optimization.

Throughout my career, I have successfully managed large-scale data processing and migration projects, implementing robust data governance practices. My experience includes transitioning legacy systems to modern, loosely coupled platforms, and I am highly skilled in microservices-based architectures. I take pride in my ability to automate workflows and enhance data retrieval processes, significantly improving efficiency and reducing latency.

Experience

Work history, roles, and key accomplishments

Current

Senior Engineer

Current

American Express

Mar 2025 - Present (4 months)

Led a team to migrate over 500 GB of mainframe data into a distributed big data environment using PySpark, reducing processing time by 40%. Developed end-to-end data pipelines using PySpark for batch data processing, leveraging optimization techniques like Partitioning, Clustering, and Sharding to handle large datasets.

Pyspark Data Migration Data Pipelines Partitioning Clustering ETL Apache Airflow

Team Lead

NucleusTeq

May 2023 - Present (2 years 2 months)

Pyspark Data Migration Data Pipelines Partitioning Clustering ETL Apache Airflow

Senior Software Engineer

LIFTLAB

Sep 2019 - Present (5 years 10 months)

Developed and maintained data management systems for scalable, fault-tolerant data pipelines, ensuring alignment with business requirements. Integrated various social media marketing APIs (Facebook, Pinterest, Snapchat, TikTok), enhancing data collection and real-time analytics capabilities.

Data Management Systems Data Pipelines Data Validation Hive AWS GCP

Senior Software Engineer

Vigoursoft

Mar 2018 - Present (7 years 4 months)

Scraped and analyzed social media data to extract insights into competitor trends. Enhanced pipeline management processes using PySpark and Scala for large-scale data processing.

Pyspark Scala Event Driven Architecture

Senior Software Engineer

Mugenesys

Apr 2016 - Present (9 years 3 months)

Architected and implemented Sentiment analysis (emotion, polarity, custom) and visualization using Natural Language Processing (NLP). Implemented Chat BOT using Api.ai.

GATE Library NLP Chatbot

Software Engineer

Cloud Insights

Aug 2013 - Present (11 years 11 months)

Designed and implemented optimized ETL pipelines for batch and real-time data using AWS Redshift and PySpark. Developed custom dashboards and reporting solutions for real-time business intelligence and data center operations.

ETL AWS RedShift Pyspark Dashboards Reporting Data Governance NoSQL MongoDB Couchbase

Software Engineer

Innoplexus

Dec 2011 - Present (13 years 7 months)

Developed a resume parsing system using the GATE library, with integration through backend APIs built on Scala Play Framework. Enhanced job matching algorithms by analyzing patterns in job descriptions and resumes, leading to a 15% increase in accuracy.

Resume Parsing GATE Library MongoDB

Education

Degrees, certifications, and relevant coursework

Vidya Pratishthan

Master of Computer Application, Computer Application

Completed a Master's degree focusing on advanced computer applications and software development. Gained expertise in various programming paradigms and system design principles.

Vidya Pratishthan

Bachelor of Computer Science, Computer Science

Acquired foundational knowledge in computer science, including programming, data structures, and algorithms. Developed strong problem-solving skills and an understanding of computing principles.