Looking for a job

Omkar Musale

@omkarmusale

Data Engineer II · GCP · Apache Beam · Airflow · BigQuery · LLM Pipelines · 4 years building production-scale data infrastructure

India

Message

What I'm looking for

I'm looking for a fully remote Data Engineering role with a US or EU company where I can own pipeline infrastructure end-to-end. I want to work on hard data problems at scale whether that's LLM data pipelines, real-time streaming, or large-scale batch processing. I thrive in async, autonomous environments where output matters more than hours logged.

I design and build production-scale data pipelines that turn raw, messy, multi-format data into clean structured datasets ready for AI models and analytical systems.

Over 4 years at Infocusp Innovations I've owned pipeline engineering across two very different domains:

Legal Document Intelligence (current): Architected Apache Beam pipelines processing millions of documents per day across 10+ file formats PDFs, emails, spreadsheets, presentations into structured LLM-ready datasets, cutting AI data preparation time by 60%. Built annotation extraction and PDF rendering libraries from scratch. Owned full Airflow + GCP Cloud Composer orchestration for 5+ production pipelines with zero-touch scheduling and automatic fault recovery.

Geospatial & Environmental Data (previous): Engineered petabyte-scale ingestion pipelines for satellite imagery from Google Earth Engine, processing 100GB+ per run. Cut pipeline run time from hours to under 40 minutes via algorithmic optimization. Built fully automated Cloud Scheduler + Cloud Workflows + Cloud Batch export pipelines powering weekly ML model training cycles.

Outside work I've shipped two open-source Python libraries to PyPI IntelliScraper (2567 + downloads), an async anti-bot web scraping library, and pdfnotate, which extracts hidden PDF annotation metadata for LLM pipelines.

Core stack: Python · Apache Beam · Airflow · GCP (Cloud Composer, GCS, Pub/Sub, Cloud Batch, Cloud Scheduler, Cloud Workflows, GKE) · BigQuery · Docker · Kubernetes · Terraform · PySpark

I care about pipelines that are reliable, observable, and don't need babysitting in production.

Experience

Work history, roles, and key accomplishments

Geospatial Data Pipeline

Infocusp Innovations

Jul 2023 - Present (3 years)

Developed a scalable data pipeline to download and process massive geospatial datasets efficiently. Worked on geospatial data, handling large geospatial datasets (TIFF, CSV) exceeding 100GB. Optimized algorithms to improve efficiency by 20% and enhanced code quality using OOP principles.

Kubernetes Terraform Docker Python OOP

Automated Data Export Pipeline

Infocusp Innovations

Jul 2023 - Present (3 years)

Designed and implemented an automated cron job to export data at regular intervals, ensuring access to the latest datasets.

Cloud Scheduler Terraform Python GCP

Accident Data Analysis & Visualization

Infocusp Innovations

Jan 2023 - Present (3 years 6 months)

Analyzed accident data to identify high-risk areas and detect patterns in accident occurrences. Used big data techniques (Pyspark) to extract insights. Created interactive visualizations using Streamlit to display accident trends and hotspots.

Pyspark Streamlit Big Data Data Analysis Data Visualization

Healthcare Device Data Processing

Infocusp Innovations

Jan 2023 - Present (3 years 6 months)

Worked on processing large volumes of healthcare data using Apache Spark (PySpark) on a distributed system to extract insights. Worked on data visualization dashboards using Streamlit for better insights.

AWS Glue Pyspark Streamlit Terraform Apache Spark Data Visualization

Education

Degrees, certifications, and relevant coursework

Pimpri Chinchwad College of Engineering

Bachelor of Engineering (B.E.), Information Technology

Grade: 8.56 CGPA

Activities and societies: Achieved 2nd Place in a Competitive Coding Competition.

Completed a Bachelor of Engineering in Information Technology, focusing on core computer science principles. Relevant coursework included Data Structures and Algorithms, Program Development, DBMS, OOP, Big Data, Devops, and Web technology.