# Mahendra Singh Rathore

Mahendra Singh Rathore is a remote worker on Himalayas, Senior Data Engineer (Team Lead) at Shell, based in India.

Source: https://himalayas.app/@mahendrasinghrathore1

```text
Senior Data Engineer at Shell who co-developed a RAG solution and replaced ADF scheduling with Databricks Asset Bundles, saving about $20K annually.
```

| Detail | Value |
| --- | --- |
| Availability | Looking for a job |
| Location | India |
| Authorized to work in | India |
| Salary expectations | 4033k+ INR |
| Job categories | Data Engineer, AI Engineer |
| Skills | Azure Data Engineering, Data Engineering, Data Ingestion, Databricks, Azure Data Factory, Unity Catalog, Databricks Asset Bundles, Delta Lake, MLOps, MLFlow, RAG, Sentence Transformers, Geospatial analysis, Apache Sedona, Uber H3 |

## About

```text
At Shell, I lead an 8-member cross-functional team delivering NESS CoPilot, an AI-backed facility management platform for optimizing OPEX and site maintenance. I co-developed a Sentence Transformer-based RAG solution for topic modelling and replaced ADF scheduling with Databricks Asset Bundles, saving about $20K annually.

Across Shell, EPAM Systems, and Boeing India Pvt. Ltd., I built and improved data pipelines and analytics platforms using Azure and Databricks. At Boeing, I scaled Phase of Flight detection on Azure Databricks, improving processing speed by 30x, and enhanced detection features with geospatial processing.
```

## Experience

### Senior Data Engineer (Team Lead) (current)

[Shell](https://himalayas.app/companies/shell). Dec 2025 - Present

```text
Leading an 8-member cross-functional team for the NESS CoPilot project, an AI-backed facility management platform. Co-developed a RAG solution, simplified orchestration with Databricks Asset Bundles, and implemented MLOps lifecycle management.
```

Skills: Databricks, Unity-Catalog, MLFlow, RAG, Sentence-Transformers, Databricks-Asset-Bundles, MLOps, Team-Leadership

### Data Engineer

[Shell](https://himalayas.app/companies/shell). Jan 2023 - Dec 2025

```text
Built ADF ingestion pipelines for multi-country data and established data validation, monitoring, and alerting. Migrated SQL DB components to Delta Tables and Unity Catalog, enabling Power BI and Copilot Studio connectivity.
```

Skills: Azure-Data-Factory, Databricks, Delta-Lake, Unity-Catalog, Power-BI, CoPilot-Studio, Data-Validation, Monitoring

### Software Engineer A2

[EPAM Systems](https://himalayas.app/companies/epam-systems). Oct 2021 - Jan 2023

```text
Worked on the SPARC-X analytics platform for Shell, ingesting and transforming 20TB of FTP and SharePoint data into ADLS using ADF and Databricks. Optimized processing logic, reducing a module runtime from 156 hours to 48 hours.
```

Skills: Azure-Data-Factory, Databricks, ADLS, FTP, SharePoint, data-transformation, Optimization

### Software Engineer (Data Science)

Boeing India Pvt. Ltd.. Jun 2020 - Sep 2021

```text
Developed real-time flight analytics platforms using Azure Databricks, Delta Lake, Event Hub, and Cosmos DB. Scaled Phase of Flight detection with Apache Spark and Pandas UDFs, improving processing speed by 30x.
```

Skills: Azure-Databricks, Delta-Lake, Event-Hub, ADLS-Gen2, Azure-Functions, Azure-Cosmos-DB, Apache-Spark, Pandas-UDFs, Scikit-Fuzzy, Apache-Sedona, Spark-SQL, Pytest, ScalaTest, GitLab-CI-CD, SonarQube, Coverity

### Project Engineer (Cyber Security)

[Wipro Technologies Ltd.](https://himalayas.app/companies/wipro-limited). Oct 2017 - Jun 2019

```text
Automated recurring cyber security tasks for a global retail client using Python and Shell scripts, reducing manual effort.
```

Skills: Python, Shell-Scripting, Automation, Cybersecurity

## Education

### CDAC

P.G. Diploma, Big Data Analytics. 2019 - 2020

```text
P.G. Diploma in Big Data Analytics from CDAC, Knowledge Park, Bengaluru, from August 2019 to February 2020.
```

### Amity University Rajasthan

B.Tech, Computer Science and Engineering. 2013 - 2017

```text
B.Tech in Computer Science and Engineering from Amity University Rajasthan, Jaipur, from June 2013 to June 2017.
```

## Tech stack

- Apache Spark
- SonarQube
- GitHub Actions
- Pandas
- PySpark
- PostgreSQL
- Databricks
- Azure DevOps
- Python
- Scala
- MLflow
- Kafka
- Azure Functions
- pytest
- SQL
- Azure Cosmos DB
- ScalaTest
- Delta Lake
- H3
- Unity Catalog
- Sentence Transformers
- Power BI
- Azure Data Factory

To contact Mahendra Singh, message them from this profile with a Himalayas employer account: [Hire on Himalayas](https://himalayas.app/hire).

## People also viewed

- [Sushant Sur](https://himalayas.app/@sushantsur): Chief Manager II, Shriram Finance
- [Lalit Namdeo](https://himalayas.app/@lalitnamdeo): Senior Consultant, Deloitte USI
- [Shreya Gupta](https://himalayas.app/@shreyagupta10): Software Development Engineer, Amadeus Software Labs
- [Deepak Singh](https://himalayas.app/@deepaksingh15): Senior Consultant, Deloitte
- [Pujitha Puppala](https://himalayas.app/@pujithapuppala): Data Analyst, Viso.ai
- [nidhi nagori](https://himalayas.app/@nidhinagori): Senior Data Engineer, A.P. Moller Maersk
- [Shariqa Naz Shaik](https://himalayas.app/@shariqashaik): Technical Lead, Wipro
- [Mohit Gaikwad](https://himalayas.app/@mohitgaikwad): Deputy Manager Data Engineer, ICICI Prudential AMC
- [Shariqa naz User](https://himalayas.app/@shariqanazuser): Technical Lead, Wipro
- [Sharmila Jagankumar](https://himalayas.app/@sharmilajagankumar): Programmer Analyst, Cognizant Technology Solutions
