# Rohit Didel

Rohit Didel is a remote worker on Himalayas, Data Engineer at BDI Plus Lab Pvt. Ltd., based in India.

Source: https://himalayas.app/@rohitdidel

```text
Data Engineer building enterprise ETL pipelines with Spark and Azure to deliver faster, business-ready analytics.
```

| Detail | Value |
| --- | --- |
| Availability | Open to opportunities |
| Location | India |
| Authorized to work in | India |
| Job categories | Data Engineer, ETL Engineer, Big Data Engineer, Cloud Data Engineer, Analytics Engineer, Data Warehouse Engineer, BI Developer, Data Engineering, Data Engineering Positions, Data Data Engineering |
| Skills | Apache Spark, Spark Structured Streaming, Scala, Hive, Apache Hive, Hadoop, Kafka, Databricks, HDInsight, Azure Data Factory, Azure ADLS Gen2, Azure Blob Storage, azure monitor, Log Analytics, Power BI, Azure, Tableau, Python, SQL, GraphX, PostgreSQL, MySQL, Git, Docker, SAP, Data Structures, Matlab, Factory, SAFe |

## What I'm looking for

```text
I’m looking for a data engineering role where I can own ETL performance and scalability using Spark/Hive/Azure, collaborate with analytics/BI teams, and build executive-ready dashboards and reliable pipelines with measurable business impact.
```

## About

```text
I’m a results-driven Data Engineer with 2+ years building enterprise-scale ETL pipelines that improve speed, reliability, and decision-ready outputs. At American Express, I reduced data ingestion time by 35% by migrating batch workflows and optimizing end-to-end processing.

I specialize in turning raw data complexity into measurable business impact—whether that’s cutting Hive query execution time by 28% through partition pruning and schema restructuring, or improving targeting accuracy by ~22% by automating Hive & Hadoop ETL for marketing campaigns. I also built pipeline refreshes that shrank data availability from 6 hrs to under 2 hrs for 500K+ daily records, and reduced team reporting workload by 40% by consolidating multiple reports into a single Tableau HR dashboard.

I bring strong distributed and cloud engineering through Apache Spark (Scala), including Spark Streaming for 50M+ events and GraphX analysis for 10M+ node graphs for anomaly and pattern detection with sub-second latency. On Azure, I’ve architected full data workflows with ADF → ADLS Gen2 → HDInsight/Databricks and delivered Power BI dashboards that cut incident response time by 50%.
```

## Experience

### Data Engineer

BDI Plus Lab Pvt. Ltd.. Aug 2022 - May 2024

```text
Reduced data ingestion time by 35% by migrating batch workflows from Hive to Apache Spark, cutting query execution time by 28% for 100+ TB customer datasets. Automated Hive/Hadoop ETL for marketing campaigns, improving targeting accuracy ~22% and reducing data availability time (6 hrs to under 2 hrs), while saving 15+ hrs/week and cutting reporting workload by 40% via Tableau dashboard consolidati
```

Skills: Apache-Spark, Apache-Hive, Hadoop, ETL, SQL, Tableau, Data-Warehouse

## Education

### Indian Institute of Technology Guwahati

Bachelor of Technology, Civil Engineering

```text
Earned a B.Tech in Civil Engineering at IIT Guwahati.
```

## Tech stack

- Apache Spark
- Microsoft Azure
- MySQL
- PostgreSQL
- Hadoop
- Gmail
- Databricks
- MATLAB
- Kafka
- Azure Monitor
- SQL
- Factory
- SAFe

To contact Rohit, message them from this profile with a Himalayas employer account: [Hire on Himalayas](https://himalayas.app/hire).

## People also viewed

- [Rajashree Sarmah](https://himalayas.app/@rajashreesarmah): Data Engineer, Zeitview
- [Jaggaa User](https://himalayas.app/@jaggaauser): Associate Analyst, Globallgoic Technologies
- [Simon Jha](https://himalayas.app/@simonjha): Azure Data Engineer, Wipro Limited
- [Dipali Kendre](https://himalayas.app/@dipalikendre): Data Engineer, German Chemical Company
- [Subhashini Damu](https://himalayas.app/@subhashinidamu): Senior Big Data Engineer, Comcast
- [Prashant Sachan](https://himalayas.app/@prashantsachan): ETL Technical Consultant, Innovya Techologies
- [Ashish Sapate](https://himalayas.app/@ashishsapate): Data Engineer, Tech Mahindra
- [Balamurali Krushna Kopparapu](https://himalayas.app/@balamuralikrushnakop): Data Engineer, Infosys
- [Akash Birajdar](https://himalayas.app/@akashbirajdar): Data Engineer, SDK Infotech Pvt. Ltd.
- [Harshit Agrawal](https://himalayas.app/@harshitagrawal): Senior Big Data Engineer, EnableData
