Open to opportunities

Shanu Prakash

@shanuprakash

Message

Data Engineer building high-volume Azure and Palantir Foundry pipelines, plus RAG chatbots and data apps.

India

Message

What I'm looking for

I’m looking to keep building reliable, high-volume data pipelines on Azure/Palantir, and apply GenAI/RAG to practical products—delivering data quality, better runtimes, and less manual work in an Agile environment.

I design, build, and optimize end-to-end data pipelines in Palantir Foundry and Azure—processing millions of records daily and supporting 50+ end users. At Merck Group (SIAL), I use Foundry Code Repository/Code Workbooks and Spark on Azure Databricks to improve pipeline runtime by up to 60% while strengthening data quality with validation checks.

Before that, at Tata Consultancy Services (TCS) for Walgreens Boots Alliance, I developed scalable ingestion and processing pipelines for big data, relational databases, and data warehouses. I used Azure Databricks, ADLS, and Azure Data Factory for migration/integration and SQL to optimize queries, cutting manual data-handling effort by ~25%, and delivered ETL processes across 10+ pipelines.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Merck Group (SIAL)

Jun 2024 - Present (2 years 1 month)

Design, develop, and maintain 20+ end-to-end data pipelines in Palantir Foundry using PySpark, processing millions of records daily. Built a GenAI-powered chatbot using OpenAI APIs and Python ML libraries in a RAG architecture.

PySpark Palantir Foundry Azure Databricks Apache Spark Snowflake OpenAI API Python Slate Streamlit Tableau Jira Agile

System Engineer

Tata Consultancy Services (TCS)

Feb 2021 - Jun 2024 (3 years 4 months)

Developed and maintained scalable data ingestion and processing pipelines for big data, relational databases, and data warehouse solutions. Designed and implemented ETL processes using Azure Databricks, ADLS, and Azure Data Factory.