Open to opportunities

Prayas Pagade

@prayaspagade

Data Engineering Lead delivering scalable multi-cloud analytics with Snowflake, dbt, Spark, and cost-optimized architectures.

United States

Message

What I'm looking for

I want to keep leading end-to-end, privacy-aware data engineering across multi-cloud platforms—building scalable pipelines, cost-optimized architectures, and GenAI-enabled analytics—while mentoring teams and partnering closely with product and data science to deliver measurable outcomes.

I’m a Senior Data Engineer and Data Engineering Lead with 8+ years designing and leading scalable real-time and batch data platforms. I focus on multi-cloud data modernization and enterprise analytics that enable decisioning, attribution insights, and privacy-aware analytics.

In my current role, I lead a Salesforce CRM migration and advisor analytics workstream, building dbt transformation models and SQL-based workflows in Snowflake. I also run automated Airflow pipelines for API-driven ingestion, validate post-ingestion data quality, and deploy an agentic GenAI approach to analyze ingested datasets and support on-demand analytics queries.

I’ve driven major cross-cloud modernization as well, leading distributed engineering teams and building scalable Spark-based applications for financial data processing and validation. I define multi-cloud architectures, performance monitoring, and cost-aware execution—partnering with product and data science to translate end-user needs into analytics workflows and UI-driven tools.

Earlier, I designed AWS-based ETL using Glue, Step Functions, EventBridge, Lambda, and Athena, including Scala utilities with 90%+ test coverage and automated Hive-to-Glue metadata migration. Before that, I built batch and real-time pipelines with Spark and Kafka, delivered Flask/Django APIs for ML model hosting, created AI-powered chatbots, and mentored 30+ engineers—earning recognition through client-facing awards.

Experience

Work history, roles, and key accomplishments

Current

Lead Data Engineer - Focus

Current

BBI (USA)

Aug 2025 - Present (11 months)

Led Salesforce CRM migration for Focus Wealth Management, building DBT transformations and Snowflake SQL workflows. Developed scheduled Airflow pipelines for API-ingested data, and implemented an agentic GenAI solution for dataset analysis and on-demand analytics, including Cortex AI cost optimization.

Salesforce Snowflake DBT SQL Airflow Agentic AI Cortex Microsoft Copilot Azure Functions Azure Blob Storage

Lead Data Engineer - TransUnion

BBI (USA)

Sep 2023 - Sep 2025 (2 years)

Led a distributed team to deliver Onetru batch cloud modernization, creating cross-cloud compatible ETL and migrating legacy apps/data across AWS and GCP. Built scalable Spark-based applications for financial-data validation and profiling, and defined multi-cloud architecture, performance monitoring, and Terraform-driven infrastructure.

S3 Spark Java Spring Boot GCP Cloudwatch Splunk Terraform AWS Lambda Jenkins Performance Monitoring Ab Initio Distributed Team Leadership

Senior Data Engineer - TransUnion

BBI (India)

Apr 2021 - Aug 2023 (2 years 4 months)

Worked on Rise Analytics cloud migration, enabling apps to publish data to cloud while maintaining BAU on-prem functionality through cloud metadata and data migration. Built AWS Glue/Lambda/Athena pipelines, created Scala utilities with strong test coverage, automated Hive-to-Glue metadata migration, and supported engineer mentoring and onboarding.

Mentorship AWS Glue Step Functions Amazon EventBridge S3 AWS Lambda Athena Scala Python Hive To Glue Metadata Migration Hadoop Spark Test Coverage

Data Engineer and Mentor

Bridgelabz Solutions

Jan 2019 - Apr 2021 (2 years 3 months)

Developed batch and real-time data pipelines for ML workflows using Spark and Kafka, including classification streaming with Spark Streaming and Python. Built and deployed Flask/Django APIs for ML model hosting with Jenkins CI/CD, created AI-powered chatbots with NLTK and scikit-learn, and mentored 30+ engineers.

Jenkins Mentorship S3 EC2 Kafka Python Flask Django Machine Learning NLTK scikit learn Pandas NumPy Shell Scripting Chatbot

CSA (Amazon)

Amazon Development Centre

Jun 2017 - Jan 2019 (1 year 7 months)

Handled customer support responsibilities, including resolving complex issues and supporting both international and domestic customers. Mentored junior associates and supported process/customer satisfaction objectives.