Open to opportunities

Rajaram Yadav

@rajaramyadav

Message

Senior Data Engineer building secure, near real-time cloud data platforms and GenAI search pipelines.

United States

Message

What I'm looking for

I’m looking for a role where I can build secure lakehouse-based data platforms and near real-time pipelines, plus RAG/LLM-powered enterprise search, with strong governance, observability, and collaboration with analytics and business teams.

I’m a Senior Data Engineer with 6 years of experience designing and scaling secure cloud native data platforms across AWS, Azure, Snowflake, Databricks, Microsoft Fabric, and dbt. I build batch and real time data pipelines with AWS Glue, S3, Lambda, Kinesis, Redshift, Azure Data Factory, Databricks, Synapse, Event Hubs, Apache Spark, Kafka, and Airflow—enabling reliable near real-time analytics for healthcare and financial teams.

Across multiple migrations, I’ve transitioned legacy systems to lakehouse architectures using Databricks/Delta Lake and Microsoft Fabric OneLake, reducing data processing time from hours to near real time. I also bring hands-on Generative AI capabilities—RAG pipelines, LLM integrations (Azure OpenAI, AWS Bedrock, LangChain), vector databases, and AI-assisted validation—along with strong governance, data quality, lineage, and observability to meet HIPAA/FHIR/SOC2 and enterprise security needs.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

The Cigna Group

Aug 2024 - Present (1 year 10 months)

Designed hybrid multi-cloud healthcare data lakehouse architecture on AWS, Azure, and Microsoft Fabric, reducing batch processing ~40% and compute costs ~25%. Built real-time streaming pipelines and dbt-based transformations, and delivered an Azure OpenAI RAG enterprise search that cut document search time ~50%, with HIPAA/GDPR governance via Purview and Lake Formation.

AWS Glue Azure Data Factory Microsoft Fabric Databricks Delta Lake DBT Kafka Azure OpenAI Terraform Data Governance (Purview & Lake Formation)

Data Engineer

Cardinal Health

Feb 2022 - Jul 2024 (2 years 5 months)

Built Databricks Spark SQL pipelines for regulated financial data and migrated legacy Redshift/Hadoop into ADLS with Delta Lake and Snowflake to improve auditability and schema evolution. Optimized Azure Synapse/Snowflake to cut query spend 35%, reduced onboarding time 40%, and delivered low-latency reporting that cut delays 60%, alongside end-to-end data observability using Monte Carlo.

Databricks Pyspark Azure Synapse Snowflake Delta Lake Dagster Airflow Terraform Monte Carlo

Data Engineer

Visa Inc.

Oct 2020 - Jan 2022 (1 year 3 months)

Built an AWS customer data platform integrating 50+ internal and external sources, and developed PySpark and Airflow pipelines running hundreds of daily jobs with retries and failure handling to improve reliability. Enabled near-real-time fraud detection using Kinesis, Lambda, and SageMaker, built REST APIs for curated payment/customer datasets, and implemented governance with Lake Formation/Glue