Data platforms that move business outcomes, not just pipelines that run green. End-to-end ownership across streaming and batch, strong engineering standards, AI-driven automation, and a team that treats data as a product. If the work connects to revenue or decisions that matter - that's where I do my best work.
Looking for a job
Pranav Shekhar
@pranavshekhar
Senior Data Engineer | 4 yrs | PySpark · Databricks · dbt · Azure · AWS | Real-time & batch pipelines at scale | Wells Fargo
IndiaWhat I'm looking for
I'm a Senior Data Engineer with 4 years at Wells Fargo, where I've built the data infrastructure powering commercial banking decisions for 260,000+ clients.
My work spans the full stack - real-time streaming pipelines with 10-minute SLAs, batch transformation across medallion architectures, a leads engine that converted data signals into $2.6M in revenue, and an ETL validation platform now adopted as standard across 10+ teams. I don't just build pipelines; I build things other engineers actually want to use.
Technically I work across PySpark, Spark SQL, dbt, Apache Airflow, Delta Lake, Databricks, and cloud platforms on both Azure and AWS. Lately I've been building AI agents to automate engineering toil — replacing manual schema changes and data contract coordination with systems that just handle it.
I'm looking for roles where data engineering connects directly to product outcomes, the bar for technical quality is high, and there's real appetite to use AI to raise what a small team can accomplish.
Experience
Work history, roles, and key accomplishments
Current
Built real-time streaming pipelines on Azure Event Hubs + Spark Structured Streaming (100 events/sec, 10-min SLA), a leads engine generating $2.6M in revenue, 20+ dbt semantic layer models enabling analyst self-service, and AI agents automating schema and data contract management.
Architected end-to-end pipelines across 15+ sources on a medallion architecture for 260K+ banking clients. Optimised PySpark execution on Databricks (75% faster), built a metadata-driven CRM onboarding framework (6 days → 1), and an ETL validation platform adopted across 10+ teams.
Contributed to a 360° customer view platform, built data mastering and validation pipeline components, and designed a POC testing framework projecting 20%+ reduction in manual testing effort.
Education
Degrees, certifications, and relevant coursework
Manipal Institute of Technology
Bachelor of Technology (B.Tech), Computer Science Engineering
2018 - 2022
Grade: CGPA 8.84/10
Completed a B.Tech in Computer Science Engineering at Manipal Institute of Technology. Earned a CGPA of 8.84/10.
Tech stack
Software and tools used professionally
Availability
Looking for a job
Location
IndiaAuthorized to work in
Salary expectations
4500k-6000k INR
Social media
Skills
Apache SparkPythonSQLPysparkSparkSQLSpark Structured StreamingData Build Tool (Dbt)Apache KafkaApache AirflowEvent HubADLSAWS S3GlueAthenaRedshiftDatabricksLambdaArchitectureStream ProcessingData ModelingDimensional ModelsETL ELT Data PipelinesData LineageData GovernanceDistributed Data ProcessingAgile ScrumGitHubGitHub ActionsAzure Data FactoryDelta Lake
Interested in hiring Pranav?
You can contact Pranav and 90k+ other talented remote workers on Himalayas.
Message PranavFind your dream job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
