Shailesh Navendu
@shaileshnavendu
Senior Data Engineer building scalable PySpark and cloud analytics platforms for reliable production data.
What I'm looking for
I’m a Senior Data Engineer with 14+ years of experience designing scalable data platforms, distributed ETL pipelines, and cloud-based analytics systems. I focus on production-grade reliability, data quality, and governance while delivering measurable performance improvements.
I’m an expert in PySpark, Python, SQL/PL-SQL, and orchestration with Apache Airflow, backed by hands-on work integrating heterogeneous data sources into centralized analytics platforms. I’ve optimized distributed data processing workloads for both performance and cost efficiency across Snowflake and Redshift.
Most recently, at Camp International, I’m designing scalable cloud-based data pipelines on AWS, integrating multiple data sources, and implementing best practices for monitoring, alerting, and data quality. Previously at ConcertAI, I led a large-scale oncology analytics platform, processing 10TB+ datasets and reducing pipeline runtime by 35% using AWS Glue, Lambda, Redshift, and S3.
I also bring leadership and mentoring experience, having mentored 4+ junior engineers and led technical decision-making for platform architecture and tooling selection. I’m now available for full-time remote roles with US/global companies and I’m looking to bring the same engineering rigor to healthcare and financial analytics at scale.
Experience
Work history, roles, and key accomplishments
Senior Data Engineer
Camp International
Jun 2025 - Present (1 year)
Designed and implemented scalable AWS cloud data pipelines with PySpark and Python, integrating heterogeneous data sources into centralized analytics platforms. Optimized distributed processing workloads for cost and performance on Snowflake and Redshift while enforcing data quality, governance, and monitoring.
Data Engineer Tech Lead
ConcertAI
Nov 2020 - Mar 2025 (4 years 4 months)
Led development of a large-scale oncology analytics platform for predictive patient analytics, processing 10TB+ of healthcare datasets. Built AWS Glue/Lambda/Redshift/S3 ETL pipelines and reduced pipeline runtime by 35%, improving orchestration reliability and observability with Apache Airflow.
Built PySpark and SparkSQL pipelines for large-scale financial data processing across loan and card products, ingesting millions of transactions daily. Optimized SQL queries to reduce batch processing time by 40%, supporting SLA compliance and regulatory reporting.
Developed validation frameworks and automated batch processes for enterprise systems handling critical financial records. Optimized SQL and PL/SQL stored procedures to improve pipeline efficiency and maintained 99.9% SLA compliance across production batch flows.
Automated data loading pipelines using Autosys job scheduling, significantly reducing manual intervention. Built PL/SQL procedures, functions, and triggers to support enterprise data transformation workflows.
Designed database schemas and optimized SQL queries for enterprise applications supporting thousands of concurrent users. Developed and maintained stored procedures and database solutions to ensure database performance and reliability.
Education
Degrees, certifications, and relevant coursework
Snowflake
Snowflake SnowPro Core, Cloud Data Warehousing
Completed the Snowflake SnowPro Core certification.
SASTRA University
Bachelor of Technology (B.Tech), Electronics & Telecommunication
Grade: CGPA: 8.7 / 10
Earned a B.Tech in Electronics & Telecommunication at SASTRA University in 2011, achieving CGPA 8.7/10.
Amazon Web Services (AWS)
AWS Certified Data Engineer – Associate, Cloud Data Engineering
Pursuing the AWS Certified Data Engineer – Associate credential (status: in progress/target).
Databricks
Databricks Certified Associate Developer for Apache Spark, Apache Spark
Completed the Databricks Certified Associate Developer for Apache Spark certification.
Availability
Location
Authorized to work in
Portfolio
github.com/shaileshnavendu/docsJob categories
Interested in hiring Shailesh?
You can contact Shailesh and 90k+ other talented remote workers on Himalayas.
Message ShaileshFind your dream job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
