SUSai Lakshmi Harshita User
@sailakshmiharshitaus
Data Engineer specialized in PySpark and AWS for scalable, cost-efficient big data pipelines.
IndiaWhat I'm looking for
I am a Data Engineer with five years of experience designing and delivering large-scale, cost-effective batch processing pipelines using PySpark on AWS EMR.
I have built and automated end-to-end workflows with Apache Airflow and AWS Step Functions, integrating EMR, S3, Lambda and other AWS services to enable event-driven and fault-tolerant data processing.
My work emphasizes performance and cost optimization — tuning Spark jobs and EMR clusters, applying partitioning, compression, caching, and Spark SQL optimizations to reduce runtimes and improve resource utilization.
I bring experience across ETL tools, data formats, and databases, and I have been recognized with internal awards for performance and professional development while delivering production-grade data solutions.
Experience
Work history, roles, and key accomplishments
Data Engineer
Tata Consultancy Services Limited
Jun 2023 - Present (2 years 8 months)
Designed and developed end-to-end batch data pipelines on AWS EMR using PySpark and Airflow, optimizing Spark jobs to reduce runtimes by 30% and integrating S3, Lambda, and Step Functions for scalable, event-driven workflows.
Spark Developer
Capgemini
Jan 2022 - May 2023 (1 year 4 months)
Developed distributed Spark applications using DataFrames and RDDs for large-scale data processing, optimized Spark SQL and Hive queries to reduce costs and improve resource utilization, and handled Sqoop-based data transfers.
ETL Developer
Capgemini
Feb 2021 - Dec 2021 (10 months)
Developed and enhanced Informatica PowerCenter ETL mappings, sessions, and workflows, performed SQL-based data validation and production support using Autosys, and applied ETL performance tuning techniques.
Education
Degrees, certifications, and relevant coursework
VCVR Siddhartha Engineering College
Bachelor of Technology, Electrical and Electronics Engineering
2016 - 2020
Grade: 7.38 CGPA
Bachelor of Technology in Electrical and Electronics Engineering completed with a CGPA of 7.38, focusing on core engineering principles and practical applications.
Tech stack
Software and tools used professionally
Availability
Location
IndiaAuthorized to work in
Job categories
Skills
Interested in hiring Sai Lakshmi Harshita?
You can contact Sai Lakshmi Harshita and 90k+ other talented remote workers on Himalayas.
Message Sai Lakshmi HarshitaFind your dream job
Sign up now and join over 100,000 remote workers who receive personalized job alerts, curated job matches, and more for free!
