PAVANI VALASAPALLI
@pavanivalasapalli
Data Engineer with expertise in scalable data pipeline design.
What I'm looking for
I am a Data Engineer with over 4 years of experience in designing and building scalable data pipelines across AWS and Azure. My expertise lies in developing high-performance ETL/ELT workflows that significantly reduce data processing time and enhance query speed in platforms like Snowflake and Redshift. I have successfully managed petabyte-scale datasets, ensuring excellent pipeline uptime while optimizing storage costs.
Throughout my career, I have leveraged technologies such as Apache Spark, Kafka, Airflow, and Terraform to support both real-time and batch data processing for analytics and business intelligence. My role involves collaborating with cross-functional teams to transform complex data challenges into scalable solutions that drive impactful business outcomes. I am passionate about implementing secure, cloud-optimized data architectures while maintaining compliance with industry regulations.
Experience
Work history, roles, and key accomplishments
Data Engineer
Progressive Corporation
Feb 2025 - Present (1 year 4 months)
Constructed Azure-based data pipelines, processing TBs of client data from on-premise & cloud sources, improving data accessibility for analytics teams. Engineered ML feature stores, streamlining data retrieval and model training workflows across teams.
Data Engineer
Aspire Technolab
Mar 2021 - Apr 2023 (2 years 1 month)
Implemented Azure Data Factory (ADF) extensively for ingesting data from different source systems like relational and semi-structured data to meet business functional requirements. Designed and developed batch processing and real-time solutions using PySpark, Kafka, and Databricks.
Data Engineer
Sage Softtech
Jun 2019 - Mar 2021 (1 year 9 months)
Created pipelines, data flows, and complex data transformations and manipulations using Azure Data Factory (ADF) and PySpark with Databricks. Developed Spark applications using PySpark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats.
Education
Degrees, certifications, and relevant coursework
Hood College
Master of Science, Information Technology
2023 - 2024
Pursued a Master of Science in Information Technology. Focused on advanced topics in IT, preparing for a career in data engineering.
Tech stack
Software and tools used professionally
Availability
Location
Authorized to work in
Job categories
Skills
Interested in hiring PAVANI?
You can contact PAVANI and 90k+ other talented remote workers on Himalayas.
Message PAVANIFind your dream job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
