Open to opportunities

pavan User

@pavanuser14

Message

I build AWS data pipelines for cloud migration, transformation, validation, and reliable analytics-ready datasets.

India

Message

What I'm looking for

I'm looking to build and support scalable cloud data pipelines, migration solutions, and data-quality processes using AWS, PySpark, SQL, and orchestration tools while collaborating with engineering, QA, and business teams.

I've supported banking, railway, and industrial data projects by building AWS-based ETL pipelines, migration workflows, and validation processes. My work spans AWS Glue, PySpark, Amazon S3, Amazon Redshift, Apache Airflow, Lambda, and Step Functions.

At iGrow Virtual IT Solutions, I contributed to a metadata-driven ETL framework for large-scale banking migration. I built Glue PySpark jobs, Spark SQL transformations, reusable JSON and CSV metadata configurations, Airflow orchestration, and SCD Type 2 validation processes that protected historical data integrity.

Previously, I worked at Test Data Technologies on IoT sensor-data processing and at YASH Technologies on Teradata and banking-system migrations. I translate legacy SQL and business logic into cloud transformation pipelines, reconcile source and target data, and work closely with engineering, QA, and business teams to resolve data-quality issues.

Experience

Work history, roles, and key accomplishments

AWS Data Engineer

iGrow Virtual IT Solutions Pvt. Ltd.

Sep 2025 - Jul 2026 (10 months)

Contributed to a metadata-driven ETL framework using AWS Glue, PySpark, Amazon S3, Apache Airflow, and Amazon Redshift for large-scale banking data migration. Built and maintained AWS Glue PySpark jobs, implemented data validation and SCD Type 2 processes, and orchestrated workflows with Airflow DAGs.

AWS Glue PySpark Python SQL S3 Amazon Redshift Airflow AWS Step Functions Data Validation Data Quality Data Migration

AWS Data Engineer

Test Data Technologies

Jan 2024 - May 2024 (4 months)

Worked on an IoT-based industrial data engineering project, processing sensor data from industrial plant environments. Developed and supported AWS Glue ETL pipelines using PySpark for data ingestion, transformation, and restructuring, and automated workflows with AWS Lambda and Step Functions.

AWS Glue PySpark Python AWS Lambda AWS Step Functions ETL data transformation

AWS Data Engineer

YASH Technologies

Apr 2022 - Nov 2023 (1 year 7 months)

Contributed to a large-scale Teradata-to-cloud migration of railway transaction data using Databricks into AWS S3, and worked on a banking data migration to Amazon Redshift. Analyzed legacy SQL and business logic, developed Spark SQL transformations, and performed data validation and reconciliation.