Open to opportunities

Tvisha Patel

@tvishapatel

Message

Senior Data Engineer with expertise in cloud-based data solutions.

Canada

Message

What I'm looking for

I am looking for a role that challenges my data engineering skills and offers opportunities for growth in cloud technologies.

I am a seasoned Data Engineering professional with over 6 years of experience in designing, developing, and optimizing large-scale data solutions. My expertise spans across the Hadoop ecosystem, Apache Spark, Kafka, and various cloud platforms including AWS, GCP, and Azure. I have a proven track record of building robust ETL pipelines and real-time streaming applications that enhance data accessibility and business insights.

Throughout my career, I have successfully implemented data architectures that leverage advanced analytics and machine learning, resulting in significant revenue increases for my employers. My hands-on experience with tools such as Informatica, Talend, and Azure Data Factory, combined with my proficiency in SQL and Python, allows me to deliver high-quality data solutions that meet complex business needs. I am passionate about data governance and have integrated best practices in data quality and security across various projects.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer-Azure

Current

Walmart

Apr 2023 - Present (2 years 3 months)

Designed and implemented a Personalized Customer Recommendation System, integrating advanced data collection, processing, and analytics techniques, enhancing customer engagement through tailored recommendations. Developed and maintained end-to-end ETL pipelines in Azure Data Factory (ADF), efficiently handling large-scale structured and unstructured data from both streaming and batch sources. Depl

Azure Data Factory Snowflake Apache Airflow Python Scala Tableau Jira Kafka SSIS

Data Engineer

TCS

Nov 2020 - Present (4 years 8 months)

Developed and optimized Spark/PySpark-based ETL pipelines for seamless data migration into an enterprise Hadoop Data Lake, implementing partitioning, broadcast joins, and performance tuning. Designed and implemented AWS-based data architecture, integrating AWS Glue, AWS EMR, AWS Lambda, Step Functions, and S3 to automate ETL processes. Extracted, transformed, and loaded data into Azure Data Lake,

AWS Glue AWS Lambda Snowflake Azure Data Factory Spark Pyspark Python Scala

AWS Data Engineer

Synechron

Feb 2019 - Present (6 years 5 months)

Designed and developed Spark/PySpark-based ETL pipelines for seamless data migration into an enterprise Hadoop Data Lake, optimizing performance with partitioning, Spark SQL, and broadcast joins. Built and maintained scalable data pipelines using Apache Spark on AWS EMR, integrating structured and semi-structured data into Hadoop and RDBMS environments. Engineered Snowflake data warehouse solution

Spark Pyspark AWS Glue Snowflake AWS RedShift Amazon Athena Apache Flink AWS Kinesis Python SQL

Education

Degrees, certifications, and relevant coursework

JNTU

Bachelors, Computer Science and Engineering

Completed a Bachelor's degree in Computer Science and Engineering. The curriculum covered fundamental concepts and advanced topics in the field, preparing for a career in technology.