Open to opportunities

Jagadish Reddy

@jagadishreddy

Message

Senior Data Engineer with extensive experience in Big Data technologies.

Zimbabwe

Message

What I'm looking for

I seek a challenging role in a dynamic environment that fosters innovation and collaboration.

I am a Senior Data Engineer with over 9 years of experience in Information Technology, specializing in Hadoop and Big Data processing. My expertise lies in implementing complex Big Data projects using technologies such as Apache Hadoop, Spark, and various Azure services. I have successfully migrated SQL databases to Azure Data Lake and developed data engineering pipelines that enhance data accessibility and usability.

Throughout my career, I have worked with industry leaders like JP Morgan Chase and Alaska Airlines, where I designed and implemented data solutions that improved system uptime and data processing efficiency. My hands-on experience with AWS and Azure cloud infrastructures, combined with my proficiency in data modeling and performance tuning, allows me to deliver high-quality results in fast-paced environments. I am passionate about leveraging data to drive business insights and support strategic decision-making.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

JP Morgan Chase

Sep 2022 - Present (2 years 8 months)

Designed technical solutions aligning with business and IT strategies, developing applications for data transformation across enterprise analytics platforms. Built an ETL framework for data migration to Azure cloud and created a new data quality check framework in Python.

Azure Data Factory Snowflake Python Apache Airflow Apache Spark Pyspark Hive

Data Engineer

Alaska Airlines

Apr 2020 - Aug 2022 (2 years 4 months)

Developed Scala Spark pipelines to transform raw data into parquet files and load data from Hive to Amazon RDS. Worked extensively with AWS services like S3, Glue, EMR, and Lambda to process data for downstream customers. Created CI/CD pipelines using Jenkins and Rundeck for job scheduling.

Spark Scala Hadoop Hive Sqoop AWS Glue Amazon EMR AWS S3 Snowflake Jenkins Rundeck Apache Airflow Amazon SQS

Data Engineer

ADP

Aug 2018 - Apr 2020 (1 year 8 months)

Designed Hadoop architecture and set up a multi-node cluster, performing performance tuning on Azure Data Factory pipelines. Extracted data from various sources using Sqoop and built a Data Discovery Platform using Azure HDInsight components. Developed ETL operations using Pig Latin scripts and Python, and created workflows on Talend.

Hadoop Hive Pig Sqoop Azure Data Factory Azure HDInsight Talend HBase Python Tableau Snowflake Power BI

Junior Software Developer

Geospatial Solutions

Jun 2014 - Jun 2015 (1 year)

Imported data from various formats to HDFS and ingested data from RDBMS sources using Sqoop. Developed real-time data ingestion and analysis using Kafka and Spark Streaming. Managed and scheduled jobs on a Hadoop cluster using Oozie and built NiFi dataflows.