Looking for a job

Anirudh Varma Penumetcha

@anirudhvarmapenumetc

Senior Data Engineer at Cognizant Technology Solutions modernizing Nationwide’s batch applications on AWS EKS and cutting a batch window from 6 hours to 3.

United States

Message

At Cognizant Technology Solutions, I modernize batch applications for Nationwide, moving legacy Linux workloads to AWS EKS and Rancher-managed Kubernetes. I also re-engineered Oracle-to-AWS Glue-to-Snowflake pipelines processing approximately 80 billion rows, reducing the end-to-end batch window from 6 hours to 3.

I designed CI/CD deployments across DEV, TEST, and PROD and integrated Stonebranch UAC with Kubernetes to orchestrate concurrent applications. I developed and optimized Python and PySpark AWS Glue jobs, including reconciliation between Oracle and Snowflake.

Previously, at Beagle Software Solutions, I designed serverless NLP pipelines that processed approximately 200,000 documents per run and reduced processing time from 4 hours to 1. I’ve also built data pipelines and reporting workflows using AWS, Azure, Spark, Snowflake, and BigQuery across my roles at NSL Hub, Appletree Software Solutions, and Countertack Inc.

Experience

Work history, roles, and key accomplishments

Current

Sr. Data Engineer

Current

Cognizant Technology Solutions

Apr 2023 - Present (3 years 7 months)

Led modernization of enterprise batch applications and Python/shell scripts to containerized workloads on AWS EKS and Rancher-managed Kubernetes. Re-engineered Oracle-to-AWS Glue/S3-to-Snowflake pipelines processing approximately 80 billion rows, reducing batch window from 6 hours to 3 hours.

AWS Amazon EKS Kubernetes Rancher Python PySpark AWS Glue Snowflake Oracle Stonebranch UAC GitHub Actions Harbor Harness Helm Docker EFS Fluent Bit Splunk Informatica PowerCenter SSH SFTP

Sr. Data Engineer

Beagle Software Solutions

Aug 2022 - Apr 2023 (8 months)

Designed serverless NLP pipelines processing approximately 200,000 documents per run, reducing processing time from 4 hours to 1 hour. Optimized distributed PySpark and Spark DataFrame workloads by 85% and accelerated EMR/HDFS-to-Snowflake processing by 35%.

NLP PySpark Spark Snowflake Airflow EMR HDFS Data Quality Anomaly Detection Governance

Sr. Data Engineer

NSL Hub

Aug 2020 - Jul 2021 (11 months)

Built end-to-end ETL/ELT pipelines using AWS Glue, PySpark, Azure Data Factory, S3, and Snowflake. Developed Airflow DAGs and Spark Streaming modules with Kafka to automate dependencies and process high-volume event and batch data.

AWS Glue PySpark Azure Data Factory S3 Snowflake Airflow Kafka Flask API Gateway Lambda EC2 EMR

Data Engineer

Appletree Software Solutions

Jun 2019 - Jul 2020 (1 year 1 month)

Designed BigQuery pipelines, geospatial datasets, and reusable spatial functions that improved data retrieval, analysis, and query performance. Implemented data-quality and governance practices and resolved geospatial processing issues.

BigQuery Geospatial Data Quality Governance Performance Optimization

Data Engineer

Countertack Inc

Aug 2017 - May 2019 (1 year 9 months)

Developed Databricks, PySpark, Spark SQL, Hive, Azure Data Factory, SSIS, and Snowflake pipelines to migrate and transform data from legacy and on-premises platforms. Built Azure and Power BI reporting workflows with incremental refresh and automated CI/CD.