Open to opportunities

Kannan Thangaraju

@kannanthangaraju

Message

Senior data engineer specialising in scalable ETL, cloud infrastructure, and real-time data pipelines.

India

Message

What I'm looking for

I’m seeking senior data engineering roles where I can lead cloud-native ETL and real-time pipeline design, drive infrastructure automation, and build scalable ML data platforms in a collaborative, engineering-focused team.

I’m a Senior Data Engineer with 8+ years building scalable ETL and real-time data pipelines, delivering measurable cost and time reductions across cloud platforms. I have designed and automated 50+ Spark-based pipelines and launched CrewAI agents that cut manual support tasks by 80%.

I modernize infrastructure—migrating EMR to AWS Glue, adopting Terraform, and implementing CI/CD—to improve deployment speed and reduce costs. I collaborate with product and engineering teams to design data models, APIs, and search solutions that power analytics and ML-driven recommendations.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Fitch Group

Oct 2022 - Present (3 years 8 months)

Designed and automated 50+ Spark-based ETL and streaming pipelines feeding real-time ML recommendations and deployed CrewAI agents that cut manual support tasks by 80%. Modernized infrastructure with Terraform and migrated EMR to AWS Glue, reducing environment setup time by 40% and ETL infrastructure costs by 60%.

Python Spark Spark Structured Streaming AWS Glue Terraform Elasticsearch fastAPI Redis

Data Engineer - Specialist Solutions

Acuity Knowledge Partners

May 2021 - Dec 2022 (1 year 7 months)

Built and modularized 50+ Spark-based data pipelines, reducing pipeline runtime and costs by 50% and enabling PowerBI dashboards and ML recommendation engines. Developed reusable Terraform modules and CI/CD with GitHub Actions and Alembic, cutting deployment time by 45% and enabling proactive monitoring that reduced downtime.

Spark Pyspark Terraform GitHub Actions)Alembic SQL Airflow CI CD GitHub Actions

Data Engineer

Tricog Health

Jun 2019 - May 2021 (1 year 11 months)

Designed Spark-based preprocessing pipelines to process 10M+ ECG readings monthly, reducing training data preparation time by 60%. Implemented a Google BigQuery data warehouse and redesigned schemas to cut OLTP query response times from 300ms to 15ms and automated reporting to save 20+ hours/week.

Spark Pyspark Google BigQuery Data Warehousing SQL ETL Partitioning Performance Optimization