Open to opportunities

Ali Khan

@alikhan16

Message

I build cloud-native data platforms, scalable pipelines, and real-time analytics systems.

United States

Message

What I'm looking for

I'm looking to build scalable cloud data platforms, real-time streaming systems, and governed Lakehouse solutions with cross-functional engineering, analytics, and data science teams.

At Nexla, I design cloud-native data platforms with Databricks, Apache Spark, Snowflake, and Python to support scalable enterprise analytics. I've built 40+ ETL/ELT pipelines processing more than 100 million records monthly and developed streaming solutions handling millions of daily events.

Previously at Redpanda Data, I led distributed batch and streaming platform development using Kafka, Spark, Python, and cloud-native technologies. I modernized 30+ production pipelines through Databricks, Delta Lake, and Snowflake migrations while improving stability, throughput, governance, and operational visibility.

At dbt Labs, I delivered analytics-ready datasets through Python, SQL, dbt, Airflow, and Snowflake. I also designed dimensional models, automated data-quality frameworks, and optimized warehouse and database performance for reporting and self-service analytics.

I'm focused on Lakehouse architecture, real-time processing, infrastructure automation, and secure data governance across Azure, AWS, and GCP.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Nexla

Jun 2024 - Present (2 years 2 months)

Designed and deployed cloud-native data platforms using Databricks, Apache Spark, Snowflake, and Python. Built 40+ enterprise ETL/ELT pipelines processing 100M+ records monthly and implemented real-time streaming solutions with Kafka and Spark Structured Streaming.

Databricks Apache Spark Snowflake Python DBT Kafka Spark Structured Streaming Delta Lake Apache Iceberg Terraform GitHub Actions Unity Catalog RBAC

Data Engineering Team Lead

Redpanda Data

Apr 2021 - May 2024 (3 years 1 month)

Led the design and development of distributed data platforms using Apache Kafka, Apache Spark, Python, and cloud-native technologies. Modernized 30+ production ETL pipelines by migrating to Databricks, Delta Lake, and Snowflake, and designed reusable Airflow workflows.

Kafka Apache Spark Python Databricks Delta Lake Snowflake Airflow SQL Data Governance Monitoring

Data Engineer

dbt Labs

Apr 2019 - Mar 2021 (1 year 11 months)

Developed 30+ scalable ETL/ELT pipelines using Python, SQL, dbt, Apache Airflow, and Snowflake. Designed dimensional data models and built automated testing and data quality frameworks to improve pipeline reliability.