Open to opportunities

Michael Carrell

@michaelcarrell

Message

Senior Data Engineer building scalable ETL pipelines and data platforms for reliable analytics and machine learning.

United States

Message

What I'm looking for

I’m looking for a role where I can build and harden ETL/data platforms end-to-end—focusing on performance, observability, data quality, and governance—while partnering with data science and product to deliver dependable analytics and ML-ready datasets.

I’m a Senior Data Engineer with 7 years building scalable ETL pipelines, data platforms, and analytical systems across AWS and on-prem environments. I focus on reliable data products by combining Python, PySpark, Spark, and SQL with strong operational practices.

At Redfin (Remote), I designed and maintained multi-source ETL pipelines that ingest into an S3 data lake using PySpark on EMR, enabling analytics on multi-terabyte datasets while improving runtime and reducing cluster cost. I orchestrate batch and near-real-time workflows with Airflow DAGs, add SLA-based monitoring, and build automated data quality checks and unit tests to detect schema drift and data anomalies early.

I also help teams move faster by collaborating on entity resolution and matching with data science, designing dimensional models and collaborative data schemas for BI, and integrating DynamoDB and Elasticsearch for low-latency lookup and search. Across roles, I’ve led incident response and postmortems, delivered observability dashboards and alerts, and supported governance, access controls, and compliance so data pipelines stay dependable in production.

Experience

Work history, roles, and key accomplishments

Senior Data Engineer

Redfin

Mar 2022 - Jun 2026 (4 years 3 months)

Designed and maintained scalable PySpark ETL pipelines ingesting diverse sources into an S3 data lake, enabling analytics on multi-terabyte datasets. Built Airflow DAGs for batch and near-real-time orchestration, implemented data quality checks, and optimized EMR PySpark jobs to reduce runtime and incident impact.

ETL Data Lake Pyspark Apache Spark Amazon S3 Airflow Orchestration Entity Resolution Matching Python Data Quality Unit Testing Schema Drift Detection Memory Tuning Data Governance DynamoDB Elasticsearch Incident Response Postmortems

Data Engineer

B3 Insight

May 2021 - Mar 2022 (10 months)

Developed Python and SQL ETL processes to load clinical and claims data into a centralized warehouse for care management and reporting. Authored dimensional models and star schemas, built monitoring for nightly ETL jobs, and implemented data validation frameworks to support compliance and reproducible analysis.

ETL Python SQL Data Warehouse Dimensional Models Star Schemas Monitoring Logging Data Validation Compliance Data Documentation

Data Architect

Blue Margin

Feb 2020 - Apr 2021 (1 year 2 months)

Designed dimensional data models and ETL mapping specifications to support executive dashboards and ad-hoc analytics. Implemented SQL and Python ETL workflows, established data lineage documentation, and performed performance tuning to reduce report generation times while supporting governance, access control, and retention requirements.

Dimensional Data Models ETL SQL Python Star Schemas Data Lineage Transformation Catalogs Indexing Data Governance Access Control

Data Platform Engineer

Spreetail

Jan 2019 - Dec 2020 (1 year 11 months)

Built batch ingestion pipelines to aggregate ecommerce transaction and inventory data into a central analytics store. Implemented schema migrations and ETL testing, automated data validation and alerting for nightly loads, and improved query performance for faster reporting and ad-hoc analytics.