Lubaba Rehman

As a Data Engineer, I built data pipelines using AWS S3, Databricks, and dbt, orchestrated with Apache Airflow. I developed Python and SQL-based dbt models, implemented Data Quality Checks for a RAG-based LLM pipeline, and created APIs using AWS API Gateway.

I developed Python-based ETL pipelines to extract data from CSV files into Google Sheets. Conducted exploratory data analysis on demographic datasets and collaborated with psychologists to identify therapy needs.

Designed and developed ETL pipelines using Apache Spark and Kafka to process large datasets. Implemented Data Quality Checks and built real-time data pipelines for condition monitoring of induction motors.

Developed data ingestion scripts using Python, PHP, and SQL. Designed database schemas for applications and conducted research on university energy consumption prediction using machine learning models.

Worked as a Software Quality Assurance professional, applying the data science pipeline on a novel dataset from the Pakistan health industry to identify risks.

Designed and developed ETL pipelines using Apache Spark and Kafka to process large datasets from various sources, including IoT sensors. Implemented DQCs using SQL and Python scripts to ensure data accuracy and consistency. Built a real-time data pipeline for condition monitoring of induction motors using Apache Kafka, PostgreSQL, and Apache Spark.

Developed a bearing fault classification model using a Fully Connected Neural Network (FCN) with Keras and TensorFlow to support predictive maintenance of induction motors. Conducted fault diagnosis on simulated data from an Aspen Plus Distillation Column model using 1D CNN, with layer-wise visualization via t-SNE.