At EG Group, I develop Databricks pipelines to migrate legacy SQL Server and SSIS workflows, processing millions of files. I also implemented a batch chunking strategy using Autoloader to ingest and process large file volumes.
At WeCrunch Ltd., I led end-to-end delivery of a data engineering project, designed its data model, and produced ERDs and technical documentation. I also designed Databricks Spark pipelines using Python and SQL that processed 100+ GB of data and improved pipeline speed by 60%.
My projects include a Databricks data warehouse and ETL pipelines for Business Penetration KPIs reporting, as well as an NLP chatbot using a custom neural network. For my final-year project, I designed an IoT air-quality monitoring device using deep learning for real-time prediction of toxic gases and particulate matter.
