At Adaptive Computer Solutions Ltd., I architect and implement distributed batch and streaming pipelines on the Databricks Lakehouse platform using PySpark and Delta Lake. I built a Bronze, Silver, and Gold Medallion Architecture with schema enforcement and transactional consistency.
I streamlined ingestion of multi-source datasets with Databricks Auto Loader and PySpark Structured Streaming. By tuning Spark clusters and applying partition pruning, cache management, and Z-Order indexing, I cut computation costs by about 35%.
I also configure Databricks Workflows with alerting, automated retries, and SLA tracking. Earlier, as a Junior Data Engineer at Adaptive Computer Solutions Ltd., I assisted with Python and SQL migration scripts and wrote PySpark notebook tests and data validation checks.
