Open to opportunities

João Silva

@joosilva7

Cloud Data Engineer at BI4ALL who delivered four on-premises-to-cloud migrations and helped build a reusable Databricks ingestion framework.

Portugal

Message

At BI4ALL, I work on Azure Databricks data platforms, including the redesign of Pordata, Portugal’s national statistics database. I’m building PySpark medallion pipelines for Eurostat and INE APIs and deploying jobs as code.

Across telecom, energy, hospitality, and industrial clients, I’ve delivered on-premises-to-cloud data migrations. At Grupo Pestana, I migrated a data warehouse to Azure Databricks and rebuilt legacy SSIS processes as PySpark notebooks and Databricks workflows.

For ADENE, I migrated the building energy certification data warehouse to Azure and built config-driven ingestion for more than 30 source tables. At Algeco, I built pipelines from IFS ERP and Salesforce and rebuilt finance models in Databricks.

I also helped build BI4ALL’s reusable Databricks ingestion framework and rolled it out at Finsolutia. Earlier, at CELFOCUS, I built Google Cloud Data Fusion pipelines for a migration to BigQuery and supported warehouse migration testing with SQL.

Experience

Work history, roles, and key accomplishments

Current

Cloud Data Engineer

Current

BI4ALL

Aug 2022 - Present (4 years 3 months)

Data engineer on the redesign of the data platform on Azure Databricks and Unity Catalog, from proof of concept to delivery. Building PySpark medallion pipelines for Eurostat and INEA APIs and curated Excel files, with change detection between releases.

Cloud Data Engineer

CELFOCUS

Jul 2020 - Jul 2022 (2 years)

Built GCP Data Fusion pipelines migrating the on-premises data warehouse to BigQuery, orchestrated with Cloud Composer (Airflow). Optimised the KPI engine.

GCP Data Fusion BigQuery Cloud Composer Airflow SQL Data Warehouse KPI Optimization

Data Analyst

CELFOCUS

Jul 2020 - Jul 2022 (2 years)

Ran UAT for the data warehouse migration to Google Cloud, validating migrated data across Oracle, Sybase and BigQuery with SQL. Supported development and business teams with root cause analysis of data issues.

UAT SQL Oracle Sybase BigQuery Data Validation Root Cause Analysis

Researcher

INOV INESC INOVAÇÃO

Mar 2019 - Dec 2019 (9 months)

Designed and built a data warehouse (SQL Server, SSIS) with PowerBI dashboards for a client. Part of a Portugal 2020 predictive maintenance research project, and the basis of my MSc thesis.

SQL Server SSIS Power BI Data Warehouse Predictive Maintenance