Open to opportunities

Pedro Oliveira

@pedrooliveira3

Message

Senior data engineer specializing in Azure, Databricks, and scalable data platform optimization.

Portugal

Message

What I'm looking for

I seek roles where I can design scalable cloud data platforms, optimize performance and costs, lead ETL/streaming projects, and collaborate in agile teams for measurable business impact.

I am a Senior Data Engineer with deep hands-on experience designing and optimizing cloud-native data platforms, primarily on Azure and Databricks. I deliver cost-efficient ETL and streaming solutions, improve query and job performance, and secure CI/CD pipelines for production-grade data processing.

At CGI in Lisbon I led migrations from Synapse to Databricks, implemented Unity Catalog, Auto Loader and Delta-based CDC, and applied tuning strategies such as Z-Order indexing and optimized Spark settings to reduce costs and accelerate jobs. I also engineered event-driven ingestion in Snowflake and automated deployments with Azure DevOps.

I hold MS and BS degrees from Universidade Nova de Lisboa (Nova IMS) and certifications including AZ-305 and DP-203, plus Agile credentials. I am collaborative, outcome-focused, and committed to building reliable, observable data platforms that serve business needs.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

CGI Inc.

Sep 2024 - Present (1 year 2 months)

Migrated Synapse pipelines to Databricks and optimized clusters to reduce costs by 15%, implemented Unity Catalog, Auto Loader, CDC with Delta Lake, and event-driven Snowflake ingestion to enable real-time processing and secure CI/CD deployments.

Databricks Azure Data Factory Delta Lake Snowflake Azure DevOps Azure Key Vault Spark Auto Loader Snowpipe

Data Engineer

CGI Inc.

Jul 2020 - Aug 2024 (4 years 1 month)

Optimized SQL Server queries reducing execution time by 40% and migrated workloads to Databricks job clusters, cutting compute costs by 30%; built automated ETL with ADF, integrated external APIs, and implemented CI/CD pipelines.

SQL Server Azure Data Factory Databricks Azure DevOps Python CI CD API Integration Azure Functions

Junior Data Engineer

CGI Inc.

Jan 2019 - Jun 2020 (1 year 5 months)

Built SQL Server data marts and dimensional models to improve Power BI reporting and collaborated with stakeholders to define data requirements for analytics solutions.