Looking for a job

Diogo Couto

@diogocouto

Message

Data Engineer at Contractor, migrating production databases from PostgreSQL to BigQuery with data integrity maintained.

Portugal

Message

At Contractor, I led the migration of production databases from PostgreSQL to Google BigQuery, redesigning schemas and maintaining data integrity while minimizing downtime.

I also implemented CI/CD pipelines with Cloud Build and ArgoCD, and developed tests to validate data accuracy and consistency throughout the migration.

At Capgemini, I optimized a real-time PySpark ETL pipeline, reducing processing time by 75%. I also created a Dataflow pipeline that loaded data into BigQuery and fed a VertexAI model.

At Spotlite, I improved and maintained an ETL pipeline using Python and distributed computing with Dask. My academic work includes evaluating GANs for InSAR phase unwrapping and contributing to a publication comparing YOLO with DETR.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Contractor

Oct 2025 - Present (1 year)

Led end-to-end migration of production databases from PostgreSQL to Google BigQuery, redesigning schemas for optimal performance. Designed CI/CD pipelines using Cloud Build and ArgoCD, and developed testing frameworks ensuring data accuracy.

PostgreSQL BigQuery Cloud Build ArgoCD CI CD Data Migration Schema Design Testing Query Optimization Partitioning

Data Engineer

Capgemini

Nov 2024 - Sep 2025 (10 months)

Maintained and optimized ETL pipelines using Oracle, Python, Spark, and Kafka, reducing processing time by 75%. Implemented real-time alerts, developed dashboards, and created GCP Dataflow pipelines feeding BigQuery and VertexAI models.

Oracle Python Spark Kafka Kibana PySpark GCP Dataflow BigQuery Vertex AI Airflow Streamlit ETL

Data Engineer

Spotlite

Jun 2022 - Nov 2024 (2 years 5 months)

Improved, expanded, and maintained ETL pipelines using Python, Pandas, Polars, Dask, Docker, and Parquet. Converted workflows from bash to Python, parallelized code with Dask, and implemented CI/CD with GitHub Actions.

Python Pandas Polars Dask Docker Parquet AWS GitHub Actions CI CD ETL

Software Developer (Internship)

Auditor

Apr 2017 - May 2017 (1 month)

Developed a MySQL database and interface for a beverage dispenser system, including login, data registration, and visualization. Implemented mechanisms to view and update saved data.

MySQL Interface Design Login System Data Management