Open to opportunities

Hudson Silva

@hudsonsilva

Message

Senior AI Data Engineer building AI-ready Snowflake/Databricks data platforms.

Brazil

Message

What I'm looking for

I’m looking for a role where I can build AI-ready, real-time data platforms on Snowflake/Databricks and take GenAI systems to production—owning RAG pipelines, LLM evaluation/observability, and reliable orchestration that scales.

Senior Data Engineer with 9+ years architecting AI-ready data platforms on Snowflake and Databricks. Built Lakehouse/DW from zero serving 600M+ users and powering AI agents at billions-of-events scale on AWS/Azure/GCP, with consistent 25-50% cost cuts via Spark optimization and Medallion. Currently building internal LLM chatbots (Databricks Genie + Snowflake Cortex), bridging data engineering with the GenAI stack — production RAG (hybrid search, reranking, RAGAS) and multi-agent orchestration (LangGraph, CrewAI, MCP). Technical leader: mentored 6+ engineers (2 promoted), led 5+ member teams, presented architecture to C-level.

Experience

Work history, roles, and key accomplishments

Current

Senior AI Data Engineer

Current

Enterprise Data Platform Consulting

Sep 2025 - Present (8 months)

Migrated 50+ Oracle DW tables (100+ TB) to Snowflake, refactoring pipelines to cut critical runtime from 8 hours to <10 minutes (48x faster). Architected RAG and agentic workflows over the Lakehouse (LangGraph, CrewAI) with RAGAS evaluation and Langfuse tracing, and built internal Databricks Genie LLM chatbots for operational data access.

Snowflake Databricks Snowpipe Pyspark Spark Structured Streaming DBT LangGraph Crew AI Ragas Langfuse

Current

Senior AI Data Engineer (Consultant)

Current

Databool Consulting

Oct 2020 - Present (5 years 7 months)

Provided senior AI/data engineering placements across enterprise clients in energy, telecom, mining, semiconductors, streaming, and fintech, delivering Snowflake/Databricks platform work as part of consulting engagements.

Snowflake Databricks Apache Spark Data Engineering ETL ELT Data Migration DBT Python SQL

Lead AI Data Engineer

Confidential Client

Mar 2025 - Aug 2025 (5 months)

Built a real-time streaming platform with Spark Structured Streaming in Databricks to feed AI recommendation agents for a 600M+ user social/video platform. Reduced incident resolution time by 40% using custom observability and cut compute costs by 25% via Spark query optimization, partitioning, and caching.

Databricks Spark Structured Streaming AWS Kinesis Pyspark DBT Observability fastAPI Deequ Pytest

Senior Data Engineer (Intel)

Intel Corporation

Jun 2024 - Mar 2025 (9 months)

Improved legal-department query performance by 40% through Snowflake/Databricks tuning, enabling faster compliance workflows. Built and maintained 15+ production pipelines on Azure Data Factory + Databricks, cutting incident response time by 50%, and led 5+ engineers to align governance, data masking, and multi-jurisdiction compliance.

Snowflake Databricks Azure Data Factory SQL Tuning Data Modeling DBT Power BI Data Governance Incident Response

DataOps Engineer

BEES

Oct 2023 - May 2024 (7 months)

Optimized Databricks/Snowflake data pipelines for +40% efficiency and -15% cost, resolving Azure/Databricks and MongoDB bottlenecks. Reduced response time by 30% and processing time by 20% through targeted performance fixes.

Databricks Snowflake Azure Data Factory DBT MongoDB Python SQL Performance Optimization DataOps Power BI

Senior Data Engineer

Vale S.A.

May 2023 - Oct 2023 (5 months)

Designed a Snowflake data warehouse with dbt and CI/CD on Azure DevOps, replacing the legacy system and processing 150+ TB/day for global mining reporting and compliance. Implemented automated data quality validation to ensure cross-system consistency for critical financial datasets.

Snowflake Databricks DBT Azure DevOps CI CD Terraform Data Validation Data Modeling Power BI

Senior Data Engineer (GCP)

Semantix Corp

Oct 2022 - Apr 2023 (6 months)

Optimized GCP queries using Hive/Trino for a 25% performance increase. Introduced Kubernetes orchestration to cut deployment time by 40% and led adoption of new data technologies, improving system performance by 40%.

BigQuery Dataflow Composer GKE Hive Trino Kubernetes Python DBT

Senior Cloud Data Engineer

Multiple Clients

Oct 2020 - Sep 2022 (1 year 11 months)

Led Azure-first cloud platform architecture supporting 200% data growth and reduced storage costs by 30% via data-lake optimization. Migrated Pandas pipelines to PySpark (-40% processing time), reduced data latency by 50%, and implemented governance/security (GDPR/LGPD) with +25% accuracy and -40% integration errors.