Databricks Pipeline Architect

Job Description

ECS is seeking a Databricks Pipeline Architect to support the design, implementation, and long‑term operation of Databricks within a complex Government enterprise environment. This role will work closely with various stakeholders during initial implementation and will support data engineering projects as Databricks is rolled out within the environment. The position is contingent upon additional funding and will be performed remotely.

Key Responsibilities

• Lead data engineering efforts within Databricks to include pipeline development within Databricks on AWS in an enterprise environment.
• Provide unification for organization and enterprise data capabilities to break down organizational data and operational silos, including mechanisms to standardize and migrate existing data to a unified data plane/warehouse.
• Support the integration of Databricks with existing analytics platforms and metadata solutions.
• Serve as a technical advisor to stakeholders and help upskill internal teams through guidance/documentation.
• Assist and develop customer use case applications of Databricks.

Salary Range: $150,000 - $180,000

Benefits

Required Skills

• Must be a US Citizen.
• Ability to successfully maintain a DHS Entry on Duty (EOD)/Public Trust.
• Minimum 7 years of hands‑on experience architecting, designing, data engineering with Databricks and AWS in a government setting.
• Strong understanding and experience in areas such as Apache Spark, Lakebase, Lakehouse architectures, Delta Lake, Delta Sharing, Medallion Architecture, Data & AI Governance, and use case acceleration.
• Demonstrated hands‑on experience with AWS Cloud Native data and ETL capabilities (S3, Lambdas, Glue, etc.).
• Demonstrated experience leveraging data to provide search, enrichment, correlation, AI/ML Model Training, in alignment with government mission and business use cases.
• Experience integrating Databricks with other analytics platforms.
• Prior consulting or customer‑facing experience in data architecture or modernization initiatives.
• Strong Python skills.
• Experience in Agile environments and managing cross‑functional technical delivery.

Desired Skills

• Prior experience supporting DHS.
• Experience with the DHS System Engineering Lifecycle (SELC).
• Experience with Unity Catalog and MLFlow within Databricks.
• Experience with agentic AI tools such as Databricks' Agent Bricks and Genie.
• Familiarity with the concepts of a data mesh.
• AWS Solutions Architect Associate or Solutions Architect Professional Certification.
• Experience building reference architectures, playbooks, or platform documentation.
• Experience with metadata platforms (e.g., DataHub, Collibra) and integrating Databricks with them.
• Databricks certifications (Architect, Data Engineer).

ECS is an equal opportunity employer and does not discriminate or allow discrimination on the basis any characteristic protected by law. All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, or local jurisdiction law.

ECS is a leading mid‑sized provider of technology services to the United States Federal Government. We are focused on people, values and purpose. Every day, our 3300+ employees focus on providing their technical talent to support the Federal Agencies and Departments of the US Government to serve, protect and defend the American People.