Lead - Fund Access Data | Offshore - HYD

Project Description:

The Fund Accounting Data Project ingests, rationalizes, and publishes fund accounting data from first‑party and third‑party Fund Accounting systems into a centralized Data Platform.

The platform supports two primary outcomes:

• A client‑facing portal with analytics and data visualizations
• The Core Accounting Model, acting as the internal canonical representation of accounting data

The initiative involves:

• SQL Server‑based source systems
• API‑based vendor data feeds
• Cloud‑based processing and transformation (Databricks)
• High requirements for data accuracy, auditability, and lineage

Role Objective

Provide a Senior Software Engineer who will act as a Data Engineer, responsible for building and maintaining scalable, reliable data ingestion and transformation workflows based on specifications produced by the Technical Analyst.

This role is accountable for the implementation, correctness, performance, and operability of data pipelines that publish trusted datasets to the Data Platform.

Role Context

• Embedded within a data platform engineering team.
• Works closely with:
  • Technical Analysts (data specifications and mappings)
  • Other Data Engineers
  • Platform / Cloud Engineers
  • QA and Product partners
• Operates in an Agile delivery model with frequent releases.
• Focused on backend engineering rather than analytics or visualization.

Core Responsibilities (What They Will Do)

• Implement data ingestion pipelines from:
  • SQL Server source systems
  • External vendor APIs
• Build data transformation workflows in Databricks (Spark) using Python and/or SQL.
• Implement source‑to‑target mappings and transformation rules defined by the Technical Analyst.
• Design and maintain reusable, modular data processing components.
• Ensure data pipelines meet requirements for:
  • Accuracy and completeness
  • Performance and scalability
  • Fault tolerance and recoverability
• Implement data quality checks, validations, and reconciliations.
• Support lineage and auditability through logging, metadata, and structured outputs.
• Troubleshoot ingestion and transformation failures across environments.
• Partner with Technical Analysts to identify gaps, edge cases, and improvements in specifications.

Explicit Non‑Responsibilities (Important for Profile Filtering)

• Not a business analyst or requirements owner.
• Not a BI or reporting developer.
• Not a front‑end or visualization engineer.
• Not a DevOps engineer, though expected to understand deployment and runtime behavior.

Required Experience (Non‑Negotiable)

• 7+ years experience as a Software Engineer, with significant experience in Data Engineering roles.
• Strong hands‑on experience with Databricks or Spark‑based data processing platforms.
• Advanced Python proficiency for data transformation and pipeline development.
• Strong SQL skills, including querying and transforming large datasets.
• Experience ingesting data from:
  • Relational databases (e.g., SQL Server)
  • REST or similar APIs
• Proven experience building production‑grade data pipelines in regulated or data‑sensitive environments.

Preferred Background (Strong Signals)

• Experience in financial services, asset servicing, or fund accounting environments.
• Familiarity with accounting data domains (positions, transactions, balances, NAV).
• Experience working with canonical or enterprise data models.
• Exposure to:
  • Data quality frameworks
  • Data observability and monitoring
  • Cloud‑native data platforms
• Experience collaborating closely with Technical Analysts or Data Architects.

Success Criteria (How We Will Judge Fit)

A successful Senior Software Engineer on this project will:

• Deliver stable, efficient data pipelines aligned with documented specifications.
• Identify and resolve data anomalies and edge cases early in implementation.
• Minimize production defects related to data correctness and transformation logic.
• Create pipelines that are understandable, supportable, and extensible by other engineers.
• Enable reliable downstream consumption by analytics platforms and core accounting models.

Seniority Guidance

• Senior individual contributor.
• Expected to own pipeline design and implementation end‑to‑end.
• Comfortable making technical decisions and tradeoffs within the data platform.
• Acts as a technical reference for data ingestion and transformation patterns.