[C3C] Senior Data Engineer (Python)

Project – the aim you’ll have

Our team works with a British insurance company on proactive cyber insurance products. The Cyber Data Services team is at the core of protecting policyholders from existing and emerging cybersecurity vulnerabilities and threats.
The role focuses on developing a proprietary data platform that aggregates, presents, and analyses large datasets. The platform supports new data capabilities such as large-scale data ingestion, robust high-performance APIs, high-frequency event streams, and data quality improvements across the estate.
This is an opportunity to have a material impact on products and solutions that help customers understand and reduce cyber risk.

Position – how you’ll contribute

As a Senior Python Software Engineer, you will play a key role in designing, building, and improving data capabilities that strengthen the company’s competitive advantage in cyber data. You’ll work closely with engineering, infrastructure, and analytics stakeholders to develop scalable data services and platform features across Azure and AWS environments.

In this role, you will:

• Design and implement new data capabilities, including large-scale data ingestion, transformation pipelines, robust high-performance APIs, and consumers for high-frequency event streams.
• Contribute to the development of the proprietary data platform and help evolve it towards a data mesh architecture for the enterprise.
• Build solutions using the standard technology stack: Python, Linux via WSL, PostgreSQL, SQL Server, Databricks, and cloud infrastructure spanning Azure and AWS.
• Develop advanced mechanisms for data ingestion, transformation, and mass-parallel orchestration of network IO.
• Model data for operational and analytical purposes in collaboration with your team, other engineering teams, and analytical data customers.
• Work with infrastructure teams to maintain Infrastructure as Code and deliver valuable features for the engineering platform.
• Improve data quality across the estate by defining and acting on indicators such as completeness, accuracy, reliability, and usability.
• Evaluate potential new data sources for specific applications, assessing their completeness, accuracy, and business value.
• Contribute to quality automation in CI/CD processes, optimising for a bug-free release cadence and a positive developer experience.
• Support test automation through platform enhancements and additional automated tests.
• Maintain clear technical documentation, starting with well-documented code and extending to materials for technical and non-technical audiences.
• Participate in a pairing-first development team by sharing expertise, learning from colleagues, and contributing to an open engineering culture.
• Mentor and coach less experienced engineers through design discussions, code reviews, and practical knowledge sharing.
• Assess application security and API quality to ensure solutions comply with relevant application security and API standards.

Expectations – the experience you need

• Senior-level commercial experience as a Software Engineer, with strong hands-on Python development skills.
• Practical experience building data platforms, data services, or backend systems that process, expose, or analyse large datasets.
• Strong understanding of data ingestion, transformation, event-driven or stream-oriented processing, and high-performance API development.
• Experience with relational databases and data modelling, especially PostgreSQL and/or SQL Server, for both operational and analytical use cases.
• Working knowledge of cloud infrastructure, preferably across Azure and AWS, and the ability to collaborate effectively with infrastructure or platform teams.
• Experience with Databricks or comparable data engineering platforms would be valuable for working with large-scale data workflows.
• Comfortable working in Linux-based development environments, including Linux via WSL.
• Experience maintaining or contributing to Infrastructure as Code and understanding how platform engineering supports product delivery.
• Strong quality mindset, including CI/CD, automated testing, test automation, debugging, code reviews, and continuous improvement of release cadence.
• Ability to evaluate data source quality using criteria such as completeness, accuracy, consistency, and suitability for specific applications.
• Good understanding of secure engineering practices, application security expectations, and API standards.
• Strong documentation habits and ability to communicate technical decisions clearly to technical and non-technical audiences.
• Collaborative, pairing-friendly approach with the ability to mentor others, challenge decisions constructively, and take ownership of outcomes.
• Fluent communication in English.

Additional skills – the edge you have

• Experience in cyber insurance, cybersecurity, fintech, or other regulated data-rich domains.
• Experience designing or contributing to data mesh architecture.
• Hands-on experience with high-frequency event streams or mass-parallel network IO.
• Cloud certifications or proven knowledge of Azure and/or AWS.
• Experience improving developer experience through CI/CD and platform automation.

Our offer – professional development, personal growth

• Flexible employment and remote work
• International projects with leading global clients
• International business trips
• Non-corporate atmosphere
• Language classes
• Internal & external training
• Private healthcare and insurance
• Multisport card
• Well-being initiatives

Software Mind develops solutions that make an impact for companies around the globe. Tech giants & unicorns, transformative projects, emerging technologies and limitless opportunities – these are a few words that describe an average day for us. Building cross-functional engineering teams that take ownership and crave more means we’re always on the lookout for talented people who bring passion and creativity to every project. Our culture embraces openness, acts with respect, shows grit & guts and combines employment with enjoyment.