Software Development Engineer 3

What You'll Do:

Platform Design & Development:

• Collaborate with stakeholders to gather requirements and translate them into a comprehensive software system. This includes designing microservices, persistence layers, and implementing deployment workflows using tools like Terraform, Ansible, Puppet, or Salt.

• Design, develop, and implement highly available, scalable, and secure platform services for our data PaaS using programming languages (e.g., Python, Go, Java).

• Contribute to the architectural evolution of our data platform, ensuring it meets the growing demands of our users and adheres to best practices.

• Develop and maintain APIs, automation tools, and internal services to streamline platform operations and enhance developer experience.

• Implement and improve data pipelines, storage solutions, and processing frameworks within the PaaS.

Infrastructure Management & Automation:

• Manage and optimize cloud infrastructure in AWS and Azure cloud services to support our data platform, focusing on cost-efficiency, performance, and reliability.

• Implement Infrastructure as Code (IaC) principles using tools like Terraform for provisioning and managing resources.

• Develop and maintain CI/CD pipelines to automate deployments, testing, and release processes for platform services.

• Monitor platform health, performance, and security, and proactively identify and address potential issues.

Data Expertise & Integration:

• Work closely with data engineers and AI engineering to understand their needs and provide platform capabilities that facilitate efficient data ingestion, processing, storage, and analysis.

• Ensure the platform supports various data technologies and paradigms (e.g., streaming, batch processing, data warehousing, data lakes).

• Implement and manage data governance, security, and compliance features within the platform.

What You'll Bring:

Experience:

• Bachelor's degree in Computer Science, Software Engineering, or related field.
  OR demonstrable equivalent deep understanding, experience, and capability.

• Master or PhD degree in Computer Science (or related field) and Strong experience in software engineering or Strong experience in software engineering at a large scale. Experience in data system/platform development.

• Proven experience managing and optimizing cloud infrastructure (AWS, Azure).

• Demonstrable experience with Infrastructure as Code (IaC) tools (e.g., Terraform, CloudFormation).

• Solid experience with containerization technologies (e.g., Docker, Kubernetes).

• Experience with CI/CD pipelines and DevOps best practices (GitHub Actions preferred)

• Familiarity with data storage and processing technologies (e.g., Kafka, Spark, Flink, Hadoop, Snowflake, MongoDB, S3/ADLS).

• Proficiency in one or more programming languages commonly used for platform development (e.g., Python, Go, Java).

• Strong understanding of networking, security, and performance principles in a cloud environment.

• Experience with monitoring and logging tools (e.g., Prometheus, Grafana, Datadog).

• Familiarity with database administration and optimization is a plus.

• Familiarity with application access control and automation

• Front-end engineering experience is advantageous.