Position Summary: Swiftly is searching for a Principal Data Engineer that will play a critical role in designing, implementing, and managing the data infrastructure of our organization. The ideal candidate will have expertise that will contribute to the availability, usability, and integrity of data, enabling data-driven decision-making and business success.
Responsibilities Include:
- Design, build, deploy, and maintain ETL (Extract, Transform, Load) processes using Azure hosted technologies
- Orchestrate data workflows to integrate structured and unstructured data from multiple sources
- Manage customer import and export processes that involve Secure File Transfer Protocol (SFTP), Azure Storage and AWS S3 for data exchange
- Ensure data security and compliance while transferring data between systems
- Support our inclusive and supportive engineering culture, Mentor engineering team
- Advocate for Agile development methodologies and participate in the software development lifecycle (SDLC) activities
- Implement and maintain security measures to protect company assets and customer data
- Develop contingency plans to address potential issues
- Collaborate with data scientists, software engineers, and other stakeholders to create efficient data pipelines using Databricks
- Optimize data processing at scale using Databricks’ capabilities while managing costs
- Author REST servers and client to enhance data and provide admin portal functionality
- Develop models and data sets to enhance personalized services for retail grocery shoppers
- Demonstrate how to leverage data to improve customer experiences through effective targeting
- Other related duties as assigned
Required Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or a related field
- 10+ years of Data Engineering experience
- 5+ years in a small team, startup environment
- Strong knowledge of Python
- Foundational understanding of Azure, Eventhubs, storage accounts and infrastructure considerations
- Experience with SQL, message queuing, stream processing, and advertising platforms
- Deep experience with data pipelines (ETL/ELT, Spark, dbt, Airflow, Databricks, etc.)
- Deep experience with OLAP database design, build, run (Star/ Snowflake, Fact / Dim, Snowflake, etc.)
- Deep experience with distributed application design, build, run (api, queues, idempotent, encapsulated, etc.)
- Experienced with cloud infrastructure (infra as code, cdk, Terraform, Kubernetes, containers & images
- Proven experience in a leadership role within engineering, with a track record of successful project delivery
- Excellent communication, and interpersonal skills
Preferred Qualifications:
- 10+ years in a customer facing, B2B or SaaS data engineering
- Large corporate experience (in addition to startup experience)
- Experience working in a distributed (remote first) environment
- Deep experience using Databricks
- Experience with Postgres
- Deep experience with OLTP database design, build, run (SQL, normalization, query optimization, Databricks, etc.)
Working For SwiftlyWe are working on a set of problems that require the best in the industry to get right. Our employees are the pillars of the future of the company and they are treated and compensated as such. We're a growing team of experienced industry professionals building an organization that can solve tough problems and values a collaborative environment.
Every Swiftly employee: -Has demonstrated the ability to work collaboratively in an ambiguous, fast-paced environment -Takes ownership of their domain from the ground up, from inception through deployment to customers -Leaves their ego at the door and ensures the best idea leaves the room -Is always experimenting with new technologies and learning new skillsets
If you've ever wanted to work on a project that blends a gorgeous consumer experience, sophisticated logistics, and scale that rivals the largest technology players, Swiftly is the place for you.
Equal Opportunity Employer Swiftly is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Swiftly will make reasonable accommodations in compliance with the Americans with Disabilities Act of 1990. Reasonable accommodations are available for qualified individuals with disabilities throughout the subsequent application process.