rockITdata is seeking an Automation Engineer to design and develop automated metadata harvesting pipelines using Python and SQL. This role would be responsible for integrating AI/ML capabilities to enable intelligent metadata extraction and implements drift detection systems for ongoing metadata maintenance.
Key Responsibilities
- Design and develop automated metadata harvesting architecture
- Build Python/SQL-based data pipelines for metadata extraction from diverse systems
- Integrate AI tools for intelligent metadata classification and tagging
- Implement Human-in-the-Loop (HITL) validation workflows for AI-assisted harvesting
- Deploy harvesting automation across enterprise systems for scalable metadata capture
- Develop drift detection and alerting mechanisms for metadata change management
- Create and maintain technical documentation for all automation components
- Support empirical testing in sandbox environments for technology evaluations
- Optimize harvesting processes to achieve efficiency gains
- Coordinate with QA teams on automation quality standards
- Troubleshoot and resolve pipeline issues across production environments
- Bachelor's degree in Computer Science, Software Engineering, Data Engineering, or related field
- Minimum 5 years of experience in Python development and SQL
- Experience with AI/ML integration or automation frameworks
- Proficiency with data pipeline tools (Apache Airflow, Prefect, or similar)
- Experience with cloud platforms (AWS, Azure) and their AI/ML services
- Strong understanding of ETL/ELT processes and data integration patterns
- Experience with version control (Git) and CI/CD practices
- Ability to obtain Public Trust clearance
- Experience with Amazon Bedrock, Azure OpenAI, or similar generative AI services
- Knowledge of RAG (Retrieval-Augmented Generation) architecture patterns
- Experience with DoD or federal government IT systems
- Familiarity with FedRAMP-compliant cloud environments (GCC-High)
- Experience with SharePoint APIs and Microsoft Graph
- Knowledge of metadata standards and data catalog integrations
#LIREMOTE
Here at rockITdata, we are committed to following our 10 Guiding Principles.
Our Guiding Principles define our culture. They’re who we are, how we work, and what inspires us to be the best. We empower our people to be themselves and encourage an entrepreneurial way of thinking. In our challenging, fast-paced environment, no day is the same.
Know the Why.
Value People Above All Else.
Transparency to a Fault.
Progress, Not Perfection.
Be Good by Doing Good.
Smart People Can Disagree.
Bend but Don’t Break.
Represent Your Brand.
Think Differently.
Be Amazing, Be Fearless, Smile, Have Fun!
Interested in joining something unique? Join us on our rockIT ship as we begin to blast off to the next chapter in our organization!
rockITdata has become aware of a recruitment scam where unauthorized individuals are impersonating rockITdata recruiters, issuing fake job offers, and attempting to setup payment for the job seeker’s new hire costs. No one is ever required to complete any monetary transactions before starting employment with rockITdata. Legitimate rockITdata communications only come from @rockitdata.com email addresses and our official Applicant Tracking system, JazzHR (@applytojob.com email addresses). Please refer to our official Careers page at www.rockitdata.com/careers to accurately submit your application with us. You may also verify any suspicious communication by contacting our recruitment team directly through our official email, dreamjob@rockitdata.com.
rockITdata is an equal opportunity employer and is committed to non-discrimination in all employment practices and decisions. All qualified applicants will receive consideration for employment without regard to disability status, veteran status or any characteristic protected by Federal, state, or local law.
rockITdata wants all interested and qualified candidates to be able to use our career site to review and apply for employment opportunities. If you have a disability and need an accommodation to access the application process, please contact us for assistance at dreamjob@rockitdata.com.
