We are hiring a Web Scraping Specialist to join a specialized technical team focused on building infrastructure for delivering massive amounts of web data for AI model training. The successful candidate will gather and analyze data, optimize scraping processes, and support scaling high-quality public web data accessibility.
Requirements
- Code Development: Write, test, and refine high-performance code to extract data from various online sources
- Data Retrieval: Manage complex data retrieval tasks, including handling pagination and dynamic content loaded via AJAX
- Data Quality: Clean and format extracted data to ensure it meets rigorous quality standards for downstream analysis and processing
- Database Management: Store and manage scraped data in appropriate databases, optimizing for both access speed and long-term data integrity
- Monitoring and Maintenance: Regularly monitor scraping processes and infrastructure to identify and resolve issues
Benefits
- Competitive Compensation: A highly competitive salary ranging from $75,000 to $150,000
- Comprehensive benefits and equity package
- Impactful Work: The opportunity to work at the forefront of AI development and web-scale knowledge graph creation
- High-Output Culture: A professional environment that prioritizes low ego, technical autonomy, and rapid execution
