Data Collection Engineer

Job not on LinkedIn

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Two Six Technologies

WebsiteLinkedInAll Job Openings

501 - 1000 employees

🔐 Security

Security • Defense

Two Six Technologies is a company that specializes in creating advanced technologies to support defense, security, and safety missions. The company develops next-generation technologies and provides solutions in areas such as information operations, cyber defense, data analytics, and secure electronic systems. Two Six Technologies partners with organizations like the US Department of Defense and Homeland Security to bolster national and global security projects. Their products encompass innovations like ZeroTrust Architecture, CBRN Detection, and reverse engineering platforms that are crucial for mission-critical operations. Driven by innovation and impact, the company focuses on tackling global challenges through research and development and operational deployment of their technologies.

📋 Description

• Design and deploy high-performance, distributed web scrapers using Python and Scrapy • Utilize Browser Scripting tools to navigate, interact with, and extract data from websites • Deploy, scale, and manage scraping workloads on Kubernetes • Define strict JSON Schemas and leverage Pydantic for data validation • Build and optimize search and storage pipelines using Elasticsearch • Architect robust pipeline workflows to manage the end-to-end data lifecycle • Manage complex proxy rotation and session handling to maintain high success rates

🎯 Requirements

• 7+ years of professional software engineering experience • Excellent reverse-engineering skills • Expert-level proficiency in Python • Deep experience with Scrapy • Proven experience with browser automation tools • Mastery of JSON and data validation using Pydantic • Hands-on experience indexing, querying, and optimizing Elasticsearch clusters • Strong proficiency in managing and scaling applications within Kubernetes environments • Experience building structured pipeline workflows • Bachelor’s degree in Computer Science or Engineering

🏖️ Benefits

• medical, dental, and vision insurance • life and disability insurance • retirement benefits • paid leave • tuition assistance • professional development