Director of Software Engineering – Node.js, Web Scraping Expert

Job not on LinkedIn

🕒 April 1

🏄 California – Remote

🏄 California – Remote

⏰ Full Time

🔴 Lead

🔙 Backend Engineer

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

PortPro

WebsiteLinkedInAll Job Openings

51 - 200 employees

🚗 Transport

☁️ SaaS

💰 $12M Series A on 2023-01

Transport • SaaS

PortPro is a privately-owned technology company that specializes in providing advanced operational software solutions tailored for drayage carriers and freight brokerages. Their platform is designed to enhance drayage operations by improving communication, operational efficiency, and customer satisfaction. Utilizing a modern Transportation Management System (TMS), PortPro focuses on helping companies streamline their processes, improve cash flow, boost operational performance, and grow their business. The company's award-winning software supports both asset-based and non-asset based divisions, enabling seamless integration and comprehensive management of drayage trucking and brokerage operations. PortPro is recognized for their customer support and innovative technology offerings that empower clients to achieve greater transparency and efficiency in their operations.

📋 Description

• Architect, develop, and maintain scalable and distributed web scraping systems using Node.js. • Design and implement data extraction pipelines to process large volumes of structured and unstructured data. • Develop solutions to bypass anti-bot mechanisms, including CAPTCHA handling, session management, fingerprinting, and IP rotation. • Optimize scraping processes for performance, reliability, and efficiency while managing proxy services (residential, datacenter, rotating). • Oversee data storage and processing strategies, ensuring high availability and consistency. • Collaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications. • Implement best practices for microservices, API integrations, and real-time data streaming. • Lead the transition to cloud-native, containerized, and serverless architectures for web scraping. • Ensure compliance with legal and ethical standards (robots.txt, GDPR, CCPA, etc.). • Optimize cloud resources (AWS, GCP, or Azure) to support high-throughput scraping. • Manage real-time monitoring and alerting systems to detect scraping failures, IP bans, or performance bottlenecks. • Work closely with DevOps teams to optimize CI/CD pipelines, automated deployments, and system scalability. • Lead, mentor, and grow a high-performance engineering team. • Define and execute the technology roadmap, aligning with business objectives. • Foster a culture of continuous learning, collaboration, and innovation. • Implement agile development methodologies (Scrum, Kanban) to optimize project execution. • Ensure code quality, security, and best practices across all engineering efforts.

🎯 Requirements

• 10+ years of experience in software engineering, with at least 5+ years in web scraping and large-scale data extraction. • Strong hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automation. • Extensive experience in handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques. • Deep knowledge of proxy management (residential, datacenter, rotating, and VPNs). • Experience with NoSQL/SQL databases (MongoDB, PostgreSQL, Redis, Elasticsearch, etc.). • Familiarity with data processing frameworks (Kafka, RabbitMQ, Spark, Airflow, etc.). • Strong experience with CI/CD, containerization (Docker, Kubernetes), and cloud deployment (AWS/GCP/Azure).