Model Test and Measurement Engineer

🕒 October 1

🇺🇸 United States – Remote

💵 $150k - $190k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

👷🏻‍♀️ Engineer

👻 Ghost score 2%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Defcon AI

WebsiteLinkedInAll Job Openings

11 - 50 employees

🤖 Artificial Intelligence

🚗 Transport

📦 Logistics

Artificial Intelligence • Transport • Logistics

Defcon AI is a company focused on transforming logistics and supply chain operations using AI-driven technologies. The company addresses disruptions caused by natural disasters, unanticipated events, and opponents through sophisticated software modeling and intelligent agents. Defcon AI aims to integrate next-generation technologies within logistics and decision-making processes to improve response planning in complex and contested environments. Positioned at the convergence of AI, mobility, and logistics, Defcon AI collaborates with partners to provide efficient, reliable, and data-driven solutions tailored to specific needs. The company is committed to enhancing resilience and efficiency in the logistics sector.

📋 Description

• Construct labeled ground truth for model evaluation • Design and run sampled audits • Gate releases against model version; maintain version inventory, evaluation records, and rollback triggers • Run drift and override review • Produce human-oversight and fairness / disparate-effect evidence • Maintain independence from build roles and produce evaluation evidence without building models, approving thresholds, or authorizing releases against that evidence • Measure workflow improvement, including review time, throughput, backlog movement, and override and rework rates • Design evaluation-phase quality control for sample selection, review units, and fair comparisons across differing methods or searched sources • Define measurement events required from other teams and confirm the review workspace emits workflow events from first use • Work closely with data scientists, AI engineers, and technical leadership • Inform release decisions, improvement efforts, and customer evaluations through independent assessments

🎯 Requirements

• 5+ years with model validation as a named responsibility, not a side task • Direct experience with ground-truth construction and sampling design • Strong Python and SQL • Comfort working independently from the teams whose models you evaluate • US Citizenship Required • Active US Secret clearance • Elevated personnel security requirements apply to portions of this work and are discussed during screening • Regulated-industry or government model-risk background • Experience with fairness / disparate-effect testing and human-oversight documentation • NIST AI RMF or comparable practice • Active Top Secret clearance

🏖️ Benefits

• Competitive salary, bonus, and equity package • A fully remote, results-based environment • 100% employer paid, comprehensive health insurance including medical, dental, and vision for you and your family • Unlimited PTO, with your manager's approval • Flexible work environment where you manage your work day • 14 weeks of fully-paid parental leave • Occasional travel to DEFCON AI headquarters, customer sites, and partner facilities as needed

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume