Inference QA Engineer

Job not on LinkedIn

🔥 1 hour ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Applied Computing

WebsiteLinkedInAll Job Openings

11 - 50 employees

🤖 Artificial Intelligence

⚡ Energy

🤝 B2B

Artificial Intelligence • Energy • B2B

Applied Computing is an artificial intelligence company that builds physics‑grounded foundational models for energy and industrial operations. Its flagship product, Orbital, connects to plant DCS, historians and LIMS to monitor thousands of sensors in real time, forecast trends, detect anomalies, recommend optimal set‑points across competing objectives, and surface early warnings while enforcing physical constraints (mass and energy balances, reaction kinetics). Applied Computing combines time‑series models, physics‑based modeling and a language model trained on chemical engineering and petrochemical knowledge to produce explainable, auditable insights for oil & gas, refining and chemical plants. The company partners with industry firms (e. g. , KBR, Wipro) and has been recognized for AI innovation in the energy sector.

📋 Description

• Own and extend tiered evaluation frameworks for data retrieval, statistical analysis, open-ended inference, and root-cause analysis • Design test sets with ground-truth rubrics, pass criteria, known failure modes, and source tables • Run large-scale prompt executions against live model endpoints on schedules and on demand • Maintain the test harness and turn raw model runs into deployment verdicts • Build observability and monitoring for multi-step agentic systems • Automatically detect silent empty responses, false refusals, tool-routing misses, latency issues, non-termination, and instability • Work with subject-matter experts to extract vetted ground truths from deployment feedback • Convert confirmed defects into permanent regression guards • Devise tier-appropriate tests after fixes, run them against deployments, and report pass rates against thresholds • Own the inference-quality gate for production deployments

🎯 Requirements

• 5+ years in software, ML, data, or QA engineering, with ownership of a quality-critical system • Strong Python skills • Comfortable with FastAPI, Postgres, and Docker • Able to read logs across services and trace requests through distributed pipelines • Fluency with LLM prompting, tool/function calling, agentic loops, RAG, hallucination, refusal, silent truncation, and non-determinism • Experience designing LLM evaluations, deterministic checks, ground-truth scoring, and statistical consistency measures • SQL literacy, including judging generated queries and table selection • Monitoring and observability experience with dashboards, alerts, and trace inspection • Rigorous handling of uncertainty and calibrated ranges • Bonus: experience evaluating or red-teaming agentic or multi-tool LLM systems • Bonus: MLflow or similar trace and experiment tooling • Bonus: experience with technical end users and translating feedback into reproducible tests • Bonus: time-series, forecasting, industrial, or operational data experience