Solutions Engineer – Language Models

Job not on LinkedIn

🔥 14 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Artificial Analysis

WebsiteLinkedInAll Job Openings

11 - 50 employees

🤖 Artificial Intelligence

☁️ SaaS

🤝 B2B

Artificial Intelligence • SaaS • B2B

Artificial Analysis is an independent AI evaluation and benchmarking organization. It publishes proprietary benchmarks and indices (e. g. , Artificial Analysis Intelligence Index v4. 1, AA-Briefcase, AA-Omniscience), leaderboards for models across text, coding/agents, image & video, and speech, and capability-specific indices. The platform measures model intelligence, speed, cost, hallucination rate, and other metrics, provides model comparisons and a personalized model recommender, and offers premium benchmark data, visualizations, industry reports, and articles to help organizations choose the best model and provider for their use case.

📋 Description

• Operate and maintain our Python-based language model benchmarking pipeline end-to-end: onboard new models, configure evaluations, execute benchmark runs, and validate results • Debug issues across the stack — from API endpoint timeouts and errors to unexpected benchmark outputs — and resolve them quickly • Serve as the primary technical contact for AI lab customers: communicate benchmarking results clearly, explain methodology, field technical questions, and troubleshoot integration issues via Slack and video conferencing • Monitor benchmark runs for anomalies, investigate discrepancies, and ensure the accuracy and integrity of published results • Maintain documentation of processes, known issues, and model-specific configurations • Collaborate with the engineering team to flag pipeline improvements and contribute to process refinements • Stay current with new model releases, API changes, and developments across the language model ecosystem

🎯 Requirements

• 5+ years of experience in a client-facing technical role — solutions engineering, support engineering, technical consulting, or similar (companies like Stripe, Vercel, Cloudflare, Datadog, Palantir, Accenture, or comparable) • Strong Python proficiency and comfort working with complex codebases you didn't write • Hands-on experience working with AI/ML model APIs (OpenAI, Anthropic, Google, Meta, etc.) • Excellent debugging skills — you can trace issues across APIs, data pipelines, and code • Strong written and verbal English communication skills, with the ability to explain technical concepts clearly to technical stakeholders • Highly responsive and reliable — you take ownership of customer issues and follow through • Comfortable with operational, repeatable work — you find satisfaction in running things well rather than building from scratch • High attention to detail and calm under pressure

🏖️ Benefits

• Competitive compensation including equity • Fast-growing team split across San Francisco, Sydney, and Melbourne