LLMOps Platform Engineer

Job not on LinkedIn

🕒 February 26

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

SumerSports

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2022

⚽ Sports

🤖 Artificial Intelligence

☁️ SaaS

Sports • Artificial Intelligence • SaaS

SumerSports is an AI-powered sports analytics and technology company focused on football (NFL and NCAA). Combining over 500 years of NFL experience with machine learning, SumerSports offers products such as SūmerBrain for film retrieval and multi-layered data, SūmerLive for game tracking, SūmerNFL and SūmerNCAA for roster building and team optimization, and a player-verified metrics and talent exposure platform. The company produces draft guides, analytics-driven content with former scouts and Hall of Famers, and tools that serve players, teams, and fans to improve scouting, roster decisions, and performance evaluation.

📋 Description

• Build and operate the LLM Platform: Develop model routing, prompt registry, and orchestration services for multi-model workflows. • Integrate external LLM APIs (OpenAI, Anthropic, Mistral) and internal finetuned models. • Enable fast, safe experimentation: Implement automated evaluation pipelines (offline + online) with golden sets, rubrics, and regression detection. • Support CI/CD for prompt and model changes, with rollback and approval gates. • Collaborate cross-functionally: Partner with product pods to instrument RAG pipelines and prompt versioning. • Work with deep learning and data teams to integrate structured and unstructured retrieval into LLM workflows. • Optimize performance and cost: Profile latency, token usage, and caching strategies. • Build observability and monitoring for LLM calls, embeddings, and agent behaviors. • Ensure reliability and safety: Implement guardrails (toxicity, PII filters, jailbreak detection). Maintain policy enforcement and audit logging for AI usage.

🎯 Requirements

• 5+ years of experience in applied ML, NLP, or ML infrastructure engineering • Strong coding skills in Python and experience with frameworks like LangChain, LlamaIndex, or Haystack • Solid understanding of retrieval-augmented generation (RAG), embeddings, vector databases, and evaluation methodologies • Experience deploying models or AI systems in production environments (AWS, GCP, or Azure) • Familiarity with prompt management, LLM observability, and CI/CD automation for AI workflows

🏖️ Benefits

• Competitive Salary and Bonus Plan • Comprehensive health insurance plan • Retirement savings plan (401k) with company match • Remote working environment • A flexible, unlimited time off policy • Generous paid holiday schedule - 13 in total including Monday after the Super Bowl