Staff Software Engineer, AI Inference

Job not on LinkedIn

🕒 July 29

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Syllo

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2019

⚖️ Legal

🤖 Artificial Intelligence

☁️ SaaS

🔥 Funding within the last year

💰 $30M Venture Round - Syllo on 2025-10

Legal • Artificial Intelligence • SaaS

Syllo is a Litigation AI company that provides a unified platform for managing complex litigation end-to-end. Its software combines case management, eDiscovery, legal research, analysis, drafting, and agentic issue tagging to help in-house teams and law firms master the record, apply issue codes at scale, and accelerate strategy and workflow across written discovery, depositions, and pretrial work. Syllo positions itself as an AI-enabled, collaborative workspace focused on high-stakes commercial litigation.

📋 Description

• Lead the design and development of our production inference platform. • Define the technical roadmap for inference infrastructure, model serving, and runtime optimization. • Build and operate scalable, cost-effective systems for serving large language models in production. • Evaluate and integrate modern inference technologies, frameworks, and serving runtimes. • Optimize latency, throughput, GPU utilization, memory efficiency, and infrastructure cost. • Develop systems for model deployment, traffic routing, autoscaling, scheduling, observability, and operational excellence. • Partner with ML engineers to productionize new models and inference techniques. • Establish benchmarking methodologies to evaluate new models, runtimes, and hardware. • Make key architectural decisions around when to build internally versus leverage open-source or commercial solutions. • Mentor engineers as the team grows and help establish engineering best practices for AI infrastructure.

🎯 Requirements

• Significant experience designing and operating production AI inference systems. • Experience building or leading production LLM serving infrastructure. • Deep experience with one or more modern inference runtimes and frameworks such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Hugging Face TGI, NVIDIA Dynamo, or comparable technologies. • Strong background in distributed systems, backend infrastructure, or high-performance platform engineering. • Experience optimizing inference performance across GPU workloads, including latency, throughput, batching, memory utilization, and serving efficiency. • Experience operating GPU infrastructure in production. • Strong proficiency in Python and at least one systems programming language (such as Go, Rust, or C++). • Proven ability to lead technical architecture for complex infrastructure initiatives. • Excellent communication skills and the ability to influence technical direction across engineering teams.

🏖️ Benefits

• Health insurance