Senior Software Engineer – Model Evaluation, AI Systems

🕒 il y a 10 jours

🇺🇸 États-Unis – Télétravail

💵 $180 000 - $240 000 / an

⏰ Temps Plein

🟠 Senior

🧑‍💻 Développeur Full-Stack

🦅 Parrain de Visa H1B

Cette entreprise a parrainé des visas H1B par le passé.

Postuler Maintenant

Trouver des Emplois à Distance Similaires

Configurer une alerte email pour les nouveaux emplois

📊 Vérifiez votre score de CV pour ce poste

Améliorez vos chances d'obtenir un entretien en vérifiant votre score de CV avant de postuler.

Télécharger le CV

Deepgram

Site WebLinkedInTous les Emplois

51 - 200 employés

Fondée en 2015

💼 Conseil

🏥 Santé

📦 Logistique

💰 €47 000 000 Series B en 2022-11

Consulting • Healthcare • Logistics

Deepgram est un leader de l’IA vocale qui propose des API puissantes pour des applications de speech-to-text, text-to-speech et de compréhension du langage. Sa plateforme permet aux développeurs de créer des solutions d’IA vocale avancées pour des cas d’usage tels que les centres de contact, la transcription médicale, l’IA conversationnelle, et plus encore. Réputée pour une précision, une rapidité et un rapport coût-efficacité inégalés, la technologie de Deepgram est plébiscitée par les plus grandes entreprises comme par les start-ups du monde entier. En offrant des capacités de transcription en temps réel et hautement précises, Deepgram aide les organisations à extraire des insights de leurs données vocales, en en faisant un outil essentiel pour transformer les interactions vocales.

Description

• Define and build evaluation methodologies for Deepgram's models, spanning speech-to-text, text-to-speech, and emerging LLM, RAG, agent, and multimodal systems. • Design, build, and maintain automated evaluation pipelines across batch and streaming (e.g. WER, runaway/hallucination detection, latency and time-to-first-byte), with a focus on correctness, reproducibility, and ease of adoption. • Build scalable, reproducible evaluation infrastructure — harnesses, orchestration, and result-aggregation pipelines — running against production models and, where needed, large GPU clusters. • Translate Research benchmarks and expected model metrics into automated, enforceable pass/fail gates. • Build and operate canaries and continuous-monitoring systems that detect quality regressions in production before they reach customers. • Partner with DevOps/Infra to stand up ephemeral test environments and results-aggregation infrastructure. • Work alongside Research, model training, inference, and product teams to provide trusted evaluation signals that inform release and optimization decisions. • Integrate evaluation and quality gates into CI/CD so quality is verified continuously, not manually. • Help raise the bar through code reviews, technical design discussions, and strong engineering and QA practices.

🎯 Exigences

• BS, MS, or PhD in Computer Science, AI, Applied Math, or a related field, or equivalent experience. • 5+ years of professional software or QA engineering experience, with a track record of shipping test infrastructure or evaluation systems (senior candidates with significantly deeper experience welcome). • Solid backend/scripting experience in a language such as Python, Rust, Go, or similar. • Experience designing and building automated test pipelines, evaluation frameworks, or data-processing systems. • Strong analytical skills and comfort reasoning about metrics, thresholds, and statistical variation in results — able to distinguish real regressions from noise. • Ability to take charge of ambiguous technical challenges and communicate effectively across research, engineering, and product teams.

🏖️ Avantages

• Offers Equity • Offers Bonus • 10% Annual Bonus