Senior Software Engineer II – Applied AI and Evaluations

🕒 vor 1 Monat

🇺🇸 Vereinigte Staaten – Remote

💵 $175.000 - $245.000 / Jahr

⏰ Vollzeit

🟠 Senior

🧑‍💻 Full-Stack-Entwickler

🦅 H1B-Visum-Sponsor

Dieses Unternehmen hat in der Vergangenheit H1B-Visa gesponsert.

Jetzt Bewerben

Ähnliche Remote-Jobs finden

E-Mails für Remote-Jobs erhalten

📊 Überprüfen Sie Ihre Lebenslauf-Bewertung für diese Stelle

Verbessern Sie Ihre Chancen auf ein Vorstellungsgespräch, indem Sie Ihre Lebenslauf-Bewertung vor der Bewerbung überprüfen.

Lebenslauf hochladen

Smartsheet

WebsiteLinkedInAlle Stellen

1001 - 5000 Mitarbeiter

Gegründet 2005

☁️ SaaS

⚡ Produktivität

🤝 B2B

SaaS • Productivity • B2B

Smartsheet ist eine Plattform, die entwickelt wurde, um Projekte zu verwalten, Arbeitsabläufe zu automatisieren und Lösungen in großem Maßstab zu erstellen. Sie bietet eine breite Palette von Funktionen, darunter Automatisierung, Teamzusammenarbeit, Dashboards und Berichterstattung sowie Integrationen, die es Unternehmen ermöglichen, ihre Betriebsvorgänge zu optimieren. Die Plattform bedient verschiedene Anwendungsfälle wie Projektmanagement, IT-Portfoliomanagement, Marketingmanagement und mehr und bedient verschiedene Branchen, darunter Regierung, Finanzen und Gesundheitswesen. Smartsheet legt zudem großen Wert auf Sicherheit und Datenschutz, um die Datensicherheit der Nutzer zu gewährleisten. Darüber hinaus bietet sie professionelle Dienstleistungen wie Beratung, Schulung und Unterstützung bei der Implementierung, um das volle Potenzial der Plattform auszuschöpfen.

Beschreibung

• Own agent quality end-to-end: diagnosis, improvement, and validation across SmartAssist's orchestrator and subagents • Identify failure modes across quality dimensions factual accuracy, completeness, tone, actionability, and latency and prioritize what to fix • Drive quality improvements through prompt engineering, context engineering, and RAG retrieval tuning • Extend and mature our evaluation framework: scorers, golden datasets, regression gates, and online evaluation for production traffic • Close the feedback loop ensure that every change has a measurable, attributable quality signal • Collaborate with our Agent Architecture lead to distinguish quality problems that require prompt/context solutions from those that require structural fixes • Establish repeatable methodology that scales beyond any single agent or subagent

🎯 Anforderungen

• 8+ years of software engineering experience, with at least 2 years working directly with LLMs in production • Deep, hands-on experience with prompt engineering and context engineering, you understand how model behavior changes with framing, structure, and input design • Strong working knowledge of RAG architectures: chunking strategies, embedding models, retrieval evaluation, and failure diagnosis • Experience building or extending LLM evaluation frameworks, you have designed scorers, worked with golden datasets, and thought carefully about what good looks like • Fluency in agent system design, you don't need to own the architecture, but you can engage as a peer on architectural tradeoffs that affect quality • Strong Python skills; comfortable working in data-heavy environments (Databricks, Delta tables, or equivalent) • Ability to communicate complex quality findings (written and verbal) to both technical and non-technical stakeholders, you can explain what’s broke, why it matters, and what needs to happen next without losing the room • Strong cross-functional judgment, you know when to escalate, when to resolve independently, and how to build credibility across engineering, product, and AI platform teams • A bias for clarity in ambiguous situations, when failure modes are murky and trade-offs are real, you bring structure and a clear point of view rather than waiting for consensus • Legally eligible to work in the U.S. on an ongoing basis • BS or MS in Computer Science, a related field, or equivalent industry experience.

🏖️ Vorteile

• Employer subsidized medical/vision and dental coverage for full-time employees • 401k Match to help you save for your future (50% of your contribution up to the first 6% of your eligible pay) • Monthly stipend to support your work and productivity • Flexible Time Away Program, plus Sick Time Off • US employees are automatically covered under Smartsheet-sponsored life insurance, short-term, and long-term disability plans • US employees receive 12 paid holidays per year • Up to 24 weeks of Parental Leave • Personal paid Volunteer Day to support our community • Opportunities for professional growth and development including access to Udemy online courses • Company Funded Perks, including a counseling membership, local retail discounts, and your own personal Smartsheet account • Teleworking options from any registered location in the U.S. (role specific)