Software Engineer – LLM Applications, Python, Java

🕒 October 29

Where this job comes from

Posted on FICO's careers page ↗FICO's careers page, not through a job board

First seen by Remote Rocketship October 29

Still open on FICO's site. Checked 34 minutes ago.

Apply Now
Logo of FICO

FICO

1001 - 5000 employees

Founded 1956

💼 Consulting

🛡️ Insurance

🏥 Healthcare

Consulting • Insurance • Healthcare

FICO is a leading analytics and software company renowned for its FICO® Score, a tool widely used by lenders to assess credit risk. The company offers a comprehensive platform that leverages data, AI, and machine learning to power intelligent decision-making and customer engagement across various industries. FICO's solutions span fraud detection, credit scoring, and customer lifecycle management, making it vital to sectors such as finance and telecommunications. Its innovative products help businesses optimize outcomes through real-time analytics, business composability, and scenario management.

📋 Description

• Own the system prompts, tool descriptions, and evaluation prompts behind Personalized Credit Insights and the AI chat assistant • Improve prompts using production evidence • Build and grow the LLM evaluation framework • Turn production failures into test cases • Run regression suites before prompt or model changes and report results • Measure and improve LLM-based evaluator reliability against human-labeled data • Design, tune, and monitor guardrails, including classifiers, PII redaction, prompt-injection defenses, and financial-advice compliance rules • Lead model upgrades by comparing quality, cost, and latency and shipping safely through A/B tests • Reduce cost and latency with prompt caching, model routing, and context management • Build tools for Conversation Analysts and QA engineers, including transcript viewers, labeling workflows, and dashboards • Partner with Analytics, Legal, and Compliance on fairness and bias evaluation • Leverage AI coding assistants and agentic workflows while reviewing and refining their output

🎯 Requirements

• Bachelor's or Master's in Computer Science, Engineering, or a related field • Demonstrated professional software engineering experience • Strong Python skills • Ability to read and contribute to Java (Spring Boot) microservices • Proven experience building and operating LLM-powered applications in production, including prompt design, tool calling, structured outputs, and streaming • Experience building evaluations for LLM systems, including test sets, regression suites, LLM-as-judge approaches, and measuring evaluator quality against human labels • Working knowledge of Amazon Bedrock or a comparable managed LLM platform • Experience with AWS services such as S3, Lambda, and DynamoDB • Proficient SQL • Comfort analyzing production data in a data warehouse such as Amazon Redshift • Production ownership including code review, CI/CD, release management, and on-call support • Experience working within Agile/Scrum teams • Strong written and verbal communication skills • Ability to explain evaluation results and tradeoffs to product, legal, and executive stakeholders • Experience with guardrail tooling, LLM security testing, RAG and retrieval evaluation, fairness and bias evaluation, model migrations, Datadog, Tableau, or regulated domains is a plus

🏖️ Benefits

• Inclusive culture reflecting core values • Professional development and valuable learning experiences • Competitive compensation, benefits and rewards programs • Work/life balance • Employee resource groups • Social events • Support to develop skills and grow your career

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 October 29

Zenlayer

201 - 500

💼 Consulting

📦 Logistics

📣 Marketing

Staff Software Engineer leading LLM inference optimization for Zenlayer’s distributed edge-cloud platform. Improving GPU goodput, token costs, kernels, KV transfer, MoE parallelism, and quantization.

🕒 October 28

Miratech

501 - 1000

🤝 B2B

💼 Consulting

☁️ SaaS

Quality Assurance Engineer validating conversational AI, IVR, and voice customer experience solutions for global IT services company Miratech. Automating tests and ensuring reliable integrations across enterprise platforms.

🕒 October 28

Miratech

501 - 1000

🤝 B2B

💼 Consulting

☁️ SaaS

Quality Assurance Engineer validating conversational AI, IVR, and voice customer experiences at global IT services company Miratech. Creating automated tests and verifying integrations, workflows, and end-to-end quality.

🕒 October 10

TechOp Solutions International

51 - 200

💼 Consulting

🎖️ Defense

📦 Logistics

AI/LLM Developer building and deploying LLM-enabled applications for TechOp Solutions International. Supporting AI pilots from use-case definition through operational implementation.

🕒 September 29

Century Interactive

51 - 200

📣 Marketing

Senior Software Engineer building real-time conversational voice AI for automotive dealerships. Owning AWS infrastructure, telephony integrations, observability, and scalable production systems.

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok