Prompt Engineer, LLM Systems, Evals, Safety

Job not on LinkedIn

October 5

Apply Now

Receive Emails with Similar Jobs

Supertech Innovation Labs

WebsiteLinkedInAll Job Openings

B2B • Technology • Consulting

Supertech Innovation Labs is a consulting and product venture company based in Saudi Arabia, focusing on innovative technology solutions. They are involved in various activities including job creation and investment fund management, aimed at enhancing the tech landscape in the region. Their commitment to innovation is evident in their slogan, 'Made with ❤️ in KSA.

11 - 50 employees

🤝 B2B

📋 Description

• Design high-quality prompts, system instructions, and tooling that make our LLM features accurate, safe, and cost-effective. • Own evaluation, prompt versioning, and continuous improvement. • Author, refactor, and chain prompts (system/tool/policy) for varied tasks. • Create offline/online evaluation harnesses (rubrics, golden sets, metrics). • Build prompt libraries with versioning, A/B testing, and telemetry. • Reduce hallucinations via verification, constrained decoding, and tool use. • Implement safety: jailbreak/prompt-injection tests, content policy checks, PII handling. • Partner with engineers to integrate prompts into production features.

🎯 Requirements

• Demonstrated prompt design across multiple task types and models. • Experience building eval datasets and automated scoring (e.g., accuracy, faithfulness, utility, cost/latency). • Familiarity with retrieval-augmented generation concepts and tool/function calling. • Strong scripting (Python/TypeScript) for data prep, evals, and analysis. • Clear writing; ability to translate business goals into measurable prompt specs. • Nice-to-Haves • Experience with LangChain/LLM orchestration, vector stores, and rerankers. • Knowledge of safety tooling and red-teaming techniques. • Experiment platforms (feature flags, A/B tests), analytics.