Python Inference Engineer

🔥 14 hours ago

🌐 Cyprus, Poland, +1 more countries – Remote

🇨🇾 Cyprus – Remote 🇵🇱 Poland – Remote 🇷🇸 Serbia – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🔙 Backend Engineer

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Gcore

WebsiteLinkedInAll Job Openings

201 - 500 employees

💼 Consulting

📦 Logistics

📣 Marketing

Consulting • Logistics • Marketing

Gcore is a global provider of cloud, edge, and AI solutions that accelerate AI training, deliver comprehensive cloud services, enhance content delivery, and protect servers and applications. With over 180 points of presence worldwide and a network capacity of 200+ Tbps, Gcore offers secure, flexible, and scalable infrastructure services. Its integrated offerings, including Edge Cloud, Edge Network, Edge Security, and AI Infrastructure, are designed to meet the needs of businesses looking to scale and control their global infrastructure efficiently. Gcore also provides robust DDoS protection and origin shielding to ensure uninterrupted online operations, making it a trusted partner for thousands of businesses worldwide.

📋 Description

• Build and improve the inference layer of the Gcore Inference platform • Integrate and operate inference frameworks such as vLLM, SGLang, NVIDIA Dynamo, and TensorRT-LLM • Bring new language and multimodal models into production • Improve inference latency, throughput, memory use, GPU utilization, and cost efficiency • Debug performance and reliability issues across model code, inference frameworks, GPU execution, networking, and Kubernetes • Work with platform, infrastructure, product, and customer-facing teams to turn inference improvements into reliable product features • Contribute improvements to open-source inference projects when appropriate

🎯 Requirements

• 5+ years of experience writing reliable, well-tested production code • Strong Python skills and experience designing production systems • Hands-on experience with PyTorch and deploying machine learning models • Experience with Linux, Docker, and Kubernetes • Experience in at least one relevant area: distributed systems, GPU computing, ML runtimes, model optimization, or cluster scheduling • Ability to debug complex problems across software, infrastructure, and hardware • Strong sense of developer experience • Genuine interest in inference engineering • Motivation to learn inference engines • Good communication and collaboration skills • Nice to have: Experience with vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM, or a similar inference framework • Nice to have: Experience running GPU workloads in production • Nice to have: Knowledge of inference techniques such as quantization, continuous batching, speculative decoding, prefix caching, chunked prefill, or LoRA serving • Nice to have: Experience with CUDA, Triton, TensorRT, or other GPU programming tools • Nice to have: Experience profiling and improving model latency, throughput, memory use, or GPU utilization • Nice to have: Experience with distributed inference, multi-GPU systems, scheduling, or autoscaling • Nice to have: Contributions to open-source ML, inference, or infrastructure projects

🏖️ Benefits

• Competitive compensation • Flexible working hours and hybrid or remote options, depending on your role • Work from anywhere in the world for up to 45 days per year • Private medical insurance for you and your family* • Extra paid vacation and sick leave days* • Support for life’s important moments and celebrations • Language courses to help you connect and grow • Modern, welcoming offices with snacks, drinks, and entertainment* • Team sports and social activities*