Software Engineer, GPU Inference

About the role

Soniox is building a real‑time speech AI platform and needs a Software Engineer to scale its cutting‑edge language models on a low‑latency, high‑throughput inference stack. You will work at the intersection of deep learning, systems engineering, and performance optimisation, squeezing every FLOP out of GPUs and delivering millisecond‑level latency at global scale.

Key responsibilities

• Collaborate with researchers, engineers, and product teams to bring state‑of‑the‑art AI models into production.
• Architect and optimise inference infrastructure for low‑latency, high‑reliability performance across thousands of concurrent requests.
• Identify and eliminate system bottlenecks, improving throughput and GPU utilisation across the fleet.
• Introduce and implement monitoring, debugging, and performance‑improvement tools for model inference at scale.
• Tune the VM fleet to maximise compute, memory and network efficiency down to the last GPU cycle.
• Build robust, scalable systems that support rapid experimentation for advanced research workflows.

Required profile

• Strong intuition for optimising modern ML architectures for inference performance.
• Deep familiarity with PyTorch, CUDA, NCCL and GPU internals, or a rapid learning ability.
• Understanding of HPC fundamentals and experience with technologies such as InfiniBand, NVLink or MPI.
• Proven experience building and scaling distributed, performance‑critical systems in production.
• History of refactoring or rebuilding systems to handle 10x+ scale increases.
• Self‑starter who thrives in fast‑moving environments and takes ownership from design to deployment.
• Commitment to reliability, simplicity and performance.
• At least 5 years of professional software engineering experience.

Required skills

• PyTorch
• CUDA
• NCCL
• GPU internals
• HPC fundamentals
• InfiniBand
• NVLink
• MPI
• Distributed systems