Senior AI Compute Infrastructure Engineer

About the role

As a Senior AI Compute Infrastructure Engineer you will join Kraken’s dedicated AI Compute and Infrastructure team. You will be responsible for designing, operating, and scaling the GPU and accelerator clusters that power model training, inference, evaluation, and experimentation across the exchange.

Key responsibilities

• Own and operate GPU/accelerator clusters, including drivers, runtimes, kernels, device plugins, and node configuration.
• Design scheduling, orchestration, placement, quota management, and utilization systems for heterogeneous accelerator environments.
• Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost using frameworks such as vLLM, Triton Inference Server, or TensorRT.
• Partner with ML engineers and researchers to remove bottlenecks in training, batch and online inference, deployment, and production debugging.
• Build observability for GPU utilization, memory pressure, queue depth, token throughput, request latency, failed workloads, capacity pressure, and spend.
• Drive reliability through incident response, alerting, runbooks, and post‑incident improvements for always‑on AI compute.

Required profile

• Several years of experience operating large‑scale GPU or accelerator infrastructure.
• Proven ability to work with AI/ML researchers, platform engineers, security, and product teams.
• Strong problem‑solving mindset with focus on performance, cost efficiency, and reliability.

Required skills

• GPU and accelerator cluster management
• Drivers, runtimes, kernels, device plugins
• Scheduling primitives, workload isolation, quota management
• Orchestration and placement systems for heterogeneous hardware
• Inference optimization using vLLM, Triton Inference Server, TensorRT
• Observability tooling for GPU utilization, memory pressure, queue depth, token throughput, latency, and cost metrics