Member of Technical Staff
True Talent Solutions Partners · San Francisco · onsite
ABOUT THE ROLE
Join a small AI infrastructure team building efficient inference systems for open-source language models. You will improve performance across diverse hardware, helping make production AI serving faster and more cost-effective.
WHAT YOU'LL DO
- Bring new open-source models into production with tuning for latency and throughput.
- Optimize inference serving, including batching, KV caching, speculative decoding, and quantization.
- Write and tune GPU kernels using CUDA, HIP, and Triton across varied accelerator platforms.
- Design, deploy, and operate heterogeneous compute clusters across infrastructure vendors.
- Run production inference systems, improving reliability, observability, and cost efficiency.
- Work with customers to understand needs and help solve infrastructure challenges.
WHAT WE'RE LOOKING FOR
- Typically 1 to 6 years of experience, with relevant early-career candidates also considered.
- Strong systems or infrastructure engineering experience, in addition to any application or full-stack work.
- Understanding of computer architecture, memory hierarchy, and operating system internals.
- Experience with Python and familiarity with CUDA, HIP, Triton, or other GPU programming tools.
- Knowledge of inference optimization techniques such as quantization, batching, KV caching, or speculative decoding.
- Experience deploying or operating infrastructure at scale, such as clusters or large GPU fleets, is valuable.
- Evidence of taking on challenging systems problems, learning quickly, and using AI coding tools to improve workflows.
- Comfort communicating with customers and working across a broad range of technical responsibilities.
COMPENSATION & BENEFITS
Base salary is $200,000 to $300,000 USD annually. Visa sponsorship is available.
LOCATION
On-site in San Francisco, California, five days per week.