Software Engineer - GPU Fabric Observability

Навыки

• Распределённые системы
• Kubernetes
• Machine Learning
• Notion
• Мониторинг и observability
• Ответственность за результат
• Маршрутизация и NAT

О компании и продукте

• Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to to ship AI products.
• Baseten is building its own GPU infrastructure for large-scale inference. As we move into large scale, high-density NVIDIA systems, the hardest failures are intermittent, cross-layer, and difficult to prove: RoCE congestion, InfiniBand stalls, ECN/DCQCN mis-tuning, bad optics, RNIC issues, host kernel stalls, GPU driver problems, and workload symptoms that look like network problems, but are not.
• We are hiring a Lead Software Engineer to build a first-class observability and root-cause analysis system for GPU fabrics. This is a hard distributed systems problem, not a dashboarding problem. The system will collect high-volume signals from switches, hosts, active probes, and inference services; reduce and correlate them in real time; understand topology and service ownership; and produce actionable diagnosis while an incident is still unfolding.

Задачи

• Own Baseten’s GPU fabric observability and root-cause analysis architecture
• Build telemetry pipelines across switches, NICs, hosts, GPUs, Kubernetes, and inference services
• Model topology, flow paths, service ownership, and failure domains
• Separate true fabric faults from host, NIC, GPU, kernel, driver, RDMA, scheduler, and workload failures
• Create clear operator workflows for triage, remediation, and post-incident learning

Требования

• Staff-level or senior staff-level experience building production infrastructure software
• Strong distributed systems background, especially streaming systems, telemetry pipelines, diagnostics, or control-plane software
• Experience building systems that process high-volume, high-cardinality, noisy operational data
• Understanding of networking fundamentals and high-performance networks
• Ability to work with low-level infrastructure signals and build practical correlation, anomaly detection, or root-cause analysis systems

Условия

• Competitive compensation, including meaningful equity
• 100% coverage of medical, dental, and vision insurance for employee and dependents
• Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
• Paid parental leave
• Fertility and family-building stipend through Carrot
• Company-facilitated 401(k)
• Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities
• If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you
• At Baseten, we are committed to fostering a diverse and inclusive workplace
• We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status
• We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable)

Паспорт вакансии

История публикации

Откуда что взялось

Отмечено то, что вывели мы. Без пометки — значение назвал работодатель.

Почему на этом месте в выдаче

Порядок выдачи объявлен контрактом: свежесть решает между днями, полнота и зарплата — внутри дня.