Software Engineer – AI Inference Engine Engineer - FriendliAI : Job Details

Software Engineer – AI Inference Engine EngineerSoftware Engineer – AI Inference Engine Engineer

Get AI-powered advice on this job and more exclusive features.

FriendliAI, a Redwood City, CA-based startup, is building the next-generation AI inference platform that accelerates the deployment of large language and multimodal models with unmatched performance and efficiency. Our infrastructure supports high-throughput, low-latency AI workloads for organizations worldwide. We are also integrated with the Hugging Face platform, allowing instant access to over 400,000 open-source models. We are on a mission to deliver the world's best platform for generative and agentic AI.

The Role

We are seeking a highly technical Inference Engine Engineer to optimize the performance and efficiency of our core inference engine. You will focus on designing, implementing, and optimizing GPU kernels and supporting infrastructure for next-generation generative and agentic AI workloads. Your work will directly power the most latency-critical and compute-intensive systems deployed by our customers.

The Person

You are an exceptional engineer with a strong foundation in GPU programming and compiler infrastructure. You enjoy pushing the performance boundaries and have experience supporting production-scale machine learning applications.

Key Responsibilities

• Design and optimize custom GPU kernels for AI (e.g., transformer and diffusion) workloads
• Contribute to the development of FriendliAI's kernel compiler, memory planner, runtime, and other core components.
• Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performance
• Analyze performance bottlenecks across the software and hardware stack, and implement targeted optimizations
• Drive support for new model architectures and tensor compute patterns
• Maintain production-grade performance infrastructure, including profiling, benchmarking, and validation tools

Qualifications

• 5+ years of experience in production or high-impact research environments
• Production-level expertise in Python and C++
• Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
• Experience developing machine learning frameworks or performance-critical runtime systems
• Hands-on experience writing and optimizing GPU kernels
• Hands-on experience profiling GPU kernels
• Experience working with generative AI models such as transformer and diffusion models

Preferred Experience

• Experience developing machine learning compilers or code generation systems
• Familiarity with dynamic shape compilation, memory planning, and kernel fusion
• Contributions to inference engines, compilers, or high-performance numerical libraries
• Understanding of multi-GPU and distributed inference strategies
• Daily lunch and dinner provided
• Unlimited snacks and beverages
• Supportive work environment

Seniority level

• Seniority levelMid-Senior level

Employment type

• Employment typeFull-time

Job function

• Job functionEngineering and Information Technology
• IndustriesSoftware Development

Referrals increase your chances of interviewing at FriendliAI by 2x

Sign in to set job alerts for “Software Engineer” roles.

New York, NY $128,000.00-$160,000.00 1 week ago

New York, NY $220,000.00-$260,000.00 1 week ago

New York, NY $130,000.00-$180,000.00 5 months ago

New York, NY $120,000.00-$180,000.00 4 months ago

New York, NY $140,000.00-$140,000.00 3 weeks ago

Want to work with us, but don't see the right job listed?Full Stack Software Engineer (All Levels)

New York, NY $235,000.00-$255,000.00 1 week ago

New York, NY $163,200.00-$223,200.00 1 week ago

New York, NY $99,500.00-$200,000.00 23 hours ago

New York, NY $110,000.00-$150,000.00 2 weeks ago

New York, NY $150,000.00-$283,000.00 2 weeks ago

New York, NY $90,000.00-$120,000.00 2 months ago

New York, NY $176,000.00-$250,000.00 2 weeks ago

New York, NY $130,000.00-$240,000.00 1 week ago

New York, NY $140,000.00-$170,000.00 2 months ago

New York, NY $100,500.00-$173,250.00 2 weeks ago

New York, NY $99,500.00-$200,000.00 23 hours ago

New York, NY $140,000.00-$200,000.00 2 weeks ago

Software Engineer - Frontend / Fullstack

New York, NY $140,000.00-$200,000.00 2 weeks ago

Don't see the right opportunity? Apply here!

New York, NY $165,000.00-$165,000.00 1 year ago

Backend Software Engineer, CloudKitchens - New York City

New York, NY $145,000.00-$260,000.00 8 months ago

New York, NY $100,000.00-$200,000.00 5 months ago

Software Engineer (Fullstack) - Payments

New York, NY $163,200.00-$223,200.00 1 week ago

New York, NY $130,000.00-$250,000.00 1 week ago

New York, NY $140,000.00-$185,000.00 1 week ago

New York, NY $140,000.00-$180,000.00 1 month ago

New York, NY $140,000.00-$180,000.00 1 month ago

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr