Featherless.ai logoFeatherless.ai logo

Senior Software Engineer - API Gateway

Featherless.aiRemote (US & Canada)
Remote Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Experience Level

Senior

Qualifications

QualificationsHands-on experience with the end users for whom we are developing solutions (familiarity with popular open LLMs and experience building with them)Proficiency with web technologies and paradigms (REST, websockets, DNS, networking, OpenTelemetry)Experience with significant components of our technology stack (Kubernetes, Node.js, MikroORM, Fastify, Redis, MongoDB, Python, Elastic Cloud, Cloudflare, Sentry, OpenTelemetry)Ability to troubleshoot complex issues across a broad tech stack and implement necessary instrumentationA collaborative spirit and a desire to work as part of a talented team

About the job

About the Role

Featherless.ai is pioneering the most dependable and extensive open-model inference platform, serving as the backbone for the next generation of AI innovators, startups, and enterprises. Our serverless inference approach maximizes GPU utilization for AI infrastructure.

We are seeking Senior Software Engineers to enhance and maintain our API gateway that supports our inference cloud, which is crucial for:

  • Authentication and inference across all models

  • Managing subscriptions and entitlements (e.g., context-length, concurrency limits)

  • Providing essential API surfaces for applications and developers

The API Gateway is continually evolving to meet the demands presented by an influx of new models, modalities, clients, and inference loads.

Key Responsibilities

As a vital member of the Platform Team, which strives to make Featherless the premier destination for discovering and utilizing models, you will:

  • Develop new features and fix bugs to accommodate client needs, address user issues, and onboard new models

  • Enhance the reliability of the existing API through improved instrumentation and monitoring, and optimizing infrastructure

  • Respond to availability incidents promptly

  • Triage and resolve issues related to inference quality and reliability

  • Manage the infrastructure that supports our API gateway

Qualifications

  • Hands-on experience with the end users for whom we are developing solutions (familiarity with popular open LLMs and experience building with them)

  • Proficiency with web technologies and paradigms (REST, websockets, DNS, networking, OpenTelemetry)

  • Experience with significant components of our technology stack (Kubernetes, Node.js, MikroORM, Fastify, Redis, MongoDB, Python, Elastic Cloud, Cloudflare, Sentry, OpenTelemetry)

  • Ability to troubleshoot complex issues across a broad tech stack and implement necessary instrumentation

  • A collaborative spirit and a desire to work as part of a talented team

Alignment with our team and company values is essential.

About Featherless.ai

Featherless.ai is at the forefront of developing the most reliable open-model inference platform, essential for AI startups and enterprises. Our innovative serverless infrastructure maximizes efficiency and GPU utilization, fostering creativity and growth in the AI landscape.

Similar jobs

Browse all companies, explore by city & role, or SEO search pages.

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.