About the job
About the Role
Featherless.ai is pioneering the most dependable and extensive open-model inference platform, serving as the backbone for the next generation of AI innovators, startups, and enterprises. Our serverless inference approach maximizes GPU utilization for AI infrastructure.
We are seeking Senior Software Engineers to enhance and maintain our API gateway that supports our inference cloud, which is crucial for:
Authentication and inference across all models
Managing subscriptions and entitlements (e.g., context-length, concurrency limits)
Providing essential API surfaces for applications and developers
The API Gateway is continually evolving to meet the demands presented by an influx of new models, modalities, clients, and inference loads.
Key Responsibilities
As a vital member of the Platform Team, which strives to make Featherless the premier destination for discovering and utilizing models, you will:
Develop new features and fix bugs to accommodate client needs, address user issues, and onboard new models
Enhance the reliability of the existing API through improved instrumentation and monitoring, and optimizing infrastructure
Respond to availability incidents promptly
Triage and resolve issues related to inference quality and reliability
Manage the infrastructure that supports our API gateway
Qualifications
Hands-on experience with the end users for whom we are developing solutions (familiarity with popular open LLMs and experience building with them)
Proficiency with web technologies and paradigms (REST, websockets, DNS, networking, OpenTelemetry)
Experience with significant components of our technology stack (Kubernetes, Node.js, MikroORM, Fastify, Redis, MongoDB, Python, Elastic Cloud, Cloudflare, Sentry, OpenTelemetry)
Ability to troubleshoot complex issues across a broad tech stack and implement necessary instrumentation
A collaborative spirit and a desire to work as part of a talented team
Alignment with our team and company values is essential.
