About the job
# About the Team
- This position joins the team at Toss Commerce, which is developing a system for real-time voice interactions with customers using AI.
- Unlike text chatbots, this involves actual phone conversations where customers may interrupt, change context suddenly, or leave pauses. The AI must manage these situations seamlessly.
- We apply cutting-edge AI technologies such as LLM, Multi-Agent systems, Automatic Prompt Optimization (APO), and LLM-as-Judge in the demanding environment of live customer calls, validating our approaches with real data every day.
- Real-time voice interactions combined with LLM are still uncharted territory in the industry. Our team goes beyond just reading papers; we create solutions directly from actual call data.
# Responsibilities
- Design and implement core features for a voice-based AI agent, including managing conversation flow, state management, tool calling, and safety measures.
- Integrate Speech-to-Text (STT) and Text-to-Speech (TTS) to create a production-level voice user experience (UX) aimed at low-latency, high-quality interactions.
- Develop and manage experimental routines that enhance the overall quality of the agent, focusing on prompts, context configuration, LLM as a Judge, and offline/online experiments.
- Explore innovative approaches to challenges that are difficult to solve with traditional methods using large language models (LLM), Retrieval-Augmented Generation (RAG), and multimodal models.
- Investigate possibilities for applying AI in areas that have not yet been clearly defined, in addition to problem-solving.
- Design solutions that consider not only technical sophistication but also business applicability and sustainability.
# Desired Candidate
- Experience in solving complex problems using advanced AI technologies such as Multi-Agent systems, LLM, RAG, and multimodal models is essential.
- Proficiency in integrating diverse data types (text, images, structured data) to design and experiment with models is required.
- A keen interest and ability to explore and technically define new problems is preferred.
- Experience leading the complete process from problem definition to model design, experimentation, and quantitative evaluation is advantageous. We seek someone who can systematically enhance quality based on datasets, metrics, and guardrails.
- Familiarity with the latest AI ecosystem tools such as PyTorch, Hugging Face Transformers, and LangChain is necessary.
- Experience designing with consideration for not just technical experiments but also service applicability and scalability is a plus.
- Understanding trade-offs between quality, latency, and cost, with demonstrated ability to make context-appropriate choices and optimizations, is highly valued.
# Resume Tips
- Rather than simply listing modeling techniques, it's more impactful to showcase specific improvements and their effects.
- Highlight experiences where you achieved meaningful results through experiments and iterative improvements in varying data conditions or constraints.
- It would be beneficial to illustrate your role in collaborative processes and how you contributed to problem-solving beyond technical aspects.
# Joining Toss
- Application submission > Job interview > Cultural fit interview > Reference check > Compensation negotiation > Final acceptance
- The job interview will include in-depth technical discussions and ML system design.
# A Note to Potential Colleagues
> “Solving complex problems with AI is just the beginning.
What we truly focus on is re-examining the problems themselves and transforming them for the better.”
> - It's not just about feeding clean data into models; it's more about contemplating how to technically resolve issues that are yet to be defined.
> - We begin every time with, “Do we really need to solve this with modeling?” and engage in a process of re-evaluating what the most impactful approach might be.
> - We can design a new flow centered around AI, rather than just attaching AI to tasks previously handled manually by service or operations teams.