About the job
About Anuvaya
At Anuvaya, we believe that conversational AI will revolutionize professional services in India, starting with astrology. Our dedicated team of engineers, designers, and product experts is committed to merging the capabilities of conversational AI with deep domain knowledge. Crafting an AI that communicates with human-like nuances and possesses expert-level domain knowledge is a challenging yet rewarding endeavor, and we are tackling both aspects simultaneously.
Supported by esteemed investors such as Accel, Arkam Ventures, and Weekend Fund, we are on an exciting journey to redefine astrology consultations.
The Role
As a member of our team, you will play a crucial role in Vaya, our consumer-focused astrology product. Vaya is not just an app that generates horoscopes or asks for your zodiac sign; it offers profound, personalized Vedic astrology consultations—covering birth charts, dasha analysis, transit readings, compatibility assessments, and muhurat timings—through engaging conversations.
Your challenge will be to ensure the quality of our AI agent's responses meets the high standards expected by users accustomed to traditional consultations with experienced pandits. This requires a discerning eye for quality and the ability to instill trust in first-time users who may have important questions about their careers or relationships.
In this role, you will collaborate closely with our engineering team to define what quality means for our AI agent. This involves creating evaluation rubrics, identifying key metrics, engaging in constant dialogue with users, and bridging the gap between our agent's output and user expectations. You will be our expert on both the domain of astrology and user needs, guiding us on where improvements are required.
What You'll Do
- Engage with users frequently to gain insights into their motivations, frustrations, and the disparities between their expectations and our agent's offerings.
- Create evaluation rubrics for assessing the quality of the agent's responses, distinguishing between a comprehensive birth chart reading and a superficial one, or a trustworthy compatibility analysis versus a generic response.
- Work alongside the engineering team to establish and monitor the metrics that truly matter—those that reveal if the agent is genuinely improving.
- Develop a thorough understanding of LLMs (Large Language Models) and their limitations to determine whether a subpar response stems from prompt issues, contextual failings, or model constraints.
- Influence the conversational style of the agent—determining its tone, depth of engagement, and sensitivity in discussions around delicate astrological topics.
- Translate user feedback into actionable improvements, transforming vague comments like 'this reading felt off' into specific, testable changes.

