AI Agent Performance Manager

Company name : Willware Technologies

Role : AI Agent Performance Manager

Experience : 7+ Years

Location : Pune

JD

Mandatory Skills 5-7 years of experience in AI/ML, Generative AI, or AI Engineering. Strong experience with AI Agents / Agentic AI. Hands-on experience with LLMs (GPT, Gemini, Claude, Llama, etc.) and Generative AI. Proficiency in Python. Experience with Google Cloud Platform (GCP). Experience in Machine Learning model deployment and production environments. Strong knowledge of AI testing, validation, and performance evaluation. Experience defining AI performance metrics, KPIs, and operational guardrails. Experience with production monitoring, issue analysis, troubleshooting, and AI performance optimization. Understanding of MLOps/LLMOps, CI/CD, and AI model lifecycle management. Excellent communication and stakeholder management skills.

Key Responsibilities Define and monitor AI agent performance metrics and operational KPIs. Design and execute testing and validation strategies for AI agents. Deploy AI agents and AI models on GCP. Monitor production performance, troubleshoot issues, and optimize AI systems. Ensure operational readiness and production stability before releases. Collaborate with product, engineering, and business teams to deliver reliable AI solutions. Drive continuous improvement in AI agent quality, scalability, and reliability.

Preferred Skills LangChain, LangGraph, CrewAI, AutoGen, or similar AI agent frameworks. Experience with Prompt Engineering, RAG, Vector Databases, and AI Observability. Experience in cloud-native AI deployments and monitoring. .