TL;DR

I recently finished a project from Udacity's Future AWS Agent Engineer Nanodegree Program, which I was able to take through the AWS AI & ML Scholarship.

I built a customer support agent using Amazon Bedrock AgentCore, AgentCore Gateway, AWS Lambda, DynamoDB, and an FAQ. The agent had to understand whether a customer was reporting a bug, asking a question that could be answered from the FAQ, or asking for something that needed human support.

I finished it in about two days and passed on my first attempt with a correctness score of 0.83. I did not have enough time to go back and refine it or run another evaluation round because there was a lot happening in my life at the time.

The part I learned the most from was actually one of my evaluation failures. For a bug report, I had told the agent that it needed 3 things before creating a ticket: a description of the problem, steps to reproduce it, and the environment where it happened. But in 2 test cases, the model understood what the customer was talking about and created the ticket even though one of those required pieces of information was missing.