Introducing AI Agents

Simon Guest

Before We Get Started

  • How many have used ChatGPT/Claude?
  • How many have built their own chatbot and/or used the OpenAI/Claude APIs?
  • How many have created something beyond this?

Chatbots are not that Smart

There are many limitations…

  • Needs constant human input every turn; No ability to plan beyond a single interaction
  • Single model with single context
  • No interaction with external systems

Introducing AI Agents!

What is an Agent?

Source: https://www.youtube.com/watch?v=bwXaJXgezf4

What is an Agent?

Source: https://www.weforum.org/stories/2025/06/cognitive-enterprise-agentic-business-revolution/

What is an Agent?

Source: https://www.crn.com/news/ai/2025/10-hottest-agentic-ai-tools-and-agents-of-2025-so-far

What is an Agent?

Source: https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027

What is an Agent?

Imagine a DigiPen Campus Assistant: An agent that can help you navigate anything and everything at DigiPen!

  • “Where can I find the ‘Hopper’ room?”
  • “Can you tell me more about FLM201?”
  • “Oh, and what’s today’s vegetarian option at the Bytes Cafe?”

Five Characteristics of Agents

  1. Agents are Planners
  • Agents are driven by goals
  • And they can put together a plan for the steps to complete that goal.
    • “First, I will discover where course information is located”
    • “Then I will search for any courses that reference FLM201”
    • “Then I summarize all of the key points for the student”

Five Characteristics of Agents

  1. Agents are Autonomous
  • Agents can then go off and execute the plan, independent of human input
  • The concept of “human in the loop” still applies for confirmation
    • e.g. “Do you really want to place this order at the Bytes Cafe?”

Five Characteristics of Agents

  1. Agents are Reactive
  • Agents can change mid-course depending on what they find and/or the environment.
    • e.g. “I couldn’t find any course information on FLM201. I’m going to whether there are other 200-level FLM courses before responding to the student.”

Five Characteristics of Agents

  1. Agents have Persistence
  • Agents often have memory systems beyond the current conversation
  • Broadly classified as short and long term memory
    • Short term memory could be the options at the Bytes Cafe
    • Long term memory could be your food preferences

Five Characteristics of Agents

  1. Agents are Interactive
  • Agents can delegate to other agents for complex tasks
    • (Or for tasks where other agents are better suited for.)
    • e.g., Campus Agent -> delegating to a Course Agent
  • Agents can also be given access to external tools
    • e.g., File search, Web search, access to the Bytes Cafe API

Let’s Try This!

Let’s Try This!

https://simonguest-campus-agent.hf.space/

Let’s Try This!

https://simonguest-campus-agent.hf.space/

Observations and Questions

Get into groups of 2 or 3

  • Q: What worked? What surprised you?
  • Q: What didn’t work? Where did the agent fail?
  • Q: What other examples of agents can you think of?

Other Examples for Agents

  • Customer service agent
  • Travel booking agent
  • Research assistant
  • Code generation agent (very popular right now)
  • Agents within games
    • Traditional NPCs are pre-written dialogue trees
    • Whereas agents can be more independent within the game environment

How Does Our Campus Assistant Work?

Gradio-Based UI

Source: https://www.gradio.app/

Gradio-Based UI

import time
import gradio as gr

def slow_echo(message, history):
    for i in range(len(message)):
        time.sleep(0.05)
        yield "You typed: " + message[: i + 1]

demo = gr.ChatInterface(
    slow_echo,
    flagging_mode="manual",
    flagging_options=["Like", "Spam", "Inappropriate", "Other"],
    save_history=True,
)

if __name__ == "__main__":
    demo.launch()

Gradio-Based UI

Source: https://gradio.app

Hosted on Hugging Face Spaces

Source: https://huggingface.co/spaces

Creating the Agents

Agent Structure

Agents and tools for the DigiPen Campus Assistant

So Many Agent Frameworks…

Source: https://e2b.dev

OpenAI Agents SDK

  • Used for our demo
  • https://openai.github.io/openai-agents-python/
  • Python and JavaScript/TypeScript
  • MIT License
  • Works with models that support OpenAI’s Responses API

LangGraph

  • https://langchain-ai.github.io/langgraph/
  • Python only
  • MIT License

Crew.ai

  • https://github.com/crewaiinc/crewai
  • Python only
  • MIT License

Microsoft

  • AutoGen
    • https://microsoft.github.io/autogen/stable/
    • Python (.NET coming soon)
    • MIT License
  • Microsoft Semantic Kernel
    • https://github.com/microsoft/semantic-kernel
    • Python, .NET, Java
    • MIT License

What Should You Be Considering?

  • Question: If most agents run in the browser, why are many of the agent frameworks written in Python? 🤔

What Should You Be Considering?

  • Where will your agent run?
    • Server or client? On the web? In game?
    • This will likely determine the language
    • How will you manage API keys?
  • Support
    • Will this agent framework be around in 3-5 years?
    • Is there a cost/hosting component to it?
  • Easy vs. Simple
    • Is your choice “easy” or “simple”?

OpenAI Agents SDK (Python)

Creating and running a new agent

from agents import Agent, Runner

agent = Agent(
    name="DigiPen Campus Assistant",
    instructions="You are a helpful campus assistant that can plan and execute tasks for students at DigiPen. Please be concise and accurate in handing off tasks to other agents as needed.",
    handoffs=[building_agent, course_agent, handbook_agent, cafe_agent],
)

messages.append({"role": "user", "content": user_msg})
result = Runner.run_streamed(agent, messages)

Multiple Agents

OpenAI Agents SDK (Python)

Specifying handoffs

from agents import Agent, Runner

agent = Agent(
    name="DigiPen Campus Assistant",
    instructions="You are a helpful campus assistant that can plan and execute tasks for students at DigiPen. Please be concise and accurate in handing off tasks to other agents as needed.",
    handoffs=[building_agent, course_agent, handbook_agent, cafe_agent],
)

messages.append({"role": "user", "content": user_msg})
result = Runner.run_streamed(agent, messages)

Why Multiple Agents?

  • Context window limitations
  • Each agent can have a different system prompt (instructions)
  • Each agent can have a different underlying model
    • Specialized models (e.g., a vision encoder)
    • Or to blend cost
  • Makes tool separation cleaner and more accurate

Example of Multiple Agents

  • Code generation
    • Agents for ‘architect’, code writer, tester, debugger, etc.
  • Content generation
    • Agent to create content, other agents to generate images, translate content, etc.
  • Travel booking
    • Agent to book flights, hotels, cars, etc. for packages

Patterns for Agents

  • As you get deeper into building agents, patterns start to emerge
    • e.g., router (which is what we used in our demo), orchestrator (using other agents as tools), parallel agents

Patterns for Agents

Source: https://www.anthropic.com/engineering/building-effective-agents

Giving Agents Access to Tools

Why Do Agents Need Tools?

  • The scope of the agents ability is contained within the model
  • Tools enable the agent to reach out to systems beyond the model
  • Examples
    • Read a file from disk or search the web (built in)
    • Calculator (because LLMs aren’t great at math)
    • Code interpreter (running code on the fly)
    • What potential tools can you think of? 🤔

OpenAI Tool Calling

  • Introduced by OpenAI in June 2023
  • Originally called Function Calling
  • Models are fine-tuned to return a structured function_call JSON object, specifying which function to call and with what arguments.
  • Tools are provided as functions
  • Option for the LLM to decide when to call the tool (always, never, auto)

OpenAI Tool Calling

from agents import Agent, function_tool

@function_tool
def get_bytes_cafe_menu(date: str) -> any:
    """Returns the menu for the Bytes Cafe for the date provided."""
    return { f"{date}": {
            "daily byte": {
                "name": "Steak Quesadilla", "price": 12, "description": "Flank steak, mixed cheese in a flour tortilla served with air fried potatoes, sour cream and salsa",
            },
        } }

cafe_agent = Agent(
    name="Cafe Agent",
    instructions="You help students locate and provide information about the Bytes Cafe.",
    tools=[
        get_bytes_cafe_menu,
    ])

OpenAI Tool Calling

from agents import Agent, FileSearchTool

VECTOR_STORE_ID = "vs_6896d8c959008191981d645850b42313"

building_agent = Agent(
    name="Building Agent",
    instructions="You help students locate and provide information about buildings and rooms on campus. Be descriptive when giving locations.",
    tools=[
        FileSearchTool(
            max_num_results=3,
            vector_store_ids=[VECTOR_STORE_ID],
            include_search_results=True,
        )
    ],
)

MCP (Model Context Protocol)

  • OpenAI’s tool/function calling works if you have local access to the tools
  • But how about external systems?
    • e.g., Access to calendar, other systems, hardware, etc.
  • You could build local tools to call APIs
  • MCP: Model Context Protocol
  • Standardized tooling interface, released by Anthropic in March 2024
  • More details: https://modelcontextprototcol.io

What Next?

Using an Agent for Your Project

  • Could your project benefit from agents?
  • One or more agents?
  • How would agents interact with the user?
  • What might be some of the challenges to overcome?

Resources (1/2)

  • This slide deck, resources, links, everything:
    • [https://simonguest.github.io/CSP(https://simonguest.github.io/CSP){.external target=“_blank”}

Resources (2/2)

Q&A