Welcome to CS-394/594!

Simon Guest

Introductions

Course Overview

Course Description

  • How Generative AI Works focuses on the practical implementation of generative AI within custom software applications and games.
  • The course covers neural network architectures, including the impact of the Transformer model, customization of large language models across multiple vendors using APIs, and experimentation with multimodal models for image and audio recognition and generation.

Course Description

  • Hands-on experience includes working with both hosted and locally run models, integrating AI with game engines such as Unity and Unreal, and developing AI agents that extend beyond simple chat-based interactions.
  • Ethical considerations and model evaluation are integrated throughout, emphasizing awareness of broader societal implications.
  • Through lectures, programming assignments, and a final project, the course provides the expertise needed to apply generative AI in creating innovative and interactive experiences.

Learning Outcomes

  • Understand the basic working principles and history of current LLMs (Large Language Models)
  • Understand ethical and safety aspects of using generative models
  • Evaluate and test generative models using industry benchmarks
  • Run generative models on local, laptop-based hardware (using CPU, GPU, NPUs)

Learning Outcomes

  • Create AI-based agents and tools based on the MCP (Model Context Protocol) specification
  • Avoid hallucinations by increasing the accuracy of models through RAG (Retrieval Augmented Generation) and fine-tuning
  • Explore and use multimodal models for image and audio recognition and generation
  • Create and deploy API-based clients, accessing LLMs hosted by different vendors (OpenAI, Meta, Google)

In Summary

  • Focus on integration/augmentation vs. automation/using
  • Provide a level of understanding beyond where most professional software developers are today
  • Build an exciting final project that you can add to your portfolio!

Syllabus

Syllabus

What to Expect in Each Class

Schedule

  • Every Friday (Curie); 2pm - 4.50pm
  • ~1.5 hours of lecture
  • ~1.5 hours for in-class hands-on lab time and assignments
  • Expectation of after-class work for assignments
  • No structured lectures for the weeks of the final project (3 hours of in-class lab time)
    • Although we may do mini-lectures for common topics

Schedule

  • Modules 1 through 4
  • Feb 6 is Founders Day, so no classes
  • Modules 5 through 8
  • Mar 9 - 13 is Spring Break
  • Weeks 9 through 14 - Final Project Work
  • Final presentations w/o Apr 20-24

During Class

  • Strive for conversation and interactivity
    • Please ask questions, even mid-slide!
    • There are no wrong or bad questions!
    • I enjoy going off on tangents / on the whiteboard
    • Use hands on lab time to seek input / troubleshoot code

Grading

Handing in Work

  • Everything submitted via GitHub
    • Recommend creating a repo for weekly assignments
      • For most weeks, submission will be one Python notebook
    • And (eventually) another repo for your final project
  • Don’t forget to give me permissions! @simonguest

Deadlines

  • Weekly Assignments
    • Assignments are due by the following week’s lesson
    • i.e., you get a week for each assignment
    • If you need more time/exception, please reach out via Teams
  • Final Project
    • Up until Week 15 presentations
    • (We’ll cover in detail later in the semester)

AI Policy

  • Permitted AI Usage
    • You may use AI tools to assist in understanding course materials
    • If AI is used to generate code, your must test and validate the code, must understand and be able to answer questions about the generated code, and include proper citations
    • If AI tools are used to assist with any part of an assignment, you must clearly cite the AI tool and explain how it was used

Tools, Languages, and Hardware

Tools

  • We will be introducing many tools
    • Colab Pro, OpenRouter, Hugging Face, etc.
    • Most will be free
    • Expect to need about $25 in API credits throughout the semester

Languages

  • We will be using (and learning) a lot of Python!
    • Most of the in-class assignments will be in Python
    • Don’t worry if you are new to Python as we’ll introduce concepts gradually
    • Although recommend investing extra time (see resources in Week 1)
  • Final Project
    • Can be any language
    • Probably depending on what you choose to create

Hardware

  • Will will be training SLMs (Small Language Models) later in the semester
  • This training will require a decent GPU and VRAM
    • Colab Pro (CUDA)
    • Your own NVIDIA-based laptop (CUDA)
    • Potential of using MLX for any Mac users

Need Help?

Need Help?

  • https://simonguest.github.io/CS-394
    • Slides (current and prior lectures), Demo code, Resources, Rubrics, Assignments
    • I will repost assignments and rubrics on the Meta-Moodle. (Grades will also be in Moodle.)
  • Office Hours
    • Thursdays 1pm - 3pm (On campus or virtually via Teams)
  • Teams (CS394/594 combined channel)
    • Primary mechanism for updates, ask questions, request office hours, etc.

Before We Get Started

Steep Learning Curve

  • We will be using the latest tools and AI models
  • Lots of new tools, acronyms, frameworks, etc.
  • Much of the curriculum builds upon itself
    • Please try not to miss lectures
    • Ask for help if you need to catch up

New Course at DigiPen!

  • There may be some minor curriculum tweaks mid-flight
    • Especially for topics that need less/more time

Fast Moving Space

  • There will be areas/questions I don’t have experience of
    • Multiple new models are launched every week
    • …or equations/algorithms that I don’t know
  • We will be learning some things together!
  • But that’s what makes it exciting!

Questions?