Building with AI Systems

What you will be able to do

• Choose an architecture based on task and evidence needs.
• Evaluate quality, latency, cost, and safety together.
• Design monitoring and incident response before launch.

Recommended first: AI Literacy Foundations, Responsible AI User

Course modules

1. 1. Language-model systems

   Outcome: Reason about tokens, context, generation, and model tradeoffs.

   Practice: Compare two model options using quality, latency, context, privacy, and cost.

   Competencies: Language-model mechanics · Models, inference, and limitations · Model cost and operational tradeoffs

   chatgpt llms →ai models →ai inference →

2. 2. Grounding with retrieval

   Outcome: Know when and how retrieval can improve evidence access.

   Practice: Design a retrieval test set with answer and citation requirements.

   Competencies: Retrieval-augmented generation · Source and claim verification

   retrieval quality →vector databases →embeddings →

3. 3. Agents and tools

   Outcome: Bound multi-step systems with permissions and checkpoints.

   Practice: Write an agent permission model and failure-recovery path.

   Competencies: Agents, tools, and long-running tasks · Automation boundaries and safeguards · AI security and misuse risk

   ai agents →ai workflow automation →

4. 4. Evaluation and operations

   Outcome: Measure system performance before and after deployment.

   Practice: Create an evaluation suite covering quality, refusal, latency, cost, and regressions.

   Competencies: Evaluation and benchmark literacy · Success metrics and monitoring · AI incident response · Experiment and pilot design

   ai benchmarks →ai safety →