This course will teach you the theory behind large language models (LLMs) and give you the tools to implement LLMs in your applications.

Contact

• Dr. Bruno Yun

  • Email: [email protected]
  • Website: https://bruno-yun.notion.site/
  • Office: Nautibus, 12.038 (second floor).
  • Working hours: 9:00 am to 5:00 pm

  Send your queries by email. For a face-to-face meeting, contact me by email to arrange one.

Structure of the course

This part of the course will have 18 hours of lectures, 12 hours of practicals.

The teaching timetable is split into three sessions as follows.

• Part 1: Zoom on the transformer architecture (3 hours, 12th of November 2024)

  The RNN Architectures

  The Transformer Model for NLG

  Continuous assessment 1 (written test, 30%) → Deadline 01/12/2024 23:59

• Part 2: A short history of large language models (3 hours, 18th of November 2024)

  A quick survey of existing LLMs

  Running LLMs on Consumer-Grade Computers

• Part 3: Pre-training and Post-training (3 hours, 25th of November 2024)

  Pre-training

  Post-training/Fine-tuning

• Part 4: Practical techniques and tools for large language models (3 hours, 2nd of December 2024)

  Prompt engineering

  Retrieval Augmented Generation

  Useful tools for large language models

  • Continuous assessment 2 (oral presentation, 30%)

Practical 0: Background on PyTorch (if needed)

Practical 1: Tokenizers, embeddings, and multi-head attention from scratch (3 hours, 2nd of December 2024)

• Part 5: Evaluation and ethical questions of large language models (3 hours, 9th of December 2024)

  Evaluating LLMs

  Security, privacy, and ethical questions of LLM

• Part 6: Exciting research of large language models (3 hours, 16th of December 2024)

  Pros and Cons of selective state space models (SSMs)

  Test-time training (TTT) models

  Mixture-of-experts

  Large Language Models in 1.58 Bits?