Game Theory and ML course -- IFT 6756


Mandatory registration.
The class will be taught in French or English, depending on attendance (all slides and class notes are in English).
A background in Deep Learning is recommended.
A background in Machine Learning, Optimization, Reinforcement Learning and Algorithmic Game Theory may be a plus.

Description

The number of Machine Learning applications related to game theory has been growing in the last couple of years. For example, two-player zero-sum games are important for generative modeling (GANs) and mastering games like Go or Poker via self-play. This course is at the interface between game theory, optimization, and machine learning. It tries to understand how to learn models to play games. It will start with some quick notions of game theory to eventually delve into machine learning problems with game formulations such as GANs or Multi-agent RL. This course will also cover the optimization (a.k.a training) of such machine learning games.

Un nombre grandissant d'applications d'apprentissage automatique liées à la théorie des jeux à vu le jour ces dernières années. Par exemple, les jeux à deux jouers et à somme nulle sont importants pour la modélisation générative (GAN) et la maîtrise de jeux comme Go ou Poker via l'appentissage autonome. Ce cours est à l'interface entre la théorie des jeux, l'optimisation et l'apprentissage automatique. Il essaie de comprendre comment apprendre des modèles pour jouer à des jeux. Il commencera par quelques notions rapides de théorie des jeux pour finalement se plonger dans les problèmes d'apprentissage automatique avec des formulations de jeux telles que les GAN ou l'apprentissage par renforcement avec plusieurs agents. Ce cours couvrira également l'optimisation (a.k.a training) de tels jeux d'apprentissage automatique.

Schedule (Tentative)

Due to the current sanitary situation the course will be given online. There will be two lectures a week, starting January 12th. The timeslots for the lectures are the following:
  • Tuesday 4:30PM--6:30PM
  • Friday 1:30PM--3:30PM
The Last timeslots will be dedicated to project presentations (probably 15 minutes talks). The Winter session ends on April 30th.
The following schedule is a tentative plan:
  • Introduction. The ML and Game Theory fundamental concepts necessary in this course. (2 weeks)
  • Generative Adversarial Networks. (2-3 weeks)
  • Optimization of Games. (3 weeks)
  • Multi-Agent RL. We will cover some theory and applications (3-4 weeks)
  • Invited talks + Projects Presentations. (last remaining weeks)


Evaluation

Project 100% - Attending at least 80% of the classes is also mandatory. The project evaluation will be based on a project report and a project presentation.

Relevant references


  • GANs: https://arxiv.org/abs/1406.2661,
  • Big GAN: https://arxiv.org/abs/1809.11096
  • WGAN: https://arxiv.org/pdf/1701.07875.pdf
  • WGAN-GP: https://arxiv.org/pdf/1704.00028.pdf
  • Poker: https://www.cs.cmu.edu/~noamb/papers/19-Science-Superhuman.pdf, https://www.cs.cmu.edu/~noamb/papers/17-Science-Superhuman.pdf
  • Diplomacy: https://arxiv.org/pdf/2010.02923.pdf, https://arxiv.org/abs/2006.04635, https://arxiv.org/abs/1909.02128
  • Hanabi: https://arxiv.org/abs/1902.00506, https://arxiv.org/abs/1811.01458
  • StarCraft II: https://www.nature.com/articles/d41586-019-03343-4
  • AlphaGo: https://www.nature.com/articles/nature16961
  • AlphaGo zero: https://www.nature.com/articles/nature24270
  • Alpha zero: https://science.sciencemag.org/content/362/6419/1140
  • Open-ended learning: https://arxiv.org/abs/1901.08106
  • Spinning-top: https://arxiv.org/abs/2004.09468
  • Unified framework: https://arxiv.org/pdf/1711.00832.pdf
  • Re-evaluating Evaluation: https://papers.nips.cc/paper/2018/file/cdf1035c34ec380218a8cc9a43d438f9-Paper.pdf
  • Lola https://arxiv.org/abs/1709.04326
  • Numerics of GANs: https://arxiv.org/abs/1705.10461
  • Optimistic methods: https://arxiv.org/abs/1711.00141
  • Extragradient methods: https://arxiv.org/abs/1802.10551, https://arxiv.org/abs/1906.05945
  • Negative momentum: https://arxiv.org/abs/1807.04740
  • Optimal convergence rates in games: https://arxiv.org/abs/2001.00602
  • Noise in Games: https://arxiv.org/abs/1904.08598