Differential Privacy Mathematical Foundations and Applications in Machine Learning
Semester: Winter 2023/2024
Added: Nov 1, 2023
Winter 2023/2024
Instructor: Franziska Boenisch and Jilles Vreeken
Contact: franziska.boenisch@cispa.de
Lectures: Seminar-based interactive hours.
Description
Since machine learning (ML) becomes increasingly prevalent in sensitive areas, such as healthcare and finance, it is crucial to ensure privacy. This seminar is centered around the mathematical framework of differential privacy, a current gold standard for privacy protection. Throughout the seminar, we will delve deep into the core principles of differential privacy, privacy accounting, and practical implementation in state-of-the-art foundational models like LLMs.
Assignments
Each student will present a topic during the seminar hours in the form of an oral presentation. Every student is supposed to read all the papers ahead of the presentations to actively participate in the discussions.
Schedule
| Topic Number | Topic Detail | Core References |
|---|---|---|
| (1) | Differential Privacy: Background and Mathematics | Dwork et al., Differential Privacy (epsilon DP); Dwork & Roth Chapter 2 & 3.3 |
| (2) | Differential Private Stochastic Gradient Descent (DPSGD) | Abadi et al., Deep Learning with Differential Privacy |
| (3) | Rényi Differential Privacy & Subsampled Gaussian Mechanisms | Mironov, Rényi DP; Wang et al., Analytical Moments Accountant |
| (4) | Private Aggregation of Teacher Ensembles (PATE) | Papernot et al., Semi-supervised knowledge transfer |
| (5) | Heterogenous/Individualized Differential Privacy | Boenisch et al., Individualized PATE / Individualized Privacy Assignment |
| (6) | Privacy Auditing in Blackbox-Access | Tramer et al., Debugging differential privacy |
| (7) | Differential Privacy for Large Language Models | Li et al., Large Language Models Can Be Strong DP Learners |
Course Staff
- Franziska Boenisch (Instructor)
- Jilles Vreeken (Instructor)