TensorFlow/Differential Privacy

Overview

The differential privacy network is a component in the tensorflow/models repository on Github.

Link to code: https://github.com/tensorflow/models/tree/master/research/differential_privacy

The networks are based on two papers:

"Deep Learning with Differential Privacy" (October 2016)

Link to paper: https://arxiv.org/pdf/1607.00133.pdf

"Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data" (March 2017)

Link to paper: https://arxiv.org/pdf/1610.05755.pdf

This repository directory is quite messy, so I'm trying to clean it up a bit.

Directories and Components

The top level directory contains three directories:

• dp_sgd - develops algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy (implementation of Deep Learning with Differential Privacy paper)
• multiple_teachers - creates student and teacher networks, transfer knowledge from teacher networks to student networks in a differentially private manner by noisily aggregating the teacher decisions before feeding them to students (implementation of Semi-Supervised Knowledge Transfer paper)
• privacy_accountant - not sure yet...

dp_sgd subdirect

Directory is focused

This directory itself contains subdirectories:

• dp_mnist
• dp_optimizer
• per_example_gradients

dp_sgd Prerequisites

Preparation steps, covered individually below:

• Install TensorFlow (old version)
• Install Bezel
• Prepare slim version of MNIST dataset

Installing old version of Tensorflow:

$ virtualenv --system-site-packages tf
$ source tf/bin/activate
(tf)$ easy_install -U pip
(tf)$ pip install tensorflow==0.12.1

Installing Bezel: using 0.5.4 (used Homebrew to install...)

Preparing slim MNIST data set:

$ cd models/research/slim/

$ python download_and_convert_data.py --dataset_name=mnist --dataset_dir="${DATA_DIR}"

$ ls ${DATA_DIR}
labels.txt
mnist_test.tfrecord
mnist_train.tfrecord

dp_sgd Building

Build instructions do not work...

See issue 2605: https://github.com/tensorflow/models/issues/2605