stargan

This repository provides a PyTorch implementation of StarGAN. StarGAN can flexibly translate an input image to any desired target domain using only a single generator and a discriminator. The demo video for StarGAN can be found here.

Paper

StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
Yunjey Choi^1,2, Minje Choi^1,2, Munyoung Kim^2,3, Jung-Woo Ha², Sung Kim^2,4, and Jaegul Choo^1,2
¹Korea University, ²Clova AI Research (NAVER Corp.), ³The College of New Jersey, ⁴HKUST
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018 (Oral)

Dependencies

Python 3.5+
PyTorch 0.3.0
TensorFlow 1.3+ (optional for tensorboard)

Usage

1. Cloning the repository

$ git clone https://github.com/yunjey/StarGAN.git
$ cd StarGAN/

2. Downloading the dataset

To download the CelebA dataset:

$ bash download.sh

To download the RaFD dataset, you must request access to the dataset from the Radboud Faces Database website. Then, you need to create a folder structure as described here.

3. Training

To train StarGAN on CelebA, run the training script below. See here for a list of selectable attributes in the CelebA dataset. If you change the selected_attrs argument, you should also change the c_dim argument accordingly.

$ python main.py --mode='train' --dataset='CelebA' --image_size=128 --c_dim=5 \
                 --sample_dir='expr_celeba/samples' --log_dir='expr_celeba/logs' \
                 --model_save_dir='expr_celeba/models' --result_dir='expr_celeba/results' \
                 --selected_attrs=['Black_Hair', 'Blond_Hair', 'Brown_Hair', 'Male', 'Young']

To train StarGAN on RaFD:

$ python main.py --mode='train' --dataset='RaFD' --image_size=128 --c_dim=8 \
                 --sample_dir='expr_rafd/samples' --log_dir='expr_rafd/logs' \
                 --model_save_dir='expr_rafd/models' --result_dir='expr_rafd/results'

To train StarGAN on both CelebA and RafD:

$ python main.py --mode='train' --dataset='Both' --image_size=256 --c_dim=5 --c2_dim=8 \
                 --sample_dir='expr_both/samples' --log_dir='expr_both/logs' \
                 --model_save_dir='expr_both/models' --result_dir='expr_both/results'

To train StarGAN on your own dataset, create a folder structure in the same format as RaFD and run the command:

$ python main.py --mode='train' --dataset='RaFD' --rafd_crop_size=CROP_SIZE --image_size=IMG_SIZE
                 --c_dim=LABEL_DIM --rafd_image_dir=TRAIN_IMG_DIR \
                 --sample_dir='expr_custom/samples' --log_dir='expr_custom/logs' \
                 --model_save_dir='expr_custom/models' --result_dir='expr_custom/results'

4. Testing

To test StarGAN on CelebA:

$ python main.py --mode='test' --dataset='CelebA' --image_size=128 --c_dim=5 \
                 --sample_dir='expr_celeba/samples' --log_dir='expr_celeba/logs' \
                 --model_save_dir='expr_celeba/models' --result_dir='expr_celeba/results' \
                 --selected_attrs=['Black_Hair', 'Blond_Hair', 'Brown_Hair', 'Male', 'Young']

To test StarGAN on RaFD:

$ python main.py --mode='test' --dataset='RaFD' --image_size=128 \
                 --c_dim=8 --rafd_image_dir='data/RaFD/test' \
                 --sample_dir='expr_rafd/samples' --log_dir='expr_rafd/logs' \
                 --model_save_dir='expr_rafd/models' --result_dir='expr_rafd/results'

To test StarGAN on both CelebA and RaFD:

$ python main.py --mode='test' --dataset='Both' --image_size=256 --c_dim=5 --c2_dim=8 \
                 --sample_dir='expr_both/samples' --log_dir='expr_both/logs' \
                 --model_save_dir='expr_both/models' --result_dir='expr_both/results'

To test StarGAN on your own dataset:

$ python main.py --mode='test' --dataset='RaFD' --rafd_crop_size=CROP_SIZE --image_size=IMG_SIZE
                 --c_dim=LABEL_DIM --rafd_image_dir=TEST_IMG_DIR \
                 --sample_dir='expr_custom/samples' --log_dir='expr_custom/logs' \
                 --model_save_dir='expr_custom/models' --result_dir='expr_custom/results'

Results

1. Facial Attribute Transfer on CelebA

2. Facial Expression Synthesis on RaFD

3. Facial Expression Synthesis on CelebA

Citation

If this work is useful for your research, please cite our paper:

@article{choi2017stargan,
 title = {StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation},    
 author = {Choi, Yunjey and Choi, Minje and Kim, Munyoung and Ha, Jung-Woo and Kim, Sunghun and Choo, Jaegul},
 journal= {arXiv preprint arXiv:1711.09020},
 Year = {2017}
}

Acknowledgement

This work was mainly done while the first author did a research intership at Clova AI Research, NAVER. We thank all the researchers at NAVER, especially Donghyun Kwak, for insightful discussions.