wellton/stargan
1
0
Fork 0
Código Issues Pull Requests Ações Pacotes Projetos Versões Wiki Atividade

Update usage section

Ver Código-Fonte
Esse commit está contido em:
yunjey
2018-04-12 12:37:38 +09:00
pai c53e59871b
commit 88cfeb0759
1 arquivos alterados com 88 adições e 74 exclusões
Baixar Arquivo de Patch Baixar Arquivo de Diff Expandir todos os arquivos Colapsar todos os arquivos
README.md
+88 -74
Ver Arquivo
@@ -1,122 +1,135 @@
<p align="center"><img width="40%" src="png/logo.png" /></p>
<p align="center"><img width="40%" src="jpg/logo.jpg" /></p>
--------------------------------------------------------------------------------
This repository provides a PyTorch implementation of [StarGAN](https://arxiv.org/abs/1711.09020). StarGAN can flexibly translate an input image to any desired target domain using only a single generator and a discriminator. The demo video for StarGAN can be found [here](https://www.youtube.com/watch?v=EYjdLppmERE).
<p align="center"><img width="100%" src="png/main.png" /></p>
<p align="center"><img width="100%" src="jpg/main.jpg" /></p>
<br/>
## Paper
[StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://arxiv.org/abs/1711.09020) <br/>
[Yunjey Choi](https://github.com/yunjey), [Minje Choi](https://github.com/mjc92), [Munyoung Kim](https://www.facebook.com/munyoung.kim.1291), [Jung-Woo Ha](https://www.facebook.com/jungwoo.ha.921), [Sung Kim](https://www.cse.ust.hk/~hunkim/), and [Jaegul Choo](https://sites.google.com/site/jaegulchoo/)    <br/>
IEEE Conference on Computer Vision and Pattern Recognition ([CVPR](http://cvpr2018.thecvf.com/)), 2018 (<b>Oral Presentation</b>)
[Yunjey Choi](https://github.com/yunjey)<sup> 1,2</sup>, [Minje Choi](https://github.com/mjc92)<sup> 1,2</sup>, [Munyoung Kim](https://www.facebook.com/munyoung.kim.1291)<sup> 2,3</sup>, [Jung-Woo Ha](https://www.facebook.com/jungwoo.ha.921)<sup> 2</sup>, [Sung Kim](https://www.cse.ust.hk/~hunkim/)<sup> 2,4</sup>, and [Jaegul Choo](https://sites.google.com/site/jaegulchoo/)<sup> 1,2</sup>    <br/>
<sup>1 </sup>Korea University, <sup>2 </sup>Clova AI Research (NAVER Corp.), <sup>3 </sup>The College of New Jersey, <sup> 4 </sup>HKUST <br/>
IEEE Conference on Computer Vision and Pattern Recognition ([CVPR](http://cvpr2018.thecvf.com/)), 2018 (<b>Oral</b>)
&nbsp;
<br/>
## Results
#### (i) Facial Attribute Transfer on CelebA
The images are generated by StarGAN trained on the CelebA dataset.
<p align="center"><img width="100%" src="png/result_celeb1.png" /></p>
#### (ii) Facial Expression Synthesis on RaFD
The images are generated by StarGAN trained on the RaFD dataset.
<p align="center"><img width="100%" src="png/result_rafd.png" /></p>
#### (iii) Facial Expression Synthesis on CelebA
The images are generated by StarGAN trained on both the CelebA and RaFD dataset.
<p align="center"><img width="100%" src="png/result_celeb2.png" /></p>
&nbsp;
## Model Description
### Training within a Single Dataset
Overview of StarGAN, consisting of two modules, a discriminator <b>D</b> and a generator <b>G</b>. <b>(a)</b> <b>D</b> learns to distinguish between real and fake images and classify the real images to its corresponding domain. <b>(b)</b> <b>G</b> takes in as input both the image and target domain label and generates an fake image. The target domain label is spatially replicated and concatenated with the input image. <b>(c)</b> <b>G</b> tries to reconstruct the original image from the fake image given the original domain label. <b>(d)</b> <b>G</b> tries to generate images indistinguishable from real images and classifiable as target domain by <b>D</b>.
<p align="center"><img width="100%" src="png/model.png" /></p>
### Training with Multiple Datasets
Overview of StarGAN when training with both CelebA and RaFD. <b>(a) ~ (d)</b> shows the training process using CelebA, and <b>(e) ~ (h)</b> shows the training process using RaFD. <b>(a), (e)</b> The discriminator <b>D</b> learns to distinguish between real and fake images and minimize the classification error only for the known label. <b>(b), (c), (f), (g)</b> When the mask vector (purple) is [1, 0], the generator <b>G</b> learns to focus on the CelebA label (yellow) and ignore the RaFD label (green) to perform image-to-image translation, and vice versa when the mask vector is [0, 1]. <b>(d), (h)</b> <b>G</b> tries to generate images that are both indistinguishable from real images and classifiable by <b>D</b> as belonging to the target domain.
<p align="center"><img width="100%" src="png/model2.png" /></p>
&nbsp;
## Prerequisites
## Dependencies
* [Python 3.5+](https://www.continuum.io/downloads)
* [PyTorch 0.2.0](http://pytorch.org/)
* [PyTorch 0.3.0](http://pytorch.org/)
* [TensorFlow 1.3+](https://www.tensorflow.org/) (optional for tensorboard)
&nbsp;
<br/>
## Usage
#### 1. Clone the repository
### 1. Cloning the repository
```bash
$ git clone https://github.com/yunjey/StarGAN.git
$ cd StarGAN/
```
#### 2. Download the dataset
##### (i) CelebA dataset
### 2. Downloading the dataset
To download the CelebA dataset:
```bash
$ bash download.sh
```
##### (ii) RaFD dataset
Because <b>RaFD</b> is not a public dataset, you must first request access to the dataset from [the Radboud Faces Database website](http://www.socsci.ru.nl:8180/RaFD2/RaFD?p=main). Then, you need to create the folder structure as decribed [here.](https://github.com/yunjey/StarGAN/blob/master/png/RaFD.md)
To download the RaFD dataset, you must request access to the dataset from [the Radboud Faces Database website](http://www.socsci.ru.nl:8180/RaFD2/RaFD?p=main). Then, you need to create a folder structure as described [here](https://github.com/yunjey/StarGAN-new2/blob/master/jpg/RaFD.md).
#### 3. Train StarGAN
##### (i) Training with CelebA
### 3. Training
To train StarGAN on CelebA, run the training script below. See [here](https://github.com/yunjey/StarGAN-new2/blob/master/jpg/CelebA.md) for a list of selectable attributes in the CelebA dataset. If you change the `selected_attrs` argument, you should also change the `c_dim` argument accordingly.
```bash
$ python main.py --mode='train' --dataset='CelebA' --c_dim=5 --image_size=128 \
--sample_path='stargan_celebA/samples' --log_path='stargan_celebA/logs' \
--model_save_path='stargan_celebA/models' --result_path='stargan_celebA/results'
$ python main.py --mode='train' --dataset='CelebA' --image_size=128 --c_dim=5 \
--sample_dir='expr_celeba/samples' --log_dir='expr_celeba/logs' \
--model_save_dir='expr_celeba/models' --result_dir='expr_celeba/results' \
--selected_attrs=['Black_Hair', 'Blond_Hair', 'Brown_Hair', 'Male', 'Young']
```
##### (ii) Training with RaFD
To train StarGAN on RaFD:
```bash
$ python main.py --mode='train' --dataset='RaFD' --c_dim=8 --image_size=128 \
--num_epochs=200 --num_epochs_decay=100 --sample_step=200 --model_save_step=200 \
--sample_path='stargan_rafd/samples' --log_path='stargan_rafd/logs' \
--model_save_path='stargan_rafd/models' --result_path='stargan_rafd/results'
$ python main.py --mode='train' --dataset='RaFD' --image_size=128 --c_dim=8 \
--sample_dir='expr_rafd/samples' --log_dir='expr_rafd/logs' \
--model_save_dir='expr_rafd/models' --result_dir='expr_rafd/results'
```
##### (iii) Training with CelebA+RaFD
To train StarGAN on both CelebA and RafD:
```bash
$ python main.py --mode='train' --dataset='Both' --image_size=256 --num_iters=200000 --num_iters_decay=100000 \
--sample_path='stargan_both/samples' --log_path='stargan_both/logs' \
--model_save_path='stargan_both/models' --result_path='stargan_both/results'
$ python main.py --mode='train' --dataset='Both' --image_size=256 --c_dim=5 --c2_dim=8 \
--sample_dir='expr_both/samples' --log_dir='expr_both/logs' \
--model_save_dir='expr_both/models' --result_dir='expr_both/results'
```
#### 4. Test StarGAN
##### (i) Facial attribute transfer on CelebA
To train StarGAN on your own dataset, create a folder structure in the same format as [RaFD](https://github.com/yunjey/StarGAN-new2/blob/master/jpg/RaFD.md) and run the command:
```bash
$ python main.py --mode='test' --dataset='CelebA' --c_dim=5 --image_size=128 --test_model='20_1000' \
--sample_path='stargan_celebA/samples' --log_path='stargan_celebA/logs' \
--model_save_path='stargan_celebA/models' --result_path='stargan_celebA/results'
$ python main.py --mode='train' --dataset='RaFD' --rafd_crop_size=CROP_SIZE --image_size=IMG_SIZE
--c_dim=LABEL_DIM --rafd_image_dir=TRAIN_IMG_DIR \
--sample_dir='expr_custom/samples' --log_dir='expr_custom/logs' \
--model_save_dir='expr_custom/models' --result_dir='expr_custom/results'
```
##### (ii) Facial expression synthesis on RaFD
### 4. Testing
To test StarGAN on CelebA:
```bash
$ python main.py --mode='test' --dataset='RaFD' --c_dim=8 --image_size=128 \
--test_model='200_200' --rafd_image_path='data/RaFD/test' \
--sample_path='stargan_rafd/samples' --log_path='stargan_rafd/logs' \
--model_save_path='stargan_rafd/models' --result_path='stargan_rafd/results'
$ python main.py --mode='test' --dataset='CelebA' --image_size=128 --c_dim=5 \
                --sample_dir='expr_celeba/samples' --log_dir='expr_celeba/logs' \
--model_save_dir='expr_celeba/models' --result_dir='expr_celeba/results' \
--selected_attrs=['Black_Hair', 'Blond_Hair', 'Brown_Hair', 'Male', 'Young']
```
##### (iii) Facial expression synthesis on CelebA
To test StarGAN on RaFD:
```bash
$ python main.py --mode='test' --dataset='Both' --image_size=256 --test_model='200000' \
--sample_path='stargan_both/samples' --log_path='stargan_both/logs' \
--model_save_path='stargan_both/models' --result_path='stargan_both/results'
$ python main.py --mode='test' --dataset='RaFD' --image_size=128 \
--c_dim=8 --rafd_image_dir='data/RaFD/test' \
                --sample_dir='expr_rafd/samples' --log_dir='expr_rafd/logs' \
--model_save_dir='expr_rafd/models' --result_dir='expr_rafd/results'
```
&nbsp;
To test StarGAN on both CelebA and RaFD:
```bash
$ python main.py --mode='test' --dataset='Both' --image_size=256 --c_dim=5 --c2_dim=8 \
                --sample_dir='expr_both/samples' --log_dir='expr_both/logs' \
--model_save_dir='expr_both/models' --result_dir='expr_both/results'
```
To test StarGAN on your own dataset:
```bash
$ python main.py --mode='test' --dataset='RaFD' --rafd_crop_size=CROP_SIZE --image_size=IMG_SIZE
--c_dim=LABEL_DIM --rafd_image_dir=TEST_IMG_DIR \
--sample_dir='expr_custom/samples' --log_dir='expr_custom/logs' \
--model_save_dir='expr_custom/models' --result_dir='expr_custom/results'
```
<br/>
## Results
### 1. Facial Attribute Transfer on CelebA
<p align="center"><img width="100%" src="jpg/result_celeba1.jpg" /></p>
### 2. Facial Expression Synthesis on RaFD
<p align="center"><img width="100%" src="jpg/result_rafd.jpg" /></p>
### 3. Facial Expression Synthesis on CelebA
<p align="center"><img width="100%" src="jpg/result_celeba2.jpg" /></p>
<br/>
## Citation
If this work is useful for your research, please cite our [paper](https://arxiv.org/abs/1711.09020).
If this work is useful for your research, please cite our [paper](https://arxiv.org/abs/1711.09020):
```
@article{choi2017stargan,
title = {StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation},
@@ -125,7 +138,8 @@ If this work is useful for your research, please cite our [paper](https://arxiv.
Year = {2017}
}
```
&nbsp;
<br/>
## Acknowledgement
This work was mainly done while the first author did a research internship at <b>Clova AI Research, NAVER (CLAIR)</b>. We also thank all the researchers at CLAIR, especially Donghyun Kwak, for insightful discussions.
This work was mainly done while the first author did a research intership at [Clova AI Research, NAVER](https://clova.ai/en/research/research-area-detail.html?id=0). We thank all the researchers at NAVER, especially Donghyun Kwak, for insightful discussions.