wellton/stargan
1
0
Fork 0
Código Issues Pull Requests Ações Pacotes Projetos Versões Wiki Atividade

Update README.md

Ver Código-Fonte
Esse commit está contido em:
Yunjey Choi
2019-12-13 21:12:55 +09:00
commit de GitHub
pai 6faf5e234f
commit e7b9994e91
1 arquivos alterados com 73 adições e 108 exclusões
Baixar Arquivo de Patch Baixar Arquivo de Diff Expandir todos os arquivos Colapsar todos os arquivos
README.md
+73 -108
Ver Arquivo
@@ -1,19 +1,19 @@
<p align="center"><img width="40%" src="jpg/logo.jpg" /></p>
## StarGAN - Official PyTorch Implementation
--------------------------------------------------------------------------------
This repository provides a PyTorch implementation of [StarGAN](https://arxiv.org/abs/1711.09020). StarGAN can flexibly translate an input image to any desired target domain using only a single generator and a discriminator. The demo video for StarGAN can be found [here](https://www.youtube.com/watch?v=EYjdLppmERE).
**\*\*\*\*\* New Dec 4th, 2019: StarGAN v2 is available at https://github.com/clovaai/stargan-v2 \*\*\*\*\***
<p align="center"><img width="100%" src="jpg/main.jpg" /></p>
<br/>
This repository provides the official PyTorch implementation of the following paper:
> **StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation**<br>
> [Yunjey Choi](https://github.com/yunjey)<sup> 1,2</sup>, [Minje Choi](https://github.com/mjc92)<sup> 1,2</sup>, [Munyoung Kim](https://www.facebook.com/munyoung.kim.1291)<sup> 2,3</sup>, [Jung-Woo Ha](https://www.facebook.com/jungwoo.ha.921)<sup> 2</sup>, [Sung Kim](https://www.cse.ust.hk/~hunkim/)<sup> 2,4</sup>, and [Jaegul Choo](https://sites.google.com/site/jaegulchoo/)<sup> 1,2</sup>    <br/>
> <sup>1 </sup>Korea University, <sup>2 </sup>Clova AI Research (NAVER Corp.), <sup>3 </sup>The College of New Jersey, <sup> 4 </sup>HKUST <br/>
> https://arxiv.org/abs/1711.09020 <br>
>
> **Abstract:** *Recent studies have shown remarkable success in image-to-image translation for two domains. However, existing approaches have limited scalability and robustness in handling more than two domains, since different models should be built independently for every pair of image domains. To address this limitation, we propose StarGAN, a novel and scalable approach that can perform image-to-image translations for multiple domains using only a single model. Such a unified model architecture of StarGAN allows simultaneous training of multiple datasets with different domains within a single network. This leads to StarGAN's superior quality of translated images compared to existing models as well as the novel capability of flexibly translating an input image to any desired target domain. We empirically demonstrate the effectiveness of our approach on a facial attribute transfer and a facial expression synthesis tasks.*
## Paper
[StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://arxiv.org/abs/1711.09020) <br/>
[Yunjey Choi](https://github.com/yunjey)<sup> 1,2</sup>, [Minje Choi](https://github.com/mjc92)<sup> 1,2</sup>, [Munyoung Kim](https://www.facebook.com/munyoung.kim.1291)<sup> 2,3</sup>, [Jung-Woo Ha](https://www.facebook.com/jungwoo.ha.921)<sup> 2</sup>, [Sung Kim](https://www.cse.ust.hk/~hunkim/)<sup> 2,4</sup>, and [Jaegul Choo](https://sites.google.com/site/jaegulchoo/)<sup> 1,2</sup>    <br/>
<sup>1 </sup>Korea University, <sup>2 </sup>Clova AI Research (NAVER Corp.), <sup>3 </sup>The College of New Jersey, <sup> 4 </sup>HKUST <br/>
IEEE Conference on Computer Vision and Pattern Recognition ([CVPR](http://cvpr2018.thecvf.com/)), 2018 (<b>Oral</b>)
<br>
<br/>
## Dependencies
* [Python 3.5+](https://www.continuum.io/downloads)
@@ -21,141 +21,106 @@ IEEE Conference on Computer Vision and Pattern Recognition ([CVPR](http://cvpr20
* [TensorFlow 1.3+](https://www.tensorflow.org/) (optional for tensorboard)
<br/>
## Usage
### 1. Cloning the repository
```bash
$ git clone https://github.com/yunjey/StarGAN.git
$ cd StarGAN/
```
### 2. Downloading the dataset
## Downloading datasets
To download the CelebA dataset:
```bash
$ bash download.sh celeba
git clone https://github.com/yunjey/StarGAN.git
cd StarGAN/
bash download.sh celeba
```
To download the RaFD dataset, you must request access to the dataset from [the Radboud Faces Database website](http://www.socsci.ru.nl:8180/RaFD2/RaFD?p=main). Then, you need to create a folder structure as described [here](https://github.com/yunjey/StarGAN/blob/master/jpg/RaFD.md).
### 3. Training
## Training networks
To train StarGAN on CelebA, run the training script below. See [here](https://github.com/yunjey/StarGAN/blob/master/jpg/CelebA.md) for a list of selectable attributes in the CelebA dataset. If you change the `selected_attrs` argument, you should also change the `c_dim` argument accordingly.
```bash
$ python main.py --mode train --dataset CelebA --image_size 128 --c_dim 5 \
--sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \
--model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \
--selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young
# Train StarGAN using the CelebA dataset
python main.py --mode train --dataset CelebA --image_size 128 --c_dim 5 \
--sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \
--model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \
--selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young
# Test StarGAN using the CelebA dataset
python main.py --mode test --dataset CelebA --image_size 128 --c_dim 5 \
              --sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \
--model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \
--selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young
```
To train StarGAN on RaFD:
```bash
$ python main.py --mode train --dataset RaFD --image_size 128 --c_dim 8 \
--sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs \
--model_save_dir stargan_rafd/models --result_dir stargan_rafd/results
# Train StarGAN using the RaFD dataset
python main.py --mode train --dataset RaFD --image_size 128 \
--c_dim 8 --rafd_image_dir data/RaFD/train \
--sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs \
--model_save_dir stargan_rafd/models --result_dir stargan_rafd/results
# Test StarGAN using the RaFD dataset
python main.py --mode test --dataset RaFD --image_size 128 \
--c_dim 8 --rafd_image_dir data/RaFD/test \
              --sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs \
--model_save_dir stargan_rafd/models --result_dir stargan_rafd/results
```
To train StarGAN on both CelebA and RafD:
```bash
$ python main.py --mode=train --dataset Both --image_size 256 --c_dim 5 --c2_dim 8 \
--sample_dir stargan_both/samples --log_dir stargan_both/logs \
--model_save_dir stargan_both/models --result_dir stargan_both/results
# Train StarGAN using both CelebA and RaFD datasets
python main.py --mode=train --dataset Both --image_size 256 --c_dim 5 --c2_dim 8 \
--sample_dir stargan_both/samples --log_dir stargan_both/logs \
--model_save_dir stargan_both/models --result_dir stargan_both/results
# Test StarGAN using both CelebA and RaFD datasets
python main.py --mode test --dataset Both --image_size 256 --c_dim 5 --c2_dim 8 \
              --sample_dir stargan_both/samples --log_dir stargan_both/logs \
--model_save_dir stargan_both/models --result_dir stargan_both/results
```
To train StarGAN on your own dataset, create a folder structure in the same format as [RaFD](https://github.com/yunjey/StarGAN/blob/master/jpg/RaFD.md) and run the command:
```bash
$ python main.py --mode train --dataset RaFD --rafd_crop_size CROP_SIZE --image_size IMG_SIZE \
--c_dim LABEL_DIM --rafd_image_dir TRAIN_IMG_DIR \
--sample_dir stargan_custom/samples --log_dir stargan_custom/logs \
--model_save_dir stargan_custom/models --result_dir stargan_custom/results
# Train StarGAN on custom datasets
python main.py --mode train --dataset RaFD --rafd_crop_size CROP_SIZE --image_size IMG_SIZE \
--c_dim LABEL_DIM --rafd_image_dir TRAIN_IMG_DIR \
--sample_dir stargan_custom/samples --log_dir stargan_custom/logs \
--model_save_dir stargan_custom/models --result_dir stargan_custom/results
# Test StarGAN on custom datasets
python main.py --mode test --dataset RaFD --rafd_crop_size CROP_SIZE --image_size IMG_SIZE \
--c_dim LABEL_DIM --rafd_image_dir TEST_IMG_DIR \
--sample_dir stargan_custom/samples --log_dir stargan_custom/logs \
--model_save_dir stargan_custom/models --result_dir stargan_custom/results
```
### 4. Testing
## Using pre-trained networks
To download a pre-trained model checkpoint, run the script below. The pre-trained model checkpoint will be downloaded and saved into `./stargan_celeba_256/models` directory.
To test StarGAN on CelebA:
```bash
$ bash download.sh pretrained-celeba-128x128
```
To translate images using the pre-trained model, run the evaluation script below. The translated images will be saved into `./stargan_celeba_128/results` directory.
```bash
$ python main.py --mode test --dataset CelebA --image_size 128 --c_dim 5 \
                --sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \
--model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \
--selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young
```
To test StarGAN on RaFD:
```bash
$ python main.py --mode test --dataset RaFD --image_size 128 \
--c_dim 8 --rafd_image_dir data/RaFD/test \
                --sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs \
--model_save_dir stargan_rafd/models --result_dir stargan_rafd/results
```
To test StarGAN on both CelebA and RaFD:
```bash
$ python main.py --mode test --dataset Both --image_size 256 --c_dim 5 --c2_dim 8 \
                --sample_dir stargan_both/samples --log_dir stargan_both/logs \
--model_save_dir stargan_both/models --result_dir stargan_both/results
```
To test StarGAN on your own dataset:
```bash
$ python main.py --mode test --dataset RaFD --rafd_crop_size CROP_SIZE --image_size IMG_SIZE \
--c_dim LABEL_DIM --rafd_image_dir TEST_IMG_DIR \
--sample_dir stargan_custom/samples --log_dir stargan_custom/logs \
--model_save_dir stargan_custom/models --result_dir stargan_custom/results
```
### 5. Pretrained model
To download a pretrained model checkpoint, run the script below. The pretrained model checkpoint will be downloaded and saved into `./stargan_celeba_256/models` directory.
```bash
$ bash download.sh pretrained-celeba-256x256
```
To translate images using the pretrained model, run the evaluation script below. The translated images will be saved into `./stargan_celeba_256/results` directory.
```bash
$ python main.py --mode test --dataset CelebA --image_size 256 --c_dim 5 \
--selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young \
--model_save_dir='stargan_celeba_256/models' \
--result_dir='stargan_celeba_256/results'
--model_save_dir='stargan_celeba_128/models' \
--result_dir='stargan_celeba_128/results'
```
<br/>
## Results
### 1. Facial Attribute Transfer on CelebA
<p align="center"><img width="100%" src="jpg/result_celeba1.jpg" /></p>
### 2. Facial Expression Synthesis on RaFD
<p align="center"><img width="100%" src="jpg/result_rafd.jpg" /></p>
### 3. Facial Expression Synthesis on CelebA
<p align="center"><img width="100%" src="jpg/result_celeba2.jpg" /></p>
<br/>
## Citation
If this work is useful for your research, please cite our [paper](https://arxiv.org/abs/1711.09020):
If you find this work is useful for your research, please consider citing our [paper](https://arxiv.org/abs/1711.09020):
```
@InProceedings{StarGAN2018,
author = {Choi, Yunjey and Choi, Minje and Kim, Munyoung and Ha, Jung-Woo and Kim, Sunghun and Choo, Jaegul},
title = {StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
@inproceedings{choi2018stargan,
author={Yunjey Choi and Minje Choi and Munyoung Kim and Jung-Woo Ha and Sunghun Kim and Jaegul Choo},
title={StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2018}
}
```
<br/>
## Acknowledgement
## Acknowledgements
This work was mainly done while the first author did a research internship at [Clova AI Research, NAVER](https://clova.ai/en/research/research-area-detail.html?id=0). We thank all the researchers at NAVER, especially Donghyun Kwak, for insightful discussions.