SWWAE Example: Conv kernel size in resid pathway to 1x1 and activation from BN+RELU to ELU (#5756)

* Changed conv kernel size in resid pathway to 1x1, and changed activation from BN+RELU to ELU.

* Added a more informative docstring decsribing elu argument and its two behaviors.

.gitignore

@@ -8,6 +8,7 @@ keras/datasets/data/*
 keras/datasets/temp/*
 docs/site/*
 docs/theme/*
+docs/sources/*
 tags
 Keras.egg-info
 

.travis.yml

@@ -4,16 +4,16 @@ language: python
 matrix:
     include:
         - python: 2.7
-          env: KERAS_BACKEND=theano TEST_MODE=PEP8
+          env: KERAS_BACKEND=tensorflow TEST_MODE=PEP8
         - python: 2.7
-          env: KERAS_BACKEND=theano TEST_MODE=INTEGRATION_TESTS
+          env: KERAS_BACKEND=tensorflow TEST_MODE=INTEGRATION_TESTS
         - python: 2.7
           env: KERAS_BACKEND=tensorflow
-        - python: 3.4
+        - python: 3.5
           env: KERAS_BACKEND=tensorflow
         - python: 2.7
           env: KERAS_BACKEND=theano
-        - python: 3.4
+        - python: 3.5
           env: KERAS_BACKEND=theano
 install:
   # code below is taken from http://conda.pydata.org/docs/travis.html
@@ -34,25 +34,20 @@ install:
 
   - conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION numpy scipy matplotlib pandas pytest h5py
   - source activate test-environment
-  - pip install pytest-cov python-coveralls pytest-xdist coverage==3.7.1 #we need this version of coverage for coveralls.io to work
-  - pip install pep8 pytest-pep8
   - pip install git+git://github.com/Theano/Theano.git
 
   # install PIL for preprocessing tests
   - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then
       conda install pil;
-    elif [[ "$TRAVIS_PYTHON_VERSION" == "3.4" ]]; then
+    elif [[ "$TRAVIS_PYTHON_VERSION" == "3.5" ]]; then
       conda install Pillow;
     fi
 
-  - python setup.py install
+  - pip install -e .[tests]
 
   # install TensorFlow
-  - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then
-      pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl;
-    elif [[ "$TRAVIS_PYTHON_VERSION" == "3.4" ]]; then
-      pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp34-cp34m-linux_x86_64.whl;
-    fi
+  - pip install tensorflow
+
 # command to run tests
 script:
   # run keras backend init to initialize backend config

CONTRIBUTING.md

@@ -43,7 +43,7 @@ We love pull requests. Here's a quick guide:
 4. Write tests. Your code should have full unit test coverage. If you want to see your PR merged promptly, this is crucial.
 
 5. Run our test suite locally. It's easy: from the Keras folder, simply run: `py.test tests/`.
-  - You will need to install `pytest`, `coveralls`, `pytest-cov`, `pytest-xdist`: `pip install pytest pytest-cov python-coveralls pytest-xdist pep8 pytest-pep8`
+  - You will need to install the test requirements as well: `pip install -e .[tests]`.
 
 6. Make sure all tests are passing:
   - with the Theano backend, on Python 2.7 and Python 3.5

README.md

@@ -3,16 +3,16 @@
 [![Build Status](https://travis-ci.org/fchollet/keras.svg?branch=master)](https://travis-ci.org/fchollet/keras)
 [![license](https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000)](https://github.com/fchollet/keras/blob/master/LICENSE)
 
+# Keras: Deep Learning library for Theano and TensorFlow
 
 ## You have just found Keras.
 
-Keras is a high-level neural networks library, written in Python and capable of running on top of either [TensorFlow](https://github.com/tensorflow/tensorflow) or [Theano](https://github.com/Theano/Theano). It was developed with a focus on enabling fast experimentation. *Being able to go from idea to result with the least possible delay is key to doing good research.*
+Keras is a high-level neural networks API, written in Python and capable of running on top of either [TensorFlow](https://github.com/tensorflow/tensorflow) or [Theano](https://github.com/Theano/Theano). It was developed with a focus on enabling fast experimentation. *Being able to go from idea to result with the least possible delay is key to doing good research.*
 
 Use Keras if you need a deep learning library that:
 
-- Allows for easy and fast prototyping (through total modularity, minimalism, and extensibility).
+- Allows for easy and fast prototyping (through user friendliness, modularity, and extensibility).
 - Supports both convolutional networks and recurrent networks, as well as combinations of the two.
-- Supports arbitrary connectivity schemes (including multi-input and multi-output training).
 - Runs seamlessly on CPU and GPU.
 
 Read the documentation at [Keras.io](http://keras.io).
@@ -25,11 +25,11 @@ Keras is compatible with: __Python 2.7-3.5__.
 
 ## Guiding principles
 
+- __User friendliness.__ Keras is an API designed for human beings, not machines. It puts user experience front and center. Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimizes the number user actions required for common use cases, and it provides clear and actionable feedback upon user error.
+
 - __Modularity.__ A model is understood as a sequence or a graph of standalone, fully-configurable modules that can be plugged together with as little restrictions as possible. In particular, neural layers, cost functions, optimizers, initialization schemes, activation functions, regularization schemes are all standalone modules that you can combine to create new models.
 
-- __Minimalism.__ Each module should be kept short and simple. Every piece of code should be transparent upon first reading. No black magic: it hurts iteration speed and ability to innovate.
-
-- __Easy extensibility.__ New modules are dead simple to add (as new classes and functions), and existing modules provide ample examples. To be able to easily create new modules allows for total expressiveness, making Keras suitable for advanced research.
+- __Easy extensibility.__ New modules are simple to add (as new classes and functions), and existing modules provide ample examples. To be able to easily create new modules allows for total expressiveness, making Keras suitable for advanced research.
 
 - __Work with Python__. No separate models configuration files in a declarative format. Models are described in Python code, which is compact, easier to debug, and allows for ease of extensibility.
 
@@ -39,9 +39,9 @@ Keras is compatible with: __Python 2.7-3.5__.
 
 ## Getting started: 30 seconds to Keras
 
-The core data structure of Keras is a __model__, a way to organize layers. The main type of model is the [`Sequential`](http://keras.io/getting-started/sequential-model-guide) model, a linear stack of layers. For more complex architectures, you should use the [Keras functional API](http://keras.io/getting-started/functional-api-guide).
+The core data structure of Keras is a __model__, a way to organize layers. The simplest type of model is the [`Sequential`](http://keras.io/getting-started/sequential-model-guide) model, a linear stack of layers. For more complex architectures, you should use the [Keras functional API](http://keras.io/getting-started/functional-api-guide), allowing to build arbitrary graphs of layers.
 
-Here's the `Sequential` model:
+Here is the `Sequential` model:
 
 ```python
 from keras.models import Sequential
@@ -54,45 +54,52 @@ Stacking layers is as easy as `.add()`:
 ```python
 from keras.layers import Dense, Activation
 
-model.add(Dense(output_dim=64, input_dim=100))
-model.add(Activation("relu"))
-model.add(Dense(output_dim=10))
-model.add(Activation("softmax"))
+model.add(Dense(units=64, input_dim=100))
+model.add(Activation('relu'))
+model.add(Dense(units=10))
+model.add(Activation('softmax'))
 ```
 
 Once your model looks good, configure its learning process with `.compile()`:
+
 ```python
-model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
+model.compile(loss='categorical_crossentropy',
+              optimizer='sgd',
+              metrics=['accuracy'])
 ```
 
 If you need to, you can further configure your optimizer. A core principle of Keras is to make things reasonably simple, while allowing the user to be fully in control when they need to (the ultimate control being the easy extensibility of the source code).
 ```python
-from keras.optimizers import SGD
-model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.01, momentum=0.9, nesterov=True))
+model.compile(loss=keras.losses.categorical_crossentropy,
+              optimizer=keras.optimizers.SGD(lr=0.01, momentum=0.9, nesterov=True))
 ```
 
 You can now iterate on your training data in batches:
+
 ```python
-model.fit(X_train, Y_train, nb_epoch=5, batch_size=32)
+# x_train and y_train are Numpy arrays --just like in the Scikit-Learn API.
+model.fit(x_train, y_train, epochs=5, batch_size=32)
 ```
 
 Alternatively, you can feed batches to your model manually:
+
 ```python
-model.train_on_batch(X_batch, Y_batch)
+model.train_on_batch(x_batch, y_batch)
 ```
 
 Evaluate your performance in one line:
+
 ```python
-loss_and_metrics = model.evaluate(X_test, Y_test, batch_size=32)
+loss_and_metrics = model.evaluate(x_test, y_test, batch_size=128)
 ```
 
 Or generate predictions on new data:
+
 ```python
-classes = model.predict_classes(X_test, batch_size=32)
-proba = model.predict_proba(X_test, batch_size=32)
+classes = model.predict(x_test, batch_size=128)
 ```
 
-Building a question answering system, an image classification model, a Neural Turing Machine, a word2vec embedder or any other model is just as fast. The ideas behind deep learning are simple, so why should their implementation be painful?
+Building a question answering system, an image classification model, a Neural Turing Machine, or any other model is just as fast. The ideas behind deep learning are simple, so why should their implementation be painful?
 
 For a more in-depth tutorial about Keras, you can check out:
 
@@ -110,7 +117,7 @@ In the [examples folder](https://github.com/fchollet/keras/tree/master/examples)
 Keras uses the following dependencies:
 
 - numpy, scipy
-- pyyaml
+- yaml
 - HDF5 and h5py (optional, required if you use model saving/loading functions)
 - Optional but recommended if you use CNNs: cuDNN.
 
@@ -118,7 +125,7 @@ Keras uses the following dependencies:
 *When using the TensorFlow backend:*
 
 - TensorFlow
-    - [See installation instructions](https://github.com/tensorflow/tensorflow#download-and-setup).
+    - [See installation instructions](https://www.tensorflow.org/install/).
 
 *When using the Theano backend:*
 

docs/autogen.py

@@ -39,7 +39,7 @@ Index
     Text preprocessing
     Sequence preprocessing
 
-Objectives
+Losses
 Metrics
 Optimizers
 Activations
@@ -65,7 +65,10 @@ if sys.version[0] == '2':
     reload(sys)
     sys.setdefaultencoding('utf8')
 
-from keras.layers import convolutional
+import keras
+from keras import utils
+from keras import layers
+from keras import initializers
 from keras.layers import pooling
 from keras.layers import local
 from keras.layers import recurrent
@@ -75,11 +78,12 @@ from keras.layers import normalization
 from keras.layers import advanced_activations
 from keras.layers import embeddings
 from keras.layers import wrappers
+from keras.layers import merge
 from keras import optimizers
 from keras import callbacks
 from keras import models
 from keras.engine import topology
-from keras import objectives
+from keras import losses
 from keras import metrics
 from keras import backend
 from keras import constraints
@@ -89,6 +93,7 @@ from keras.utils import data_utils
 from keras.utils import io_utils
 from keras.utils import layer_utils
 from keras.utils import np_utils
+from keras.utils import generic_utils
 
 
 EXCLUDE = {
@@ -97,6 +102,9 @@ EXCLUDE = {
     'get_session',
     'set_session',
     'CallbackList',
+    'serialize',
+    'deserialize',
+    'get',
 }
 
 PAGES = [
@@ -136,44 +144,35 @@ PAGES = [
     {
         'page': 'layers/core.md',
         'classes': [
-            core.Dense,
-            core.Activation,
-            core.Dropout,
-            core.SpatialDropout1D,
-            core.SpatialDropout2D,
-            core.SpatialDropout3D,
-            core.Flatten,
-            core.Reshape,
-            core.Permute,
-            core.RepeatVector,
-            topology.Merge,
-            core.Lambda,
-            core.ActivityRegularization,
-            core.Masking,
-            core.Highway,
-            core.MaxoutDense,
-            core.TimeDistributedDense,
+            layers.Dense,
+            layers.Activation,
+            layers.Dropout,
+            layers.Flatten,
+            layers.Reshape,
+            layers.Permute,
+            layers.RepeatVector,
+            layers.Lambda,
+            layers.ActivityRegularization,
+            layers.Masking,
         ],
     },
     {
         'page': 'layers/convolutional.md',
         'classes': [
-            convolutional.Convolution1D,
-            convolutional.AtrousConvolution1D,
-            convolutional.Convolution2D,
-            convolutional.AtrousConvolution2D,
-            convolutional.SeparableConvolution2D,
-            convolutional.Deconvolution2D,
-            convolutional.Convolution3D,
-            convolutional.Cropping1D,
-            convolutional.Cropping2D,
-            convolutional.Cropping3D,
-            convolutional.UpSampling1D,
-            convolutional.UpSampling2D,
-            convolutional.UpSampling3D,
-            convolutional.ZeroPadding1D,
-            convolutional.ZeroPadding2D,
-            convolutional.ZeroPadding3D,
+            layers.Conv1D,
+            layers.Conv2D,
+            layers.SeparableConv2D,
+            layers.Conv2DTranspose,
+            layers.Conv3D,
+            layers.Cropping1D,
+            layers.Cropping2D,
+            layers.Cropping3D,
+            layers.UpSampling1D,
+            layers.UpSampling2D,
+            layers.UpSampling3D,
+            layers.ZeroPadding1D,
+            layers.ZeroPadding2D,
+            layers.ZeroPadding3D,
         ],
     },
     {
@@ -227,6 +226,10 @@ PAGES = [
         'page': 'layers/noise.md',
         'all_module_classes': [noise],
     },
+    {
+        'page': 'layers/merge.md',
+        'all_module_classes': [merge],
+    },
     {
         'page': 'layers/wrappers.md',
         'all_module_classes': [wrappers],
@@ -235,6 +238,15 @@ PAGES = [
         'page': 'metrics.md',
         'all_module_functions': [metrics],
     },
+    {
+        'page': 'losses.md',
+        'all_module_functions': [losses],
+    },
+    {
+        'page': 'initializers.md',
+        'all_module_functions': [initializers],
+        'all_module_classes': [initializers],
+    },
     {
         'page': 'optimizers.md',
         'all_module_classes': [optimizers],
@@ -243,31 +255,19 @@ PAGES = [
         'page': 'callbacks.md',
         'all_module_classes': [callbacks],
     },
+    {
+        'page': 'activations.md',
+        'all_module_functions': [activations],
+    },
     {
         'page': 'backend.md',
         'all_module_functions': [backend],
     },
     {
-        'page': 'utils/data_utils.md',
-        'functions': [
-            data_utils.get_file,
-        ]
-    },
-    {
-        'page': 'utils/io_utils.md',
-        'classes': [
-            io_utils.HDF5Matrix
-        ],
-    },
-    {
-        'page': 'utils/layer_utils.md',
-        'functions': [
-            layer_utils.layer_from_config,
-        ]
-    },
-    {
-        'page': 'utils/np_utils.md',
-        'all_module_functions': [np_utils]
+        'page': 'utils.md',
+        'all_module_functions': [utils],
+        'classes': [utils.CustomObjectScope,
+                    utils.HDF5Matrix]
     },
 ]
 
@@ -316,7 +316,7 @@ def get_function_signature(function, method=True):
     for a in args:
         st += str(a) + ', '
     for a, v in kwargs:
-        if type(v) == str:
+        if isinstance(v, str):
             v = '\'' + v + '\''
         st += str(a) + '=' + str(v) + ', '
     if kwargs or args:
@@ -461,7 +461,7 @@ for page_data in PAGES:
         docstring = function.__doc__
         if docstring:
             subblocks.append(process_function_docstring(docstring))
-            blocks.append('\n\n'.join(subblocks))
+        blocks.append('\n\n'.join(subblocks))
 
     mkdown = '\n----\n\n'.join(blocks)
     # save module page.

docs/mkdocs.yml

@@ -1,15 +1,14 @@
 site_name: Keras Documentation
 theme: readthedocs
+# theme_dir: theme
 docs_dir: sources
 repo_url: http://github.com/fchollet/keras
 site_url: http://keras.io/
-# theme_dir: theme
 site_description: 'Documentation for Keras, the Python Deep Learning library.'
 
 dev_addr: '0.0.0.0:8000'
 google_analytics: ['UA-61785484-1', 'keras.io']
 
-
 pages:
 - Home: index.md
 - Getting started:
@@ -28,6 +27,7 @@ pages:
   - Locally-connected Layers: layers/local.md
   - Recurrent Layers: layers/recurrent.md
   - Embedding Layers: layers/embeddings.md
+  - Merge Layers: layers/merge.md
   - Advanced Activations Layers: layers/advanced-activations.md
   - Normalization Layers: layers/normalization.md
   - Noise layers: layers/noise.md
@@ -37,7 +37,7 @@ pages:
   - Sequence Preprocessing: preprocessing/sequence.md
   - Text Preprocessing: preprocessing/text.md
   - Image Preprocessing: preprocessing/image.md
-- Objectives: objectives.md
+- Losses: losses.md
 - Metrics: metrics.md
 - Optimizers: optimizers.md
 - Activations: activations.md
@@ -45,16 +45,9 @@ pages:
 - Datasets: datasets.md
 - Applications: applications.md
 - Backend: backend.md
-- Initializations: initializations.md
+- Initializers: initializers.md
 - Regularizers: regularizers.md
 - Constraints: constraints.md
 - Visualization: visualization.md
 - Scikit-learn API: scikit-learn-api.md
-- Utils:
-  - Data Utils: utils/data_utils.md
-  - I/O Utils: utils/io_utils.md
-  - Layer Utils: utils/layer_utils.md
-  - Numpy Utils: utils/np_utils.md
-
-
-
+- Utils: utils.md

docs/templates/activations.md

@@ -4,39 +4,31 @@
 Activations can either be used through an `Activation` layer, or through the `activation` argument supported by all forward layers:
 
 ```python
-from keras.layers.core import Activation, Dense
+from keras.layers import Activation, Dense
 
 model.add(Dense(64))
 model.add(Activation('tanh'))
 ```
-is equivalent to:
+
+This is equivalent to:
+
 ```python
 model.add(Dense(64, activation='tanh'))
 ```
 
-You can also pass an element-wise Theano/TensorFlow function as an activation:
+You can also pass an element-wise Tensorflow/Theano function as an activation:
 
 ```python
 from keras import backend as K
 
-def tanh(x):
-    return K.tanh(x)
-
-model.add(Dense(64, activation=tanh))
-model.add(Activation(tanh))
+model.add(Dense(64, activation=K.tanh))
+model.add(Activation(K.tanh))
 ```
 
 ## Available activations
 
-- __softmax__: Softmax applied across inputs last dimension. Expects shape either `(nb_samples, nb_timesteps, nb_dims)` or `(nb_samples, nb_dims)`.
-- __softplus__
-- __softsign__
-- __relu__
-- __tanh__
-- __sigmoid__
-- __hard_sigmoid__
-- __linear__
+{{autogenerated}}
 
-## On Advanced Activations
+## On "Advanced Activations"
 
-Activations that are more complex than a simple Theano/TensorFlow function (eg. learnable activations, configurable activations, etc.) are available as [Advanced Activation layers](layers/advanced-activations.md), and can be found in the module `keras.layers.advanced_activations`. These include PReLU and LeakyReLU.
+Activations that are more complex than a simple Tensorflow/Theano function (eg. learnable activations, which maintain a state) are available as [Advanced Activation layers](layers/advanced-activations.md), and can be found in the module `keras.layers.advanced_activations`. These include `PReLU` and `LeakyReLU`.

docs/templates/applications.md

@@ -15,14 +15,10 @@ Weights are downloaded automatically when instantiating a model. They are stored
 - [ResNet50](#resnet50)
 - [InceptionV3](#inceptionv3)
 
-All of these architectures (except Xception) are compatible with both TensorFlow and Theano, and upon instantiation the models will be built according to the image dimension ordering set in your Keras configuration file at `~/.keras/keras.json`. For instance, if you have set `image_dim_ordering=tf`, then any model loaded from this repository will get built according to the TensorFlow dimension ordering convention, "Width-Height-Depth".
+All of these architectures (except Xception) are compatible with both TensorFlow and Theano, and upon instantiation the models will be built according to the image data format set in your Keras configuration file at `~/.keras/keras.json`. For instance, if you have set `image_data_format=tf`, then any model loaded from this repository will get built according to the TensorFlow data format convention, "Width-Height-Depth".
 
 The Xception model is only available for TensorFlow, due to its reliance on `SeparableConvolution` layers.
 
-### Model for music audio file auto-tagging (taking as input Mel-spectrograms):
-
-- [MusicTaggerCRNN](#musictaggercrnn)
-
 -----
 
 ## Usage examples for image classification models
@@ -158,7 +154,7 @@ from keras.applications.inception_v3 import InceptionV3
 from keras.layers import Input
 
 # this could also be the output a different Keras model or layer
-input_tensor = Input(shape=(224, 224, 3))  # this assumes K.image_dim_ordering() == 'tf'
+input_tensor = Input(shape=(224, 224, 3))  # this assumes K.image_data_format() == 'channels_last'
 
 model = InceptionV3(input_tensor=input_tensor, weights='imagenet', include_top=True)
 ```
@@ -172,7 +168,6 @@ model = InceptionV3(input_tensor=input_tensor, weights='imagenet', include_top=T
 - [VGG19](#vgg19)
 - [ResNet50](#resnet50)
 - [InceptionV3](#inceptionv3)
-- [MusicTaggerCRNN](#musictaggercrnn)
 
 -----
 
@@ -191,7 +186,7 @@ and a top-5 validation accuracy of 0.945.
 
 Note that this model is only available for the TensorFlow backend,
 due to its reliance on `SeparableConvolution` layers. Additionally it only supports
-the dimension ordering "tf" (width, height, channels).
+the data format "channels_last" (height, width, channels).
 
 The default input size for this model is 299x299.
 
@@ -200,12 +195,26 @@ The default input size for this model is 299x299.
 - include_top: whether to include the fully-connected layer at the top of the network.
 - weights: one of `None` (random initialization) or "imagenet" (pre-training on ImageNet).
 - input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) to use as image input for the model.
-- inputs_shape: optional shape tuple, only to be specified
+- input_shape: optional shape tuple, only to be specified
     if `include_top` is False (otherwise the input shape
     has to be `(299, 299, 3)`.
     It should have exactly 3 inputs channels,
     and width and height should be no smaller than 71.
     E.g. `(150, 150, 3)` would be one valid value.
+- pooling: Optional pooling mode for feature extraction
+    when `include_top` is `False`.
+    - `None` means that the output of the model will be
+        the 4D tensor output of the
+        last convolutional layer.
+    - `avg` means that global average pooling
+        will be applied to the output of the
+        last convolutional layer, and thus
+        the output of the model will be a 2D tensor.
+    - `max` means that global max pooling will
+        be applied.
+- classes: optional number of classes to classify images 
+    into, only to be specified if `include_top` is True, and 
+    if no `weights` argument is specified.
 
 ### Returns
 
@@ -232,7 +241,7 @@ keras.applications.vgg16.VGG16(include_top=True, weights='imagenet', input_tenso
 VGG16 model, with weights pre-trained on ImageNet.
 
 This model is available for both the Theano and TensorFlow backend, and can be built both
-with "th" dim ordering (channels, width, height) or "tf" dim ordering (width, height, channels).
+with "channels_first" data format (channels, height, width) or "channels_last" data format (height, width, channels).
 
 The default input size for this model is 224x224.
 
@@ -241,14 +250,28 @@ The default input size for this model is 224x224.
 - include_top: whether to include the 3 fully-connected layers at the top of the network.
 - weights: one of `None` (random initialization) or "imagenet" (pre-training on ImageNet).
 - input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) to use as image input for the model.
-- inputs_shape: optional shape tuple, only to be specified
+- input_shape: optional shape tuple, only to be specified
     if `include_top` is False (otherwise the input shape
-    has to be `(224, 224, 3)` (with `tf` dim ordering)
-    or `(3, 224, 244)` (with `th` dim ordering).
+    has to be `(224, 224, 3)` (with `channels_last` data format)
+    or `(3, 224, 244)` (with `channels_first` data format).
     It should have exactly 3 inputs channels,
     and width and height should be no smaller than 48.
     E.g. `(200, 200, 3)` would be one valid value.
-
+- pooling: Optional pooling mode for feature extraction
+    when `include_top` is `False`.
+    - `None` means that the output of the model will be
+        the 4D tensor output of the
+        last convolutional layer.
+    - `avg` means that global average pooling
+        will be applied to the output of the
+        last convolutional layer, and thus
+        the output of the model will be a 2D tensor.
+    - `max` means that global max pooling will
+        be applied.
+- classes: optional number of classes to classify images 
+    into, only to be specified if `include_top` is True, and 
+    if no `weights` argument is specified.
+    
 ### Returns
 
 A Keras model instance.
@@ -274,7 +297,7 @@ keras.applications.vgg19.VGG19(include_top=True, weights='imagenet', input_tenso
 VGG19 model, with weights pre-trained on ImageNet.
 
 This model is available for both the Theano and TensorFlow backend, and can be built both
-with "th" dim ordering (channels, width, height) or "tf" dim ordering (width, height, channels).
+with "channels_first" data format (channels, height, width) or "channels_last" data format (height, width, channels).
 
 The default input size for this model is 224x224.
 
@@ -283,14 +306,28 @@ The default input size for this model is 224x224.
 - include_top: whether to include the 3 fully-connected layers at the top of the network.
 - weights: one of `None` (random initialization) or "imagenet" (pre-training on ImageNet).
 - input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) to use as image input for the model.
-- inputs_shape: optional shape tuple, only to be specified
+- input_shape: optional shape tuple, only to be specified
     if `include_top` is False (otherwise the input shape
-    has to be `(224, 224, 3)` (with `tf` dim ordering)
-    or `(3, 224, 244)` (with `th` dim ordering).
+    has to be `(224, 224, 3)` (with `channels_last` data format)
+    or `(3, 224, 244)` (with `channels_first` data format).
     It should have exactly 3 inputs channels,
     and width and height should be no smaller than 48.
     E.g. `(200, 200, 3)` would be one valid value.
-
+- pooling: Optional pooling mode for feature extraction
+    when `include_top` is `False`.
+    - `None` means that the output of the model will be
+        the 4D tensor output of the
+        last convolutional layer.
+    - `avg` means that global average pooling
+        will be applied to the output of the
+        last convolutional layer, and thus
+        the output of the model will be a 2D tensor.
+    - `max` means that global max pooling will
+        be applied.
+- classes: optional number of classes to classify images 
+    into, only to be specified if `include_top` is True, and 
+    if no `weights` argument is specified.
+    
 ### Returns
 
 A Keras model instance.
@@ -317,7 +354,7 @@ keras.applications.resnet50.ResNet50(include_top=True, weights='imagenet', input
 ResNet50 model, with weights pre-trained on ImageNet.
 
 This model is available for both the Theano and TensorFlow backend, and can be built both
-with "th" dim ordering (channels, width, height) or "tf" dim ordering (width, height, channels).
+with "channels_first" data format (channels, height, width) or "channels_last" data format (height, width, channels).
 
 The default input size for this model is 224x224.
 
@@ -327,14 +364,28 @@ The default input size for this model is 224x224.
 - include_top: whether to include the fully-connected layer at the top of the network.
 - weights: one of `None` (random initialization) or "imagenet" (pre-training on ImageNet).
 - input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) to use as image input for the model.
-- inputs_shape: optional shape tuple, only to be specified
+- input_shape: optional shape tuple, only to be specified
     if `include_top` is False (otherwise the input shape
-    has to be `(224, 224, 3)` (with `tf` dim ordering)
-    or `(3, 224, 244)` (with `th` dim ordering).
+    has to be `(224, 224, 3)` (with `channels_last` data format)
+    or `(3, 224, 244)` (with `channels_first` data format).
     It should have exactly 3 inputs channels,
     and width and height should be no smaller than 197.
     E.g. `(200, 200, 3)` would be one valid value.
-
+- pooling: Optional pooling mode for feature extraction
+    when `include_top` is `False`.
+    - `None` means that the output of the model will be
+        the 4D tensor output of the
+        last convolutional layer.
+    - `avg` means that global average pooling
+        will be applied to the output of the
+        last convolutional layer, and thus
+        the output of the model will be a 2D tensor.
+    - `max` means that global max pooling will
+        be applied.
+- classes: optional number of classes to classify images 
+    into, only to be specified if `include_top` is True, and 
+    if no `weights` argument is specified.
+    
 ### Returns
 
 A Keras model instance.
@@ -359,7 +410,7 @@ keras.applications.inception_v3.InceptionV3(include_top=True, weights='imagenet'
 Inception V3 model, with weights pre-trained on ImageNet.
 
 This model is available for both the Theano and TensorFlow backend, and can be built both
-with "th" dim ordering (channels, width, height) or "tf" dim ordering (width, height, channels).
+with "channels_first" data format (channels, height, width) or "channels_last" data format (height, width, channels).
 
 The default input size for this model is 299x299.
 
@@ -369,14 +420,28 @@ The default input size for this model is 299x299.
 - include_top: whether to include the fully-connected layer at the top of the network.
 - weights: one of `None` (random initialization) or "imagenet" (pre-training on ImageNet).
 - input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) to use as image input for the model.
-- inputs_shape: optional shape tuple, only to be specified
+- input_shape: optional shape tuple, only to be specified
     if `include_top` is False (otherwise the input shape
-    has to be `(299, 299, 3)` (with `tf` dim ordering)
-    or `(3, 299, 299)` (with `th` dim ordering).
+    has to be `(299, 299, 3)` (with `channels_last` data format)
+    or `(3, 299, 299)` (with `channels_first` data format).
     It should have exactly 3 inputs channels,
     and width and height should be no smaller than 139.
     E.g. `(150, 150, 3)` would be one valid value.
-
+- pooling: Optional pooling mode for feature extraction
+    when `include_top` is `False`.
+    - `None` means that the output of the model will be
+        the 4D tensor output of the
+        last convolutional layer.
+    - `avg` means that global average pooling
+        will be applied to the output of the
+        last convolutional layer, and thus
+        the output of the model will be a 2D tensor.
+    - `max` means that global max pooling will
+        be applied.
+- classes: optional number of classes to classify images 
+    into, only to be specified if `include_top` is True, and 
+    if no `weights` argument is specified.
+    
 ### Returns
 
 A Keras model instance.
@@ -387,65 +452,4 @@ A Keras model instance.
 
 ### License
 
-These weights are trained by ourselves and are released under the MIT license.
-
------
-
-## MusicTaggerCRNN
-
-
-```python
-keras.applications.music_tagger_crnn.MusicTaggerCRNN(weights='msd', input_tensor=None, include_top=True)
-```
-
-A convolutional-recurrent model taking as input a vectorized representation of the MelSpectrogram of a music track and capable of outputting the musical genre of the track. You can use `keras.applications.music_tagger_crnn.preprocess_input` to convert a sound file to a vectorized spectrogram. This requires to have installed the [Librosa](http://librosa.github.io/librosa/) library. See [the usage example](#music-tagging-and-feature-extraction-with-musictaggercrnn).
-
-### Arguments
-
-- weights: one of `None` (random initialization) or "msd" (pre-training on [Million Song Dataset](http://labrosa.ee.columbia.edu/millionsong/)).
-- input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) to use as image input for the model.
-- include_top: whether to include the 1 fully-connected layer (output layer) at the top of the network. If False, the network outputs 32-dim features.
-
-### Returns
-
-A Keras model instance.
-
-### References
-
-- [Convolutional Recurrent Neural Networks for Music Classification](https://arxiv.org/abs/1609.04243)
-
-### License
-
-These weights are ported from the ones [released by Keunwoo Choi](https://github.com/keunwoochoi/music-auto_tagging-keras) under the [MIT license](https://github.com/keunwoochoi/music-auto_tagging-keras/blob/master/LICENSE.md).
-
-### Examples: music tagging and audio feature extraction
-
-```python
-from keras.applications.music_tagger_crnn import MusicTaggerCRNN
-from keras.applications.music_tagger_crnn import preprocess_input, decode_predictions
-import numpy as np
-
-# 1. Tagging
-model = MusicTaggerCRNN(weights='msd')
-
-audio_path = 'audio_file.mp3'
-melgram = preprocess_input(audio_path)
-melgrams = np.expand_dims(melgram, axis=0)
-
-preds = model.predict(melgrams)
-print('Predicted:')
-print(decode_predictions(preds))
-# print: ('Predicted:', [[('rock', 0.097071797), ('pop', 0.042456303), ('alternative', 0.032439161), ('indie', 0.024491295), ('female vocalists', 0.016455274)]])
-
-#. 2. Feature extraction
-model = MusicTaggerCRNN(weights='msd', include_top=False)
-
-audio_path = 'audio_file.mp3'
-melgram = preprocess_input(audio_path)
-melgrams = np.expand_dims(melgram, axis=0)
-
-feats = model.predict(melgrams)
-print('Features:')
-print(feats[0, :10])
-# print: ('Features:', [-0.19160545 0.94259131 -0.9991011 0.47644514 -0.19089699 0.99033844 0.1103896 -0.00340496 0.14823607 0.59856361])
-```
+These weights are released under [the Apache License](https://github.com/tensorflow/models/blob/master/LICENSE).

docs/templates/backend.md

@@ -9,7 +9,7 @@ At this time, Keras has two backend implementations available: the **TensorFlow*
 - [TensorFlow](http://www.tensorflow.org/) is an open-source symbolic tensor manipulation framework developed by Google, Inc.
 - [Theano](http://deeplearning.net/software/theano/) is an open-source symbolic tensor manipulation framework developed by LISA/MILA Lab at Université de Montréal.
 
-In the future, we are likely to add more backend options. If you are interested in developing a new backend, get in touch!
+In the future, we are likely to add more backend options. Go ask Microsoft about how their CNTK backend project is doing.
 
 ----
 
@@ -25,7 +25,7 @@ The default configuration file looks like this:
 
 ```
 {
-    "image_dim_ordering": "tf",
+    "image_data_format": "channels_last",
     "epsilon": 1e-07,
     "floatx": "float32",
     "backend": "tensorflow"
@@ -49,7 +49,7 @@ Using TensorFlow backend.
 
 ```
 {
-    "image_dim_ordering": "tf",
+    "image_data_format": "channels_last",
     "epsilon": 1e-07,
     "floatx": "float32",
     "backend": "tensorflow"
@@ -58,9 +58,9 @@ Using TensorFlow backend.
 
 You can change these settings by editing `~/.keras/keras.json`. 
 
-* `image_dim_ordering`: string, either `"tf"` or `"th"`. It specifies which dimension ordering convention Keras will follow. (`keras.backend.image_dim_ordering()` returns it.)
-  - For 2D data (e.g. image), `"tf"` assumes `(rows, cols, channels)` while `"th"` assumes `(channels, rows, cols)`. 
-  - For 3D data, `"tf"` assumes `(conv_dim1, conv_dim2, conv_dim3, channels)` while `"th"` assumes `(channels, conv_dim1, conv_dim2, conv_dim3)`.
+* `image_data_format`: string, either `"channels_last"` or `"channels_first"`. It specifies which data format convention Keras will follow. (`keras.backend.image_data_format()` returns it.)
+  - For 2D data (e.g. image), `"channels_last"` assumes `(rows, cols, channels)` while `"channels_first"` assumes `(channels, rows, cols)`. 
+  - For 3D data, `"channels_last"` assumes `(conv_dim1, conv_dim2, conv_dim3, channels)` while `"channels_first"` assumes `(channels, conv_dim1, conv_dim2, conv_dim3)`.
 * `epsilon`: float, a numeric fuzzing constant used to avoid dividing by zero in some operations.
 * `floatx`: string, `"float16"`, `"float32"`, or `"float64"`. Default float precision.
 * `backend`: string, `"tensorflow"` or `"theano"`.

docs/templates/callbacks.md

@@ -1,6 +1,6 @@
 ## Usage of callbacks
 
-A callback is a set of functions to be applied at given stages of the training procedure. You can use callbacks to get a view on internal states and statistics of the model during training. You can pass a list of callbacks (as the keyword argument `callbacks`) to the `.fit()` method of the `Sequential` model. The relevant methods of the callbacks will then be called at each stage of the training. 
+A callback is a set of functions to be applied at given stages of the training procedure. You can use callbacks to get a view on internal states and statistics of the model during training. You can pass a list of callbacks (as the keyword argument `callbacks`) to the `.fit()` method of the `Sequential` or `Model` classes. The relevant methods of the callbacks will then be called at each stage of the training. 
 
 ---
 
@@ -41,7 +41,7 @@ model.add(Activation('softmax'))
 model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
 
 history = LossHistory()
-model.fit(X_train, Y_train, batch_size=128, nb_epoch=20, verbose=0, callbacks=[history])
+model.fit(X_train, Y_train, batch_size=128, epochs=20, verbose=0, callbacks=[history])
 
 print history.losses
 # outputs
@@ -66,7 +66,7 @@ model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
 saves the model weights after each epoch if the validation loss decreased
 '''
 checkpointer = ModelCheckpoint(filepath="/tmp/weights.hdf5", verbose=1, save_best_only=True)
-model.fit(X_train, Y_train, batch_size=128, nb_epoch=20, verbose=0, validation_data=(X_test, Y_test), callbacks=[checkpointer])
+model.fit(X_train, Y_train, batch_size=128, epochs=20, verbose=0, validation_data=(X_test, Y_test), callbacks=[checkpointer])
 
 ```
 

docs/templates/constraints.md

@@ -2,21 +2,21 @@
 
 Functions from the `constraints` module allow setting constraints (eg. non-negativity) on network parameters during optimization.
 
-The penalties are applied on a per-layer basis. The exact API will depend on the layer, but the layers `Dense`, `TimeDistributedDense`, `MaxoutDense`, `Convolution1D`, `Convolution2D` and `Convolution3D` have a unified API.
+The penalties are applied on a per-layer basis. The exact API will depend on the layer, but the layers `Dense`, `Convolution1D`, `Convolution2D` and `Convolution3D` have a unified API.
 
 These layers expose 2 keyword arguments:
 
-- `W_constraint` for the main weights matrix
-- `b_constraint` for the bias.
+- `kernel_constraint` for the main weights matrix
+- `bias_constraint` for the bias.
 
 
 ```python
 from keras.constraints import maxnorm
-model.add(Dense(64, W_constraint = maxnorm(2)))
+model.add(Dense(64, kernel_constraint=max_norm(2.)))
 ```
 
 ## Available constraints
 
-- __maxnorm__(m=2): maximum-norm constraint
-- __nonneg__(): non-negativity constraint
-- __unitnorm__(): unit-norm constraint, enforces the matrix to have unit norm along the last axis
+- __max_norm__(m=2): maximum-norm constraint
+- __non_neg__(): non-negativity constraint
+- __unit_norm__(): unit-norm constraint, enforces the matrix to have unit norm along the last axis

docs/templates/datasets.md

@@ -9,13 +9,14 @@ Dataset of 50,000 32x32 color training images, labeled over 10 categories, and 1
 ```python
 from keras.datasets import cifar10
 
-(X_train, y_train), (X_test, y_test) = cifar10.load_data()
+(x_train, y_train), (x_test, y_test) = cifar10.load_data()
 ```
 
-- __Return:__
+- __Returns:__
     - 2 tuples:
-        - __X_train, X_test__: uint8 array of RGB image data with shape (nb_samples, 3, 32, 32).
-        - __y_train, y_test__: uint8 array of category labels (integers in range 0-9) with shape (nb_samples,).
+        - __x_train, x_test__: uint8 array of RGB image data with shape (num_samples, 3, 32, 32).
+        - __y_train, y_test__: uint8 array of category labels (integers in range 0-9) with shape (num_samples,).
+
 
 ---
 
@@ -28,18 +29,19 @@ Dataset of 50,000 32x32 color training images, labeled over 100 categories, and
 ```python
 from keras.datasets import cifar100
 
-(X_train, y_train), (X_test, y_test) = cifar100.load_data(label_mode='fine')
+(x_train, y_train), (x_test, y_test) = cifar100.load_data(label_mode='fine')
 ```
 
-- __Return:__
+- __Returns:__
     - 2 tuples:
-        - __X_train, X_test__: uint8 array of RGB image data with shape (nb_samples, 3, 32, 32).
-        - __y_train, y_test__: uint8 array of category labels with shape (nb_samples,).
+        - __x_train, x_test__: uint8 array of RGB image data with shape (num_samples, 3, 32, 32).
+        - __y_train, y_test__: uint8 array of category labels with shape (num_samples,).
 
 - __Arguments:__
 
     - __label_mode__: "fine" or "coarse".
 
+
 ---
 
 ## IMDB Movie reviews sentiment classification
@@ -53,8 +55,8 @@ As a convention, "0" does not stand for a specific word, but instead is used to
 ```python
 from keras.datasets import imdb
 
-(X_train, y_train), (X_test, y_test) = imdb.load_data(path="imdb_full.pkl",
-                                                      nb_words=None,
+(x_train, y_train), (x_test, y_test) = imdb.load_data(path="imdb_full.pkl",
+                                                      num_words=None,
                                                       skip_top=0,
                                                       maxlen=None,
                                                       seed=113,
@@ -62,24 +64,25 @@ from keras.datasets import imdb
                                                       oov_char=2,
                                                       index_from=3)
 ```
-- __Return:__
+- __Returns:__
     - 2 tuples:
-        - __X_train, X_test__: list of sequences, which are lists of indexes (integers). If the nb_words argument was specific, the maximum possible index value is nb_words-1. If the maxlen argument was specified, the largest possible sequence length is maxlen.
+        - __x_train, x_test__: list of sequences, which are lists of indexes (integers). If the num_words argument was specific, the maximum possible index value is num_words-1. If the maxlen argument was specified, the largest possible sequence length is maxlen.
         - __y_train, y_test__: list of integer labels (1 or 0). 
 
 - __Arguments:__
 
-    - __path__: if you do have the data locally (at `'~/.keras/datasets/' + path`), if will be downloaded to this location (in cPickle format).
-    - __nb_words__: integer or None. Top most frequent words to consider. Any less frequent word will appear as 0 in the sequence data.
+    - __path__: if you do not have the data locally (at `'~/.keras/datasets/' + path`), it will be downloaded to this location.
+    - __num_words__: integer or None. Top most frequent words to consider. Any less frequent word will appear as 0 in the sequence data.
     - __skip_top__: integer. Top most frequent words to ignore (they will appear as 0s in the sequence data).
     - __maxlen__: int. Maximum sequence length. Any longer sequence will be truncated.
     - __seed__: int. Seed for reproducible data shuffling.
     - __start_char__: char. The start of a sequence will be marked with this character.
         Set to 1 because 0 is usually the padding character.
-    - __oov_char__: char. words that were cut out because of the `nb_words`
+    - __oov_char__: char. words that were cut out because of the `num_words`
         or `skip_top` limit will be replaced with this character.
     - __index_from__: int. Index actual words with this index and higher.
 
+
 ---
 
 ## Reuters newswire topics classification
@@ -91,8 +94,8 @@ Dataset of 11,228 newswires from Reuters, labeled over 46 topics. As with the IM
 ```python
 from keras.datasets import reuters
 
-(X_train, y_train), (X_test, y_test) = reuters.load_data(path="reuters.pkl",
-                                                         nb_words=None,
+(x_train, y_train), (x_test, y_test) = reuters.load_data(path="reuters.pkl",
+                                                         num_words=None,
                                                          skip_top=0,
                                                          maxlen=None,
                                                          test_split=0.2,
@@ -112,12 +115,15 @@ This dataset also makes available the word index used for encoding the sequences
 word_index = reuters.get_word_index(path="reuters_word_index.pkl")
 ```
 
-- __Return:__ A dictionary where key are words (str) and values are indexes (integer). eg. `word_index["giraffe"]` might return `1234`. 
+- __Returns:__ A dictionary where key are words (str) and values are indexes (integer). eg. `word_index["giraffe"]` might return `1234`. 
 
 - __Arguments:__
 
-    - __path__: if you do have the index file locally (at `'~/.keras/datasets/' + path`), if will be downloaded to this location (in cPickle format).
+    - __path__: if you do not have the index file locally (at `'~/.keras/datasets/' + path`), it will be downloaded to this location.
     
+
+---
+
 ## MNIST database of handwritten digits
 
 Dataset of 60,000 28x28 grayscale images of the 10 digits, along with a test set of 10,000 images.
@@ -127,14 +133,44 @@ Dataset of 60,000 28x28 grayscale images of the 10 digits, along with a test set
 ```python
 from keras.datasets import mnist
 
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
 ```
 
-- __Return:__
+- __Returns:__
     - 2 tuples:
-        - __X_train, X_test__: uint8 array of grayscale image data with shape (nb_samples, 28, 28).
-        - __y_train, y_test__: uint8 array of digit labels (integers in range 0-9) with shape (nb_samples,).
+        - __x_train, x_test__: uint8 array of grayscale image data with shape (num_samples, 28, 28).
+        - __y_train, y_test__: uint8 array of digit labels (integers in range 0-9) with shape (num_samples,).
 
 - __Arguments:__
 
-    - __path__: if you do have the index file locally (at `'~/.keras/datasets/' + path`), if will be downloaded to this location (in cPickle format).
+    - __path__: if you do not have the index file locally (at `'~/.keras/datasets/' + path`), it will be downloaded to this location.
+
+
+---
+
+## Boston housing price regression dataset
+
+
+Dataset taken from the StatLib library which is maintained at Carnegie Mellon University. 
+
+Samples contain 13 attributes of houses at different locations around the Boston suburbs in the late 1970s.
+Targets are the median values of the houses at a location (in k$).
+
+
+### Usage:
+
+```python
+from keras.datasets import boston_housing
+
+(x_train, y_train), (x_test, y_test) = boston_housing.load_data()
+```
+
+- __Arguments:__
+    - __path__: path where to cache the dataset locally
+        (relative to ~/.keras/datasets).
+    - __seed__: Random seed for shuffling the data
+        before computing the test split.
+    - __test_split__: fraction of the data to reserve as test set.
+
+- __Returns:__
+    Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.

docs/templates/getting-started/faq.md

@@ -2,6 +2,7 @@
 
 - [How should I cite Keras?](#how-should-i-cite-keras)
 - [How can I run Keras on GPU?](#how-can-i-run-keras-on-gpu)
+- [What does \["sample", "batch", "epoch"\] mean?](#what-does-sample-batch-epoch-mean)
 - [How can I save a Keras model?](#how-can-i-save-a-keras-model)
 - [Why is the training loss much higher than the testing loss?](#why-is-the-training-loss-much-higher-than-the-testing-loss)
 - [How can I obtain the output of an intermediate layer?](#how-can-i-obtain-the-output-of-an-intermediate-layer)
@@ -31,9 +32,12 @@ Please cite Keras in your publications if it helps your research. Here is an exa
 }
 ```
 
+---
+
 ### How can I run Keras on GPU?
 
 If you are running on the TensorFlow backend, your code will automatically run on GPU if any available GPU is detected.
+
 If you are running on the Theano backend, you can use one of the following methods:
 
 Method 1: use Theano flags.
@@ -54,6 +58,21 @@ theano.config.floatX = 'float32'
 
 ---
 
+### What does \["sample", "batch", "epoch"\] mean?
+
+Below are some common definitions that are necessary to know and understand to correctly utilize Keras:
+
+- **Sample**: one element of a dataset.
+  - *Example:* one image is a **sample** in a convolutional network
+  - *Example:* one audio file is a **sample** for a speech recognition model
+- **Batch**: a set of *N* samples. The samples in a **batch** are processed independently, in parallel. If training, a batch results in only one update to the model.
+  - A **batch** generally approximates the distribution of the input data better than a single input. The larger the batch, the better the approximation; however, it is also true that the batch will take longer to processes and will still result in only one update. For inference (evaluate/predict), it is recommended to pick a batch size that is as large as you can afford without going out of memory (since larger batches will usually result in faster evaluating/prediction).
+- **Epoch**: an arbitrary cutoff, generally defined as "one pass over the entire dataset", used to separate training into distinct phases, which is useful for logging and periodic evaluation.
+  - When using `evaluation_data` or `evaluation_split` with the `fit` method of Keras models, evaluation will be run at the end of every **epoch**.
+  - Within Keras, there is the ability to add [callbacks](https://keras.io/callbacks/) specifically designed to be run at the end of an **epoch**. Examples of these are learning rate changes and model checkpointing (saving).
+
+---
+
 ### How can I save a Keras model?
 
 *It is not recommended to use pickle or cPickle to save a Keras model.*
@@ -102,6 +121,7 @@ from keras.models import model_from_json
 model = model_from_json(json_string)
 
 # model reconstruction from YAML
+from keras.models import model_from_yaml
 model = model_from_yaml(yaml_string)
 ```
 
@@ -166,8 +186,8 @@ from keras.models import Model
 model = ...  # create the original model
 
 layer_name = 'my_layer'
-intermediate_layer_model = Model(input=model.input,
-                                 output=model.get_layer(layer_name).output)
+intermediate_layer_model = Model(inputs=model.input,
+                                 outputs=model.get_layer(layer_name).output)
 intermediate_output = intermediate_layer_model.predict(data)
 ```
 
@@ -204,7 +224,7 @@ layer_output = get_3rd_layer_output([X, 1])[0]
 
 You can do batch training using `model.train_on_batch(X, y)` and `model.test_on_batch(X, y)`. See the [models documentation](/models/sequential).
 
-Alternatively, you can write a generator that yields batches of training data and use the method `model.fit_generator(data_generator, samples_per_epoch, nb_epoch)`.
+Alternatively, you can write a generator that yields batches of training data and use the method `model.fit_generator(data_generator, samples_per_epoch, epochs)`.
 
 You can see batch training in action in our [CIFAR10 example](https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py).
 
@@ -297,7 +317,7 @@ When using stateful RNNs, it is therefore assumed that:
 
 To use statefulness in RNNs, you need to:
 
-- explicitly specify the batch size you are using, by passing a `batch_input_shape` argument to the first layer in your model. It should be a tuple of integers, e.g. `(32, 10, 16)` for a 32-samples batch of sequences of 10 timesteps with 16 features per timestep.
+- explicitly specify the batch size you are using, by passing a `batch_size` argument to the first layer in your model. E.g. `batch_size=32` for a 32-samples batch of sequences of 10 timesteps with 16 features per timestep.
 - set `stateful=True` in your RNN layer(s).
 
 To reset the states accumulated:
@@ -313,7 +333,7 @@ X  # this is our input data, of shape (32, 21, 16)
 # we will feed it to our model in sequences of length 10
 
 model = Sequential()
-model.add(LSTM(32, batch_input_shape=(32, 10, 16), stateful=True))
+model.add(LSTM(32, input_shape=(10, 16), batch_size=32, stateful=True))
 model.add(Dense(16, activation='softmax'))
 
 model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
@@ -356,6 +376,7 @@ print(len(model.layers))  # "1"
 
 Code and pre-trained weights are available for the following image classification models:
 
+- Xception
 - VGG16
 - VGG19
 - ResNet50
@@ -364,6 +385,7 @@ Code and pre-trained weights are available for the following image classificatio
 They can be imported from the module `keras.applications`:
 
 ```python
+from keras.applications.xception import Xception
 from keras.applications.vgg16 import VGG16
 from keras.applications.vgg19 import VGG19
 from keras.applications.resnet50 import ResNet50

docs/templates/getting-started/functional-api-guide.md

@@ -8,7 +8,7 @@ Let's start with something simple.
 
 -----
 
-## First example: fully connected network
+## First example: a densely-connected network
 
 The `Sequential` model is probably a better choice to implement such a network, but it helps to start with something really simple.
 
@@ -20,7 +20,7 @@ The `Sequential` model is probably a better choice to implement such a network,
 from keras.layers import Input, Dense
 from keras.models import Model
 
-# this returns a tensor
+# This returns a tensor
 inputs = Input(shape=(784,))
 
 # a layer instance is callable on a tensor, and returns a tensor
@@ -28,9 +28,9 @@ x = Dense(64, activation='relu')(inputs)
 x = Dense(64, activation='relu')(x)
 predictions = Dense(10, activation='softmax')(x)
 
-# this creates a model that includes
+# This creates a model that includes
 # the Input layer and three Dense layers
-model = Model(input=inputs, output=predictions)
+model = Model(inputs=inputs, outputs=predictions)
 model.compile(optimizer='rmsprop',
               loss='categorical_crossentropy',
               metrics=['accuracy'])
@@ -45,7 +45,7 @@ With the functional API, it is easy to re-use trained models: you can treat any
 
 ```python
 x = Input(shape=(784,))
-# this works, and returns the 10-way softmax we defined above.
+# This works, and returns the 10-way softmax we defined above.
 y = model(x)
 ```
 
@@ -54,11 +54,11 @@ This can allow, for instance, to quickly create models that can process *sequenc
 ```python
 from keras.layers import TimeDistributed
 
-# input tensor for sequences of 20 timesteps,
+# Input tensor for sequences of 20 timesteps,
 # each containing a 784-dimensional vector
 input_sequences = Input(shape=(20, 784))
 
-# this applies our previous model to every timestep in the input sequences.
+# This applies our previous model to every timestep in the input sequences.
 # the output of the previous model was a 10-way softmax,
 # so the output of the layer below will be a sequence of 20 vectors of size 10.
 processed_sequences = TimeDistributed(model)(input_sequences)
@@ -83,18 +83,18 @@ The main input will receive the headline, as a sequence of integers (each intege
 The integers will be between 1 and 10,000 (a vocabulary of 10,000 words) and the sequences will be 100 words long.
 
 ```python
-from keras.layers import Input, Embedding, LSTM, Dense, merge
+from keras.layers import Input, Embedding, LSTM, Dense
 from keras.models import Model
 
-# headline input: meant to receive sequences of 100 integers, between 1 and 10000.
-# note that we can name any layer by passing it a "name" argument.
+# Headline input: meant to receive sequences of 100 integers, between 1 and 10000.
+# Note that we can name any layer by passing it a "name" argument.
 main_input = Input(shape=(100,), dtype='int32', name='main_input')
 
-# this embedding layer will encode the input sequence
+# This embedding layer will encode the input sequence
 # into a sequence of dense 512-dimensional vectors.
 x = Embedding(output_dim=512, input_dim=10000, input_length=100)(main_input)
 
-# a LSTM will transform the vector sequence into a single vector,
+# A LSTM will transform the vector sequence into a single vector,
 # containing information about the entire sequence
 lstm_out = LSTM(32)(x)
 ```
@@ -109,21 +109,21 @@ At this point, we feed into the model our auxiliary input data by concatenating
 
 ```python
 auxiliary_input = Input(shape=(5,), name='aux_input')
-x = merge([lstm_out, auxiliary_input], mode='concat')
+x = keras.layers.concatenate([lstm_out, auxiliary_input])
 
-# we stack a deep fully-connected network on top
+# We stack a deep densely-connected network on top
 x = Dense(64, activation='relu')(x)
 x = Dense(64, activation='relu')(x)
 x = Dense(64, activation='relu')(x)
 
-# and finally we add the main logistic regression layer
+# And finally we add the main logistic regression layer
 main_output = Dense(1, activation='sigmoid', name='main_output')(x)
 ```
 
 This defines a model with two inputs and two outputs:
 
 ```python
-model = Model(input=[main_input, auxiliary_input], output=[main_output, auxiliary_output])
+model = Model(inputs=[main_input, auxiliary_input], outputs=[main_output, auxiliary_output])
 ```
 
 We compile the model and assign a weight of 0.2 to the auxiliary loss.
@@ -139,7 +139,7 @@ We can train the model by passing it lists of input arrays and target arrays:
 
 ```python
 model.fit([headline_data, additional_data], [labels, labels],
-          nb_epoch=50, batch_size=32)
+          epochs=50, batch_size=32)
 ```
 
 Since our inputs and outputs are named (we passed them a "name" argument),
@@ -150,10 +150,10 @@ model.compile(optimizer='rmsprop',
               loss={'main_output': 'binary_crossentropy', 'aux_output': 'binary_crossentropy'},
               loss_weights={'main_output': 1., 'aux_output': 0.2})
 
-# and trained it via:
+# And trained it via:
 model.fit({'main_input': headline_data, 'aux_input': additional_data},
           {'main_output': labels, 'aux_output': labels},
-          nb_epoch=50, batch_size=32)
+          epochs=50, batch_size=32)
 ```
 
 -----
@@ -171,7 +171,8 @@ Because the problem is symmetric, the mechanism that encodes the first tweet sho
 Let's build this with the functional API. We will take as input for a tweet a binary matrix of shape `(140, 256)`, i.e. a sequence of 140 vectors of size 256, where each dimension in the 256-dimensional vector encodes the presence/absence of a character (out of an alphabet of 256 frequent characters).
 
 ```python
-from keras.layers import Input, LSTM, Dense, merge
+import keras
+from keras.layers import Input, LSTM, Dense
 from keras.models import Model
 
 tweet_a = Input(shape=(140, 256))
@@ -181,31 +182,31 @@ tweet_b = Input(shape=(140, 256))
 To share a layer across different inputs, simply instantiate the layer once, then call it on as many inputs as you want:
 
 ```python
-# this layer can take as input a matrix
+# This layer can take as input a matrix
 # and will return a vector of size 64
 shared_lstm = LSTM(64)
 
-# when we reuse the same layer instance
+# When we reuse the same layer instance
 # multiple times, the weights of the layer
 # are also being reused
 # (it is effectively *the same* layer)
 encoded_a = shared_lstm(tweet_a)
 encoded_b = shared_lstm(tweet_b)
 
-# we can then concatenate the two vectors:
-merged_vector = merge([encoded_a, encoded_b], mode='concat', concat_axis=-1)
+# We can then concatenate the two vectors:
+merged_vector = keras.layers.concatenate([encoded_a, encoded_b], axis=-1)
 
-# and add a logistic regression on top
+# And add a logistic regression on top
 predictions = Dense(1, activation='sigmoid')(merged_vector)
 
-# we define a trainable model linking the
+# We define a trainable model linking the
 # tweet inputs to the predictions
-model = Model(input=[tweet_a, tweet_b], output=predictions)
+model = Model(inputs=[tweet_a, tweet_b], outputs=predictions)
 
 model.compile(optimizer='rmsprop',
               loss='binary_crossentropy',
               metrics=['accuracy'])
-model.fit([data_a, data_b], labels, nb_epoch=10)
+model.fit([data_a, data_b], labels, epochs=10)
 ```
 
 Let's pause to take a look at how to read the shared layer's output or output shape.
@@ -255,16 +256,16 @@ assert lstm.get_output_at(1) == encoded_b
 
 Simple enough, right?
 
-The same is true for the properties `input_shape` and `output_shape`: as long as the layer has only one node, or as long as all nodes have the same input/output shape, then the notion of "layer output/input shape" is well defined, and that one shape will be returned by `layer.output_shape`/`layer.input_shape`. But if, for instance, you apply a same `Convolution2D` layer to an input of shape `(3, 32, 32)`, and then to an input of shape `(3, 64, 64)`, the layer will have multiple input/output shapes, and you will have to fetch them by specifying the index of the node they belong to:
+The same is true for the properties `input_shape` and `output_shape`: as long as the layer has only one node, or as long as all nodes have the same input/output shape, then the notion of "layer output/input shape" is well defined, and that one shape will be returned by `layer.output_shape`/`layer.input_shape`. But if, for instance, you apply a same `Conv2D` layer to an input of shape `(3, 32, 32)`, and then to an input of shape `(3, 64, 64)`, the layer will have multiple input/output shapes, and you will have to fetch them by specifying the index of the node they belong to:
 
 ```python
 a = Input(shape=(3, 32, 32))
 b = Input(shape=(3, 64, 64))
 
-conv = Convolution2D(16, 3, 3, border_mode='same')
+conv = Conv2D(16, (3, 3), padding='same')
 conved_a = conv(a)
 
-# only one input so far, the following will work:
+# Only one input so far, the following will work:
 assert conv.input_shape == (None, 3, 32, 32)
 
 conved_b = conv(b)
@@ -284,20 +285,20 @@ Code examples are still the best way to get started, so here are a few more.
 For more information about the Inception architecture, see [Going Deeper with Convolutions](http://arxiv.org/abs/1409.4842).
 
 ```python
-from keras.layers import merge, Convolution2D, MaxPooling2D, Input
+from keras.layers import Conv2D, MaxPooling2D, Input
 
 input_img = Input(shape=(3, 256, 256))
 
-tower_1 = Convolution2D(64, 1, 1, border_mode='same', activation='relu')(input_img)
-tower_1 = Convolution2D(64, 3, 3, border_mode='same', activation='relu')(tower_1)
+tower_1 = Conv2D(64, (1, 1), padding='same', activation='relu')(input_img)
+tower_1 = Conv2D(64, (3, 3), padding='same', activation='relu')(tower_1)
 
-tower_2 = Convolution2D(64, 1, 1, border_mode='same', activation='relu')(input_img)
-tower_2 = Convolution2D(64, 5, 5, border_mode='same', activation='relu')(tower_2)
+tower_2 = Conv2D(64, (1, 1), padding='same', activation='relu')(input_img)
+tower_2 = Conv2D(64, (5, 5), padding='same', activation='relu')(tower_2)
 
-tower_3 = MaxPooling2D((3, 3), strides=(1, 1), border_mode='same')(input_img)
-tower_3 = Convolution2D(64, 1, 1, border_mode='same', activation='relu')(tower_3)
+tower_3 = MaxPooling2D((3, 3), strides=(1, 1), padding='same')(input_img)
+tower_3 = Conv2D(64, (1, 1), padding='same', activation='relu')(tower_3)
 
-output = merge([tower_1, tower_2, tower_3], mode='concat', concat_axis=1)
+output = keras.layers.concatenate([tower_1, tower_2, tower_3], axis=1)
 ```
 
 ### Residual connection on a convolution layer
@@ -305,14 +306,14 @@ output = merge([tower_1, tower_2, tower_3], mode='concat', concat_axis=1)
 For more information about residual networks, see [Deep Residual Learning for Image Recognition](http://arxiv.org/abs/1512.03385).
 
 ```python
-from keras.layers import merge, Convolution2D, Input
+from keras.layers import Conv2D, Input
 
 # input tensor for a 3-channel 256x256 image
 x = Input(shape=(3, 256, 256))
 # 3x3 conv with 3 output channels (same as input channels)
-y = Convolution2D(3, 3, 3, border_mode='same')(x)
+y = Conv2D(3, (3, 3), padding='same')(x)
 # this returns x + y.
-z = merge([x, y], mode='sum')
+z = keras.layers.add([x, y])
 ```
 
 ### Shared vision model
@@ -320,27 +321,27 @@ z = merge([x, y], mode='sum')
 This model re-uses the same image-processing module on two inputs, to classify whether two MNIST digits are the same digit or different digits.
 
 ```python
-from keras.layers import merge, Convolution2D, MaxPooling2D, Input, Dense, Flatten
+from keras.layers import Conv2D, MaxPooling2D, Input, Dense, Flatten
 from keras.models import Model
 
-# first, define the vision modules
+# First, define the vision modules
 digit_input = Input(shape=(1, 27, 27))
-x = Convolution2D(64, 3, 3)(digit_input)
-x = Convolution2D(64, 3, 3)(x)
+x = Conv2D(64, (3, 3))(digit_input)
+x = Conv2D(64, (3, 3))(x)
 x = MaxPooling2D((2, 2))(x)
 out = Flatten()(x)
 
 vision_model = Model(digit_input, out)
 
-# then define the tell-digits-apart model
+# Then define the tell-digits-apart model
 digit_a = Input(shape=(1, 27, 27))
 digit_b = Input(shape=(1, 27, 27))
 
-# the vision model will be shared, weights and all
+# The vision model will be shared, weights and all
 out_a = vision_model(digit_a)
 out_b = vision_model(digit_b)
 
-concatenated = merge([out_a, out_b], mode='concat')
+concatenated = keras.layers.concatenate([out_a, out_b])
 out = Dense(1, activation='sigmoid')(concatenated)
 
 classification_model = Model([digit_a, digit_b], out)
@@ -353,46 +354,46 @@ This model can select the correct one-word answer when asked a natural-language
 It works by encoding the question into a vector, encoding the image into a vector, concatenating the two, and training on top a logistic regression over some vocabulary of potential answers.
 
 ```python
-from keras.layers import Convolution2D, MaxPooling2D, Flatten
-from keras.layers import Input, LSTM, Embedding, Dense, merge
+from keras.layers import Conv2D, MaxPooling2D, Flatten
+from keras.layers import Input, LSTM, Embedding, Dense
 from keras.models import Model, Sequential
 
-# first, let's define a vision model using a Sequential model.
-# this model will encode an image into a vector.
+# First, let's define a vision model using a Sequential model.
+# This model will encode an image into a vector.
 vision_model = Sequential()
-vision_model.add(Convolution2D(64, 3, 3, activation='relu', border_mode='same', input_shape=(3, 224, 224)))
-vision_model.add(Convolution2D(64, 3, 3, activation='relu'))
+vision_model.add(Conv2D(64, (3, 3) activation='relu', padding='same', input_shape=(3, 224, 224)))
+vision_model.add(Conv2D(64, (3, 3), activation='relu'))
 vision_model.add(MaxPooling2D((2, 2)))
-vision_model.add(Convolution2D(128, 3, 3, activation='relu', border_mode='same'))
-vision_model.add(Convolution2D(128, 3, 3, activation='relu'))
+vision_model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
+vision_model.add(Conv2D(128, (3, 3), activation='relu'))
 vision_model.add(MaxPooling2D((2, 2)))
-vision_model.add(Convolution2D(256, 3, 3, activation='relu', border_mode='same'))
-vision_model.add(Convolution2D(256, 3, 3, activation='relu'))
-vision_model.add(Convolution2D(256, 3, 3, activation='relu'))
+vision_model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
+vision_model.add(Conv2D(256, (3, 3), activation='relu'))
+vision_model.add(Conv2D(256, (3, 3), activation='relu'))
 vision_model.add(MaxPooling2D((2, 2)))
 vision_model.add(Flatten())
 
-# now let's get a tensor with the output of our vision model:
+# Now let's get a tensor with the output of our vision model:
 image_input = Input(shape=(3, 224, 224))
 encoded_image = vision_model(image_input)
 
-# next, let's define a language model to encode the question into a vector.
-# each question will be at most 100 word long,
+# Next, let's define a language model to encode the question into a vector.
+# Each question will be at most 100 word long,
 # and we will index words as integers from 1 to 9999.
 question_input = Input(shape=(100,), dtype='int32')
 embedded_question = Embedding(input_dim=10000, output_dim=256, input_length=100)(question_input)
 encoded_question = LSTM(256)(embedded_question)
 
-# let's concatenate the question vector and the image vector:
-merged = merge([encoded_question, encoded_image], mode='concat')
+# Let's concatenate the question vector and the image vector:
+merged = keras.layers.concatenate([encoded_question, encoded_image])
 
-# and let's train a logistic regression over 1000 words on top:
+# And let's train a logistic regression over 1000 words on top:
 output = Dense(1000, activation='softmax')(merged)
 
-# this is our final model:
-vqa_model = Model(input=[image_input, question_input], output=output)
+# This is our final model:
+vqa_model = Model(inputs=[image_input, question_input], outputs=output)
 
-# the next stage would be training this model on actual data.
+# The next stage would be training this model on actual data.
 ```
 
 ### Video question answering model
@@ -403,19 +404,19 @@ Now that we have trained our image QA model, we can quickly turn it into a video
 from keras.layers import TimeDistributed
 
 video_input = Input(shape=(100, 3, 224, 224))
-# this is our video encoded via the previously trained vision_model (weights are reused)
+# This is our video encoded via the previously trained vision_model (weights are reused)
 encoded_frame_sequence = TimeDistributed(vision_model)(video_input)  # the output will be a sequence of vectors
 encoded_video = LSTM(256)(encoded_frame_sequence)  # the output will be a vector
 
-# this is a model-level representation of the question encoder, reusing the same weights as before:
-question_encoder = Model(input=question_input, output=encoded_question)
+# This is a model-level representation of the question encoder, reusing the same weights as before:
+question_encoder = Model(inputs=question_input, outputs=encoded_question)
 
-# let's use it to encode the question:
+# Let's use it to encode the question:
 video_question_input = Input(shape=(100,), dtype='int32')
 encoded_video_question = question_encoder(video_question_input)
 
-# and this is our video question answering model:
-merged = merge([encoded_video, encoded_video_question], mode='concat')
+# And this is our video question answering model:
+merged = keras.layers.concatenate([encoded_video, encoded_video_question])
 output = Dense(1000, activation='softmax')(merged)
-video_qa_model = Model(input=[video_input, video_question_input], output=output)
+video_qa_model = Model(inputs=[video_input, video_question_input], outputs=output)
 ```

docs/templates/getting-started/sequential-model-guide.md

@@ -30,192 +30,101 @@ model.add(Activation('relu'))
 
 The model needs to know what input shape it should expect. For this reason, the first layer in a `Sequential` model (and only the first, because following layers can do automatic shape inference) needs to receive information about its input shape. There are several possible ways to do this:
 
-- pass an `input_shape` argument to the first layer. This is a shape tuple (a tuple of integers or `None` entries, where `None` indicates that any positive integer may be expected). In `input_shape`, the batch dimension is not included.
-- pass instead a `batch_input_shape` argument, where the batch dimension is included. This is useful for specifying a fixed batch size (e.g. with stateful RNNs).
-- some 2D layers, such as `Dense`, support the specification of their input shape via the argument `input_dim`, and some 3D temporal layers support the arguments `input_dim` and `input_length`.
+- Pass an `input_shape` argument to the first layer. This is a shape tuple (a tuple of integers or `None` entries, where `None` indicates that any positive integer may be expected). In `input_shape`, the batch dimension is not included.
+- Some 2D layers, such as `Dense`, support the specification of their input shape via the argument `input_dim`, and some 3D temporal layers support the arguments `input_dim` and `input_length`.
+- If you ever need to specify a fixed batch size for your inputs (this is useful for stateful recurrent networks), you can pass a `batch_size` argument to a layer. If you pass both `batch_size=32` and `input_shape=(6, 8)` to a layer, it will then expect every batch of inputs to have the batch shape `(32, 6, 8)`.
 
-As such, the following three snippets are strictly equivalent:
+As such, the following snippets are strictly equivalent:
 ```python
 model = Sequential()
 model.add(Dense(32, input_shape=(784,)))
 ```
 ```python
 model = Sequential()
-model.add(Dense(32, batch_input_shape=(None, 784)))
-# note that batch dimension is "None" here,
-# so the model will be able to process batches of any size.
-```
-```python
-model = Sequential()
 model.add(Dense(32, input_dim=784))
 ```
 
-And so are the following three snippets:
-```python
-model = Sequential()
-model.add(LSTM(32, input_shape=(10, 64)))
-```
-```python
-model = Sequential()
-model.add(LSTM(32, batch_input_shape=(None, 10, 64)))
-```
-```python
-model = Sequential()
-model.add(LSTM(32, input_length=10, input_dim=64))
-```
-
-----
-
-## The Merge layer
-
-Multiple `Sequential` instances can be merged into a single output via a `Merge` layer. The output is a layer that can be added as first layer in a new `Sequential` model. For instance, here's a model with two separate input branches getting merged:
-
-```python
-from keras.layers import Merge
-
-left_branch = Sequential()
-left_branch.add(Dense(32, input_dim=784))
-
-right_branch = Sequential()
-right_branch.add(Dense(32, input_dim=784))
-
-merged = Merge([left_branch, right_branch], mode='concat')
-
-final_model = Sequential()
-final_model.add(merged)
-final_model.add(Dense(10, activation='softmax'))
-```
-
-<img src="https://s3.amazonaws.com/keras.io/img/two_branches_sequential_model.png" alt="two branch Sequential" style="width: 400px;"/>
-
-Such a two-branch model can then be trained via e.g.:
-
-```python
-final_model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
-final_model.fit([input_data_1, input_data_2], targets)  # we pass one data array per model input
-```
-
-The `Merge` layer supports a number of pre-defined modes:
-
-- `sum` (default): element-wise sum
-- `concat`: tensor concatenation. You can specify the concatenation axis via the argument `concat_axis`.
-- `mul`: element-wise multiplication
-- `ave`: tensor average
-- `dot`: dot product. You can specify which axes to reduce along via the argument `dot_axes`.
-- `cos`: cosine proximity between vectors in 2D tensors.
-
-You can also pass a function as the `mode` argument, allowing for arbitrary transformations:
-
-```python
-merged = Merge([left_branch, right_branch], mode=lambda x: x[0] - x[1])
-```
-
-Now you know enough to be able to define *almost* any model with Keras. For complex models that cannot be expressed via `Sequential` and `Merge`, you can use [the functional API](/getting-started/functional-api-guide).
-
-
 ----
 
 ## Compilation
 
 Before training a model, you need to configure the learning process, which is done via the `compile` method. It receives three arguments:
 
-- an optimizer. This could be the string identifier of an existing optimizer (such as `rmsprop` or `adagrad`), or an instance of the `Optimizer` class. See: [optimizers](/optimizers).
-- a loss function. This is the objective that the model will try to minimize. It can be the string identifier of an existing loss function (such as `categorical_crossentropy` or `mse`), or it can be an objective function. See: [objectives](/objectives).
-- a list of metrics. For any classification problem you will want to set this to `metrics=['accuracy']`. A metric could be the string identifier of an existing metric or a custom metric function.  Custom metric function should return either a single tensor value or a dict `metric_name -> metric_value`. See: [metrics](/metrics).
+- An optimizer. This could be the string identifier of an existing optimizer (such as `rmsprop` or `adagrad`), or an instance of the `Optimizer` class. See: [optimizers](/optimizers).
+- A loss function. This is the objective that the model will try to minimize. It can be the string identifier of an existing loss function (such as `categorical_crossentropy` or `mse`), or it can be an objective function. See: [losses](/losses).
+- A list of metrics. For any classification problem you will want to set this to `metrics=['accuracy']`. A metric could be the string identifier of an existing metric or a custom metric function.
 
 ```python
-# for a multi-class classification problem
+# For a multi-class classification problem
 model.compile(optimizer='rmsprop',
               loss='categorical_crossentropy',
               metrics=['accuracy'])
 
-# for a binary classification problem
+# For a binary classification problem
 model.compile(optimizer='rmsprop',
               loss='binary_crossentropy',
               metrics=['accuracy'])
 
-# for a mean squared error regression problem
+# For a mean squared error regression problem
 model.compile(optimizer='rmsprop',
               loss='mse')
 
-# for custom metrics
+# For custom metrics
 import keras.backend as K
 
 def mean_pred(y_true, y_pred):
     return K.mean(y_pred)
 
-def false_rates(y_true, y_pred):
-    false_neg = ...
-    false_pos = ...
-    return {
-        'false_neg': false_neg,
-        'false_pos': false_pos,
-    }
-
 model.compile(optimizer='rmsprop',
               loss='binary_crossentropy',
-              metrics=['accuracy', mean_pred, false_rates])
+              metrics=['accuracy', mean_pred])
 ```
 
 ----
 
 ## Training
 
-Keras models are trained on Numpy arrays of input data and labels. For training a model, you will typically use the `fit` function. [Read its documentation here](/models/sequential). 
+Keras models are trained on Numpy arrays of input data and labels. For training a model, you will typically use the `fit` function. [Read its documentation here](/models/sequential).
 
 ```python
-# for a single-input model with 2 classes (binary):
+# For a single-input model with 2 classes (binary classification):
 
 model = Sequential()
-model.add(Dense(1, input_dim=784, activation='sigmoid'))
+model.add(Dense(32, activation='relu', input_dim=100))
+model.add(Dense(1, activation='sigmoid'))
 model.compile(optimizer='rmsprop',
               loss='binary_crossentropy',
               metrics=['accuracy'])
 
-# generate dummy data
+# Generate dummy data
 import numpy as np
 data = np.random.random((1000, 784))
 labels = np.random.randint(2, size=(1000, 1))
 
-# train the model, iterating on the data in batches
-# of 32 samples
-model.fit(data, labels, nb_epoch=10, batch_size=32)
+# Train the model, iterating on the data in batches of 32 samples
+model.fit(data, labels, epochs=10, batch_size=32)
 ```
+
 ```python
-# for a multi-input model with 10 classes:
-
-left_branch = Sequential()
-left_branch.add(Dense(32, input_dim=784))
-
-right_branch = Sequential()
-right_branch.add(Dense(32, input_dim=784))
-
-merged = Merge([left_branch, right_branch], mode='concat')
+# For a single-input model with 10 classes (categorical classification):
 
 model = Sequential()
-model.add(merged)
+model.add(Dense(32, activation='relu', input_dim=100))
 model.add(Dense(10, activation='softmax'))
-
 model.compile(optimizer='rmsprop',
               loss='categorical_crossentropy',
               metrics=['accuracy'])
 
-# generate dummy data
+# Generate dummy data
 import numpy as np
-from keras.utils.np_utils import to_categorical
-data_1 = np.random.random((1000, 784))
-data_2 = np.random.random((1000, 784))
+data = np.random.random((1000, 784))
+labels = np.random.randint(10, size=(1000, 10))
 
-# these are integers between 0 and 9
-labels = np.random.randint(10, size=(1000, 1))
-# we convert the labels to a binary matrix of size (1000, 10)
-# for use with categorical_crossentropy
-labels = to_categorical(labels, 10)
+# Convert labels to categorical one-hot encoding
+binary_labels = keras.utils.to_categorical(labels, num_classes=10)
 
-# train the model
-# note that we are passing a list of Numpy arrays as training data
-# since the model has 2 inputs
-model.fit([data_1, data_2], labels, nb_epoch=10, batch_size=32)
+# Train the model, iterating on the data in batches of 32 samples
+model.fit(data, binary_labels, epochs=10, batch_size=32)
 ```
 
 ----
@@ -225,7 +134,7 @@ model.fit([data_1, data_2], labels, nb_epoch=10, batch_size=32)
 
 Here are a few examples to get you started!
 
-In the examples folder, you will also find example models for real datasets:
+In the [examples folder](https://github.com/fchollet/keras/tree/master/examples), you will also find example models for real datasets:
 
 - CIFAR10 small images classification: Convolutional Neural Network (CNN) with realtime data augmentation
 - IMDB movie review sentiment classification: LSTM over sequences of words
@@ -247,47 +156,29 @@ model = Sequential()
 # Dense(64) is a fully-connected layer with 64 hidden units.
 # in the first layer, you must specify the expected input data shape:
 # here, 20-dimensional vectors.
-model.add(Dense(64, input_dim=20, init='uniform'))
-model.add(Activation('tanh'))
-model.add(Dropout(0.5))
-model.add(Dense(64, init='uniform'))
-model.add(Activation('tanh'))
-model.add(Dropout(0.5))
-model.add(Dense(10, init='uniform'))
-model.add(Activation('softmax'))
-
-sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
-model.compile(loss='categorical_crossentropy',
-              optimizer=sgd,
-              metrics=['accuracy'])
-
-model.fit(X_train, y_train,
-          nb_epoch=20,
-          batch_size=16)
-score = model.evaluate(X_test, y_test, batch_size=16)
-```
-
-
-### Alternative implementation of a similar MLP:
-
-```python
-model = Sequential()
-model.add(Dense(64, input_dim=20, activation='relu'))
+model.add(Dense(64, activation='relu', input_dim=20))
 model.add(Dropout(0.5))
 model.add(Dense(64, activation='relu'))
 model.add(Dropout(0.5))
 model.add(Dense(10, activation='softmax'))
 
+sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
 model.compile(loss='categorical_crossentropy',
-              optimizer='adadelta',
+              optimizer=sgd,
               metrics=['accuracy'])
+
+model.fit(x_train, y_train,
+          epochs=20,
+          batch_size=128)
+score = model.evaluate(x_test, y_test, batch_size=128)
 ```
 
 
 ### MLP for binary classification:
+
 ```python
 model = Sequential()
-model.add(Dense(64, input_dim=20, init='uniform', activation='relu'))
+model.add(Dense(64, input_dim=20, activation='relu'))
 model.add(Dropout(0.5))
 model.add(Dense(64, activation='relu'))
 model.add(Dropout(0.5))
@@ -303,40 +194,32 @@ model.compile(loss='binary_crossentropy',
 
 ```python
 from keras.models import Sequential
-from keras.layers import Dense, Dropout, Activation, Flatten
-from keras.layers import Convolution2D, MaxPooling2D
+from keras.layers import Dense, Dropout, Flatten
+from keras.layers import Conv2D, MaxPooling2D
 from keras.optimizers import SGD
 
 model = Sequential()
 # input: 100x100 images with 3 channels -> (3, 100, 100) tensors.
 # this applies 32 convolution filters of size 3x3 each.
-model.add(Convolution2D(32, 3, 3, border_mode='valid', input_shape=(3, 100, 100)))
-model.add(Activation('relu'))
-model.add(Convolution2D(32, 3, 3))
-model.add(Activation('relu'))
+model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(3, 100, 100)))
+model.add(Conv2D(32, (3, 3), activation='relu'))
 model.add(MaxPooling2D(pool_size=(2, 2)))
 model.add(Dropout(0.25))
 
-model.add(Convolution2D(64, 3, 3, border_mode='valid'))
-model.add(Activation('relu'))
-model.add(Convolution2D(64, 3, 3))
-model.add(Activation('relu'))
+model.add(Conv2D(64, (3, 3), activation='relu'))
+model.add(Conv2D(64, (3, 3), activation='relu'))
 model.add(MaxPooling2D(pool_size=(2, 2)))
 model.add(Dropout(0.25))
 
 model.add(Flatten())
-# Note: Keras does automatic shape inference.
-model.add(Dense(256))
-model.add(Activation('relu'))
+model.add(Dense(256, activation='relu'))
 model.add(Dropout(0.5))
+model.add(Dense(10, activation='softmax'))
 
-model.add(Dense(10))
-model.add(Activation('softmax'))
-
-sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
+sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
 model.compile(loss='categorical_crossentropy', optimizer=sgd)
 
-model.fit(X_train, Y_train, batch_size=32, nb_epoch=1)
+model.fit(x_train, y_train, batch_size=32, epochs=10)
 ```
 
 
@@ -344,89 +227,50 @@ model.fit(X_train, Y_train, batch_size=32, nb_epoch=1)
 
 ```python
 from keras.models import Sequential
-from keras.layers import Dense, Dropout, Activation
+from keras.layers import Dense, Dropout
 from keras.layers import Embedding
 from keras.layers import LSTM
 
 model = Sequential()
-model.add(Embedding(max_features, 256, input_length=maxlen))
-model.add(LSTM(output_dim=128, activation='sigmoid', inner_activation='hard_sigmoid'))
+model.add(Embedding(max_features, output_dim=256))
+model.add(LSTM(128))
 model.add(Dropout(0.5))
-model.add(Dense(1))
-model.add(Activation('sigmoid'))
+model.add(Dense(1, activation='sigmoid'))
 
 model.compile(loss='binary_crossentropy',
               optimizer='rmsprop',
               metrics=['accuracy'])
 
-model.fit(X_train, Y_train, batch_size=16, nb_epoch=10)
-score = model.evaluate(X_test, Y_test, batch_size=16)
+model.fit(x_train, y_train, batch_size=16, epochs=10)
+score = model.evaluate(x_test, y_test, batch_size=16)
 ```
 
-### Architecture for learning image captions with a convnet and a Gated Recurrent Unit:
-(word-level embedding, caption of maximum length 16 words).
-
-Note that getting this to work well will require using a bigger convnet, initialized with pre-trained weights.
+### Sequence classification with 1D convolutions:
 
 ```python
-max_caption_len = 16
-vocab_size = 10000
+from keras.models import Sequential
+from keras.layers import Dense, Dropout
+from keras.layers import Embedding
+from keras.layers import Conv1D, GlobalAveragePooling1D
 
-# first, let's define an image model that
-# will encode pictures into 128-dimensional vectors.
-# it should be initialized with pre-trained weights.
-image_model = Sequential()
-image_model.add(Convolution2D(32, 3, 3, border_mode='valid', input_shape=(3, 100, 100)))
-image_model.add(Activation('relu'))
-image_model.add(Convolution2D(32, 3, 3))
-image_model.add(Activation('relu'))
-image_model.add(MaxPooling2D(pool_size=(2, 2)))
-
-image_model.add(Convolution2D(64, 3, 3, border_mode='valid'))
-image_model.add(Activation('relu'))
-image_model.add(Convolution2D(64, 3, 3))
-image_model.add(Activation('relu'))
-image_model.add(MaxPooling2D(pool_size=(2, 2)))
-
-image_model.add(Flatten())
-image_model.add(Dense(128))
-
-# let's load the weights from a save file.
-image_model.load_weights('weight_file.h5')
-
-# next, let's define a RNN model that encodes sequences of words
-# into sequences of 128-dimensional word vectors.
-language_model = Sequential()
-language_model.add(Embedding(vocab_size, 256, input_length=max_caption_len))
-language_model.add(GRU(output_dim=128, return_sequences=True))
-language_model.add(TimeDistributed(Dense(128)))
-
-# let's repeat the image vector to turn it into a sequence.
-image_model.add(RepeatVector(max_caption_len))
-
-# the output of both models will be tensors of shape (samples, max_caption_len, 128).
-# let's concatenate these 2 vector sequences.
 model = Sequential()
-model.add(Merge([image_model, language_model], mode='concat', concat_axis=-1))
-# let's encode this vector sequence into a single vector
-model.add(GRU(256, return_sequences=False))
-# which will be used to compute a probability
-# distribution over what the next word in the caption should be!
-model.add(Dense(vocab_size))
-model.add(Activation('softmax'))
+model.add(Conv1D(64, 3, activation='relu', input_shape=(seq_length, 100)))
+model.add(Conv1D(64, 3, activation='relu'))
+model.add(MaxPooling1D((3, 3)))
+model.add(Conv1D(128, 3, activation='relu'))
+model.add(Conv1D(128, 3, activation='relu'))
+model.add(GlobalAveragePooling1D())
+model.add(Dropout(0.5))
+model.add(Dense(1, activation='sigmoid'))
 
-model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
+model.compile(loss='binary_crossentropy',
+              optimizer='rmsprop',
+              metrics=['accuracy'])
 
-# "images" is a numpy float array of shape (nb_samples, nb_channels=3, width, height).
-# "captions" is a numpy integer array of shape (nb_samples, max_caption_len)
-# containing word index sequences representing partial captions.
-# "next_words" is a numpy float array of shape (nb_samples, vocab_size)
-# containing a categorical encoding (0s and 1s) of the next word in the corresponding
-# partial caption.
-model.fit([images, partial_captions], next_words, batch_size=16, nb_epoch=100)
+model.fit(x_train, y_train, batch_size=16, epochs=10)
+score = model.evaluate(x_test, y_test, batch_size=16)
 ```
 
-
 ### Stacked LSTM for sequence classification
 
 In this model, we stack 3 LSTM layers on top of each other,
@@ -445,7 +289,7 @@ import numpy as np
 
 data_dim = 16
 timesteps = 8
-nb_classes = 10
+num_classes = 10
 
 # expected input data shape: (batch_size, timesteps, data_dim)
 model = Sequential()
@@ -459,16 +303,16 @@ model.compile(loss='categorical_crossentropy',
               optimizer='rmsprop',
               metrics=['accuracy'])
 
-# generate dummy training data
+# Generate dummy training data
 x_train = np.random.random((1000, timesteps, data_dim))
-y_train = np.random.random((1000, nb_classes))
+y_train = np.random.random((1000, num_classes))
 
-# generate dummy validation data
+# Generate dummy validation data
 x_val = np.random.random((100, timesteps, data_dim))
-y_val = np.random.random((100, nb_classes))
+y_val = np.random.random((100, num_classes))
 
 model.fit(x_train, y_train,
-          batch_size=64, nb_epoch=5,
+          batch_size=64, epochs=5,
           validation_data=(x_val, y_val))
 ```
 
@@ -488,11 +332,11 @@ import numpy as np
 
 data_dim = 16
 timesteps = 8
-nb_classes = 10
+num_classes = 10
 batch_size = 32
 
-# expected input batch shape: (batch_size, timesteps, data_dim)
-# note that we have to provide the full batch_input_shape since the network is stateful.
+# Expected input batch shape: (batch_size, timesteps, data_dim)
+# Note that we have to provide the full batch_input_shape since the network is stateful.
 # the sample of index i in batch k is the follow-up for the sample i in batch k-1.
 model = Sequential()
 model.add(LSTM(32, return_sequences=True, stateful=True,
@@ -505,63 +349,15 @@ model.compile(loss='categorical_crossentropy',
               optimizer='rmsprop',
               metrics=['accuracy'])
 
-# generate dummy training data
+# Generate dummy training data
 x_train = np.random.random((batch_size * 10, timesteps, data_dim))
-y_train = np.random.random((batch_size * 10, nb_classes))
+y_train = np.random.random((batch_size * 10, num_classes))
 
-# generate dummy validation data
+# Generate dummy validation data
 x_val = np.random.random((batch_size * 3, timesteps, data_dim))
-y_val = np.random.random((batch_size * 3, nb_classes))
+y_val = np.random.random((batch_size * 3, num_classes))
 
 model.fit(x_train, y_train,
-          batch_size=batch_size, nb_epoch=5,
+          batch_size=batch_size, epochs=5,
           validation_data=(x_val, y_val))
 ```
-
-
-### Two merged LSTM encoders for classification over two parallel sequences
-
-In this model, two input sequences are encoded into vectors by two separate LSTM modules.
-
-These two vectors are then concatenated, and a fully connected network is trained on top of the concatenated representations.
-
-<img src="https://keras.io/img/dual_lstm.png" alt="Dual LSTM" style="width: 600px;"/>
-
-```python
-from keras.models import Sequential
-from keras.layers import Merge, LSTM, Dense
-import numpy as np
-
-data_dim = 16
-timesteps = 8
-nb_classes = 10
-
-encoder_a = Sequential()
-encoder_a.add(LSTM(32, input_shape=(timesteps, data_dim)))
-
-encoder_b = Sequential()
-encoder_b.add(LSTM(32, input_shape=(timesteps, data_dim)))
-
-decoder = Sequential()
-decoder.add(Merge([encoder_a, encoder_b], mode='concat'))
-decoder.add(Dense(32, activation='relu'))
-decoder.add(Dense(nb_classes, activation='softmax'))
-
-decoder.compile(loss='categorical_crossentropy',
-                optimizer='rmsprop',
-                metrics=['accuracy'])
-
-# generate dummy training data
-x_train_a = np.random.random((1000, timesteps, data_dim))
-x_train_b = np.random.random((1000, timesteps, data_dim))
-y_train = np.random.random((1000, nb_classes))
-
-# generate dummy validation data
-x_val_a = np.random.random((100, timesteps, data_dim))
-x_val_b = np.random.random((100, timesteps, data_dim))
-y_val = np.random.random((100, nb_classes))
-
-decoder.fit([x_train_a, x_train_b], y_train,
-            batch_size=64, nb_epoch=5,
-            validation_data=([x_val_a, x_val_b], y_val))
-```

docs/templates/index.md

@@ -2,13 +2,12 @@
 
 ## You have just found Keras.
 
-Keras is a high-level neural networks library, written in Python and capable of running on top of either [TensorFlow](https://github.com/tensorflow/tensorflow) or [Theano](https://github.com/Theano/Theano). It was developed with a focus on enabling fast experimentation. *Being able to go from idea to result with the least possible delay is key to doing good research.*
+Keras is a high-level neural networks API, written in Python and capable of running on top of either [TensorFlow](https://github.com/tensorflow/tensorflow) or [Theano](https://github.com/Theano/Theano). It was developed with a focus on enabling fast experimentation. *Being able to go from idea to result with the least possible delay is key to doing good research.*
 
 Use Keras if you need a deep learning library that:
 
-- Allows for easy and fast prototyping (through total modularity, minimalism, and extensibility).
+- Allows for easy and fast prototyping (through user friendliness, modularity, and extensibility).
 - Supports both convolutional networks and recurrent networks, as well as combinations of the two.
-- Supports arbitrary connectivity schemes (including multi-input and multi-output training).
 - Runs seamlessly on CPU and GPU.
 
 Read the documentation at [Keras.io](http://keras.io).
@@ -21,11 +20,11 @@ Keras is compatible with: __Python 2.7-3.5__.
 
 ## Guiding principles
 
+- __User friendliness.__ Keras is an API designed for human beings, not machines. It puts user experience front and center. Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimizes the number user actions required for common use cases, and it provides clear and actionable feedback upon user error.
+
 - __Modularity.__ A model is understood as a sequence or a graph of standalone, fully-configurable modules that can be plugged together with as little restrictions as possible. In particular, neural layers, cost functions, optimizers, initialization schemes, activation functions, regularization schemes are all standalone modules that you can combine to create new models.
 
-- __Minimalism.__ Each module should be kept short and simple. Every piece of code should be transparent upon first reading. No black magic: it hurts iteration speed and ability to innovate.
-
-- __Easy extensibility.__ New modules are dead simple to add (as new classes and functions), and existing modules provide ample examples. To be able to easily create new modules allows for total expressiveness, making Keras suitable for advanced research.
+- __Easy extensibility.__ New modules are simple to add (as new classes and functions), and existing modules provide ample examples. To be able to easily create new modules allows for total expressiveness, making Keras suitable for advanced research.
 
 - __Work with Python__. No separate models configuration files in a declarative format. Models are described in Python code, which is compact, easier to debug, and allows for ease of extensibility.
 
@@ -35,9 +34,9 @@ Keras is compatible with: __Python 2.7-3.5__.
 
 ## Getting started: 30 seconds to Keras
 
-The core data structure of Keras is a __model__, a way to organize layers. The main type of model is the [`Sequential`](http://keras.io/getting-started/sequential-model-guide) model, a linear stack of layers. For more complex architectures, you should use the [Keras functional API](http://keras.io/getting-started/functional-api-guide).
+The core data structure of Keras is a __model__, a way to organize layers. The simplest type of model is the [`Sequential`](http://keras.io/getting-started/sequential-model-guide) model, a linear stack of layers. For more complex architectures, you should use the [Keras functional API](http://keras.io/getting-started/functional-api-guide), allowing to build arbitrary graphs of layers.
 
-Here's the `Sequential` model:
+Here is the `Sequential` model:
 
 ```python
 from keras.models import Sequential
@@ -50,45 +49,52 @@ Stacking layers is as easy as `.add()`:
 ```python
 from keras.layers import Dense, Activation
 
-model.add(Dense(output_dim=64, input_dim=100))
-model.add(Activation("relu"))
-model.add(Dense(output_dim=10))
-model.add(Activation("softmax"))
+model.add(Dense(units=64, input_dim=100))
+model.add(Activation('relu'))
+model.add(Dense(units=10))
+model.add(Activation('softmax'))
 ```
 
 Once your model looks good, configure its learning process with `.compile()`:
+
 ```python
-model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
+model.compile(loss='categorical_crossentropy',
+              optimizer='sgd',
+              metrics=['accuracy'])
 ```
 
 If you need to, you can further configure your optimizer. A core principle of Keras is to make things reasonably simple, while allowing the user to be fully in control when they need to (the ultimate control being the easy extensibility of the source code).
 ```python
-from keras.optimizers import SGD
-model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.01, momentum=0.9, nesterov=True))
+model.compile(loss=keras.losses.categorical_crossentropy,
+              optimizer=keras.optimizers.SGD(lr=0.01, momentum=0.9, nesterov=True))
 ```
 
 You can now iterate on your training data in batches:
+
 ```python
-model.fit(X_train, Y_train, nb_epoch=5, batch_size=32)
+# x_train and y_train are Numpy arrays --just like in the Scikit-Learn API.
+model.fit(x_train, y_train, epochs=5, batch_size=32)
 ```
 
 Alternatively, you can feed batches to your model manually:
+
 ```python
-model.train_on_batch(X_batch, Y_batch)
+model.train_on_batch(x_batch, y_batch)
 ```
 
 Evaluate your performance in one line:
+
 ```python
-loss_and_metrics = model.evaluate(X_test, Y_test, batch_size=32)
+loss_and_metrics = model.evaluate(x_test, y_test, batch_size=128)
 ```
 
 Or generate predictions on new data:
+
 ```python
-classes = model.predict_classes(X_test, batch_size=32)
-proba = model.predict_proba(X_test, batch_size=32)
+classes = model.predict(x_test, batch_size=128)
 ```
 
-Building a question answering system, an image classification model, a Neural Turing Machine, a word2vec embedder or any other model is just as fast. The ideas behind deep learning are simple, so why should their implementation be painful?
+Building a question answering system, an image classification model, a Neural Turing Machine, or any other model is just as fast. The ideas behind deep learning are simple, so why should their implementation be painful?
 
 For a more in-depth tutorial about Keras, you can check out:
 
@@ -106,7 +112,7 @@ In the [examples folder](https://github.com/fchollet/keras/tree/master/examples)
 Keras uses the following dependencies:
 
 - numpy, scipy
-- pyyaml
+- yaml
 - HDF5 and h5py (optional, required if you use model saving/loading functions)
 - Optional but recommended if you use CNNs: cuDNN.
 

docs/templates/initializations.md

@@ -1,50 +0,0 @@
-
-## Usage of initializations
-
-Initializations define the way to set the initial random weights of Keras layers.
-
-The keyword arguments used for passing initializations to layers will depend on the layer. Usually it is simply `init`:
-
-```python
-model.add(Dense(64, init='uniform'))
-```
-
-## Available initializations
-
-- __uniform__
-- __lecun_uniform__: Uniform initialization scaled by the square root of the number of inputs (LeCun 98).
-- __normal__
-- __identity__: Use with square 2D layers (`shape[0] == shape[1]`).
-- __orthogonal__: Use with square 2D layers (`shape[0] == shape[1]`).
-- __zero__
-- __glorot_normal__: Gaussian initialization scaled by fan_in + fan_out (Glorot 2010)
-- __glorot_uniform__
-- __he_normal__: Gaussian initialization scaled by fan_in (He et al., 2014)
-- __he_uniform__
-
-
-An initialization may be passed as a string (must match one of the available initializations above), or as a callable.
-If a callable, then it must take two arguments: `shape` (shape of the variable to initialize) and `name` (name of the variable),
-and it must return a variable (e.g. output of `K.variable()`):
-
-```python
-from keras import backend as K
-import numpy as np
-
-def my_init(shape, name=None):
-    value = np.random.random(shape)
-    return K.variable(value, name=name)
-
-model.add(Dense(64, init=my_init))
-```
-
-You could also use functions from `keras.initializations` in this way:
-
-```python
-from keras import initializations
-
-def my_init(shape, name=None):
-    return initializations.normal(shape, scale=0.01, name=name)
-
-model.add(Dense(64, init=my_init))
-```

docs/templates/initializers.md

@@ -0,0 +1,43 @@
+## Usage of initializers
+
+Initializations define the way to set the initial random weights of Keras layers.
+
+The keyword arguments used for passing initializers to layers will depend on the layer. Usually it is simply `kernel_initializer` and `bias_initializer`:
+
+```python
+model.add(Dense(64,
+                kernel_initializer='random_uniform',
+                bias_initializer='zeros'))
+```
+
+## Available initializers
+
+The following built-in initializers are available as part of the `keras.initializers` module:
+
+{{autogenerated}}
+
+
+An initializer may be passed as a string (must match one of the available initializers above), or as a callable:
+
+```python
+from keras import initializers
+
+model.add(Dense(64, kernel_initializer=initializers.random_normal(stddev=0.01)))
+
+# also works; will use the default parameters.
+model.add(Dense(64, kernel_initializer='random_normal'))
+```
+
+
+## Using custom initializers
+
+If passing a custom callable, then it must take the argument `shape` (shape of the variable to initialize) and `dtype` (dtype of generated values):
+
+```python
+from keras import backend as K
+
+def my_init(shape, dtype=None):
+    return K.random_normal(shape, dtype=dtype)
+
+model.add(Dense(64, init=my_init))
+```

docs/templates/layers/about-keras-layers.md

@@ -5,11 +5,21 @@ All Keras layers have a number of methods in common:
 - `layer.get_weights()`: returns the weights of the layer as a list of Numpy arrays.
 - `layer.set_weights(weights)`: sets the weights of the layer from a list of Numpy arrays (with the same shapes as the output of `get_weights`).
 - `layer.get_config()`: returns a dictionary containing the configuration of the layer. The layer can be reinstantiated from its config via:
+
+```python
+layer = Dense(32)
+config = layer.get_config()
+reconstructed_layer = Dense.from_config(config)
+```
+
+Or:
+
 ```python
 from keras.utils.layer_utils import layer_from_config
 
 config = layer.get_config()
-layer = layer_from_config(config)
+layer = layer_from_config({'class_name': layer.__class__.__name__,
+                           'config': config})
 ```
 
 If a layer has a single node (i.e. if it isn't a shared layer), you can get its input tensor, output tensor, input shape and output shape via:

docs/templates/layers/writing-your-own-keras-layers.md

@@ -2,11 +2,11 @@
 
 For simple, stateless custom operations, you are probably better off using `layers.core.Lambda` layers. But for any custom operation that has trainable weights, you should implement your own layer.
 
-Here is the skeleton of a Keras layer, **as of Keras 1.1.3** (if you have an older version, please upgrade). There are only three methods you need to implement:
+Here is the skeleton of a Keras layer, **as of Keras 2.0** (if you have an older version, please upgrade). There are only three methods you need to implement:
 
 - `build(input_shape)`: this is where you will define your weights. This method must set `self.built = True`, which can be done by calling `super([Layer], self).build()`.
 - `call(x)`: this is where the layer's logic lives. Unless you want your layer to support masking, you only have to care about the first argument passed to `call`: the input tensor.
-- `get_output_shape_for(input_shape)`: in case your layer modifies the shape of its input, you should specify here the shape transformation logic. This allows Keras to do automatic shape inference.
+- `compute_output_shape(input_shape)`: in case your layer modifies the shape of its input, you should specify here the shape transformation logic. This allows Keras to do automatic shape inference.
 
 ```python
 from keras import backend as K
@@ -14,22 +14,23 @@ from keras.engine.topology import Layer
 import numpy as np
 
 class MyLayer(Layer):
+
     def __init__(self, output_dim, **kwargs):
         self.output_dim = output_dim
         super(MyLayer, self).__init__(**kwargs)
 
     def build(self, input_shape):
         # Create a trainable weight variable for this layer.
-        self.W = self.add_weight(shape=(input_shape[1], self.output_dim),
-                                 initializer='random_uniform',
-                                 trainable=True)
-        super(MyLayer, self).build()  # Be sure to call this somewhere!
+        self.kernel = self.add_weight(shape=(input_shape[1], self.output_dim),
+                                      initializer='uniform',
+                                      trainable=True)
+        super(MyLayer, self).build(input_shape)  # Be sure to call this somewhere!
 
-    def call(self, x, mask=None):
-        return K.dot(x, self.W)
+    def call(self, x):
+        return K.dot(x, self.kernel)
 
-    def get_output_shape_for(self, input_shape):
+    def compute_output_shape(self, input_shape):
         return (input_shape[0], self.output_dim)
 ```
 
-The existing Keras layers provide ample examples of how to implement almost anything. Never hesitate to read the source code!
+The existing Keras layers provide examples of how to implement almost anything. Never hesitate to read the source code!

docs/templates/losses.md

@@ -0,0 +1,37 @@
+
+## Usage of loss functions
+
+A loss function (or objective function, or optimization score function) is one of the two parameters required to compile a model:
+
+```python
+model.compile(loss='mean_squared_error', optimizer='sgd')
+```
+
+```python
+from keras import losses
+
+model.compile(loss=losses.mean_squared_error, optimizer='sgd')
+```
+
+You can either pass the name of an existing loss function, or pass a TensorFlow/Theano symbolic function that returns a scalar for each data-point and takes the following two arguments:
+
+- __y_true__: True labels. TensorFlow/Theano tensor.
+- __y_pred__: Predictions. TensorFlow/Theano tensor of the same shape as y_true.
+
+The actual optimized objective is the mean of the output array across all datapoints.
+
+For a few examples of such functions, check out the [losses source](https://github.com/fchollet/keras/blob/master/keras/losses.py).
+
+## Available loss functions
+
+{{autogenerated}}
+
+----
+
+**Note**: when using the `categorical_crossentropy` loss, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros expect for a 1 at the index corresponding to the class of the sample). In order to convert *integer targets* into *categorical targets*, you can use the Keras utility `to_categorical`:
+
+```python
+from keras.utils.np_utils import to_categorical
+
+categorical_labels = to_categorical(int_labels, num_classes=None)
+```

docs/templates/metrics.md

@@ -3,6 +3,20 @@
 
 A metric is a function that is used to judge the performance of your model. Metric functions are to be supplied in the `metrics` parameter when a model is compiled.
 
+```python
+model.compile(loss='mean_squared_error',
+              optimizer='sgd',
+              metrics=['mae', 'acc'])
+```
+
+```python
+from keras import metrics
+
+model.compile(loss='mean_squared_error',
+              optimizer='sgd',
+              metrics=[metrics.mae, metrics.categorical_accuracy])
+```
+
 A metric function is similar to an [objective function](/objectives), except that the results from evaluating a metric are not used when training the model.
 
 You can either pass the name of an existing metric, or pass a Theano/TensorFlow symbolic function (see [Custom metrics](#custom-metrics)).
@@ -26,26 +40,17 @@ You can either pass the name of an existing metric, or pass a Theano/TensorFlow
 
 ## Custom metrics
 
-Custom metrics can be defined and passed via the compilation step. The
+Custom metrics can be passed at the compilation step. The
 function would need to take `(y_true, y_pred)` as arguments and return
-either a single tensor value or a dict `metric_name -> metric_value`.
+a single tensor value.
 
 ```python
-# for custom metrics
 import keras.backend as K
 
 def mean_pred(y_true, y_pred):
     return K.mean(y_pred)
 
-def false_rates(y_true, y_pred):
-    false_neg = ...
-    false_pos = ...
-    return {
-        'false_neg': false_neg,
-        'false_pos': false_pos,
-    }
-
 model.compile(optimizer='rmsprop',
               loss='binary_crossentropy',
-              metrics=['accuracy', mean_pred, false_rates])
-```
+              metrics=['accuracy', mean_pred])
+```

docs/templates/models/model.md

@@ -1,6 +1,6 @@
 # Model class API
 
-In the functional API, given an input tensor and output tensor, you can instantiate a `Model` via:
+In the functional API, given some input tensor(s) and output tensor(s), you can instantiate a `Model` via:
 
 ```python
 from keras.models import Model
@@ -8,7 +8,7 @@ from keras.layers import Input, Dense
 
 a = Input(shape=(32,))
 b = Dense(32)(a)
-model = Model(input=a, output=b)
+model = Model(inputs=a, outputs=b)
 ```
 
 This model will include all layers required in the computation of `b` given `a`.
@@ -16,7 +16,7 @@ This model will include all layers required in the computation of `b` given `a`.
 In the case of multi-input or multi-output models, you can use lists as well:
 
 ```python
-model = Model(input=[a1, a2], output=[b1, b3, b3])
+model = Model(inputs=[a1, a2], outputs=[b1, b3, b3])
 ```
 
 For a detailed introduction of what `Model` can do, read [this guide to the Keras functional API](/getting-started/functional-api-guide).

docs/templates/objectives.md

@@ -1,40 +0,0 @@
-
-## Usage of objectives
-
-An objective function (or loss function, or optimization score function) is one of the two parameters required to compile a model:
-
-```python
-model.compile(loss='mean_squared_error', optimizer='sgd')
-```
-
-You can either pass the name of an existing objective, or pass a Theano/TensorFlow symbolic function that returns a scalar for each data-point and takes the following two arguments:
-
-- __y_true__: True labels. Theano/TensorFlow tensor.
-- __y_pred__: Predictions. Theano/TensorFlow tensor of the same shape as y_true.
-
-The actual optimized objective is the mean of the output array across all datapoints.
-
-For a few examples of such functions, check out the [objectives source](https://github.com/fchollet/keras/blob/master/keras/objectives.py).
-
-## Available objectives
-
-- __mean_squared_error__ / __mse__
-- __mean_absolute_error__ / __mae__
-- __mean_absolute_percentage_error__ / __mape__
-- __mean_squared_logarithmic_error__ / __msle__
-- __squared_hinge__
-- __hinge__
-- __binary_crossentropy__: Also known as logloss. 
-- __categorical_crossentropy__: Also known as multiclass logloss. __Note__: using this objective requires that your labels are binary arrays of shape `(nb_samples, nb_classes)`.
-- __sparse_categorical_crossentropy__: As above but accepts sparse labels. __Note__: this objective still requires that your labels have the same number of dimensions as your outputs; you may need to add a length-1 dimension to the shape of your labels, e.g with `np.expand_dims(y, -1)`.
-- __kullback_leibler_divergence__ / __kld__: Information gain from a predicted probability distribution Q to a true probability distribution P. Gives a measure of difference between both distributions.
-- __poisson__: Mean of `(predictions - targets * log(predictions))`
-- __cosine_proximity__: The opposite (negative) of the mean cosine proximity between predictions and targets.
-
-**Note**: when using the `categorical_crossentropy` objective, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros expect for a 1 at the index corresponding to the class of the sample). In order to convert *integer targets* into *categorical targets*, you can use the Keras utility `to_categorical`:
-
-```python
-from keras.utils.np_utils import to_categorical
-
-categorical_labels = to_categorical(int_labels, nb_classes=None)
-```

docs/templates/optimizers.md

@@ -4,12 +4,14 @@
 An optimizer is one of the two arguments required for compiling a Keras model:
 
 ```python
+from keras import optimizers
+
 model = Sequential()
-model.add(Dense(64, init='uniform', input_dim=10))
+model.add(Dense(64, init='uniform', input_shape=(10,)))
 model.add(Activation('tanh'))
 model.add(Activation('softmax'))
 
-sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
+sgd = optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
 model.compile(loss='mean_squared_error', optimizer=sgd)
 ```
 
@@ -27,16 +29,20 @@ model.compile(loss='mean_squared_error', optimizer='sgd')
 The parameters `clipnorm` and `clipvalue` can be used with all optimizers to control gradient clipping:
 
 ```python
-# all parameter gradients will be clipped to
+from keras import optimizers
+
+# All parameter gradients will be clipped to
 # a maximum norm of 1.
-sgd = SGD(lr=0.01, clipnorm=1.)
+sgd = optimizers.SGD(lr=0.01, clipnorm=1.)
 ```
 
 ```python
-# all parameter gradients will be clipped to
+from keras import optimizers
+
+# All parameter gradients will be clipped to
 # a maximum value of 0.5 and
 # a minimum value of -0.5.
-sgd = SGD(lr=0.01, clipvalue=0.5)
+sgd = optimizers.SGD(lr=0.01, clipvalue=0.5)
 ```
 
 ---

docs/templates/preprocessing/image.md

@@ -18,7 +18,7 @@ keras.preprocessing.image.ImageDataGenerator(featurewise_center=False,
     horizontal_flip=False,
     vertical_flip=False,
     rescale=None,
-    dim_ordering=K.image_dim_ordering())
+    data_format=K.image_data_format())
 ```
 
 Generate batches of tensor image data with real-time data augmentation. The data will be looped over (in batches) indefinitely.
@@ -42,12 +42,12 @@ Generate batches of tensor image data with real-time data augmentation. The data
     - __rescale__: rescaling factor. Defaults to None. If None or 0, no rescaling is applied,
             otherwise we multiply the data by the value provided (before applying
             any other transformation).
-    - __dim_ordering__: One of {"th", "tf"}.
-        "tf" mode means that the images should have shape `(samples, height, width, channels)`,
-        "th" mode means that the images should have shape `(samples, channels, height, width)`.
-        It defaults to the `image_dim_ordering` value found in your
+    - _data_format_: One of {"channels_first", "channels_last"}.
+        "channels_last" mode means that the images should have shape `(samples, height, width, channels)`,
+        "channels_first" mode means that the images should have shape `(samples, channels, height, width)`.
+        It defaults to the `image_data_format` value found in your
         Keras config file at `~/.keras/keras.json`.
-        If you never set it, then it will be "tf".
+        If you never set it, then it will be "channels_last".
 
 - __Methods__:
     - __fit(X)__: Compute the internal data stats related to the data-dependent transformations, based on an array of sample data.
@@ -78,7 +78,7 @@ Generate batches of tensor image data with real-time data augmentation. The data
     - __flow_from_directory(directory)__: Takes the path to a directory, and generates batches of augmented/normalized data. Yields batches indefinitely, in an infinite loop.
         - __Arguments__:
             - __directory__: path to the target directory. It should contain one subdirectory per class.
-                Any PNG, JPG or BNP images inside each of the subdirectories directory tree will be included in the generator.
+                Any PNG, JPG or BMP images inside each of the subdirectories directory tree will be included in the generator.
                 See [this script](https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d) for more details.
             - __target_size__: tuple of integers, default: `(256, 256)`. The dimensions to which all images found will be resized.
             - __color_mode__: one of "grayscale", "rbg". Default: "rgb". Whether the images will be converted to have 1 or 3 color channels.
@@ -99,8 +99,8 @@ Example of using `.flow(X, y)`:
 
 ```python
 (X_train, y_train), (X_test, y_test) = cifar10.load_data()
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
+Y_train = np_utils.to_categorical(y_train, num_classes)
+Y_test = np_utils.to_categorical(y_test, num_classes)
 
 datagen = ImageDataGenerator(
     featurewise_center=True,
@@ -116,10 +116,10 @@ datagen.fit(X_train)
 
 # fits the model on batches with real-time data augmentation:
 model.fit_generator(datagen.flow(X_train, Y_train, batch_size=32),
-                    samples_per_epoch=len(X_train), nb_epoch=nb_epoch)
+                    samples_per_epoch=len(X_train), epochs=epochs)
 
 # here's a more "manual" example
-for e in range(nb_epoch):
+for e in range(epochs):
     print 'Epoch', e
     batches = 0
     for X_batch, Y_batch in datagen.flow(X_train, Y_train, batch_size=32):
@@ -157,9 +157,9 @@ validation_generator = test_datagen.flow_from_directory(
 model.fit_generator(
         train_generator,
         samples_per_epoch=2000,
-        nb_epoch=50,
+        epochs=50,
         validation_data=validation_generator,
-        nb_val_samples=800)
+        num_val_samples=800)
 ```
 
 Example of transforming images and masks together.
@@ -196,5 +196,5 @@ train_generator = zip(image_generator, mask_generator)
 model.fit_generator(
     train_generator,
     samples_per_epoch=2000,
-    nb_epoch=50)
+    epochs=50)
 ```

docs/templates/preprocessing/sequence.md

@@ -1,12 +1,13 @@
 ## pad_sequences
 
 ```python
-keras.preprocessing.sequence.pad_sequences(sequences, maxlen=None, dtype='int32')
+keras.preprocessing.sequence.pad_sequences(sequences, maxlen=None, dtype='int32',
+    padding='pre', truncating='pre', value=0.)
 ```
 
-Transform a list of `nb_samples sequences` (lists of scalars) into a 2D Numpy array of shape `(nb_samples, nb_timesteps)`. `nb_timesteps` is either the `maxlen` argument if provided, or the length of the longest sequence otherwise. Sequences that are shorter than `nb_timesteps` are padded with zeros at the end.
+Transform a list of `num_samples` sequences (lists of scalars) into a 2D Numpy array of shape `(num_samples, num_timesteps)`. `num_timesteps` is either the `maxlen` argument if provided, or the length of the longest sequence otherwise. Sequences that are shorter than `num_timesteps` are padded with `value` at the end. Sequences longer than `num_timesteps` are truncated so that it fits the desired length. Position where padding or truncation happens is determined by `padding` or `truncating`, respectively.
 
-- __Return__: 2D Numpy array of shape `(nb_samples, nb_timesteps)`.
+- __Return__: 2D Numpy array of shape `(num_samples, num_timesteps)`.
 
 - __Arguments__:
     - __sequences__: List of lists of int or float.

docs/templates/preprocessing/text.md

@@ -33,14 +33,14 @@ One-hot encode a text into a list of word indexes in a vocabulary of size n.
 ## Tokenizer
 
 ```python
-keras.preprocessing.text.Tokenizer(nb_words=None, filters=base_filter(), 
+keras.preprocessing.text.Tokenizer(num_words=None, filters=base_filter(), 
     lower=True, split=" ")
 ```
 
 Class for vectorizing texts, or/and turning texts into sequences (=list of word indexes, where the word of rank i in the dataset (starting at 1) has index i).
 
 - __Arguments__: Same as `text_to_word_sequence` above.
-    - __nb_words__: None or int. Maximum number of words to work with (if set, tokenization will be restricted to the top nb_words most common words in the dataset).
+    - __num_words__: None or int. Maximum number of words to work with (if set, tokenization will be restricted to the top num_words most common words in the dataset).
 
 - __Methods__:
 
@@ -57,7 +57,7 @@ Class for vectorizing texts, or/and turning texts into sequences (=list of word
         - __Return__: yield one sequence per input text.
 
     - __texts_to_matrix(texts)__:
-        - __Return__: numpy array of shape `(len(texts), nb_words)`.
+        - __Return__: numpy array of shape `(len(texts), num_words)`.
         - __Arguments__:
             - __texts__: list of texts to vectorize.
             - __mode__: one of "binary", "count", "tfidf", "freq" (default: "binary").
@@ -67,7 +67,7 @@ Class for vectorizing texts, or/and turning texts into sequences (=list of word
             - __sequences__: list of sequences to train on. 
 
     - __sequences_to_matrix(sequences)__:
-        - __Return__: numpy array of shape `(len(sequences), nb_words)`.
+        - __Return__: numpy array of shape `(len(sequences), num_words)`.
         - __Arguments__:
             - __sequences__: list of sequences to vectorize.
             - __mode__: one of "binary", "count", "tfidf", "freq" (default: "binary").

docs/templates/regularizers.md

@@ -2,39 +2,45 @@
 
 Regularizers allow to apply penalties on layer parameters or layer activity during optimization. These penalties are incorporated in the loss function that the network optimizes.
 
-The penalties are applied on a per-layer basis. The exact API will depend on the layer, but the layers `Dense`, `TimeDistributedDense`, `MaxoutDense`, `Convolution1D`, `Convolution2D` and `Convolution3D` have a unified API.
+The penalties are applied on a per-layer basis. The exact API will depend on the layer, but the layers `Dense`, `Conv1D`, `Conv2D` and `Conv3D` have a unified API.
 
 These layers expose 3 keyword arguments:
 
-- `W_regularizer`: instance of `keras.regularizers.WeightRegularizer`
-- `b_regularizer`: instance of `keras.regularizers.WeightRegularizer`
-- `activity_regularizer`: instance of `keras.regularizers.ActivityRegularizer`
+- `kernel_regularizer`: instance of `keras.regularizers.Regularizer`
+- `bias_regularizer`: instance of `keras.regularizers.Regularizer`
+- `activity_regularizer`: instance of `keras.regularizers.Regularizer`
 
 
 ## Example
 
 ```python
 from keras.regularizers import l2, activity_l2
-model.add(Dense(64, input_dim=64, W_regularizer=l2(0.01), activity_regularizer=activity_l2(0.01)))
+model.add(Dense(64, input_dim=64,
+                kernel_regularizer=l2(0.01),
+                activity_regularizer=activity_l2(0.01)))
 ```
 
 ## Available penalties
 
 ```python
-keras.regularizers.WeightRegularizer(l1=0., l2=0.)
+keras.regularizers.l1(0.)
+keras.regularizers.l2(0.)
+keras.regularizers.l1_l2(0.)
 ```
 
+## Developing new regularizers
+
+Any function that takes in a weight matrix and returns a loss contribution tensor can be used as a regularizer, e.g.:
+
 ```python
-keras.regularizers.ActivityRegularizer(l1=0., l2=0.)
+from keras import backend as K
+
+def l1_reg(weight_matrix):
+    return 0.01 * K.sum(K.abs(weight_matrix))
+
+model.add(Dense(64, input_dim=64,
+                kernel_regularizer=l1_reg)
 ```
 
-## Shortcuts
-
-These are shortcut functions available in `keras.regularizers`.
-
-- __l1__(l=0.01): L1 weight regularization penalty, also known as LASSO
-- __l2__(l=0.01): L2 weight regularization penalty, also known as weight decay, or Ridge
-- __l1l2__(l1=0.01, l2=0.01): L1-L2 weight regularization penalty, also known as ElasticNet
-- __activity_l1__(l=0.01): L1 activity regularization
-- __activity_l2__(l=0.01): L2 activity regularization
-- __activity_l1l2__(l1=0.01, l2=0.01): L1+L2 activity regularization
+Alternatively, you can write your regularizers in an object-oriented way;
+see the [keras/regularizers.py](https://github.com/fchollet/keras/blob/master/keras/regularizers.py) module for examples.

docs/templates/scikit-learn-api.md

@@ -30,7 +30,7 @@ its arguments, so that you could create the estimator without passing any
 values to `sk_params`.
 
 `sk_params` could also accept parameters for calling `fit`, `predict`,
-`predict_proba`, and `score` methods (e.g., `nb_epoch`, `batch_size`).
+`predict_proba`, and `score` methods (e.g., `epochs`, `batch_size`).
 fitting (predicting) parameters are selected in the following order:
 
 1. Values passed to the dictionary arguments of
@@ -42,4 +42,4 @@ fitting (predicting) parameters are selected in the following order:
 When using scikit-learn's `grid_search` API, legal tunable parameters are
 those you could pass to `sk_params`, including fitting parameters.
 In other words, you could use `grid_search` to search for the best
-`batch_size` or `nb_epoch` as well as the model parameters.
+`batch_size` or `epochs` as well as the model parameters.

docs/templates/visualization.md

@@ -1,16 +1,16 @@
 
 ## Model visualization
 
-The `keras.utils.visualize_util` module provides utility functions to plot
-a Keras model (using graphviz).
+The `keras.utils.vis_utils` module provides utility functions to plot
+a Keras model (using `graphviz`).
 
 This will plot a graph of the model and save it to a file:
 ```python
-from keras.utils.visualize_util import plot
-plot(model, to_file='model.png')
+from keras.utils import plot_model
+plot_model(model, to_file='model.png')
 ```
 
-`plot` takes two optional arguments:
+`plot_model` takes two optional arguments:
 
 - `show_shapes` (defaults to False) controls whether output shapes are shown in the graph.
 - `show_layer_names` (defaults to True) controls whether layer names are shown in the graph.

examples/README.md

@@ -48,6 +48,9 @@ Compares different LSTM implementations on the IMDB sentiment classification tas
 [lstm_text_generation.py](lstm_text_generation.py)
 Generates text from Nietzsche's writings.
 
+[mnist_acgan.py](mnist_acgan.py)
+Implementation of AC-GAN ( Auxiliary Classifier GAN ) on the MNIST dataset
+
 [mnist_cnn.py](mnist_cnn.py)
 Trains a simple convnet on the MNIST dataset.
 

examples/addition_rnn.py

@@ -23,41 +23,47 @@ Four digits inverted:
 
 Five digits inverted:
 + One layer LSTM (128 HN), 550k training examples = 99% train/test accuracy in 30 epochs
-
 '''
 
 from __future__ import print_function
 from keras.models import Sequential
-from keras.engine.training import slice_X
-from keras.layers import Activation, TimeDistributed, Dense, RepeatVector, recurrent
+from keras import layers
 import numpy as np
 from six.moves import range
 
 
 class CharacterTable(object):
-    '''
-    Given a set of characters:
+    """Given a set of characters:
     + Encode them to a one hot integer representation
     + Decode the one hot integer representation to their character output
     + Decode a vector of probabilities to their character output
-    '''
-    def __init__(self, chars, maxlen):
+    """
+    def __init__(self, chars):
+        """Initialize character table.
+
+        # Arguments
+            chars: Characters that can appear in the input.
+        """
         self.chars = sorted(set(chars))
         self.char_indices = dict((c, i) for i, c in enumerate(self.chars))
         self.indices_char = dict((i, c) for i, c in enumerate(self.chars))
-        self.maxlen = maxlen
 
-    def encode(self, C, maxlen=None):
-        maxlen = maxlen if maxlen else self.maxlen
-        X = np.zeros((maxlen, len(self.chars)))
+    def encode(self, C, num_rows):
+        """One hot encode given string C.
+
+        # Arguments
+            num_rows: Number of rows in the returned one hot encoding. This is
+                used to keep the # of rows for each data the same.
+        """
+        x = np.zeros((num_rows, len(self.chars)))
         for i, c in enumerate(C):
-            X[i, self.char_indices[c]] = 1
-        return X
+            x[i, self.char_indices[c]] = 1
+        return x
 
-    def decode(self, X, calc_argmax=True):
+    def decode(self, x, calc_argmax=True):
         if calc_argmax:
-            X = X.argmax(axis=-1)
-        return ''.join(self.indices_char[x] for x in X)
+            x = x.argmax(axis=-1)
+        return ''.join(self.indices_char[x] for x in x)
 
 
 class colors:
@@ -65,104 +71,130 @@ class colors:
     fail = '\033[91m'
     close = '\033[0m'
 
-# Parameters for the model and dataset
+# Parameters for the model and dataset.
 TRAINING_SIZE = 50000
 DIGITS = 3
 INVERT = True
-# Try replacing GRU, or SimpleRNN
-RNN = recurrent.LSTM
-HIDDEN_SIZE = 128
-BATCH_SIZE = 128
-LAYERS = 1
-MAXLEN = DIGITS + 1 + DIGITS
 
+# Maximum length of input is 'int + int' (e.g., '345+678'). Maximum length of
+# int is DIGITS.
+MAxLEN = DIGITS + 1 + DIGITS
+
+# All the numbers, plus sign and space for padding.
 chars = '0123456789+ '
-ctable = CharacterTable(chars, MAXLEN)
+ctable = CharacterTable(chars)
 
 questions = []
 expected = []
 seen = set()
 print('Generating data...')
 while len(questions) < TRAINING_SIZE:
-    f = lambda: int(''.join(np.random.choice(list('0123456789')) for i in range(np.random.randint(1, DIGITS + 1))))
+    f = lambda: int(''.join(np.random.choice(list('0123456789'))
+                    for i in range(np.random.randint(1, DIGITS + 1))))
     a, b = f(), f()
     # Skip any addition questions we've already seen
-    # Also skip any such that X+Y == Y+X (hence the sorting)
+    # Also skip any such that x+Y == Y+x (hence the sorting).
     key = tuple(sorted((a, b)))
     if key in seen:
         continue
     seen.add(key)
-    # Pad the data with spaces such that it is always MAXLEN
+    # Pad the data with spaces such that it is always MAxLEN.
     q = '{}+{}'.format(a, b)
-    query = q + ' ' * (MAXLEN - len(q))
+    query = q + ' ' * (MAxLEN - len(q))
     ans = str(a + b)
-    # Answers can be of maximum size DIGITS + 1
+    # Answers can be of maximum size DIGITS + 1.
     ans += ' ' * (DIGITS + 1 - len(ans))
     if INVERT:
+        # Reverse the query, e.g., '12+345  ' becomes '  543+21'. (Note the
+        # space used for padding.)
         query = query[::-1]
     questions.append(query)
     expected.append(ans)
 print('Total addition questions:', len(questions))
 
 print('Vectorization...')
-X = np.zeros((len(questions), MAXLEN, len(chars)), dtype=np.bool)
+x = np.zeros((len(questions), MAxLEN, len(chars)), dtype=np.bool)
 y = np.zeros((len(questions), DIGITS + 1, len(chars)), dtype=np.bool)
 for i, sentence in enumerate(questions):
-    X[i] = ctable.encode(sentence, maxlen=MAXLEN)
+    x[i] = ctable.encode(sentence, MAxLEN)
 for i, sentence in enumerate(expected):
-    y[i] = ctable.encode(sentence, maxlen=DIGITS + 1)
+    y[i] = ctable.encode(sentence, DIGITS + 1)
 
-# Shuffle (X, y) in unison as the later parts of X will almost all be larger digits
+# Shuffle (x, y) in unison as the later parts of x will almost all be larger
+# digits.
 indices = np.arange(len(y))
 np.random.shuffle(indices)
-X = X[indices]
+x = x[indices]
 y = y[indices]
 
-# Explicitly set apart 10% for validation data that we never train over
-split_at = len(X) - len(X) / 10
-(X_train, X_val) = (slice_X(X, 0, split_at), slice_X(X, split_at))
-(y_train, y_val) = (y[:split_at], y[split_at:])
+# Explicitly set apart 10% for validation data that we never train over.
+split_at = len(x) - len(x) // 10
+(x_train, x_val) = x[:split_at], x[split_at:]
+(y_train, y_val) = y[:split_at], y[split_at:]
 
-print(X_train.shape)
+print('Training Data:')
+print(x_train.shape)
 print(y_train.shape)
 
+print('Validation Data:')
+print(x_val.shape)
+print(y_val.shape)
+
+# Try replacing GRU, or SimpleRNN.
+RNN = layers.LSTM
+HIDDEN_SIZE = 128
+BATCH_SIZE = 128
+LAYERS = 1
+
 print('Build model...')
 model = Sequential()
-# "Encode" the input sequence using an RNN, producing an output of HIDDEN_SIZE
-# note: in a situation where your input sequences have a variable length,
-# use input_shape=(None, nb_feature).
-model.add(RNN(HIDDEN_SIZE, input_shape=(MAXLEN, len(chars))))
-# For the decoder's input, we repeat the encoded input for each time step
-model.add(RepeatVector(DIGITS + 1))
-# The decoder RNN could be multiple layers stacked or a single layer
+# "Encode" the input sequence using an RNN, producing an output of HIDDEN_SIZE.
+# Note: In a situation where your input sequences have a variable length,
+# use input_shape=(None, num_feature).
+model.add(RNN(HIDDEN_SIZE, input_shape=(MAxLEN, len(chars))))
+# As the decoder RNN's input, repeatedly provide with the last hidden state of
+# RNN for each time step. Repeat 'DIGITS + 1' times as that's the maximum
+# length of output, e.g., when DIGITS=3, max output is 999+999=1998.
+model.add(layers.RepeatVector(DIGITS + 1))
+# The decoder RNN could be multiple layers stacked or a single layer.
 for _ in range(LAYERS):
+    # By setting return_sequences to True, return not only the last output but
+    # all the outputs so far in the form of (num_samples, timesteps,
+    # output_dim). This is necessary as TimeDistributed in the below expects
+    # the first dimension to be the timesteps.
     model.add(RNN(HIDDEN_SIZE, return_sequences=True))
 
-# For each of step of the output sequence, decide which character should be chosen
-model.add(TimeDistributed(Dense(len(chars))))
-model.add(Activation('softmax'))
-
+# Apply a dense layer to the every temporal slice of an input. For each of step
+# of the output sequence, decide which character should be chosen.
+model.add(layers.TimeDistributed(layers.Dense(len(chars))))
+model.add(layers.Activation('softmax'))
 model.compile(loss='categorical_crossentropy',
               optimizer='adam',
               metrics=['accuracy'])
+model.summary()
 
-# Train the model each generation and show predictions against the validation dataset
+# Train the model each generation and show predictions against the validation
+# dataset.
 for iteration in range(1, 200):
     print()
     print('-' * 50)
     print('Iteration', iteration)
-    model.fit(X_train, y_train, batch_size=BATCH_SIZE, nb_epoch=1,
-              validation_data=(X_val, y_val))
-    ###
-    # Select 10 samples from the validation set at random so we can visualize errors
+    model.fit(x_train, y_train, batch_size=BATCH_SIZE, epochs=1,
+              validation_data=(x_val, y_val))
+    # Select 10 samples from the validation set at random so we can visualize
+    # errors.
     for i in range(10):
-        ind = np.random.randint(0, len(X_val))
-        rowX, rowy = X_val[np.array([ind])], y_val[np.array([ind])]
-        preds = model.predict_classes(rowX, verbose=0)
-        q = ctable.decode(rowX[0])
+        ind = np.random.randint(0, len(x_val))
+        rowx, rowy = x_val[np.array([ind])], y_val[np.array([ind])]
+        preds = model.predict_classes(rowx, verbose=0)
+        q = ctable.decode(rowx[0])
         correct = ctable.decode(rowy[0])
         guess = ctable.decode(preds[0], calc_argmax=False)
         print('Q', q[::-1] if INVERT else q)
         print('T', correct)
-        print(colors.ok + '☑' + colors.close if correct == guess else colors.fail + '☒' + colors.close, guess)
+        if correct == guess:
+            print(colors.ok + '☑' + colors.close, end=" ")
+        else:
+            print(colors.fail + '☒' + colors.close, end=" ")
+        print(guess)
         print('---')

examples/antirectifier.py

@@ -11,14 +11,14 @@ backend (`K`), our code can run both on TensorFlow and Theano.
 '''
 
 from __future__ import print_function
+import keras
 from keras.models import Sequential
-from keras.layers import Dense, Dropout, Layer, Activation
+from keras import layers
 from keras.datasets import mnist
 from keras import backend as K
-from keras.utils import np_utils
 
 
-class Antirectifier(Layer):
+class Antirectifier(layers.Layer):
     '''This is the combination of a sample-wise
     L2 normalization with the concatenation of the
     positive part of the input with the negative part
@@ -45,50 +45,51 @@ class Antirectifier(Layer):
         with twice less parameters yet with comparable
         classification accuracy as an equivalent ReLU-based network.
     '''
-    def get_output_shape_for(self, input_shape):
+
+    def compute_output_shape(self, input_shape):
         shape = list(input_shape)
         assert len(shape) == 2  # only valid for 2D tensors
         shape[-1] *= 2
         return tuple(shape)
 
-    def call(self, x, mask=None):
-        x -= K.mean(x, axis=1, keepdims=True)
-        x = K.l2_normalize(x, axis=1)
-        pos = K.relu(x)
-        neg = K.relu(-x)
+    def call(self, inputs):
+        inputs -= K.mean(inputs, axis=1, keepdims=True)
+        inputs = K.l2_normalize(inputs, axis=1)
+        pos = K.relu(inputs)
+        neg = K.relu(-inputs)
         return K.concatenate([pos, neg], axis=1)
 
 # global parameters
 batch_size = 128
-nb_classes = 10
-nb_epoch = 40
+num_classes = 10
+epochs = 40
 
 # the data, shuffled and split between train and test sets
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
 
-X_train = X_train.reshape(60000, 784)
-X_test = X_test.reshape(10000, 784)
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
-print(X_train.shape[0], 'train samples')
-print(X_test.shape[0], 'test samples')
+x_train = x_train.reshape(60000, 784)
+x_test = x_test.reshape(10000, 784)
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
+print(x_train.shape[0], 'train samples')
+print(x_test.shape[0], 'test samples')
 
 # convert class vectors to binary class matrices
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
 
 # build the model
 model = Sequential()
-model.add(Dense(256, input_shape=(784,)))
+model.add(layers.Dense(256, input_shape=(784,)))
 model.add(Antirectifier())
-model.add(Dropout(0.1))
-model.add(Dense(256))
+model.add(layers.Dropout(0.1))
+model.add(layers.Dense(256))
 model.add(Antirectifier())
-model.add(Dropout(0.1))
-model.add(Dense(10))
-model.add(Activation('softmax'))
+model.add(layers.Dropout(0.1))
+model.add(layers.Dense(10))
+model.add(layers.Activation('softmax'))
 
 # compile the model
 model.compile(loss='categorical_crossentropy',
@@ -96,9 +97,9 @@ model.compile(loss='categorical_crossentropy',
               metrics=['accuracy'])
 
 # train the model
-model.fit(X_train, Y_train,
-          batch_size=batch_size, nb_epoch=nb_epoch,
-          verbose=1, validation_data=(X_test, Y_test))
+model.fit(x_train, y_train,
+          batch_size=batch_size, epochs=epochs,
+          verbose=1, validation_data=(x_test, y_test))
 
 # next, compare with an equivalent network
 # with2x bigger Dense layers and ReLU

examples/babi_memnn.py

@@ -12,8 +12,8 @@ References:
 Reaches 98.6% accuracy on task 'single_supporting_fact_10k' after 120 epochs.
 Time per epoch: 3s on CPU (core i7).
 '''
-
 from __future__ import print_function
+
 from keras.models import Sequential
 from keras.layers.embeddings import Embedding
 from keras.layers import Activation, Dense, Merge, Permute, Dropout
@@ -206,5 +206,5 @@ answer.compile(optimizer='rmsprop', loss='categorical_crossentropy',
 # Note: you could use a Graph model to avoid repeat the input twice
 answer.fit([inputs_train, queries_train, inputs_train], answers_train,
            batch_size=32,
-           nb_epoch=120,
+           epochs=120,
            validation_data=([inputs_test, queries_test, inputs_test], answers_test))

examples/babi_rnn.py

@@ -12,7 +12,7 @@ QA2 - Two Supporting Facts   | 20               | 50.0
 QA3 - Three Supporting Facts | 20               | 20.5
 QA4 - Two Arg. Relations     | 61               | 62.9
 QA5 - Three Arg. Relations   | 70               | 61.9
-QA6 - Yes/No Questions       | 48               | 50.7
+QA6 - yes/No Questions       | 48               | 50.7
 QA7 - Counting               | 49               | 78.9
 QA8 - Lists/Sets             | 45               | 77.2
 QA9 - Simple Negation        | 64               | 64.0
@@ -62,13 +62,12 @@ import re
 import tarfile
 
 import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 from keras.utils.data_utils import get_file
 from keras.layers.embeddings import Embedding
-from keras.layers import Dense, Merge, Dropout, RepeatVector
+from keras import layers
 from keras.layers import recurrent
-from keras.models import Sequential
+from keras.models import Model
 from keras.preprocessing.sequence import pad_sequences
 
 
@@ -125,26 +124,26 @@ def get_stories(f, only_supporting=False, max_length=None):
 
 
 def vectorize_stories(data, word_idx, story_maxlen, query_maxlen):
-    X = []
-    Xq = []
-    Y = []
+    xs = []
+    xqs = []
+    ys = []
     for story, query, answer in data:
         x = [word_idx[w] for w in story]
         xq = [word_idx[w] for w in query]
         y = np.zeros(len(word_idx) + 1)  # let's not forget that index 0 is reserved
         y[word_idx[answer]] = 1
-        X.append(x)
-        Xq.append(xq)
-        Y.append(y)
-    return pad_sequences(X, maxlen=story_maxlen), pad_sequences(Xq, maxlen=query_maxlen), np.array(Y)
+        xs.append(x)
+        xqs.append(xq)
+        ys.append(y)
+    return pad_sequences(xs, maxlen=story_maxlen), pad_sequences(xqs, maxlen=query_maxlen), np.array(ys)
 
 RNN = recurrent.LSTM
 EMBED_HIDDEN_SIZE = 50
 SENT_HIDDEN_SIZE = 100
-QUERY_HIDDEN_SIZE = 100
+QUERy_HIDDEN_SIZE = 100
 BATCH_SIZE = 32
 EPOCHS = 40
-print('RNN / Embed / Sent / Query = {}, {}, {}, {}'.format(RNN, EMBED_HIDDEN_SIZE, SENT_HIDDEN_SIZE, QUERY_HIDDEN_SIZE))
+print('RNN / Embed / Sent / Query = {}, {}, {}, {}'.format(RNN, EMBED_HIDDEN_SIZE, SENT_HIDDEN_SIZE, QUERy_HIDDEN_SIZE))
 
 try:
     path = get_file('babi-tasks-v1-2.tar.gz', origin='https://s3.amazonaws.com/text-datasets/babi_tasks_1-20_v1-2.tar.gz')
@@ -172,40 +171,38 @@ word_idx = dict((c, i + 1) for i, c in enumerate(vocab))
 story_maxlen = max(map(len, (x for x, _, _ in train + test)))
 query_maxlen = max(map(len, (x for _, x, _ in train + test)))
 
-X, Xq, Y = vectorize_stories(train, word_idx, story_maxlen, query_maxlen)
-tX, tXq, tY = vectorize_stories(test, word_idx, story_maxlen, query_maxlen)
+x, xq, y = vectorize_stories(train, word_idx, story_maxlen, query_maxlen)
+tx, txq, ty = vectorize_stories(test, word_idx, story_maxlen, query_maxlen)
 
 print('vocab = {}'.format(vocab))
-print('X.shape = {}'.format(X.shape))
-print('Xq.shape = {}'.format(Xq.shape))
-print('Y.shape = {}'.format(Y.shape))
+print('x.shape = {}'.format(x.shape))
+print('xq.shape = {}'.format(xq.shape))
+print('y.shape = {}'.format(y.shape))
 print('story_maxlen, query_maxlen = {}, {}'.format(story_maxlen, query_maxlen))
 
 print('Build model...')
 
-sentrnn = Sequential()
-sentrnn.add(Embedding(vocab_size, EMBED_HIDDEN_SIZE,
-                      input_length=story_maxlen))
-sentrnn.add(Dropout(0.3))
+sentence = layers.Input(shape=(story_maxlen,), dtype='int32')
+encoded_sentence = layers.Embedding(vocab_size, EMBED_HIDDEN_SIZE)(sentence)
+encoded_sentence = layers.Dropout(0.3)(encoded_sentence)
 
-qrnn = Sequential()
-qrnn.add(Embedding(vocab_size, EMBED_HIDDEN_SIZE,
-                   input_length=query_maxlen))
-qrnn.add(Dropout(0.3))
-qrnn.add(RNN(EMBED_HIDDEN_SIZE, return_sequences=False))
-qrnn.add(RepeatVector(story_maxlen))
+question = layers.Input(shape=(query_maxlen,), dtype='int32')
+encoded_question = layers.Embedding(vocab_size, EMBED_HIDDEN_SIZE)(question)
+encoded_question = layers.Dropout(0.3)(encoded_question)
+encoded_question = RNN(EMBED_HIDDEN_SIZE)(encoded_question)
+encoded_question = layers.RepeatVector(story_maxlen)(encoded_question)
 
-model = Sequential()
-model.add(Merge([sentrnn, qrnn], mode='sum'))
-model.add(RNN(EMBED_HIDDEN_SIZE, return_sequences=False))
-model.add(Dropout(0.3))
-model.add(Dense(vocab_size, activation='softmax'))
+merged = layers.add([encoded_sentence, encoded_question])
+merged = RNN(EMBED_HIDDEN_SIZE)(merged)
+merged = layers.Dropout(0.3)(merged)
+preds = layers.Dense(vocab_size, activation='softmax')(merged)
 
+model = Model([sentence, question], preds)
 model.compile(optimizer='adam',
               loss='categorical_crossentropy',
               metrics=['accuracy'])
 
 print('Training')
-model.fit([X, Xq], Y, batch_size=BATCH_SIZE, nb_epoch=EPOCHS, validation_split=0.05)
-loss, acc = model.evaluate([tX, tXq], tY, batch_size=BATCH_SIZE)
+model.fit([x, xq], y, batch_size=BATCH_SIZE, epochs=EPOCHS, validation_split=0.05)
+loss, acc = model.evaluate([tx, txq], ty, batch_size=BATCH_SIZE)
 print('Test loss / test accuracy = {:.4f} / {:.4f}'.format(loss, acc))

examples/cifar10_cnn.py

@@ -1,23 +1,23 @@
 '''Train a simple deep CNN on the CIFAR10 small images dataset.
 
 GPU run command with Theano backend (with TensorFlow, the GPU is automatically used):
-    THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python cifar10_cnn.py
+    THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatx=float32 python cifar10_cnn.py
 
 It gets down to 0.65 test logloss in 25 epochs, and down to 0.55 after 50 epochs.
 (it's still underfitting at that point, though).
 '''
 
 from __future__ import print_function
+import keras
 from keras.datasets import cifar10
 from keras.preprocessing.image import ImageDataGenerator
 from keras.models import Sequential
 from keras.layers import Dense, Dropout, Activation, Flatten
-from keras.layers import Convolution2D, MaxPooling2D
-from keras.utils import np_utils
+from keras.layers import Conv2D, MaxPooling2D
 
 batch_size = 32
-nb_classes = 10
-nb_epoch = 200
+num_classes = 10
+epochs = 200
 data_augmentation = True
 
 # input image dimensions
@@ -26,28 +26,28 @@ img_rows, img_cols = 32, 32
 img_channels = 3
 
 # The data, shuffled and split between train and test sets:
-(X_train, y_train), (X_test, y_test) = cifar10.load_data()
-print('X_train shape:', X_train.shape)
-print(X_train.shape[0], 'train samples')
-print(X_test.shape[0], 'test samples')
+(x_train, y_train), (x_test, y_test) = cifar10.load_data()
+print('x_train shape:', x_train.shape)
+print(x_train.shape[0], 'train samples')
+print(x_test.shape[0], 'test samples')
 
 # Convert class vectors to binary class matrices.
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
 
 model = Sequential()
 
-model.add(Convolution2D(32, 3, 3, border_mode='same',
-                        input_shape=X_train.shape[1:]))
+model.add(Conv2D(32, (3, 3), padding='same',
+                 input_shape=x_train.shape[1:]))
 model.add(Activation('relu'))
-model.add(Convolution2D(32, 3, 3))
+model.add(Conv2D(32, (3, 3)))
 model.add(Activation('relu'))
 model.add(MaxPooling2D(pool_size=(2, 2)))
 model.add(Dropout(0.25))
 
-model.add(Convolution2D(64, 3, 3, border_mode='same'))
+model.add(Conv2D(64, (3, 3), padding='same'))
 model.add(Activation('relu'))
-model.add(Convolution2D(64, 3, 3))
+model.add(Conv2D(64, (3, 3)))
 model.add(Activation('relu'))
 model.add(MaxPooling2D(pool_size=(2, 2)))
 model.add(Dropout(0.25))
@@ -56,7 +56,7 @@ model.add(Flatten())
 model.add(Dense(512))
 model.add(Activation('relu'))
 model.add(Dropout(0.5))
-model.add(Dense(nb_classes))
+model.add(Dense(num_classes))
 model.add(Activation('softmax'))
 
 # Let's train the model using RMSprop
@@ -64,17 +64,17 @@ model.compile(loss='categorical_crossentropy',
               optimizer='rmsprop',
               metrics=['accuracy'])
 
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
 
 if not data_augmentation:
     print('Not using data augmentation.')
-    model.fit(X_train, Y_train,
+    model.fit(x_train, y_train,
               batch_size=batch_size,
-              nb_epoch=nb_epoch,
-              validation_data=(X_test, Y_test),
+              epochs=epochs,
+              validation_data=(x_test, y_test),
               shuffle=True)
 else:
     print('Using real-time data augmentation.')
@@ -93,11 +93,11 @@ else:
 
     # Compute quantities required for featurewise normalization
     # (std, mean, and principal components if ZCA whitening is applied).
-    datagen.fit(X_train)
+    datagen.fit(x_train)
 
     # Fit the model on the batches generated by datagen.flow().
-    model.fit_generator(datagen.flow(X_train, Y_train,
-                        batch_size=batch_size),
-                        samples_per_epoch=X_train.shape[0],
-                        nb_epoch=nb_epoch,
-                        validation_data=(X_test, Y_test))
+    model.fit_generator(datagen.flow(x_train, y_train,
+                                     batch_size=batch_size),
+                        steps_per_epoch=x_train.shape[0] // batch_size,
+                        epochs=epochs,
+                        validation_data=(x_test, y_test))

examples/conv_filter_visualization.py

@@ -5,6 +5,7 @@ This script can run on CPU in a few minutes (with the TensorFlow backend).
 Results example: http://i.imgur.com/4nj4KjN.jpg
 '''
 from __future__ import print_function
+
 from scipy.misc import imsave
 import numpy as np
 import time
@@ -20,6 +21,8 @@ img_height = 128
 layer_name = 'block5_conv1'
 
 # util function to convert a tensor into a valid image
+
+
 def deprocess_image(x):
     # normalize tensor: center on 0., ensure std is 0.1
     x -= x.mean()
@@ -32,7 +35,7 @@ def deprocess_image(x):
 
     # convert to RGB array
     x *= 255
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         x = x.transpose((1, 2, 0))
     x = np.clip(x, 0, 255).astype('uint8')
     return x
@@ -65,7 +68,7 @@ for filter_index in range(0, 200):
     # we build a loss function that maximizes the activation
     # of the nth filter of the layer considered
     layer_output = layer_dict[layer_name].output
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         loss = K.mean(layer_output[:, filter_index, :, :])
     else:
         loss = K.mean(layer_output[:, :, :, filter_index])
@@ -83,7 +86,7 @@ for filter_index in range(0, 200):
     step = 1.
 
     # we start from a gray image with some random noise
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         input_img_data = np.random.random((1, 3, img_width, img_height))
     else:
         input_img_data = np.random.random((1, img_width, img_height, 3))

examples/conv_lstm.py

@@ -3,7 +3,7 @@ This network is used to predict the next frame of an artificially
 generated movie which contains moving squares.
 """
 from keras.models import Sequential
-from keras.layers.convolutional import Convolution3D
+from keras.layers.convolutional import Conv3D
 from keras.layers.convolutional_recurrent import ConvLSTM2D
 from keras.layers.normalization import BatchNormalization
 import numpy as np
@@ -14,27 +14,26 @@ import pylab as plt
 # of identical shape.
 
 seq = Sequential()
-seq.add(ConvLSTM2D(nb_filter=40, nb_row=3, nb_col=3,
+seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
                    input_shape=(None, 40, 40, 1),
-                   border_mode='same', return_sequences=True))
+                   padding='same', return_sequences=True))
 seq.add(BatchNormalization())
 
-seq.add(ConvLSTM2D(nb_filter=40, nb_row=3, nb_col=3,
-                   border_mode='same', return_sequences=True))
+seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
+                   padding='same', return_sequences=True))
 seq.add(BatchNormalization())
 
-seq.add(ConvLSTM2D(nb_filter=40, nb_row=3, nb_col=3,
-                   border_mode='same', return_sequences=True))
+seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
+                   padding='same', return_sequences=True))
 seq.add(BatchNormalization())
 
-seq.add(ConvLSTM2D(nb_filter=40, nb_row=3, nb_col=3,
-                   border_mode='same', return_sequences=True))
+seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
+                   padding='same', return_sequences=True))
 seq.add(BatchNormalization())
 
-seq.add(Convolution3D(nb_filter=1, kernel_dim1=1, kernel_dim2=3,
-                      kernel_dim3=3, activation='sigmoid',
-                      border_mode='same', dim_ordering='tf'))
-
+seq.add(Conv3D(filters=1, kernel_size=(3, 3, 3),
+               activation='sigmoid',
+               padding='same', data_format='channels_last'))
 seq.compile(loss='binary_crossentropy', optimizer='adadelta')
 
 
@@ -101,7 +100,7 @@ def generate_movies(n_samples=1200, n_frames=15):
 # Train the network
 noisy_movies, shifted_movies = generate_movies(n_samples=1200)
 seq.fit(noisy_movies[:1000], shifted_movies[:1000], batch_size=10,
-        nb_epoch=300, validation_split=0.05)
+        epochs=300, validation_split=0.05)
 
 # Testing the network on one movie
 # feed it with the first 7 positions and then

examples/deep_dream.py

@@ -15,6 +15,7 @@ If running on CPU, prefer the TensorFlow backend (much faster).
 Example results: http://i.imgur.com/FX6ROg9.jpg
 '''
 from __future__ import print_function
+
 from keras.preprocessing.image import load_img, img_to_array
 import numpy as np
 from scipy.misc import imsave
@@ -37,11 +38,8 @@ base_image_path = args.base_image_path
 result_prefix = args.result_prefix
 
 # dimensions of the generated picture.
-img_width = 600
 img_height = 600
-
-# path to the model weights file.
-weights_path = 'vgg16_weights.h5'
+img_width = 600
 
 # some settings we found interesting
 saved_settings = {
@@ -60,21 +58,24 @@ saved_settings = {
 # the settings we will use in this experiment
 settings = saved_settings['dreamy']
 
-# util function to open, resize and format pictures into appropriate tensors
+
 def preprocess_image(image_path):
-    img = load_img(image_path, target_size=(img_width, img_height))
+    # util function to open, resize and format pictures
+    # into appropriate tensors
+    img = load_img(image_path, target_size=(img_height, img_width))
     img = img_to_array(img)
     img = np.expand_dims(img, axis=0)
     img = vgg16.preprocess_input(img)
     return img
 
-# util function to convert a tensor into a valid image
+
 def deprocess_image(x):
-    if K.image_dim_ordering() == 'th':
-        x = x.reshape((3, img_width, img_height))
+    # util function to convert a tensor into a valid image
+    if K.image_data_format() == 'channels_first':
+        x = x.reshape((3, img_height, img_width))
         x = x.transpose((1, 2, 0))
     else:
-        x = x.reshape((img_width, img_height, 3))
+        x = x.reshape((img_height, img_width, 3))
     # Remove zero-center by mean pixel
     x[:, :, 0] += 103.939
     x[:, :, 1] += 116.779
@@ -84,10 +85,10 @@ def deprocess_image(x):
     x = np.clip(x, 0, 255).astype('uint8')
     return x
 
-if K.image_dim_ordering() == 'th':
-    img_size = (3, img_width, img_height)
+if K.image_data_format() == 'channels_first':
+    img_size = (3, img_height, img_width)
 else:
-    img_size = (img_width, img_height, 3)
+    img_size = (img_height, img_width, 3)
 # this will contain our generated image
 dream = Input(batch_shape=(1,) + img_size)
 
@@ -100,19 +101,20 @@ print('Model loaded.')
 # get the symbolic outputs of each "key" layer (we gave them unique names).
 layer_dict = dict([(layer.name, layer) for layer in model.layers])
 
-# continuity loss util function
+
 def continuity_loss(x):
+    # continuity loss util function
     assert K.ndim(x) == 4
-    if K.image_dim_ordering() == 'th':
-        a = K.square(x[:, :, :img_width - 1, :img_height - 1] -
-                     x[:, :, 1:, :img_height - 1])
-        b = K.square(x[:, :, :img_width - 1, :img_height - 1] -
-                     x[:, :, :img_width - 1, 1:])
+    if K.image_data_format() == 'channels_first':
+        a = K.square(x[:, :, :img_height - 1, :img_width - 1] -
+                     x[:, :, 1:, :img_width - 1])
+        b = K.square(x[:, :, :img_height - 1, :img_width - 1] -
+                     x[:, :, :img_height - 1, 1:])
     else:
-        a = K.square(x[:, :img_width - 1, :img_height-1, :] -
-                     x[:, 1:, :img_height - 1, :])
-        b = K.square(x[:, :img_width - 1, :img_height-1, :] -
-                     x[:, :img_width - 1, 1:, :])
+        a = K.square(x[:, :img_height - 1, :img_width - 1, :] -
+                     x[:, 1:, :img_width - 1, :])
+        b = K.square(x[:, :img_height - 1, :img_width - 1, :] -
+                     x[:, :img_height - 1, 1:, :])
     return K.sum(K.pow(a + b, 1.25))
 
 # define the loss
@@ -124,7 +126,7 @@ for layer_name in settings['features']:
     x = layer_dict[layer_name].output
     shape = layer_dict[layer_name].output_shape
     # we avoid border artifacts by only involving non-border pixels in the loss
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         loss -= coeff * K.sum(K.square(x[:, :, 2: shape[2] - 2, 2: shape[3] - 2])) / np.prod(shape[1:])
     else:
         loss -= coeff * K.sum(K.square(x[:, 2: shape[1] - 2, 2: shape[2] - 2, :])) / np.prod(shape[1:])
@@ -140,12 +142,14 @@ loss += settings['dream_l2'] * K.sum(K.square(dream)) / np.prod(img_size)
 grads = K.gradients(loss, dream)
 
 outputs = [loss]
-if type(grads) in {list, tuple}:
+if isinstance(grads, (list, tuple)):
     outputs += grads
 else:
     outputs.append(grads)
 
 f_outputs = K.function([dream], outputs)
+
+
 def eval_loss_and_grads(x):
     x = x.reshape((1,) + img_size)
     outs = f_outputs([x])
@@ -156,13 +160,18 @@ def eval_loss_and_grads(x):
         grad_values = np.array(outs[1:]).flatten().astype('float64')
     return loss_value, grad_values
 
-# this Evaluator class makes it possible
-# to compute loss and gradients in one pass
-# while retrieving them via two separate functions,
-# "loss" and "grads". This is done because scipy.optimize
-# requires separate functions for loss and gradients,
-# but computing them separately would be inefficient.
+
 class Evaluator(object):
+    """Loss and gradients evaluator.
+
+    This Evaluator class makes it possible
+    to compute loss and gradients in one pass
+    while retrieving them via two separate functions,
+    "loss" and "grads". This is done because scipy.optimize
+    requires separate functions for loss and gradients,
+    but computing them separately would be inefficient.
+    """
+
     def __init__(self):
         self.loss_value = None
         self.grad_values = None
@@ -183,22 +192,23 @@ class Evaluator(object):
 
 evaluator = Evaluator()
 
-# run scipy-based optimization (L-BFGS) over the pixels of the generated image
+# Run scipy-based optimization (L-BFGS) over the pixels of the generated image
 # so as to minimize the loss
 x = preprocess_image(base_image_path)
 for i in range(5):
     print('Start of iteration', i)
     start_time = time.time()
 
-    # add a random jitter to the initial image. This will be reverted at decoding time
+    # Add a random jitter to the initial image.
+    # This will be reverted at decoding time
     random_jitter = (settings['jitter'] * 2) * (np.random.random(img_size) - 0.5)
     x += random_jitter
 
-    # run L-BFGS for 7 steps
+    # Run L-BFGS for 7 steps
     x, min_val, info = fmin_l_bfgs_b(evaluator.loss, x.flatten(),
                                      fprime=evaluator.grads, maxfun=7)
     print('Current loss value:', min_val)
-    # decode the dream and save it
+    # Decode the dream and save it
     x = x.reshape(img_size)
     x -= random_jitter
     img = deprocess_image(np.copy(x))

examples/image_ocr.py

@@ -6,36 +6,31 @@ the different fonts thrown at it...the purpose is more to demonstrate CTC
 inside of Keras.  Note that the font list may need to be updated
 for the particular OS in use.
 
-This starts off with 4 letter words. After 10 or so epochs, CTC
-learns translational invariance, so longer words and groups of words
-with spaces are gradually fed in.  This gradual increase in difficulty
-is handled using the TextImageGenerator class which is both a generator
-class for test/train data and a Keras callback class. Every 10 epochs
-the wordlist that the generator draws from increases in difficulty.
+This starts off with 4 letter words.  For the first 12 epochs, the
+difficulty is gradually increased using the TextImageGenerator class
+which is both a generator class for test/train data and a Keras
+callback class. After 20 epochs, longer sequences are thrown at it
+by recompiling the model to handle a wider image and rebuilding
+the word list to include two words separated by a space.
 
 The table below shows normalized edit distance values. Theano uses
-a slightly different CTC implementation, so some Theano-specific
-hyperparameter tuning would be needed to get it to match Tensorflow.
+a slightly different CTC implementation, hence the different results.
 
             Norm. ED
 Epoch |   TF   |   TH
 ------------------------
-    10   0.072    0.272
-    20   0.032    0.115
-    30   0.024    0.098
-    40   0.023    0.108
+    10   0.027   0.064
+    15   0.038   0.035
+    20   0.043   0.045
+    25   0.014   0.019
 
 This requires cairo and editdistance packages:
 pip install cairocffi
 pip install editdistance
 
-Due to the use of a dummy loss function, Theano requires the following flags:
-on_unused_input='ignore'
-
 Created by Mike Henry
 https://github.com/mbhenry/
 '''
-
 import os
 import itertools
 import re
@@ -47,17 +42,17 @@ from scipy import ndimage
 import pylab
 from keras import backend as K
 from keras.layers.convolutional import Convolution2D, MaxPooling2D
-from keras.layers import Input, Layer, Dense, Activation, Flatten
-from keras.layers import Reshape, Lambda, merge, Permute, TimeDistributed
+from keras.layers import Input, Dense, Activation
+from keras.layers import Reshape, Lambda, merge
 from keras.models import Model
 from keras.layers.recurrent import GRU
 from keras.optimizers import SGD
-from keras.utils import np_utils
 from keras.utils.data_utils import get_file
 from keras.preprocessing import image
 import keras.callbacks
 
-OUTPUT_DIR = "image_ocr"
+
+OUTPUT_DIR = 'image_ocr'
 
 np.random.seed(55)
 
@@ -79,28 +74,33 @@ def speckle(img):
 # also uses a random font, a slight random rotation,
 # and a random amount of speckle noise
 
-def paint_text(text, w, h):
+def paint_text(text, w, h, rotate=False, ud=False, multi_fonts=False):
     surface = cairo.ImageSurface(cairo.FORMAT_RGB24, w, h)
     with cairo.Context(surface) as context:
         context.set_source_rgb(1, 1, 1)  # White
         context.paint()
         # this font list works in Centos 7
-        fonts = ['Century Schoolbook', 'Courier', 'STIX', 'URW Chancery L', 'FreeMono']
-        context.select_font_face(np.random.choice(fonts), cairo.FONT_SLANT_NORMAL,
-                                 np.random.choice([cairo.FONT_WEIGHT_BOLD, cairo.FONT_WEIGHT_NORMAL]))
-        context.set_font_size(40)
+        if multi_fonts:
+            fonts = ['Century Schoolbook', 'Courier', 'STIX', 'URW Chancery L', 'FreeMono']
+            context.select_font_face(np.random.choice(fonts), cairo.FONT_SLANT_NORMAL,
+                                     np.random.choice([cairo.FONT_WEIGHT_BOLD, cairo.FONT_WEIGHT_NORMAL]))
+        else:
+            context.select_font_face('Courier', cairo.FONT_SLANT_NORMAL, cairo.FONT_WEIGHT_BOLD)
+        context.set_font_size(25)
         box = context.text_extents(text)
-        if box[2] > w or box[3] > h:
+        border_w_h = (4, 4)
+        if box[2] > (w - 2 * border_w_h[1]) or box[3] > (h - 2 * border_w_h[0]):
             raise IOError('Could not fit string into image. Max char count is too large for given image width.')
 
         # teach the RNN translational invariance by
         # fitting text box randomly on canvas, with some room to rotate
-        border_w_h = (10, 16)
         max_shift_x = w - box[2] - border_w_h[0]
         max_shift_y = h - box[3] - border_w_h[1]
         top_left_x = np.random.randint(0, int(max_shift_x))
-        top_left_y = np.random.randint(0, int(max_shift_y))
-
+        if ud:
+            top_left_y = np.random.randint(0, int(max_shift_y))
+        else:
+            top_left_y = h // 2
         context.move_to(top_left_x - int(box[0]), top_left_y - int(box[1]))
         context.set_source_rgb(0, 0, 0)
         context.show_text(text)
@@ -111,8 +111,9 @@ def paint_text(text, w, h):
     a = a[:, :, 0]  # grab single channel
     a = a.astype(np.float32) / 255
     a = np.expand_dims(a, 0)
+    if rotate:
+        a = image.random_rotation(a, 3 * (w - top_left_x) / w + 1)
     a = speckle(a)
-    a = image.random_rotation(a, 3 * (w - top_left_x) / w + 1)
 
     return a
 
@@ -164,7 +165,7 @@ def is_valid_str(in_str):
 class TextImageGenerator(keras.callbacks.Callback):
 
     def __init__(self, monogram_file, bigram_file, minibatch_size,
-                 img_w, img_h, downsample_width, val_split,
+                 img_w, img_h, downsample_factor, val_split,
                  absolute_max_string_len=16):
 
         self.minibatch_size = minibatch_size
@@ -172,7 +173,7 @@ class TextImageGenerator(keras.callbacks.Callback):
         self.img_h = img_h
         self.monogram_file = monogram_file
         self.bigram_file = bigram_file
-        self.downsample_width = downsample_width
+        self.downsample_factor = downsample_factor
         self.val_split = val_split
         self.blank_label = self.get_output_size() - 1
         self.absolute_max_string_len = absolute_max_string_len
@@ -187,7 +188,8 @@ class TextImageGenerator(keras.callbacks.Callback):
         assert num_words % self.minibatch_size == 0
         assert (self.val_split * num_words) % self.minibatch_size == 0
         self.num_words = num_words
-        self.string_list = []
+        self.string_list = [''] * self.num_words
+        tmp_string_list = []
         self.max_string_len = max_string_len
         self.Y_data = np.ones([self.num_words, self.absolute_max_string_len]) * -1
         self.X_text = []
@@ -196,25 +198,28 @@ class TextImageGenerator(keras.callbacks.Callback):
         # monogram file is sorted by frequency in english speech
         with open(self.monogram_file, 'rt') as f:
             for line in f:
-                if len(self.string_list) == int(self.num_words * mono_fraction):
+                if len(tmp_string_list) == int(self.num_words * mono_fraction):
                     break
                 word = line.rstrip()
                 if max_string_len == -1 or max_string_len is None or len(word) <= max_string_len:
-                    self.string_list.append(word)
+                    tmp_string_list.append(word)
 
         # bigram file contains common word pairings in english speech
         with open(self.bigram_file, 'rt') as f:
             lines = f.readlines()
             for line in lines:
-                if len(self.string_list) == self.num_words:
+                if len(tmp_string_list) == self.num_words:
                     break
                 columns = line.lower().split()
                 word = columns[0] + ' ' + columns[1]
                 if is_valid_str(word) and \
                         (max_string_len == -1 or max_string_len is None or len(word) <= max_string_len):
-                    self.string_list.append(word)
-        if len(self.string_list) != self.num_words:
+                    tmp_string_list.append(word)
+        if len(tmp_string_list) != self.num_words:
             raise IOError('Could not pull enough words from supplied monogram and bigram files. ')
+        # interlace to mix up the easy and hard words
+        self.string_list[::2] = tmp_string_list[:self.num_words // 2]
+        self.string_list[1::2] = tmp_string_list[self.num_words // 2:]
 
         for i, word in enumerate(self.string_list):
             self.Y_len[i] = len(word)
@@ -228,37 +233,38 @@ class TextImageGenerator(keras.callbacks.Callback):
     # each time an image is requested from train/val/test, a new random
     # painting of the text is performed
     def get_batch(self, index, size, train):
-        if K.image_dim_ordering() == 'th':
-            X_data = np.ones([size, 1, self.img_h, self.img_w])
+        # width and height are backwards from typical Keras convention
+        # because width is the time dimension when it gets fed into the RNN
+        if K.image_data_format() == 'channels_first':
+            X_data = np.ones([size, 1, self.img_w, self.img_h])
         else:
-            X_data = np.ones([size, self.img_h, self.img_w, 1])
+            X_data = np.ones([size, self.img_w, self.img_h, 1])
+
         labels = np.ones([size, self.absolute_max_string_len])
         input_length = np.zeros([size, 1])
         label_length = np.zeros([size, 1])
         source_str = []
-
         for i in range(0, size):
             # Mix in some blank inputs.  This seems to be important for
             # achieving translational invariance
             if train and i > size - 4:
-                if K.image_dim_ordering() == 'th':
-                    X_data[i, 0, :, :] = paint_text('', self.img_w, self.img_h)
+                if K.image_data_format() == 'channels_first':
+                    X_data[i, 0, 0:self.img_w, :] = self.paint_func('')[0, :, :].T
                 else:
-                    X_data[i, :, :, 0] = paint_text('', self.img_w, self.img_h)
+                    X_data[i, 0:self.img_w, :, 0] = self.paint_func('',)[0, :, :].T
                 labels[i, 0] = self.blank_label
-                input_length[i] = self.downsample_width
+                input_length[i] = self.img_w // self.downsample_factor - 2
                 label_length[i] = 1
                 source_str.append('')
             else:
-                if K.image_dim_ordering() == 'th':
-                    X_data[i, 0, :, :] = paint_text(self.X_text[index + i], self.img_w, self.img_h)
+                if K.image_data_format() == 'channels_first':
+                    X_data[i, 0, 0:self.img_w, :] = self.paint_func(self.X_text[index + i])[0, :, :].T
                 else:
-                    X_data[i, :, :, 0] = paint_text(self.X_text[index + i], self.img_w, self.img_h)
+                    X_data[i, 0:self.img_w, :, 0] = self.paint_func(self.X_text[index + i])[0, :, :].T
                 labels[i, :] = self.Y_data[index + i]
-                input_length[i] = self.downsample_width
+                input_length[i] = self.img_w // self.downsample_factor - 2
                 label_length[i] = self.Y_len[index + i]
                 source_str.append(self.X_text[index + i])
-
         inputs = {'the_input': X_data,
                   'the_labels': labels,
                   'input_length': input_length,
@@ -287,19 +293,23 @@ class TextImageGenerator(keras.callbacks.Callback):
             yield ret
 
     def on_train_begin(self, logs={}):
-        # translational invariance seems to be the hardest thing
-        # for the RNN to learn, so start with <= 4 letter words.
         self.build_word_list(16000, 4, 1)
+        self.paint_func = lambda text: paint_text(text, self.img_w, self.img_h,
+                                                  rotate=False, ud=False, multi_fonts=False)
 
     def on_epoch_begin(self, epoch, logs={}):
-        # After 10 epochs, translational invariance should be learned
-        # so start feeding longer words and eventually multiple words with spaces
-        if epoch == 10:
-            self.build_word_list(32000, 8, 1)
-        if epoch == 20:
-            self.build_word_list(32000, 8, 0.6)
-        if epoch == 30:
-            self.build_word_list(64000, 12, 0.5)
+        # rebind the paint function to implement curriculum learning
+        if epoch >= 3 and epoch < 6:
+            self.paint_func = lambda text: paint_text(text, self.img_w, self.img_h,
+                                                      rotate=False, ud=True, multi_fonts=False)
+        elif epoch >= 6 and epoch < 9:
+            self.paint_func = lambda text: paint_text(text, self.img_w, self.img_h,
+                                                      rotate=False, ud=True, multi_fonts=True)
+        elif epoch >= 9:
+            self.paint_func = lambda text: paint_text(text, self.img_w, self.img_h,
+                                                      rotate=True, ud=True, multi_fonts=True)
+        if epoch >= 21 and self.max_string_len < 12:
+            self.build_word_list(32000, 12, 0.5)
 
 
 # the actual loss calc occurs here despite it not being
@@ -335,13 +345,14 @@ def decode_batch(test_func, word_batch):
 
 class VizCallback(keras.callbacks.Callback):
 
-    def __init__(self, test_func, text_img_gen, num_display_words=6):
+    def __init__(self, run_name, test_func, text_img_gen, num_display_words=6):
         self.test_func = test_func
         self.output_dir = os.path.join(
-            OUTPUT_DIR, datetime.datetime.now().strftime('%A, %d. %B %Y %I.%M%p'))
+            OUTPUT_DIR, run_name)
         self.text_img_gen = text_img_gen
         self.num_display_words = num_display_words
-        os.makedirs(self.output_dir)
+        if not os.path.exists(self.output_dir):
+            os.makedirs(self.output_dir)
 
     def show_edit_distance(self, num):
         num_left = num
@@ -362,109 +373,116 @@ class VizCallback(keras.callbacks.Callback):
               % (num, mean_ed, mean_norm_ed))
 
     def on_epoch_end(self, epoch, logs={}):
-        self.model.save_weights(os.path.join(self.output_dir, 'weights%02d.h5' % epoch))
+        self.model.save_weights(os.path.join(self.output_dir, 'weights%02d.h5' % (epoch)))
         self.show_edit_distance(256)
         word_batch = next(self.text_img_gen)[0]
         res = decode_batch(self.test_func, word_batch['the_input'][0:self.num_display_words])
-
+        if word_batch['the_input'][0].shape[0] < 256:
+            cols = 2
+        else:
+            cols = 1
         for i in range(self.num_display_words):
-            pylab.subplot(self.num_display_words, 1, i + 1)
-            if K.image_dim_ordering() == 'th':
+            pylab.subplot(self.num_display_words // cols, cols, i + 1)
+            if K.image_data_format() == 'channels_first':
                 the_input = word_batch['the_input'][i, 0, :, :]
             else:
                 the_input = word_batch['the_input'][i, :, :, 0]
-            pylab.imshow(the_input, cmap='Greys_r')
-            pylab.xlabel('Truth = \'%s\' Decoded = \'%s\'' % (word_batch['source_str'][i], res[i]))
+            pylab.imshow(the_input.T, cmap='Greys_r')
+            pylab.xlabel('Truth = \'%s\'\nDecoded = \'%s\'' % (word_batch['source_str'][i], res[i]))
         fig = pylab.gcf()
-        fig.set_size_inches(10, 12)
-        pylab.savefig(os.path.join(self.output_dir, 'e%02d.png' % epoch))
+        fig.set_size_inches(10, 13)
+        pylab.savefig(os.path.join(self.output_dir, 'e%02d.png' % (epoch)))
         pylab.close()
 
-# Input Parameters
-img_h = 64
-img_w = 512
-nb_epoch = 50
-minibatch_size = 32
-words_per_epoch = 16000
-val_split = 0.2
-val_words = int(words_per_epoch * (val_split))
 
-# Network parameters
-conv_num_filters = 16
-filter_size = 3
-pool_size_1 = 4
-pool_size_2 = 2
-time_dense_size = 32
-rnn_size = 512
-time_steps = img_w // (pool_size_1 * pool_size_2)
+def train(run_name, start_epoch, stop_epoch, img_w):
+    # Input Parameters
+    img_h = 64
+    words_per_epoch = 16000
+    val_split = 0.2
+    val_words = int(words_per_epoch * (val_split))
 
-if K.image_dim_ordering() == 'th':
-    input_shape = (1, img_h, img_w)
-else:
-    input_shape = (img_h, img_w, 1)
+    # Network parameters
+    conv_filterss = 16
+    filter_size = 3
+    pool_size = 2
+    time_dense_size = 32
+    rnn_size = 512
 
-fdir = os.path.dirname(get_file('wordlists.tgz',
-                                origin='http://www.isosemi.com/datasets/wordlists.tgz', untar=True))
+    if K.image_data_format() == 'channels_first':
+        input_shape = (1, img_w, img_h)
+    else:
+        input_shape = (img_w, img_h, 1)
 
-img_gen = TextImageGenerator(monogram_file=os.path.join(fdir, 'wordlist_mono_clean.txt'),
-                             bigram_file=os.path.join(fdir, 'wordlist_bi_clean.txt'),
-                             minibatch_size=32,
-                             img_w=img_w,
-                             img_h=img_h,
-                             downsample_width=img_w // (pool_size_1 * pool_size_2) - 2,
-                             val_split=words_per_epoch - val_words)
+    fdir = os.path.dirname(get_file('wordlists.tgz',
+                                    origin='http://www.isosemi.com/datasets/wordlists.tgz', untar=True))
 
-act = 'relu'
-input_data = Input(name='the_input', shape=input_shape, dtype='float32')
-inner = Convolution2D(conv_num_filters, filter_size, filter_size, border_mode='same',
-                      activation=act, name='conv1')(input_data)
-inner = MaxPooling2D(pool_size=(pool_size_1, pool_size_1), name='max1')(inner)
-inner = Convolution2D(conv_num_filters, filter_size, filter_size, border_mode='same',
-                      activation=act, name='conv2')(inner)
-inner = MaxPooling2D(pool_size=(pool_size_2, pool_size_2), name='max2')(inner)
+    img_gen = TextImageGenerator(monogram_file=os.path.join(fdir, 'wordlist_mono_clean.txt'),
+                                 bigram_file=os.path.join(fdir, 'wordlist_bi_clean.txt'),
+                                 minibatch_size=32,
+                                 img_w=img_w,
+                                 img_h=img_h,
+                                 downsample_factor=(pool_size ** 2),
+                                 val_split=words_per_epoch - val_words
+                                 )
+    act = 'relu'
+    input_data = Input(name='the_input', shape=input_shape, dtype='float32')
+    inner = Convolution2D(conv_filterss, filter_size, filter_size, border_mode='same',
+                          activation=act, init='he_normal', name='conv1')(input_data)
+    inner = MaxPooling2D(pool_size=(pool_size, pool_size), name='max1')(inner)
+    inner = Convolution2D(conv_filterss, filter_size, filter_size, border_mode='same',
+                          activation=act, init='he_normal', name='conv2')(inner)
+    inner = MaxPooling2D(pool_size=(pool_size, pool_size), name='max2')(inner)
 
-conv_to_rnn_dims = ((img_h // (pool_size_1 * pool_size_2)) * conv_num_filters, img_w // (pool_size_1 * pool_size_2))
-inner = Reshape(target_shape=conv_to_rnn_dims, name='reshape')(inner)
-inner = Permute(dims=(2, 1), name='permute')(inner)
+    conv_to_rnn_dims = (img_w // (pool_size ** 2), (img_h // (pool_size ** 2)) * conv_filterss)
+    inner = Reshape(target_shape=conv_to_rnn_dims, name='reshape')(inner)
 
-# cuts down input size going into RNN:
-inner = TimeDistributed(Dense(time_dense_size, activation=act, name='dense1'))(inner)
+    # cuts down input size going into RNN:
+    inner = Dense(time_dense_size, activation=act, name='dense1')(inner)
 
-# Two layers of bidirecitonal GRUs
-# GRU seems to work as well, if not better than LSTM:
-gru_1 = GRU(rnn_size, return_sequences=True, name='gru1')(inner)
-gru_1b = GRU(rnn_size, return_sequences=True, go_backwards=True, name='gru1_b')(inner)
-gru1_merged = merge([gru_1, gru_1b], mode='sum')
-gru_2 = GRU(rnn_size, return_sequences=True, name='gru2')(gru1_merged)
-gru_2b = GRU(rnn_size, return_sequences=True, go_backwards=True)(gru1_merged)
+    # Two layers of bidirecitonal GRUs
+    # GRU seems to work as well, if not better than LSTM:
+    gru_1 = GRU(rnn_size, return_sequences=True, init='he_normal', name='gru1')(inner)
+    gru_1b = GRU(rnn_size, return_sequences=True, go_backwards=True, init='he_normal', name='gru1_b')(inner)
+    gru1_merged = merge([gru_1, gru_1b], mode='sum')
+    gru_2 = GRU(rnn_size, return_sequences=True, init='he_normal', name='gru2')(gru1_merged)
+    gru_2b = GRU(rnn_size, return_sequences=True, go_backwards=True, init='he_normal', name='gru2_b')(gru1_merged)
 
-# transforms RNN output to character activations:
-inner = TimeDistributed(Dense(img_gen.get_output_size(), name='dense2'))(merge([gru_2, gru_2b], mode='concat'))
-y_pred = Activation('softmax', name='softmax')(inner)
-Model(input=[input_data], output=y_pred).summary()
+    # transforms RNN output to character activations:
+    inner = Dense(img_gen.get_output_size(), init='he_normal',
+                  name='dense2')(merge([gru_2, gru_2b], mode='concat'))
+    y_pred = Activation('softmax', name='softmax')(inner)
+    Model(input=[input_data], output=y_pred).summary()
 
-labels = Input(name='the_labels', shape=[img_gen.absolute_max_string_len], dtype='float32')
-input_length = Input(name='input_length', shape=[1], dtype='int64')
-label_length = Input(name='label_length', shape=[1], dtype='int64')
-# Keras doesn't currently support loss funcs with extra parameters
-# so CTC loss is implemented in a lambda layer
-loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name="ctc")([y_pred, labels, input_length, label_length])
+    labels = Input(name='the_labels', shape=[img_gen.absolute_max_string_len], dtype='float32')
+    input_length = Input(name='input_length', shape=[1], dtype='int64')
+    label_length = Input(name='label_length', shape=[1], dtype='int64')
+    # Keras doesn't currently support loss funcs with extra parameters
+    # so CTC loss is implemented in a lambda layer
+    loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')([y_pred, labels, input_length, label_length])
 
-lr = 0.03
-# clipnorm seems to speeds up convergence
-clipnorm = 5
-sgd = SGD(lr=lr, decay=3e-7, momentum=0.9, nesterov=True, clipnorm=clipnorm)
+    # clipnorm seems to speeds up convergence
+    sgd = SGD(lr=0.02, decay=1e-6, momentum=0.9, nesterov=True, clipnorm=5)
 
-model = Model(input=[input_data, labels, input_length, label_length], output=[loss_out])
+    model = Model(input=[input_data, labels, input_length, label_length], output=[loss_out])
 
-# the loss calc occurs elsewhere, so use a dummy lambda func for the loss
-model.compile(loss={'ctc': lambda y_true, y_pred: y_pred}, optimizer=sgd)
+    # the loss calc occurs elsewhere, so use a dummy lambda func for the loss
+    model.compile(loss={'ctc': lambda y_true, y_pred: y_pred}, optimizer=sgd)
+    if start_epoch > 0:
+        weight_file = os.path.join(OUTPUT_DIR, os.path.join(run_name, 'weights%02d.h5' % (start_epoch - 1)))
+        model.load_weights(weight_file)
+    # captures output of softmax so we can decode the output during visualization
+    test_func = K.function([input_data], [y_pred])
 
-# captures output of softmax so we can decode the output during visualization
-test_func = K.function([input_data], [y_pred])
+    viz_cb = VizCallback(run_name, test_func, img_gen.next_val())
 
-viz_cb = VizCallback(test_func, img_gen.next_val())
+    model.fit_generator(generator=img_gen.next_train(), samples_per_epoch=(words_per_epoch - val_words),
+                        epochs=stop_epoch, validation_data=img_gen.next_val(), num_val_samples=val_words,
+                        callbacks=[viz_cb, img_gen], initial_epoch=start_epoch)
 
-model.fit_generator(generator=img_gen.next_train(), samples_per_epoch=(words_per_epoch - val_words),
-                    nb_epoch=nb_epoch, validation_data=img_gen.next_val(), nb_val_samples=val_words,
-                    callbacks=[viz_cb, img_gen])
+
+if __name__ == '__main__':
+    run_name = datetime.datetime.now().strftime('%Y:%m:%d:%H:%M:%S')
+    train(run_name, 0, 20, 128)
+    # increase to wider images and start at epoch 20. The learned weights are reloaded
+    train(run_name, 20, 25, 512)

examples/imdb_bidirectional_lstm.py

@@ -6,28 +6,29 @@ Time per epoch on CPU (Core i7): ~150s.
 
 from __future__ import print_function
 import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 from keras.preprocessing import sequence
 from keras.models import Sequential
-from keras.layers import Dense, Dropout, Embedding, LSTM, Input, Bidirectional
+from keras.layers import Dense, Dropout, Embedding, LSTM, Bidirectional
 from keras.datasets import imdb
 
 
 max_features = 20000
-maxlen = 100  # cut texts after this number of words (among top max_features most common words)
+# cut texts after this number of words
+# (among top max_features most common words)
+maxlen = 100
 batch_size = 32
 
 print('Loading data...')
-(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features)
-print(len(X_train), 'train sequences')
-print(len(X_test), 'test sequences')
+(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
+print(len(x_train), 'train sequences')
+print(len(x_test), 'test sequences')
 
 print("Pad sequences (samples x time)")
-X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
-X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
-print('X_train shape:', X_train.shape)
-print('X_test shape:', X_test.shape)
+x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
+x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
+print('x_train shape:', x_train.shape)
+print('x_test shape:', x_test.shape)
 y_train = np.array(y_train)
 y_test = np.array(y_test)
 
@@ -41,7 +42,7 @@ model.add(Dense(1, activation='sigmoid'))
 model.compile('adam', 'binary_crossentropy', metrics=['accuracy'])
 
 print('Train...')
-model.fit(X_train, y_train,
+model.fit(x_train, y_train,
           batch_size=batch_size,
-          nb_epoch=4,
-          validation_data=[X_test, y_test])
+          epochs=4,
+          validation_data=[x_test, y_test])

examples/imdb_cnn.py

@@ -7,38 +7,34 @@ Gets to 0.89 test accuracy after 2 epochs.
 '''
 
 from __future__ import print_function
-import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 from keras.preprocessing import sequence
 from keras.models import Sequential
 from keras.layers import Dense, Dropout, Activation
 from keras.layers import Embedding
-from keras.layers import Convolution1D, GlobalMaxPooling1D
+from keras.layers import Conv1D, GlobalMaxPooling1D
 from keras.datasets import imdb
-from keras import backend as K
-
 
 # set parameters:
 max_features = 5000
 maxlen = 400
 batch_size = 32
 embedding_dims = 50
-nb_filter = 250
-filter_length = 3
+filters = 250
+kernel_size = 3
 hidden_dims = 250
-nb_epoch = 2
+epochs = 2
 
 print('Loading data...')
-(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features)
-print(len(X_train), 'train sequences')
-print(len(X_test), 'test sequences')
+(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
+print(len(x_train), 'train sequences')
+print(len(x_test), 'test sequences')
 
 print('Pad sequences (samples x time)')
-X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
-X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
-print('X_train shape:', X_train.shape)
-print('X_test shape:', X_test.shape)
+x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
+x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
+print('x_train shape:', x_train.shape)
+print('x_test shape:', x_test.shape)
 
 print('Build model...')
 model = Sequential()
@@ -47,16 +43,16 @@ model = Sequential()
 # our vocab indices into embedding_dims dimensions
 model.add(Embedding(max_features,
                     embedding_dims,
-                    input_length=maxlen,
-                    dropout=0.2))
+                    input_length=maxlen))
+model.add(Dropout(0.2))
 
-# we add a Convolution1D, which will learn nb_filter
+# we add a Convolution1D, which will learn filters
 # word group filters of size filter_length:
-model.add(Convolution1D(nb_filter=nb_filter,
-                        filter_length=filter_length,
-                        border_mode='valid',
-                        activation='relu',
-                        subsample_length=1))
+model.add(Conv1D(filters,
+                 kernel_size,
+                 padding='valid',
+                 activation='relu',
+                 strides=1))
 # we use max pooling:
 model.add(GlobalMaxPooling1D())
 
@@ -72,7 +68,7 @@ model.add(Activation('sigmoid'))
 model.compile(loss='binary_crossentropy',
               optimizer='adam',
               metrics=['accuracy'])
-model.fit(X_train, y_train,
+model.fit(x_train, y_train,
           batch_size=batch_size,
-          nb_epoch=nb_epoch,
-          validation_data=(X_test, y_test))
+          epochs=epochs,
+          validation_data=(x_test, y_test))

examples/imdb_cnn_lstm.py

@@ -4,34 +4,31 @@ classification task.
 Gets to 0.8498 test accuracy after 2 epochs. 41s/epoch on K520 GPU.
 '''
 from __future__ import print_function
-import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 from keras.preprocessing import sequence
 from keras.models import Sequential
 from keras.layers import Dense, Dropout, Activation
 from keras.layers import Embedding
 from keras.layers import LSTM
-from keras.layers import Convolution1D, MaxPooling1D
+from keras.layers import Conv1D, MaxPooling1D
 from keras.datasets import imdb
 
-
 # Embedding
 max_features = 20000
 maxlen = 100
 embedding_size = 128
 
 # Convolution
-filter_length = 5
-nb_filter = 64
-pool_length = 4
+kernel_size = 5
+filters = 64
+pool_size = 4
 
 # LSTM
 lstm_output_size = 70
 
 # Training
 batch_size = 30
-nb_epoch = 2
+epochs = 2
 
 '''
 Note:
@@ -40,27 +37,27 @@ Only 2 epochs are needed as the dataset is very small.
 '''
 
 print('Loading data...')
-(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features)
-print(len(X_train), 'train sequences')
-print(len(X_test), 'test sequences')
+(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
+print(len(x_train), 'train sequences')
+print(len(x_test), 'test sequences')
 
 print('Pad sequences (samples x time)')
-X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
-X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
-print('X_train shape:', X_train.shape)
-print('X_test shape:', X_test.shape)
+x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
+x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
+print('x_train shape:', x_train.shape)
+print('x_test shape:', x_test.shape)
 
 print('Build model...')
 
 model = Sequential()
 model.add(Embedding(max_features, embedding_size, input_length=maxlen))
 model.add(Dropout(0.25))
-model.add(Convolution1D(nb_filter=nb_filter,
-                        filter_length=filter_length,
-                        border_mode='valid',
-                        activation='relu',
-                        subsample_length=1))
-model.add(MaxPooling1D(pool_length=pool_length))
+model.add(Conv1D(filters,
+                 kernel_size,
+                 padding='valid',
+                 activation='relu',
+                 strides=1))
+model.add(MaxPooling1D(pool_size=pool_size))
 model.add(LSTM(lstm_output_size))
 model.add(Dense(1))
 model.add(Activation('sigmoid'))
@@ -70,8 +67,8 @@ model.compile(loss='binary_crossentropy',
               metrics=['accuracy'])
 
 print('Train...')
-model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=nb_epoch,
-          validation_data=(X_test, y_test))
-score, acc = model.evaluate(X_test, y_test, batch_size=batch_size)
+model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs,
+          validation_data=(x_test, y_test))
+score, acc = model.evaluate(x_test, y_test, batch_size=batch_size)
 print('Test score:', score)
 print('Test accuracy:', acc)

examples/imdb_fasttext.py

@@ -7,12 +7,11 @@ https://arxiv.org/abs/1607.01759
 
 Results on IMDB datasets with uni and bi-gram embeddings:
     Uni-gram: 0.8813 test accuracy after 5 epochs. 8s/epoch on i7 cpu.
-    Bi-gram : 0.9056 test accuracy after 5 epochs. 2s/epoch on GTX 980M gpu.
+    Bi-gram : 0.9056 test accuracy after 5 epochs. 2s/epoch on GTx 980M gpu.
 '''
 
 from __future__ import print_function
 import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 from keras.preprocessing import sequence
 from keras.models import Sequential
@@ -54,9 +53,9 @@ def add_ngram(sequences, token_indice, ngram_range=2):
     new_sequences = []
     for input_list in sequences:
         new_list = input_list[:]
-        for i in range(len(new_list)-ngram_range+1):
-            for ngram_value in range(2, ngram_range+1):
-                ngram = tuple(new_list[i:i+ngram_value])
+        for i in range(len(new_list) - ngram_range + 1):
+            for ngram_value in range(2, ngram_range + 1):
+                ngram = tuple(new_list[i:i + ngram_value])
                 if ngram in token_indice:
                     new_list.append(token_indice[ngram])
         new_sequences.append(new_list)
@@ -70,21 +69,21 @@ max_features = 20000
 maxlen = 400
 batch_size = 32
 embedding_dims = 50
-nb_epoch = 5
+epochs = 5
 
 print('Loading data...')
-(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features)
-print(len(X_train), 'train sequences')
-print(len(X_test), 'test sequences')
-print('Average train sequence length: {}'.format(np.mean(list(map(len, X_train)), dtype=int)))
-print('Average test sequence length: {}'.format(np.mean(list(map(len, X_test)), dtype=int)))
+(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
+print(len(x_train), 'train sequences')
+print(len(x_test), 'test sequences')
+print('Average train sequence length: {}'.format(np.mean(list(map(len, x_train)), dtype=int)))
+print('Average test sequence length: {}'.format(np.mean(list(map(len, x_test)), dtype=int)))
 
 if ngram_range > 1:
     print('Adding {}-gram features'.format(ngram_range))
     # Create set of unique n-gram from the training set.
     ngram_set = set()
-    for input_list in X_train:
-        for i in range(2, ngram_range+1):
+    for input_list in x_train:
+        for i in range(2, ngram_range + 1):
             set_of_ngram = create_ngram_set(input_list, ngram_value=i)
             ngram_set.update(set_of_ngram)
 
@@ -92,23 +91,23 @@ if ngram_range > 1:
     # Integer values are greater than max_features in order
     # to avoid collision with existing features.
     start_index = max_features + 1
-    token_indice = {v: k+start_index for k, v in enumerate(ngram_set)}
+    token_indice = {v: k + start_index for k, v in enumerate(ngram_set)}
     indice_token = {token_indice[k]: k for k in token_indice}
 
     # max_features is the highest integer that could be found in the dataset.
     max_features = np.max(list(indice_token.keys())) + 1
 
-    # Augmenting X_train and X_test with n-grams features
-    X_train = add_ngram(X_train, token_indice, ngram_range)
-    X_test = add_ngram(X_test, token_indice, ngram_range)
-    print('Average train sequence length: {}'.format(np.mean(list(map(len, X_train)), dtype=int)))
-    print('Average test sequence length: {}'.format(np.mean(list(map(len, X_test)), dtype=int)))
+    # Augmenting x_train and x_test with n-grams features
+    x_train = add_ngram(x_train, token_indice, ngram_range)
+    x_test = add_ngram(x_test, token_indice, ngram_range)
+    print('Average train sequence length: {}'.format(np.mean(list(map(len, x_train)), dtype=int)))
+    print('Average test sequence length: {}'.format(np.mean(list(map(len, x_test)), dtype=int)))
 
 print('Pad sequences (samples x time)')
-X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
-X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
-print('X_train shape:', X_train.shape)
-print('X_test shape:', X_test.shape)
+x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
+x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
+print('x_train shape:', x_train.shape)
+print('x_test shape:', x_test.shape)
 
 print('Build model...')
 model = Sequential()
@@ -130,7 +129,7 @@ model.compile(loss='binary_crossentropy',
               optimizer='adam',
               metrics=['accuracy'])
 
-model.fit(X_train, y_train,
+model.fit(x_train, y_train,
           batch_size=batch_size,
-          nb_epoch=nb_epoch,
-          validation_data=(X_test, y_test))
+          epochs=epochs,
+          validation_data=(x_test, y_test))

examples/imdb_lstm.py

@@ -11,14 +11,11 @@ Some configurations won't converge.
 from what you see with CNNs/MLPs/etc.
 '''
 from __future__ import print_function
-import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 from keras.preprocessing import sequence
-from keras.utils import np_utils
 from keras.models import Sequential
-from keras.layers import Dense, Dropout, Activation, Embedding
-from keras.layers import LSTM, SimpleRNN, GRU
+from keras.layers import Dense, Embedding
+from keras.layers import LSTM
 from keras.datasets import imdb
 
 max_features = 20000
@@ -26,22 +23,21 @@ maxlen = 80  # cut texts after this number of words (among top max_features most
 batch_size = 32
 
 print('Loading data...')
-(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features)
-print(len(X_train), 'train sequences')
-print(len(X_test), 'test sequences')
+(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
+print(len(x_train), 'train sequences')
+print(len(x_test), 'test sequences')
 
 print('Pad sequences (samples x time)')
-X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
-X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
-print('X_train shape:', X_train.shape)
-print('X_test shape:', X_test.shape)
+x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
+x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
+print('x_train shape:', x_train.shape)
+print('x_test shape:', x_test.shape)
 
 print('Build model...')
 model = Sequential()
-model.add(Embedding(max_features, 128, dropout=0.2))
-model.add(LSTM(128, dropout_W=0.2, dropout_U=0.2))  # try using a GRU instead, for fun
-model.add(Dense(1))
-model.add(Activation('sigmoid'))
+model.add(Embedding(max_features, 128))
+model.add(LSTM(128, dropout=0.2, recurrent_dropout=0.2))
+model.add(Dense(1, activation='sigmoid'))
 
 # try using different optimizers and different optimizer configs
 model.compile(loss='binary_crossentropy',
@@ -49,9 +45,9 @@ model.compile(loss='binary_crossentropy',
               metrics=['accuracy'])
 
 print('Train...')
-model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=15,
-          validation_data=(X_test, y_test))
-score, acc = model.evaluate(X_test, y_test,
+model.fit(x_train, y_train, batch_size=batch_size, epochs=15,
+          validation_data=(x_test, y_test))
+score, acc = model.evaluate(x_test, y_test,
                             batch_size=batch_size)
 print('Test score:', score)
 print('Test accuracy:', acc)

examples/lstm_benchmark.py

@@ -1,13 +1,13 @@
 '''Compare LSTM implementations on the IMDB sentiment classification task.
 
-consume_less='cpu' preprocesses input to the LSTM which typically results in
+implementation=0 preprocesses input to the LSTM which typically results in
 faster computations at the expense of increased peak memory usage as the
 preprocessed input must be kept in memory.
 
-consume_less='mem' does away with the preprocessing, meaning that it might take
+implementation=1 does away with the preprocessing, meaning that it might take
 a little longer, but should require less peak memory.
 
-consume_less='gpu' concatenates the input, output and forget gate's weights
+implementation=2 concatenates the input, output and forget gate's weights
 into one, large matrix, resulting in faster computation time as the GPU can
 utilize more cores, at the expense of reduced regularization because the same
 dropout is shared across the gates.
@@ -22,7 +22,7 @@ import matplotlib.pyplot as plt
 
 from keras.preprocessing import sequence
 from keras.models import Sequential
-from keras.layers import Embedding, Dense, LSTM
+from keras.layers import Embedding, Dense, LSTM, Dropout
 from keras.datasets import imdb
 
 max_features = 20000
@@ -30,21 +30,26 @@ max_length = 80
 embedding_dim = 256
 batch_size = 128
 epochs = 10
-modes = ['cpu', 'mem', 'gpu']
+modes = [0, 1, 2]
 
 print('Loading data...')
-(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features)
+(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=max_features)
 X_train = sequence.pad_sequences(X_train, max_length)
 X_test = sequence.pad_sequences(X_test, max_length)
 
 # Compile and train different models while meauring performance.
 results = []
 for mode in modes:
-    print('Testing mode: consume_less="{}"'.format(mode))
+    print('Testing mode: implementation={}'.format(mode))
 
     model = Sequential()
-    model.add(Embedding(max_features, embedding_dim, input_length=max_length, dropout=0.2))
-    model.add(LSTM(embedding_dim, dropout_W=0.2, dropout_U=0.2, consume_less=mode))
+    model.add(Embedding(max_features, embedding_dim,
+                        input_length=max_length))
+    model.add(Dropout(0.2))
+    model.add(LSTM(embedding_dim,
+                   dropout=0.2,
+                   recurrent_dropout=0.2,
+                   implementation=mode))
     model.add(Dense(1, activation='sigmoid'))
     model.compile(loss='binary_crossentropy',
                   optimizer='adam',
@@ -53,7 +58,7 @@ for mode in modes:
     start_time = time.time()
     history = model.fit(X_train, y_train,
                         batch_size=batch_size,
-                        nb_epoch=epochs,
+                        epochs=epochs,
                         validation_data=(X_test, y_test))
     average_time_per_epoch = (time.time() - start_time) / epochs
 

examples/lstm_text_generation.py

@@ -12,7 +12,7 @@ has at least ~100k characters. ~1M is better.
 
 from __future__ import print_function
 from keras.models import Sequential
-from keras.layers import Dense, Activation, Dropout
+from keras.layers import Dense, Activation
 from keras.layers import LSTM
 from keras.optimizers import RMSprop
 from keras.utils.data_utils import get_file
@@ -20,7 +20,7 @@ import numpy as np
 import random
 import sys
 
-path = get_file('nietzsche.txt', origin="https://s3.amazonaws.com/text-datasets/nietzsche.txt")
+path = get_file('nietzsche.txt', origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
 text = open(path).read().lower()
 print('corpus length:', len(text))
 
@@ -73,7 +73,7 @@ for iteration in range(1, 60):
     print()
     print('-' * 50)
     print('Iteration', iteration)
-    model.fit(X, y, batch_size=128, nb_epoch=1)
+    model.fit(X, y, batch_size=128, epochs=1)
 
     start_index = random.randint(0, len(text) - maxlen - 1)
 

examples/mnist_acgan.py

@@ -6,8 +6,8 @@ MNIST dataset. See https://arxiv.org/abs/1610.09585 for more details.
 
 You should start to see reasonable images after ~5 epochs, and good images
 by ~15 epochs. You should use a GPU, as the convolution-heavy operations are
-very slow on the CPU. Prefer the TensorFlow backend if you plan on iterating, as
-the compilation time can be a blocker using Theano.
+very slow on the CPU. Prefer the TensorFlow backend if you plan on iterating,
+as the compilation time can be a blocker using Theano.
 
 Timings:
 
@@ -33,9 +33,10 @@ from six.moves import range
 
 import keras.backend as K
 from keras.datasets import mnist
-from keras.layers import Input, Dense, Reshape, Flatten, Embedding, merge, Dropout
+from keras import layers
+from keras.layers import Input, Dense, Reshape, Flatten, Embedding, Dropout
 from keras.layers.advanced_activations import LeakyReLU
-from keras.layers.convolutional import UpSampling2D, Convolution2D
+from keras.layers.convolutional import UpSampling2D, Conv2D
 from keras.models import Sequential, Model
 from keras.optimizers import Adam
 from keras.utils.generic_utils import Progbar
@@ -43,7 +44,7 @@ import numpy as np
 
 np.random.seed(1337)
 
-K.set_image_dim_ordering('th')
+K.set_image_data_format('channels_first')
 
 
 def build_generator(latent_size):
@@ -57,17 +58,20 @@ def build_generator(latent_size):
 
     # upsample to (..., 14, 14)
     cnn.add(UpSampling2D(size=(2, 2)))
-    cnn.add(Convolution2D(256, 5, 5, border_mode='same',
-                          activation='relu', init='glorot_normal'))
+    cnn.add(Conv2D(256, 5, padding='same',
+                   activation='relu',
+                   kernel_initializer='glorot_normal'))
 
     # upsample to (..., 28, 28)
     cnn.add(UpSampling2D(size=(2, 2)))
-    cnn.add(Convolution2D(128, 5, 5, border_mode='same',
-                          activation='relu', init='glorot_normal'))
+    cnn.add(Conv2D(128, 5, padding='same',
+                   activation='relu',
+                   kernel_initializer='glorot_normal'))
 
     # take a channel axis reduction
-    cnn.add(Convolution2D(1, 2, 2, border_mode='same',
-                          activation='tanh', init='glorot_normal'))
+    cnn.add(Conv2D(1, 2, padding='same',
+                   activation='tanh',
+                   kernel_initializer='glorot_normal'))
 
     # this is the z space commonly refered to in GAN papers
     latent = Input(shape=(latent_size, ))
@@ -77,14 +81,14 @@ def build_generator(latent_size):
 
     # 10 classes in MNIST
     cls = Flatten()(Embedding(10, latent_size,
-                              init='glorot_normal')(image_class))
+                              embeddings_initializer='glorot_normal')(image_class))
 
     # hadamard product between z-space and a class conditional embedding
-    h = merge([latent, cls], mode='mul')
+    h = layers.multiply([latent, cls])
 
     fake_image = cnn(h)
 
-    return Model(input=[latent, image_class], output=fake_image)
+    return Model([latent, image_class], fake_image)
 
 
 def build_discriminator():
@@ -92,20 +96,20 @@ def build_discriminator():
     # the reference paper
     cnn = Sequential()
 
-    cnn.add(Convolution2D(32, 3, 3, border_mode='same', subsample=(2, 2),
-                          input_shape=(1, 28, 28)))
+    cnn.add(Conv2D(32, 3, padding='same', strides=2,
+                   input_shape=(1, 28, 28)))
     cnn.add(LeakyReLU())
     cnn.add(Dropout(0.3))
 
-    cnn.add(Convolution2D(64, 3, 3, border_mode='same', subsample=(1, 1)))
+    cnn.add(Conv2D(64, 3, padding='same', strides=2))
     cnn.add(LeakyReLU())
     cnn.add(Dropout(0.3))
 
-    cnn.add(Convolution2D(128, 3, 3, border_mode='same', subsample=(2, 2)))
+    cnn.add(Conv2D(128, 3, padding='same', strides=2))
     cnn.add(LeakyReLU())
     cnn.add(Dropout(0.3))
 
-    cnn.add(Convolution2D(256, 3, 3, border_mode='same', subsample=(1, 1)))
+    cnn.add(Conv2D(256, 3, padding='same', strides=1))
     cnn.add(LeakyReLU())
     cnn.add(Dropout(0.3))
 
@@ -122,12 +126,12 @@ def build_discriminator():
     fake = Dense(1, activation='sigmoid', name='generation')(features)
     aux = Dense(10, activation='softmax', name='auxiliary')(features)
 
-    return Model(input=image, output=[fake, aux])
+    return Model(image, [fake, aux])
 
 if __name__ == '__main__':
 
     # batch and latent size taken from the paper
-    nb_epochs = 50
+    epochs = 50
     batch_size = 100
     latent_size = 100
 
@@ -156,7 +160,7 @@ if __name__ == '__main__':
     # we only want to be able to train generation for the combined model
     discriminator.trainable = False
     fake, aux = discriminator(fake)
-    combined = Model(input=[latent, image_class], output=[fake, aux])
+    combined = Model([latent, image_class], [fake, aux])
 
     combined.compile(
         optimizer=Adam(lr=adam_lr, beta_1=adam_beta_1),
@@ -172,21 +176,21 @@ if __name__ == '__main__':
     X_test = (X_test.astype(np.float32) - 127.5) / 127.5
     X_test = np.expand_dims(X_test, axis=1)
 
-    nb_train, nb_test = X_train.shape[0], X_test.shape[0]
+    num_train, num_test = X_train.shape[0], X_test.shape[0]
 
     train_history = defaultdict(list)
     test_history = defaultdict(list)
 
-    for epoch in range(nb_epochs):
-        print('Epoch {} of {}'.format(epoch + 1, nb_epochs))
+    for epoch in range(epochs):
+        print('Epoch {} of {}'.format(epoch + 1, epochs))
 
-        nb_batches = int(X_train.shape[0] / batch_size)
-        progress_bar = Progbar(target=nb_batches)
+        num_batches = int(X_train.shape[0] / batch_size)
+        progress_bar = Progbar(target=num_batches)
 
         epoch_gen_loss = []
         epoch_disc_loss = []
 
-        for index in range(nb_batches):
+        for index in range(num_batches):
             progress_bar.update(index)
             # generate a new batch of noise
             noise = np.random.uniform(-1, 1, (batch_size, latent_size))
@@ -224,22 +228,23 @@ if __name__ == '__main__':
             trick = np.ones(2 * batch_size)
 
             epoch_gen_loss.append(combined.train_on_batch(
-                [noise, sampled_labels.reshape((-1, 1))], [trick, sampled_labels]))
+                [noise, sampled_labels.reshape((-1, 1))],
+                [trick, sampled_labels]))
 
         print('\nTesting for epoch {}:'.format(epoch + 1))
 
         # evaluate the testing loss here
 
         # generate a new batch of noise
-        noise = np.random.uniform(-1, 1, (nb_test, latent_size))
+        noise = np.random.uniform(-1, 1, (num_test, latent_size))
 
         # sample some labels from p_c and generate images from them
-        sampled_labels = np.random.randint(0, 10, nb_test)
+        sampled_labels = np.random.randint(0, 10, num_test)
         generated_images = generator.predict(
             [noise, sampled_labels.reshape((-1, 1))], verbose=False)
 
         X = np.concatenate((X_test, generated_images))
-        y = np.array([1] * nb_test + [0] * nb_test)
+        y = np.array([1] * num_test + [0] * num_test)
         aux_y = np.concatenate((y_test, sampled_labels), axis=0)
 
         # see if the discriminator can figure itself out...
@@ -249,10 +254,10 @@ if __name__ == '__main__':
         discriminator_train_loss = np.mean(np.array(epoch_disc_loss), axis=0)
 
         # make new noise
-        noise = np.random.uniform(-1, 1, (2 * nb_test, latent_size))
-        sampled_labels = np.random.randint(0, 10, 2 * nb_test)
+        noise = np.random.uniform(-1, 1, (2 * num_test, latent_size))
+        sampled_labels = np.random.randint(0, 10, 2 * num_test)
 
-        trick = np.ones(2 * nb_test)
+        trick = np.ones(2 * num_test)
 
         generator_test_loss = combined.evaluate(
             [noise, sampled_labels.reshape((-1, 1))],

examples/mnist_cnn.py

@@ -6,77 +6,62 @@ Gets to 99.25% test accuracy after 12 epochs
 '''
 
 from __future__ import print_function
-import numpy as np
-np.random.seed(1337)  # for reproducibility
-
+import keras
 from keras.datasets import mnist
 from keras.models import Sequential
-from keras.layers import Dense, Dropout, Activation, Flatten
-from keras.layers import Convolution2D, MaxPooling2D
-from keras.utils import np_utils
+from keras.layers import Dense, Dropout, Flatten
+from keras.layers import Conv2D, MaxPooling2D
 from keras import backend as K
 
 batch_size = 128
-nb_classes = 10
-nb_epoch = 12
+num_classes = 10
+epochs = 12
 
 # input image dimensions
 img_rows, img_cols = 28, 28
-# number of convolutional filters to use
-nb_filters = 32
-# size of pooling area for max pooling
-pool_size = (2, 2)
-# convolution kernel size
-kernel_size = (3, 3)
 
 # the data, shuffled and split between train and test sets
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
 
-if K.image_dim_ordering() == 'th':
-    X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
-    X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
+if K.image_data_format() == 'channels_first':
+    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
+    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
     input_shape = (1, img_rows, img_cols)
 else:
-    X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)
-    X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)
+    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
+    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
     input_shape = (img_rows, img_cols, 1)
 
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
-print('X_train shape:', X_train.shape)
-print(X_train.shape[0], 'train samples')
-print(X_test.shape[0], 'test samples')
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
+print('x_train shape:', x_train.shape)
+print(x_train.shape[0], 'train samples')
+print(x_test.shape[0], 'test samples')
 
 # convert class vectors to binary class matrices
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
 
 model = Sequential()
-
-model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1],
-                        border_mode='valid',
-                        input_shape=input_shape))
-model.add(Activation('relu'))
-model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1]))
-model.add(Activation('relu'))
-model.add(MaxPooling2D(pool_size=pool_size))
+model.add(Conv2D(32, kernel_size=(3, 3),
+                 activation='relu',
+                 input_shape=input_shape))
+model.add(Conv2D(64, (3, 3), activation='relu'))
+model.add(MaxPooling2D(pool_size=(2, 2)))
 model.add(Dropout(0.25))
-
 model.add(Flatten())
-model.add(Dense(128))
-model.add(Activation('relu'))
+model.add(Dense(128, activation='relu'))
 model.add(Dropout(0.5))
-model.add(Dense(nb_classes))
-model.add(Activation('softmax'))
+model.add(Dense(num_classes, activation='softmax'))
 
-model.compile(loss='categorical_crossentropy',
-              optimizer='adadelta',
+model.compile(loss=keras.losses.categorical_crossentropy,
+              optimizer=keras.optimizers.Adadelta(),
               metrics=['accuracy'])
 
-model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,
-          verbose=1, validation_data=(X_test, Y_test))
-score = model.evaluate(X_test, Y_test, verbose=0)
-print('Test score:', score[0])
+model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs,
+          verbose=1, validation_data=(x_test, y_test))
+score = model.evaluate(x_test, y_test, verbose=0)
+print('Test loss:', score[0])
 print('Test accuracy:', score[1])

examples/mnist_hierarchical_rnn.py

@@ -8,7 +8,7 @@ document vector is considered to preserve both the word-level and
 sentence-level structure of the context.
 
 # References
-    - [A Hierarchical Neural Autoencoder for Paragraphs and Documents](https://web.stanford.edu/~jurafsky/pubs/P15-1107.pdf)
+    - [A Hierarchical Neural Autoencoder for Paragraphs and Documents](https://arxiv.org/abs/1506.01057)
         Encodes paragraphs and documents with HRNN.
         Results have shown that HRNN outperforms standard
         RNNs and may play some role in more sophisticated generation tasks like
@@ -26,62 +26,63 @@ After 5 epochs: train acc: 0.9858, val acc: 0.9864
 """
 from __future__ import print_function
 
+import keras
 from keras.datasets import mnist
-from keras.models import Sequential, Model
+from keras.models import Model
 from keras.layers import Input, Dense, TimeDistributed
 from keras.layers import LSTM
-from keras.utils import np_utils
 
 # Training parameters.
 batch_size = 32
-nb_classes = 10
-nb_epochs = 5
+num_classes = 10
+epochs = 5
 
 # Embedding dimensions.
 row_hidden = 128
 col_hidden = 128
 
 # The data, shuffled and split between train and test sets.
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
 
 # Reshapes data to 4D for Hierarchical RNN.
-X_train = X_train.reshape(X_train.shape[0], 28, 28, 1)
-X_test = X_test.reshape(X_test.shape[0], 28, 28, 1)
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
-print('X_train shape:', X_train.shape)
-print(X_train.shape[0], 'train samples')
-print(X_test.shape[0], 'test samples')
+x_train = x_train.reshape(x_train.shape[0], 28, 28, 1)
+x_test = x_test.reshape(x_test.shape[0], 28, 28, 1)
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
+print('x_train shape:', x_train.shape)
+print(x_train.shape[0], 'train samples')
+print(x_test.shape[0], 'test samples')
 
 # Converts class vectors to binary class matrices.
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
 
-row, col, pixel = X_train.shape[1:]
+row, col, pixel = x_train.shape[1:]
 
 # 4D input.
 x = Input(shape=(row, col, pixel))
 
 # Encodes a row of pixels using TimeDistributed Wrapper.
-encoded_rows = TimeDistributed(LSTM(output_dim=row_hidden))(x)
+encoded_rows = TimeDistributed(LSTM(row_hidden))(x)
 
 # Encodes columns of encoded rows.
 encoded_columns = LSTM(col_hidden)(encoded_rows)
 
 # Final predictions and model.
-prediction = Dense(nb_classes, activation='softmax')(encoded_columns)
-model = Model(input=x, output=prediction)
+prediction = Dense(num_classes, activation='softmax')(encoded_columns)
+model = Model(x, prediction)
 model.compile(loss='categorical_crossentropy',
               optimizer='rmsprop',
               metrics=['accuracy'])
 
 # Training.
-model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epochs,
-          verbose=1, validation_data=(X_test, Y_test))
+model.fit(x_train, y_train,
+          batch_size=batch_size, epochs=epochs,
+          verbose=1, validation_data=(x_test, y_test))
 
 # Evaluation.
-scores = model.evaluate(X_test, Y_test, verbose=0)
+scores = model.evaluate(x_test, y_test, verbose=0)
 print('Test loss:', scores[0])
 print('Test accuracy:', scores[1])

examples/mnist_irnn.py

@@ -3,7 +3,7 @@ with pixel-by-pixel sequential MNIST in
 "A Simple Way to Initialize Recurrent Networks of Rectified Linear Units"
 by Quoc V. Le, Navdeep Jaitly, Geoffrey E. Hinton
 
-arXiv:1504.00941v2 [cs.NE] 7 Apr 2015
+arxiv:1504.00941v2 [cs.NE] 7 Apr 2015
 http://arxiv.org/pdf/1504.00941v2.pdf
 
 Optimizer is replaced with RMSprop which yields more stable and steady
@@ -15,56 +15,56 @@ Reaches 0.93 train/test accuracy after 900 epochs
 
 from __future__ import print_function
 
+import keras
 from keras.datasets import mnist
 from keras.models import Sequential
 from keras.layers import Dense, Activation
 from keras.layers import SimpleRNN
-from keras.initializations import normal, identity
+from keras import initializers
 from keras.optimizers import RMSprop
-from keras.utils import np_utils
 
 batch_size = 32
-nb_classes = 10
-nb_epochs = 200
+num_classes = 10
+epochs = 200
 hidden_units = 100
 
 learning_rate = 1e-6
 clip_norm = 1.0
 
 # the data, shuffled and split between train and test sets
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
 
-X_train = X_train.reshape(X_train.shape[0], -1, 1)
-X_test = X_test.reshape(X_test.shape[0], -1, 1)
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
-print('X_train shape:', X_train.shape)
-print(X_train.shape[0], 'train samples')
-print(X_test.shape[0], 'test samples')
+x_train = x_train.reshape(x_train.shape[0], -1, 1)
+x_test = x_test.reshape(x_test.shape[0], -1, 1)
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
+print('x_train shape:', x_train.shape)
+print(x_train.shape[0], 'train samples')
+print(x_test.shape[0], 'test samples')
 
 # convert class vectors to binary class matrices
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
 
 print('Evaluate IRNN...')
 model = Sequential()
-model.add(SimpleRNN(output_dim=hidden_units,
-                    init=lambda shape, name: normal(shape, scale=0.001, name=name),
-                    inner_init=lambda shape, name: identity(shape, scale=1.0, name=name),
+model.add(SimpleRNN(hidden_units,
+                    kernel_initializer=initializers.RandomNormal(stddev=0.001),
+                    recurrent_initializer=initializers.Identity(gain=1.0),
                     activation='relu',
-                    input_shape=X_train.shape[1:]))
-model.add(Dense(nb_classes))
+                    input_shape=x_train.shape[1:]))
+model.add(Dense(num_classes))
 model.add(Activation('softmax'))
 rmsprop = RMSprop(lr=learning_rate)
 model.compile(loss='categorical_crossentropy',
               optimizer=rmsprop,
               metrics=['accuracy'])
 
-model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epochs,
-          verbose=1, validation_data=(X_test, Y_test))
+model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs,
+          verbose=1, validation_data=(x_test, y_test))
 
-scores = model.evaluate(X_test, Y_test, verbose=0)
+scores = model.evaluate(x_test, y_test, verbose=0)
 print('IRNN test score:', scores[0])
 print('IRNN test accuracy:', scores[1])

examples/mnist_mlp.py

@@ -6,45 +6,40 @@ Gets to 98.40% test accuracy after 20 epochs
 '''
 
 from __future__ import print_function
-import numpy as np
-np.random.seed(1337)  # for reproducibility
 
+import keras
 from keras.datasets import mnist
 from keras.models import Sequential
-from keras.layers.core import Dense, Dropout, Activation
-from keras.optimizers import SGD, Adam, RMSprop
-from keras.utils import np_utils
+from keras.layers import Dense, Dropout
+from keras.optimizers import RMSprop
 
 
 batch_size = 128
-nb_classes = 10
-nb_epoch = 20
+num_classes = 10
+epochs = 20
 
 # the data, shuffled and split between train and test sets
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
 
-X_train = X_train.reshape(60000, 784)
-X_test = X_test.reshape(10000, 784)
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
-print(X_train.shape[0], 'train samples')
-print(X_test.shape[0], 'test samples')
+x_train = x_train.reshape(60000, 784)
+x_test = x_test.reshape(10000, 784)
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
+print(x_train.shape[0], 'train samples')
+print(x_test.shape[0], 'test samples')
 
 # convert class vectors to binary class matrices
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
 
 model = Sequential()
-model.add(Dense(512, input_shape=(784,)))
-model.add(Activation('relu'))
+model.add(Dense(512, activation='relu', input_shape=(784,)))
 model.add(Dropout(0.2))
-model.add(Dense(512))
-model.add(Activation('relu'))
+model.add(Dense(512, activation='relu'))
 model.add(Dropout(0.2))
-model.add(Dense(10))
-model.add(Activation('softmax'))
+model.add(Dense(10, activation='softmax'))
 
 model.summary()
 
@@ -52,9 +47,9 @@ model.compile(loss='categorical_crossentropy',
               optimizer=RMSprop(),
               metrics=['accuracy'])
 
-history = model.fit(X_train, Y_train,
-                    batch_size=batch_size, nb_epoch=nb_epoch,
-                    verbose=1, validation_data=(X_test, Y_test))
-score = model.evaluate(X_test, Y_test, verbose=0)
-print('Test score:', score[0])
+history = model.fit(x_train, y_train,
+                    batch_size=batch_size, epochs=epochs,
+                    verbose=1, validation_data=(x_test, y_test))
+score = model.evaluate(x_test, y_test, verbose=0)
+print('Test loss:', score[0])
 print('Test accuracy:', score[1])

examples/mnist_net2net.py

@@ -44,7 +44,7 @@ Experiments
     when a Dropout layer is used.
 
 Results
-- Tested with 'Theano' backend and 'th' image_dim_ordering.
+- Tested with 'Theano' backend and 'channels_first' image_data_format.
 - Running on GPU GeForce GTX 980M
 - Performance Comparisons - validation loss values during first 3 epochs:
 (1) teacher_model:             0.075    0.041    0.041
@@ -57,16 +57,17 @@ Results
 from __future__ import print_function
 from six.moves import xrange
 import numpy as np
-np.random.seed(1337)
-
+import keras
 from keras.models import Sequential
 from keras.layers import Conv2D, MaxPooling2D, Dense, Flatten
 from keras.optimizers import SGD
-from keras.utils import np_utils
 from keras.datasets import mnist
 
-input_shape = (1, 28, 28)  # image shape
-nb_class = 10  # number of class
+if keras.backend.image_data_format() == 'channels_first':
+    input_shape = (1, 28, 28)  # image shape
+else:
+    input_shape = (28, 28, 1)  # image shape
+num_class = 10  # number of class
 
 
 # load and pre-process data
@@ -75,7 +76,7 @@ def preprocess_input(x):
 
 
 def preprocess_output(y):
-    return np_utils.to_categorical(y)
+    return keras.utils.to_categorical(y)
 
 (train_x, train_y), (validation_x, validation_y) = mnist.load_data()
 train_x, validation_x = map(preprocess_input, [train_x, validation_x])
@@ -88,17 +89,17 @@ print('validation_x shape:', validation_x.shape,
 
 # knowledge transfer algorithms
 def wider2net_conv2d(teacher_w1, teacher_b1, teacher_w2, new_width, init):
-    '''Get initial weights for a wider conv2d layer with a bigger nb_filter,
+    '''Get initial weights for a wider conv2d layer with a bigger filters,
     by 'random-padding' or 'net2wider'.
 
     # Arguments
         teacher_w1: `weight` of conv2d layer to become wider,
-          of shape (nb_filter1, nb_channel1, kh1, kw1)
+          of shape (filters1, num_channel1, kh1, kw1)
         teacher_b1: `bias` of conv2d layer to become wider,
-          of shape (nb_filter1, )
+          of shape (filters1, )
         teacher_w2: `weight` of next connected conv2d layer,
-          of shape (nb_filter2, nb_channel2, kh2, kw2)
-        new_width: new `nb_filter` for the wider conv2d layer
+          of shape (filters2, num_channel2, kh2, kw2)
+        new_width: new `filters` for the wider conv2d layer
         init: initialization algorithm for new weights,
           either 'random-pad' or 'net2wider'
     '''
@@ -107,7 +108,7 @@ def wider2net_conv2d(teacher_w1, teacher_b1, teacher_w2, new_width, init):
     assert teacher_w1.shape[0] == teacher_b1.shape[0], (
         'weight and bias from same layer should have compatible shapes')
     assert new_width > teacher_w1.shape[0], (
-        'new width (nb_filter) should be bigger than the existing one')
+        'new width (filters) should be bigger than the existing one')
 
     n = new_width - teacher_w1.shape[0]
     if init == 'random-pad':
@@ -193,13 +194,13 @@ def deeper2net_conv2d(teacher_w):
 
     # Arguments
         teacher_w: `weight` of previous conv2d layer,
-          of shape (nb_filter, nb_channel, kh, kw)
+          of shape (filters, num_channel, kh, kw)
     '''
-    nb_filter, nb_channel, kh, kw = teacher_w.shape
-    student_w = np.zeros((nb_filter, nb_filter, kh, kw))
-    for i in xrange(nb_filter):
+    filters, num_channel, kh, kw = teacher_w.shape
+    student_w = np.zeros((filters, filters, kh, kw))
+    for i in xrange(filters):
         student_w[i, i, (kh - 1) / 2, (kw - 1) / 2] = 1.
-    student_b = np.zeros(nb_filter)
+    student_b = np.zeros(filters)
     return student_w, student_b
 
 
@@ -213,30 +214,30 @@ def copy_weights(teacher_model, student_model, layer_names):
 
 
 # methods to construct teacher_model and student_models
-def make_teacher_model(train_data, validation_data, nb_epoch=3):
+def make_teacher_model(train_data, validation_data, epochs=3):
     '''Train a simple CNN as teacher model.
     '''
     model = Sequential()
-    model.add(Conv2D(64, 3, 3, input_shape=input_shape,
-                     border_mode='same', name='conv1'))
-    model.add(MaxPooling2D(name='pool1'))
-    model.add(Conv2D(64, 3, 3, border_mode='same', name='conv2'))
-    model.add(MaxPooling2D(name='pool2'))
+    model.add(Conv2D(64, 3, input_shape=input_shape,
+                     padding='same', name='conv1'))
+    model.add(MaxPooling2D(2, name='pool1'))
+    model.add(Conv2D(64, 3, padding='same', name='conv2'))
+    model.add(MaxPooling2D(2, name='pool2'))
     model.add(Flatten(name='flatten'))
     model.add(Dense(64, activation='relu', name='fc1'))
-    model.add(Dense(nb_class, activation='softmax', name='fc2'))
+    model.add(Dense(num_class, activation='softmax', name='fc2'))
     model.compile(loss='categorical_crossentropy',
                   optimizer=SGD(lr=0.01, momentum=0.9),
                   metrics=['accuracy'])
 
     train_x, train_y = train_data
-    history = model.fit(train_x, train_y, nb_epoch=nb_epoch,
+    history = model.fit(train_x, train_y, epochs=epochs,
                         validation_data=validation_data)
     return model, history
 
 
 def make_wider_student_model(teacher_model, train_data,
-                             validation_data, init, nb_epoch=3):
+                             validation_data, init, epochs=3):
     '''Train a wider student model based on teacher_model,
        with either 'random-pad' (baseline) or 'net2wider'
     '''
@@ -245,15 +246,15 @@ def make_wider_student_model(teacher_model, train_data,
 
     model = Sequential()
     # a wider conv1 compared to teacher_model
-    model.add(Conv2D(new_conv1_width, 3, 3, input_shape=input_shape,
-                     border_mode='same', name='conv1'))
-    model.add(MaxPooling2D(name='pool1'))
-    model.add(Conv2D(64, 3, 3, border_mode='same', name='conv2'))
-    model.add(MaxPooling2D(name='pool2'))
+    model.add(Conv2D(new_conv1_width, 3, input_shape=input_shape,
+                     padding='same', name='conv1'))
+    model.add(MaxPooling2D(2, name='pool1'))
+    model.add(Conv2D(64, 3, padding='same', name='conv2'))
+    model.add(MaxPooling2D(2, name='pool2'))
     model.add(Flatten(name='flatten'))
     # a wider fc1 compared to teacher model
     model.add(Dense(new_fc1_width, activation='relu', name='fc1'))
-    model.add(Dense(nb_class, activation='softmax', name='fc2'))
+    model.add(Dense(num_class, activation='softmax', name='fc2'))
 
     # The weights for other layers need to be copied from teacher_model
     # to student_model, except for widened layers
@@ -279,32 +280,32 @@ def make_wider_student_model(teacher_model, train_data,
                   metrics=['accuracy'])
 
     train_x, train_y = train_data
-    history = model.fit(train_x, train_y, nb_epoch=nb_epoch,
+    history = model.fit(train_x, train_y, epochs=epochs,
                         validation_data=validation_data)
     return model, history
 
 
 def make_deeper_student_model(teacher_model, train_data,
-                              validation_data, init, nb_epoch=3):
+                              validation_data, init, epochs=3):
     '''Train a deeper student model based on teacher_model,
        with either 'random-init' (baseline) or 'net2deeper'
     '''
     model = Sequential()
-    model.add(Conv2D(64, 3, 3, input_shape=input_shape,
-                     border_mode='same', name='conv1'))
-    model.add(MaxPooling2D(name='pool1'))
-    model.add(Conv2D(64, 3, 3, border_mode='same', name='conv2'))
+    model.add(Conv2D(64, 3, input_shape=input_shape,
+                     padding='same', name='conv1'))
+    model.add(MaxPooling2D(2, name='pool1'))
+    model.add(Conv2D(64, 3, padding='same', name='conv2'))
     # add another conv2d layer to make original conv2 deeper
     if init == 'net2deeper':
         prev_w, _ = model.get_layer('conv2').get_weights()
         new_weights = deeper2net_conv2d(prev_w)
-        model.add(Conv2D(64, 3, 3, border_mode='same',
+        model.add(Conv2D(64, 3, padding='same',
                          name='conv2-deeper', weights=new_weights))
     elif init == 'random-init':
-        model.add(Conv2D(64, 3, 3, border_mode='same', name='conv2-deeper'))
+        model.add(Conv2D(64, 3, padding='same', name='conv2-deeper'))
     else:
         raise ValueError('Unsupported weight initializer: %s' % init)
-    model.add(MaxPooling2D(name='pool2'))
+    model.add(MaxPooling2D(2, name='pool2'))
     model.add(Flatten(name='flatten'))
     model.add(Dense(64, activation='relu', name='fc1'))
     # add another fc layer to make original fc1 deeper
@@ -316,7 +317,7 @@ def make_deeper_student_model(teacher_model, train_data,
         model.add(Dense(64, activation='relu', name='fc1-deeper'))
     else:
         raise ValueError('Unsupported weight initializer: %s' % init)
-    model.add(Dense(nb_class, activation='softmax', name='fc2'))
+    model.add(Dense(num_class, activation='softmax', name='fc2'))
 
     # copy weights for other layers
     copy_weights(teacher_model, model, layer_names=[
@@ -327,7 +328,7 @@ def make_deeper_student_model(teacher_model, train_data,
                   metrics=['accuracy'])
 
     train_x, train_y = train_data
-    history = model.fit(train_x, train_y, nb_epoch=nb_epoch,
+    history = model.fit(train_x, train_y, epochs=epochs,
                         validation_data=validation_data)
     return model, history
 
@@ -345,16 +346,16 @@ def net2wider_experiment():
     print('\nbuilding teacher model ...')
     teacher_model, _ = make_teacher_model(train_data,
                                           validation_data,
-                                          nb_epoch=3)
+                                          epochs=3)
 
     print('\nbuilding wider student model by random padding ...')
     make_wider_student_model(teacher_model, train_data,
                              validation_data, 'random-pad',
-                             nb_epoch=3)
+                             epochs=3)
     print('\nbuilding wider student model by net2wider ...')
     make_wider_student_model(teacher_model, train_data,
                              validation_data, 'net2wider',
-                             nb_epoch=3)
+                             epochs=3)
 
 
 def net2deeper_experiment():
@@ -369,16 +370,16 @@ def net2deeper_experiment():
     print('\nbuilding teacher model ...')
     teacher_model, _ = make_teacher_model(train_data,
                                           validation_data,
-                                          nb_epoch=3)
+                                          epochs=3)
 
     print('\nbuilding deeper student model by random init ...')
     make_deeper_student_model(teacher_model, train_data,
                               validation_data, 'random-init',
-                              nb_epoch=3)
+                              epochs=3)
     print('\nbuilding deeper student model by net2deeper ...')
     make_deeper_student_model(teacher_model, train_data,
                               validation_data, 'net2deeper',
-                              nb_epoch=3)
+                              epochs=3)
 
 # run the experiments
 net2wider_experiment()

examples/mnist_siamese_graph.py

@@ -13,13 +13,12 @@ Gets to 99.5% test accuracy after 20 epochs.
 from __future__ import absolute_import
 from __future__ import print_function
 import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 import random
 from keras.datasets import mnist
 from keras.models import Sequential, Model
 from keras.layers import Dense, Dropout, Input, Lambda
-from keras.optimizers import SGD, RMSprop
+from keras.optimizers import RMSprop
 from keras import backend as K
 
 
@@ -38,7 +37,8 @@ def contrastive_loss(y_true, y_pred):
     http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf
     '''
     margin = 1
-    return K.mean(y_true * K.square(y_pred) + (1 - y_true) * K.square(K.maximum(margin - y_pred, 0)))
+    return K.mean(y_true * K.square(y_pred) +
+                  (1 - y_true) * K.square(K.maximum(margin - y_pred, 0)))
 
 
 def create_pairs(x, digit_indices):
@@ -50,7 +50,7 @@ def create_pairs(x, digit_indices):
     n = min([len(digit_indices[d]) for d in range(10)]) - 1
     for d in range(10):
         for i in range(n):
-            z1, z2 = digit_indices[d][i], digit_indices[d][i+1]
+            z1, z2 = digit_indices[d][i], digit_indices[d][i + 1]
             pairs += [[x[z1], x[z2]]]
             inc = random.randrange(1, 10)
             dn = (d + inc) % 10
@@ -75,26 +75,26 @@ def create_base_network(input_dim):
 def compute_accuracy(predictions, labels):
     '''Compute classification accuracy with a fixed threshold on distances.
     '''
-    return np.mean(labels == (predictions.ravel() > 0.5))
+    return labels[predictions.ravel() < 0.5].mean()
 
 
 # the data, shuffled and split between train and test sets
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
-X_train = X_train.reshape(60000, 784)
-X_test = X_test.reshape(10000, 784)
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
+x_train = x_train.reshape(60000, 784)
+x_test = x_test.reshape(10000, 784)
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
 input_dim = 784
-nb_epoch = 20
+epochs = 20
 
 # create training+test positive and negative pairs
 digit_indices = [np.where(y_train == i)[0] for i in range(10)]
-tr_pairs, tr_y = create_pairs(X_train, digit_indices)
+tr_pairs, tr_y = create_pairs(x_train, digit_indices)
 
 digit_indices = [np.where(y_test == i)[0] for i in range(10)]
-te_pairs, te_y = create_pairs(X_test, digit_indices)
+te_pairs, te_y = create_pairs(x_test, digit_indices)
 
 # network definition
 base_network = create_base_network(input_dim)
@@ -108,9 +108,10 @@ input_b = Input(shape=(input_dim,))
 processed_a = base_network(input_a)
 processed_b = base_network(input_b)
 
-distance = Lambda(euclidean_distance, output_shape=eucl_dist_output_shape)([processed_a, processed_b])
+distance = Lambda(euclidean_distance,
+                  output_shape=eucl_dist_output_shape)([processed_a, processed_b])
 
-model = Model(input=[input_a, input_b], output=distance)
+model = Model([input_a, input_b], distance)
 
 # train
 rms = RMSprop()
@@ -118,7 +119,7 @@ model.compile(loss=contrastive_loss, optimizer=rms)
 model.fit([tr_pairs[:, 0], tr_pairs[:, 1]], tr_y,
           validation_data=([te_pairs[:, 0], te_pairs[:, 1]], te_y),
           batch_size=128,
-          nb_epoch=nb_epoch)
+          epochs=epochs)
 
 # compute final accuracy on training and test sets
 pred = model.predict([tr_pairs[:, 0], tr_pairs[:, 1]])

examples/mnist_sklearn_wrapper.py

@@ -4,54 +4,62 @@ Builds simple CNN models on MNIST and uses sklearn's GridSearchCV to find best m
 '''
 
 from __future__ import print_function
-import numpy as np
-np.random.seed(1337)  # for reproducibility
 
+import keras
 from keras.datasets import mnist
 from keras.models import Sequential
 from keras.layers import Dense, Dropout, Activation, Flatten
-from keras.layers import Convolution2D, MaxPooling2D
-from keras.utils import np_utils
+from keras.layers import Conv2D, MaxPooling2D
 from keras.wrappers.scikit_learn import KerasClassifier
+from keras import backend as K
 from sklearn.grid_search import GridSearchCV
 
 
-nb_classes = 10
+num_classes = 10
 
 # input image dimensions
 img_rows, img_cols = 28, 28
 
 # load training data and do basic data normalization
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
-X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
-X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
+
+if K.image_data_format() == 'channels_first':
+    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
+    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
+    input_shape = (1, img_rows, img_cols)
+else:
+    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
+    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
+    input_shape = (img_rows, img_cols, 1)
+
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
 
 # convert class vectors to binary class matrices
-y_train = np_utils.to_categorical(y_train, nb_classes)
-y_test = np_utils.to_categorical(y_test, nb_classes)
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
 
-def make_model(dense_layer_sizes, nb_filters, nb_conv, nb_pool):
+
+def make_model(dense_layer_sizes, filters, kernel_size, pool_size):
     '''Creates model comprised of 2 convolutional layers followed by dense layers
 
-    dense_layer_sizes: List of layer sizes. This list has one number for each layer
-    nb_filters: Number of convolutional filters in each convolutional layer
-    nb_conv: Convolutional kernel size
-    nb_pool: Size of pooling area for max pooling
+    dense_layer_sizes: List of layer sizes.
+        This list has one number for each layer
+    filters: Number of convolutional filters in each convolutional layer
+    kernel_size: Convolutional kernel size
+    pool_size: Size of pooling area for max pooling
     '''
 
     model = Sequential()
-
-    model.add(Convolution2D(nb_filters, nb_conv, nb_conv,
-                            border_mode='valid',
-                            input_shape=(1, img_rows, img_cols)))
+    model.add(Conv2D(filters, kernel_size,
+                     padding='valid',
+                     input_shape=input_shape))
     model.add(Activation('relu'))
-    model.add(Convolution2D(nb_filters, nb_conv, nb_conv))
+    model.add(Conv2D(filters, kernel_size))
     model.add(Activation('relu'))
-    model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))
+    model.add(MaxPooling2D(pool_size=pool_size))
     model.add(Dropout(0.25))
 
     model.add(Flatten())
@@ -59,7 +67,7 @@ def make_model(dense_layer_sizes, nb_filters, nb_conv, nb_pool):
         model.add(Dense(layer_size))
     model.add(Activation('relu'))
     model.add(Dropout(0.5))
-    model.add(Dense(nb_classes))
+    model.add(Dense(num_classes))
     model.add(Activation('softmax'))
 
     model.compile(loss='categorical_crossentropy',
@@ -72,15 +80,15 @@ dense_size_candidates = [[32], [64], [32, 32], [64, 64]]
 my_classifier = KerasClassifier(make_model, batch_size=32)
 validator = GridSearchCV(my_classifier,
                          param_grid={'dense_layer_sizes': dense_size_candidates,
-                                     # nb_epoch is avail for tuning even when not
+                                     # epochs is avail for tuning even when not
                                      # an argument to model building function
-                                     'nb_epoch': [3, 6],
-                                     'nb_filters': [8],
-                                     'nb_conv': [3],
-                                     'nb_pool': [2]},
-                         scoring='log_loss',
+                                     'epochs': [3, 6],
+                                     'filters': [8],
+                                     'kernel_size': [3],
+                                     'pool_size': [2]},
+                         scoring='neg_log_loss',
                          n_jobs=1)
-validator.fit(X_train, y_train)
+validator.fit(x_train, y_train)
 
 print('The parameters of the best model are: ')
 print(validator.best_params_)
@@ -89,6 +97,6 @@ print(validator.best_params_)
 # validator.best_estimator_.model returns the (unwrapped) keras model
 best_model = validator.best_estimator_.model
 metric_names = best_model.metrics_names
-metric_values = best_model.evaluate(X_test, y_test)
+metric_values = best_model.evaluate(x_test, y_test)
 for metric, value in zip(metric_names, metric_values):
     print(metric, ': ', value)

examples/mnist_swwae.py

@@ -2,21 +2,21 @@
 MNIST dataset.  It exemplifies two influential methods that have been developed
 in the past few years.
 
-The first is the idea of properly "unpooling." During any max pool, the
-exact location (the "where") of the maximal value in a pooled receptive field
+The first is the idea of properly 'unpooling.' During any max pool, the
+exact location (the 'where') of the maximal value in a pooled receptive field
 is lost, however it can be very useful in the overall reconstruction of an
-input image.  Therefore, if the "where" is handed from the encoder
-to the corresponding decoder layer, features being decoded can be "placed" in
+input image.  Therefore, if the 'where' is handed from the encoder
+to the corresponding decoder layer, features being decoded can be 'placed' in
 the right location, allowing for reconstructions of much higher fidelity.
 
 References:
 [1]
-"Visualizing and Understanding Convolutional Networks"
+'Visualizing and Understanding Convolutional Networks'
 Matthew D Zeiler, Rob Fergus
 https://arxiv.org/abs/1311.2901v3
 
 [2]
-"Stacked What-Where Auto-encoders"
+'Stacked What-Where Auto-encoders'
 Junbo Zhao, Michael Mathieu, Ross Goroshin, Yann LeCun
 https://arxiv.org/abs/1506.02351v8
 
@@ -34,72 +34,97 @@ applied as a bias because we know the MNIST digits are mapped to [0,1].
 
 References:
 [3]
-"Deep Residual Learning for Image Recognition"
-Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
+'Deep Residual Learning for Image Recognition'
+Kaiming He, xiangyu Zhang, Shaoqing Ren, Jian Sun
 https://arxiv.org/abs/1512.03385v1
 
 [4]
-"Identity Mappings in Deep Residual Networks"
-Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
+'Identity Mappings in Deep Residual Networks'
+Kaiming He, xiangyu Zhang, Shaoqing Ren, Jian Sun
 https://arxiv.org/abs/1603.05027v3
 
 '''
 from __future__ import print_function
 import numpy as np
-np.random.seed(1337)  # for reproducibility
 
 from keras.datasets import mnist
 from keras.models import Model
-from keras.layers import Activation, merge
-from keras.layers import UpSampling2D, Convolution2D, MaxPooling2D
-from keras.layers import Input, BatchNormalization
+from keras.layers import Activation
+from keras.layers import UpSampling2D, Conv2D, MaxPooling2D
+from keras.layers import Input, BatchNormalization, ELU
 import matplotlib.pyplot as plt
 import keras.backend as K
+from keras import layers
 
 
-def convresblock(x, nfeats=8, ksize=3, nskipped=2):
-    ''' The proposed residual block from [4]'''
-    y0 = Convolution2D(nfeats, ksize, ksize, border_mode='same')(x)
+def convresblock(x, nfeats=8, ksize=3, nskipped=2, elu=True):
+    """The proposed residual block from [4].
+    
+    Running with elu=True will use ELU nonlinearity and running with
+    elu=False will use BatchNorm + RELU nonlinearity.  While ELU's are fast
+    due to the fact they do not suffer from BatchNorm overhead, they may
+    overfit because they do not offer the stochastic element of the batch
+    formation process of BatchNorm, which acts as a good regularizer.
+
+    # Arguments
+        x: 4D tensor, the tensor to feed through the block
+        nfeats: Integer, number of feature maps for conv layers.
+        ksize: Integer, width and height of conv kernels in first convolution.
+        nskipped: Integer, number of conv layers for the residual function.
+        elu: Boolean, whether to use ELU or BN+RELU.
+
+    # Input shape
+        4D tensor with shape:
+        `(batch, channels, rows, cols)`
+
+    # Output shape
+        4D tensor with shape:
+        `(batch, filters, rows, cols)`
+    """                                            
+    y0 = Conv2D(nfeats, ksize, padding='same')(x)
     y = y0
     for i in range(nskipped):
-        y = BatchNormalization(mode=0, axis=1)(y)
-        y = Activation('relu')(y)
-        y = Convolution2D(nfeats, ksize, ksize, border_mode='same')(y)
-    return merge([y0, y], mode='sum')
+        if elu:
+            y = ELU()(y)
+        else:
+            y = BatchNormalization(axis=1)(y)
+            y = Activation('relu')(y)
+        y = Conv2D(nfeats, 1, padding='same')(y)
+    return layers.add([y0, y])
 
 
 def getwhere(x):
-    ''' Calculate the "where" mask that contains switches indicating which
+    ''' Calculate the 'where' mask that contains switches indicating which
     index contained the max value when MaxPool2D was applied.  Using the
     gradient of the sum is a nice trick to keep everything high level.'''
     y_prepool, y_postpool = x
     return K.gradients(K.sum(y_postpool), y_prepool)
 
 if K.backend() == 'tensorflow':
-    raise Exception('This example can only run with the '
-                    'Theano backend for the time being, '
-                    'because it requires taking the gradient '
-                    'of a gradient, which isn\'t '
-                    'supported for all TF ops.')
+    raise RuntimeError('This example can only run with the '
+                       'Theano backend for the time being, '
+                       'because it requires taking the gradient '
+                       'of a gradient, which isn\'t '
+                       'supported for all TF ops.')
 
-# This example assume 'th' dim ordering.
-K.set_image_dim_ordering('th')
+# This example assume 'channels_first' data format.
+K.set_image_data_format('channels_first')
 
 # input image dimensions
 img_rows, img_cols = 28, 28
 
 # the data, shuffled and split between train and test sets
-(X_train, _), (X_test, _) = mnist.load_data()
+(x_train, _), (x_test, _) = mnist.load_data()
 
-X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
-X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
-X_train = X_train.astype('float32')
-X_test = X_test.astype('float32')
-X_train /= 255
-X_test /= 255
-print('X_train shape:', X_train.shape)
-print(X_train.shape[0], 'train samples')
-print(X_test.shape[0], 'test samples')
+x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
+x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
+x_train = x_train.astype('float32')
+x_test = x_test.astype('float32')
+x_train /= 255
+x_test /= 255
+print('x_train shape:', x_train.shape)
+print(x_train.shape[0], 'train samples')
+print(x_test.shape[0], 'test samples')
 
 # The size of the kernel used for the MaxPooling2D
 pool_size = 2
@@ -110,47 +135,46 @@ pool_sizes = np.array([1, 1, 1, 1, 1]) * pool_size
 # The convolution kernel size
 ksize = 3
 # Number of epochs to train for
-nb_epoch = 5
+epochs = 5
 # Batch size during training
 batch_size = 128
 
 if pool_size == 2:
     # if using a 5 layer net of pool_size = 2
-    X_train = np.pad(X_train, [[0, 0], [0, 0], [2, 2], [2, 2]],
+    x_train = np.pad(x_train, [[0, 0], [0, 0], [2, 2], [2, 2]],
                      mode='constant')
-    X_test = np.pad(X_test, [[0, 0], [0, 0], [2, 2], [2, 2]], mode='constant')
+    x_test = np.pad(x_test, [[0, 0], [0, 0], [2, 2], [2, 2]], mode='constant')
     nlayers = 5
 elif pool_size == 3:
     # if using a 3 layer net of pool_size = 3
-    X_train = X_train[:, :, :-1, :-1]
-    X_test = X_test[:, :, :-1, :-1]
+    x_train = x_train[:, :, :-1, :-1]
+    x_test = x_test[:, :, :-1, :-1]
     nlayers = 3
 else:
     import sys
-    sys.exit("Script supports pool_size of 2 and 3.")
+    sys.exit('Script supports pool_size of 2 and 3.')
 
 # Shape of input to train on (note that model is fully convolutional however)
-input_shape = X_train.shape[1:]
+input_shape = x_train.shape[1:]
 # The final list of the size of axis=1 for all layers, including input
 nfeats_all = [input_shape[0]] + nfeats
 
-# First build the encoder, all the while keeping track of the "where" masks
+# First build the encoder, all the while keeping track of the 'where' masks
 img_input = Input(shape=input_shape)
 
-# We push the "where" masks to the following list
+# We push the 'where' masks to the following list
 wheres = [None] * nlayers
 y = img_input
 for i in range(nlayers):
     y_prepool = convresblock(y, nfeats=nfeats_all[i + 1], ksize=ksize)
     y = MaxPooling2D(pool_size=(pool_sizes[i], pool_sizes[i]))(y_prepool)
-    wheres[i] = merge([y_prepool, y], mode=getwhere,
-                      output_shape=lambda x: x[0])
+    wheres[i] = layers.Lambda(getwhere, output_shape=lambda x: x[0])([y_prepool, y])
 
-# Now build the decoder, and use the stored "where" masks to place the features
+# Now build the decoder, and use the stored 'where' masks to place the features
 for i in range(nlayers):
     ind = nlayers - 1 - i
     y = UpSampling2D(size=(pool_sizes[ind], pool_sizes[ind]))(y)
-    y = merge([y, wheres[ind]], mode='mul')
+    y = layers.multiply([y, wheres[ind]])
     y = convresblock(y, nfeats=nfeats_all[ind], ksize=ksize)
 
 # Use hard_simgoid to clip range of reconstruction
@@ -161,16 +185,16 @@ model = Model(img_input, y)
 model.compile('adam', 'mse')
 
 # Fit the model
-model.fit(X_train, X_train, validation_data=(X_test, X_test),
-          batch_size=batch_size, nb_epoch=nb_epoch)
+model.fit(x_train, x_train, validation_data=(x_test, x_test),
+          batch_size=batch_size, epochs=epochs)
 
 # Plot
-X_recon = model.predict(X_test[:25])
-X_plot = np.concatenate((X_test[:25], X_recon), axis=1)
-X_plot = X_plot.reshape((5, 10, input_shape[-2], input_shape[-1]))
-X_plot = np.vstack([np.hstack(x) for x in X_plot])
+x_recon = model.predict(x_test[:25])
+x_plot = np.concatenate((x_test[:25], x_recon), axis=1)
+x_plot = x_plot.reshape((5, 10, input_shape[-2], input_shape[-1]))
+x_plot = np.vstack([np.hstack(x) for x in x_plot])
 plt.figure()
 plt.axis('off')
 plt.title('Test Samples: Originals/Reconstructions')
-plt.imshow(X_plot, interpolation='none', cmap='gray')
+plt.imshow(x_plot, interpolation='none', cmap='gray')
 plt.savefig('reconstructions.png')

examples/mnist_transfer_cnn.py

@@ -12,100 +12,98 @@ and 99.2% for the last five digits after transfer + fine-tuning.
 '''
 
 from __future__ import print_function
-import numpy as np
+
 import datetime
-
-np.random.seed(1337)  # for reproducibility
-
+import keras
 from keras.datasets import mnist
 from keras.models import Sequential
 from keras.layers import Dense, Dropout, Activation, Flatten
-from keras.layers import Convolution2D, MaxPooling2D
-from keras.utils import np_utils
+from keras.layers import Conv2D, MaxPooling2D
 from keras import backend as K
 
 now = datetime.datetime.now
 
 batch_size = 128
-nb_classes = 5
-nb_epoch = 5
+num_classes = 5
+epochs = 5
 
 # input image dimensions
 img_rows, img_cols = 28, 28
 # number of convolutional filters to use
-nb_filters = 32
+filters = 32
 # size of pooling area for max pooling
 pool_size = 2
 # convolution kernel size
 kernel_size = 3
 
-if K.image_dim_ordering() == 'th':
+if K.image_data_format() == 'channels_first':
     input_shape = (1, img_rows, img_cols)
 else:
     input_shape = (img_rows, img_cols, 1)
 
 
-def train_model(model, train, test, nb_classes):
-    X_train = train[0].reshape((train[0].shape[0],) + input_shape)
-    X_test = test[0].reshape((test[0].shape[0],) + input_shape)
-    X_train = X_train.astype('float32')
-    X_test = X_test.astype('float32')
-    X_train /= 255
-    X_test /= 255
-    print('X_train shape:', X_train.shape)
-    print(X_train.shape[0], 'train samples')
-    print(X_test.shape[0], 'test samples')
+def train_model(model, train, test, num_classes):
+    x_train = train[0].reshape((train[0].shape[0],) + input_shape)
+    x_test = test[0].reshape((test[0].shape[0],) + input_shape)
+    x_train = x_train.astype('float32')
+    x_test = x_test.astype('float32')
+    x_train /= 255
+    x_test /= 255
+    print('x_train shape:', x_train.shape)
+    print(x_train.shape[0], 'train samples')
+    print(x_test.shape[0], 'test samples')
 
     # convert class vectors to binary class matrices
-    Y_train = np_utils.to_categorical(train[1], nb_classes)
-    Y_test = np_utils.to_categorical(test[1], nb_classes)
+    y_train = keras.utils.to_categorical(train[1], num_classes)
+    y_test = keras.utils.to_categorical(test[1], num_classes)
 
     model.compile(loss='categorical_crossentropy',
                   optimizer='adadelta',
                   metrics=['accuracy'])
 
     t = now()
-    model.fit(X_train, Y_train,
-              batch_size=batch_size, nb_epoch=nb_epoch,
+    model.fit(x_train, y_train,
+              batch_size=batch_size, epochs=epochs,
               verbose=1,
-              validation_data=(X_test, Y_test))
+              validation_data=(x_test, y_test))
     print('Training time: %s' % (now() - t))
-    score = model.evaluate(X_test, Y_test, verbose=0)
+    score = model.evaluate(x_test, y_test, verbose=0)
     print('Test score:', score[0])
     print('Test accuracy:', score[1])
 
 
 # the data, shuffled and split between train and test sets
-(X_train, y_train), (X_test, y_test) = mnist.load_data()
+(x_train, y_train), (x_test, y_test) = mnist.load_data()
 
 # create two datasets one with digits below 5 and one with 5 and above
-X_train_lt5 = X_train[y_train < 5]
+x_train_lt5 = x_train[y_train < 5]
 y_train_lt5 = y_train[y_train < 5]
-X_test_lt5 = X_test[y_test < 5]
+x_test_lt5 = x_test[y_test < 5]
 y_test_lt5 = y_test[y_test < 5]
 
-X_train_gte5 = X_train[y_train >= 5]
-y_train_gte5 = y_train[y_train >= 5] - 5  # make classes start at 0 for
-X_test_gte5 = X_test[y_test >= 5]         # np_utils.to_categorical
+x_train_gte5 = x_train[y_train >= 5]
+y_train_gte5 = y_train[y_train >= 5] - 5
+x_test_gte5 = x_test[y_test >= 5]
 y_test_gte5 = y_test[y_test >= 5] - 5
 
 # define two groups of layers: feature (convolutions) and classification (dense)
 feature_layers = [
-    Convolution2D(nb_filters, kernel_size, kernel_size,
-                  border_mode='valid',
-                  input_shape=input_shape),
+    Conv2D(filters, kernel_size,
+           padding='valid',
+           input_shape=input_shape),
     Activation('relu'),
-    Convolution2D(nb_filters, kernel_size, kernel_size),
+    Conv2D(filters, kernel_size),
     Activation('relu'),
-    MaxPooling2D(pool_size=(pool_size, pool_size)),
+    MaxPooling2D(pool_size=pool_size),
     Dropout(0.25),
     Flatten(),
 ]
+
 classification_layers = [
     Dense(128),
     Activation('relu'),
     Dropout(0.5),
-    Dense(nb_classes),
+    Dense(num_classes),
     Activation('softmax')
 ]
 
@@ -114,8 +112,8 @@ model = Sequential(feature_layers + classification_layers)
 
 # train model for 5-digit classification [0..4]
 train_model(model,
-            (X_train_lt5, y_train_lt5),
-            (X_test_lt5, y_test_lt5), nb_classes)
+            (x_train_lt5, y_train_lt5),
+            (x_test_lt5, y_test_lt5), num_classes)
 
 # freeze feature layers and rebuild model
 for l in feature_layers:
@@ -123,5 +121,5 @@ for l in feature_layers:
 
 # transfer: train dense layers for new classification task [5..9]
 train_model(model,
-            (X_train_gte5, y_train_gte5),
-            (X_test_gte5, y_test_gte5), nb_classes)
+            (x_train_gte5, y_train_gte5),
+            (x_test_gte5, y_test_gte5), num_classes)

examples/neural_doodle.py

@@ -47,7 +47,7 @@ from scipy.optimize import fmin_l_bfgs_b
 from scipy.misc import imread, imsave
 
 from keras import backend as K
-from keras.layers import Input, Convolution2D, MaxPooling2D, AveragePooling2D
+from keras.layers import Input, AveragePooling2D
 from keras.models import Model
 from keras.preprocessing.image import load_img, img_to_array
 from keras.applications import vgg19
@@ -77,8 +77,8 @@ content_img_path = args.content_image
 target_img_prefix = args.target_image_prefix
 use_content_img = content_img_path is not None
 
-nb_labels = args.nlabels
-nb_colors = 3  # RGB
+num_labels = args.nlabels
+num_colors = 3  # RGB
 # determine image sizes based on target_mask
 ref_img = imread(target_mask_path)
 img_nrows, img_ncols = ref_img.shape[:2]
@@ -103,7 +103,7 @@ def preprocess_image(image_path):
 
 
 def deprocess_image(x):
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         x = x.reshape((3, img_nrows, img_ncols))
         x = x.transpose((1, 2, 0))
     else:
@@ -122,7 +122,7 @@ def kmeans(xs, k):
     assert xs.ndim == 2
     try:
         from sklearn.cluster import k_means
-        _, labels, _ = k_means(xs.astype("float64"), k)
+        _, labels, _ = k_means(xs.astype('float64'), k)
     except ImportError:
         from scipy.cluster.vq import kmeans2
         _, labels = kmeans2(xs, k, missing='raise')
@@ -132,7 +132,7 @@ def kmeans(xs, k):
 def load_mask_labels():
     '''Load both target and style masks.
     A mask image (nr x nc) with m labels/colors will be loaded
-    as a 4D boolean tensor: (1, m, nr, nc) for 'th' or (1, nr, nc, m) for 'tf'
+    as a 4D boolean tensor: (1, m, nr, nc) for 'channels_first' or (1, nr, nc, m) for 'channels_last'
     '''
     target_mask_img = load_img(target_mask_path,
                                target_size=(img_nrows, img_ncols))
@@ -140,33 +140,33 @@ def load_mask_labels():
     style_mask_img = load_img(style_mask_path,
                               target_size=(img_nrows, img_ncols))
     style_mask_img = img_to_array(style_mask_img)
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         mask_vecs = np.vstack([style_mask_img.reshape((3, -1)).T,
                                target_mask_img.reshape((3, -1)).T])
     else:
         mask_vecs = np.vstack([style_mask_img.reshape((-1, 3)),
                                target_mask_img.reshape((-1, 3))])
 
-    labels = kmeans(mask_vecs, nb_labels)
+    labels = kmeans(mask_vecs, num_labels)
     style_mask_label = labels[:img_nrows *
                               img_ncols].reshape((img_nrows, img_ncols))
     target_mask_label = labels[img_nrows *
                                img_ncols:].reshape((img_nrows, img_ncols))
 
-    stack_axis = 0 if K.image_dim_ordering() == 'th' else -1
-    style_mask = np.stack([style_mask_label == r for r in xrange(nb_labels)],
+    stack_axis = 0 if K.image_data_format() == 'channels_first' else -1
+    style_mask = np.stack([style_mask_label == r for r in xrange(num_labels)],
                           axis=stack_axis)
-    target_mask = np.stack([target_mask_label == r for r in xrange(nb_labels)],
+    target_mask = np.stack([target_mask_label == r for r in xrange(num_labels)],
                            axis=stack_axis)
 
     return (np.expand_dims(style_mask, axis=0),
             np.expand_dims(target_mask, axis=0))
 
 # Create tensor variables for images
-if K.image_dim_ordering() == 'th':
-    shape = (1, nb_colors, img_nrows, img_ncols)
+if K.image_data_format() == 'channels_first':
+    shape = (1, num_colors, img_nrows, img_ncols)
 else:
-    shape = (1, img_nrows, img_ncols, nb_colors)
+    shape = (1, img_nrows, img_ncols, num_colors)
 
 style_image = K.variable(preprocess_image(style_img_path))
 target_image = K.placeholder(shape=shape)
@@ -179,8 +179,8 @@ images = K.concatenate([style_image, target_image, content_image], axis=0)
 
 # Create tensor variables for masks
 raw_style_mask, raw_target_mask = load_mask_labels()
-style_mask = K.variable(raw_style_mask.astype("float32"))
-target_mask = K.variable(raw_target_mask.astype("float32"))
+style_mask = K.variable(raw_style_mask.astype('float32'))
+target_mask = K.variable(raw_target_mask.astype('float32'))
 masks = K.concatenate([style_mask, target_mask], axis=0)
 
 # index constants for images and tasks variables
@@ -191,13 +191,13 @@ STYLE, TARGET, CONTENT = 0, 1, 2
 image_model = vgg19.VGG19(include_top=False, input_tensor=images)
 
 # mask model as a series of pooling
-mask_input = Input(tensor=masks, shape=(None, None, None), name="mask_input")
+mask_input = Input(tensor=masks, shape=(None, None, None), name='mask_input')
 x = mask_input
 for layer in image_model.layers[1:]:
     name = 'mask_%s' % layer.name
     if 'conv' in layer.name:
         x = AveragePooling2D((3, 3), strides=(
-            1, 1), name=name, border_mode="same")(x)
+            1, 1), name=name, border_mode='same')(x)
     elif 'pool' in layer.name:
         x = AveragePooling2D((2, 2), name=name)(x)
 mask_model = Model(mask_input, x)
@@ -228,18 +228,18 @@ def region_style_loss(style_image, target_image, style_mask, target_mask):
     '''
     assert 3 == K.ndim(style_image) == K.ndim(target_image)
     assert 2 == K.ndim(style_mask) == K.ndim(target_mask)
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         masked_style = style_image * style_mask
         masked_target = target_image * target_mask
-        nb_channels = K.shape(style_image)[0]
+        num_channels = K.shape(style_image)[0]
     else:
         masked_style = K.permute_dimensions(
             style_image, (2, 0, 1)) * style_mask
         masked_target = K.permute_dimensions(
             target_image, (2, 0, 1)) * target_mask
-        nb_channels = K.shape(style_image)[-1]
-    s = gram_matrix(masked_style) / K.mean(style_mask) / nb_channels
-    c = gram_matrix(masked_target) / K.mean(target_mask) / nb_channels
+        num_channels = K.shape(style_image)[-1]
+    s = gram_matrix(masked_style) / K.mean(style_mask) / num_channels
+    c = gram_matrix(masked_target) / K.mean(target_mask) / num_channels
     return K.mean(K.square(s - c))
 
 
@@ -250,8 +250,8 @@ def style_loss(style_image, target_image, style_masks, target_masks):
     assert 3 == K.ndim(style_image) == K.ndim(target_image)
     assert 3 == K.ndim(style_masks) == K.ndim(target_masks)
     loss = K.variable(0)
-    for i in xrange(nb_labels):
-        if K.image_dim_ordering() == 'th':
+    for i in xrange(num_labels):
+        if K.image_data_format() == 'channels_first':
             style_mask = style_masks[i, :, :]
             target_mask = target_masks[i, :, :]
         else:
@@ -268,7 +268,7 @@ def content_loss(content_image, target_image):
 
 def total_variation_loss(x):
     assert 4 == K.ndim(x)
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         a = K.square(x[:, :, :img_nrows - 1, :img_ncols - 1] -
                      x[:, :, 1:, :img_ncols - 1])
         b = K.square(x[:, :, :img_nrows - 1, :img_ncols - 1] -
@@ -301,7 +301,7 @@ loss_grads = K.gradients(loss, target_image)
 
 # Evaluator class for computing efficiency
 outputs = [loss]
-if type(loss_grads) in {list, tuple}:
+if isinstance(loss_grads, (list, tuple)):
     outputs += loss_grads
 else:
     outputs.append(loss_grads)
@@ -310,7 +310,7 @@ f_outputs = K.function([target_image], outputs)
 
 
 def eval_loss_and_grads(x):
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         x = x.reshape((1, 3, img_nrows, img_ncols))
     else:
         x = x.reshape((1, img_nrows, img_ncols, 3))
@@ -346,7 +346,7 @@ class Evaluator(object):
 evaluator = Evaluator()
 
 # Generate images by iterative optimization
-if K.image_dim_ordering() == 'th':
+if K.image_data_format() == 'channels_first':
     x = np.random.uniform(0, 255, (1, 3, img_nrows, img_ncols)) - 128.
 else:
     x = np.random.uniform(0, 255, (1, img_nrows, img_ncols, 3)) - 128.

examples/neural_style_transfer.py

@@ -88,11 +88,13 @@ style_weight = args.style_weight
 content_weight = args.content_weight
 
 # dimensions of the generated picture.
+width, height = load_img(base_image_path).size
 img_nrows = 400
-img_ncols = 400
-assert img_ncols == img_nrows, 'Due to the use of the Gram matrix, width and height must match.'
+img_ncols = int(width * img_nrows / height)
 
 # util function to open, resize and format pictures into appropriate tensors
+
+
 def preprocess_image(image_path):
     img = load_img(image_path, target_size=(img_nrows, img_ncols))
     img = img_to_array(img)
@@ -101,8 +103,10 @@ def preprocess_image(image_path):
     return img
 
 # util function to convert a tensor into a valid image
+
+
 def deprocess_image(x):
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         x = x.reshape((3, img_nrows, img_ncols))
         x = x.transpose((1, 2, 0))
     else:
@@ -121,7 +125,7 @@ base_image = K.variable(preprocess_image(base_image_path))
 style_reference_image = K.variable(preprocess_image(style_reference_image_path))
 
 # this will contain our generated image
-if K.image_dim_ordering() == 'th':
+if K.image_data_format() == 'channels_first':
     combination_image = K.placeholder((1, 3, img_nrows, img_ncols))
 else:
     combination_image = K.placeholder((1, img_nrows, img_ncols, 3))
@@ -144,9 +148,11 @@ outputs_dict = dict([(layer.name, layer.output) for layer in model.layers])
 # first we need to define 4 util functions
 
 # the gram matrix of an image tensor (feature-wise outer product)
+
+
 def gram_matrix(x):
     assert K.ndim(x) == 3
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         features = K.batch_flatten(x)
     else:
         features = K.batch_flatten(K.permute_dimensions(x, (2, 0, 1)))
@@ -158,6 +164,8 @@ def gram_matrix(x):
 # It is based on the gram matrices (which capture style) of
 # feature maps from the style reference image
 # and from the generated image
+
+
 def style_loss(style, combination):
     assert K.ndim(style) == 3
     assert K.ndim(combination) == 3
@@ -170,19 +178,23 @@ def style_loss(style, combination):
 # an auxiliary loss function
 # designed to maintain the "content" of the
 # base image in the generated image
+
+
 def content_loss(base, combination):
     return K.sum(K.square(combination - base))
 
 # the 3rd loss function, total variation loss,
 # designed to keep the generated image locally coherent
+
+
 def total_variation_loss(x):
     assert K.ndim(x) == 4
-    if K.image_dim_ordering() == 'th':
-        a = K.square(x[:, :, :img_nrows-1, :img_ncols-1] - x[:, :, 1:, :img_ncols-1])
-        b = K.square(x[:, :, :img_nrows-1, :img_ncols-1] - x[:, :, :img_nrows-1, 1:])
+    if K.image_data_format() == 'channels_first':
+        a = K.square(x[:, :, :img_nrows - 1, :img_ncols - 1] - x[:, :, 1:, :img_ncols - 1])
+        b = K.square(x[:, :, :img_nrows - 1, :img_ncols - 1] - x[:, :, :img_nrows - 1, 1:])
     else:
-        a = K.square(x[:, :img_nrows-1, :img_ncols-1, :] - x[:, 1:, :img_ncols-1, :])
-        b = K.square(x[:, :img_nrows-1, :img_ncols-1, :] - x[:, :img_nrows-1, 1:, :])
+        a = K.square(x[:, :img_nrows - 1, :img_ncols - 1, :] - x[:, 1:, :img_ncols - 1, :])
+        b = K.square(x[:, :img_nrows - 1, :img_ncols - 1, :] - x[:, :img_nrows - 1, 1:, :])
     return K.sum(K.pow(a + b, 1.25))
 
 # combine these loss functions into a single scalar
@@ -208,15 +220,16 @@ loss += total_variation_weight * total_variation_loss(combination_image)
 grads = K.gradients(loss, combination_image)
 
 outputs = [loss]
-if type(grads) in {list, tuple}:
+if isinstance(grads, (list, tuple)):
     outputs += grads
 else:
     outputs.append(grads)
 
 f_outputs = K.function([combination_image], outputs)
 
+
 def eval_loss_and_grads(x):
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         x = x.reshape((1, 3, img_nrows, img_ncols))
     else:
         x = x.reshape((1, img_nrows, img_ncols, 3))
@@ -234,7 +247,10 @@ def eval_loss_and_grads(x):
 # "loss" and "grads". This is done because scipy.optimize
 # requires separate functions for loss and gradients,
 # but computing them separately would be inefficient.
+
+
 class Evaluator(object):
+
     def __init__(self):
         self.loss_value = None
         self.grads_values = None
@@ -257,7 +273,7 @@ evaluator = Evaluator()
 
 # run scipy-based optimization (L-BFGS) over the pixels of the generated image
 # so as to minimize the neural style loss
-if K.image_dim_ordering() == 'th':
+if K.image_data_format() == 'channels_first':
     x = np.random.uniform(0, 255, (1, 3, img_nrows, img_ncols)) - 128.
 else:
     x = np.random.uniform(0, 255, (1, img_nrows, img_ncols, 3)) - 128.

examples/pretrained_word_embeddings.py

@@ -12,17 +12,17 @@ http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/data/news20.html
 '''
 
 from __future__ import print_function
-import os
-import numpy as np
-np.random.seed(1337)
 
+import os
+import sys
+import numpy as np
 from keras.preprocessing.text import Tokenizer
 from keras.preprocessing.sequence import pad_sequences
-from keras.utils.np_utils import to_categorical
+from keras.utils import to_categorical
 from keras.layers import Dense, Input, Flatten
 from keras.layers import Conv1D, MaxPooling1D, Embedding
 from keras.models import Model
-import sys
+
 
 BASE_DIR = ''
 GLOVE_DIR = BASE_DIR + '/glove.6B/'
@@ -66,14 +66,18 @@ for name in sorted(os.listdir(TEXT_DATA_DIR)):
                     f = open(fpath)
                 else:
                     f = open(fpath, encoding='latin-1')
-                texts.append(f.read())
+                t = f.read()
+                i = t.find('\n\n')  # skip header
+                if 0 < i:
+                    t = t[i:]
+                texts.append(t)
                 f.close()
                 labels.append(label_id)
 
 print('Found %s texts.' % len(texts))
 
 # finally, vectorize the text samples into a 2D integer tensor
-tokenizer = Tokenizer(nb_words=MAX_NB_WORDS)
+tokenizer = Tokenizer(num_words=MAX_NB_WORDS)
 tokenizer.fit_on_texts(texts)
 sequences = tokenizer.texts_to_sequences(texts)
 
@@ -91,20 +95,20 @@ indices = np.arange(data.shape[0])
 np.random.shuffle(indices)
 data = data[indices]
 labels = labels[indices]
-nb_validation_samples = int(VALIDATION_SPLIT * data.shape[0])
+num_validation_samples = int(VALIDATION_SPLIT * data.shape[0])
 
-x_train = data[:-nb_validation_samples]
-y_train = labels[:-nb_validation_samples]
-x_val = data[-nb_validation_samples:]
-y_val = labels[-nb_validation_samples:]
+x_train = data[:-num_validation_samples]
+y_train = labels[:-num_validation_samples]
+x_val = data[-num_validation_samples:]
+y_val = labels[-num_validation_samples:]
 
 print('Preparing embedding matrix.')
 
 # prepare embedding matrix
-nb_words = min(MAX_NB_WORDS, len(word_index))
-embedding_matrix = np.zeros((nb_words + 1, EMBEDDING_DIM))
+num_words = min(MAX_NB_WORDS, len(word_index))
+embedding_matrix = np.zeros((num_words, EMBEDDING_DIM))
 for word, i in word_index.items():
-    if i > MAX_NB_WORDS:
+    if i >= MAX_NB_WORDS:
         continue
     embedding_vector = embeddings_index.get(word)
     if embedding_vector is not None:
@@ -113,7 +117,7 @@ for word, i in word_index.items():
 
 # load pre-trained word embeddings into an Embedding layer
 # note that we set trainable = False so as to keep the embeddings fixed
-embedding_layer = Embedding(nb_words + 1,
+embedding_layer = Embedding(num_words,
                             EMBEDDING_DIM,
                             weights=[embedding_matrix],
                             input_length=MAX_SEQUENCE_LENGTH,
@@ -141,4 +145,4 @@ model.compile(loss='categorical_crossentropy',
 
 # happy learning!
 model.fit(x_train, y_train, validation_data=(x_val, y_val),
-          nb_epoch=2, batch_size=128)
+          epochs=10, batch_size=128)

examples/reuters_mlp.py

@@ -1,58 +1,58 @@
 '''Trains and evaluate a simple MLP
 on the Reuters newswire topic classification task.
 '''
-
 from __future__ import print_function
-import numpy as np
-np.random.seed(1337)  # for reproducibility
 
+import numpy as np
+import keras
 from keras.datasets import reuters
 from keras.models import Sequential
 from keras.layers import Dense, Dropout, Activation
-from keras.utils import np_utils
 from keras.preprocessing.text import Tokenizer
 
 max_words = 1000
 batch_size = 32
-nb_epoch = 5
+epochs = 5
 
 print('Loading data...')
-(X_train, y_train), (X_test, y_test) = reuters.load_data(nb_words=max_words, test_split=0.2)
-print(len(X_train), 'train sequences')
-print(len(X_test), 'test sequences')
+(x_train, y_train), (x_test, y_test) = reuters.load_data(num_words=max_words,
+                                                         test_split=0.2)
+print(len(x_train), 'train sequences')
+print(len(x_test), 'test sequences')
 
-nb_classes = np.max(y_train)+1
-print(nb_classes, 'classes')
+num_classes = np.max(y_train) + 1
+print(num_classes, 'classes')
 
 print('Vectorizing sequence data...')
-tokenizer = Tokenizer(nb_words=max_words)
-X_train = tokenizer.sequences_to_matrix(X_train, mode='binary')
-X_test = tokenizer.sequences_to_matrix(X_test, mode='binary')
-print('X_train shape:', X_train.shape)
-print('X_test shape:', X_test.shape)
+tokenizer = Tokenizer(num_words=max_words)
+x_train = tokenizer.sequences_to_matrix(x_train, mode='binary')
+x_test = tokenizer.sequences_to_matrix(x_test, mode='binary')
+print('x_train shape:', x_train.shape)
+print('x_test shape:', x_test.shape)
 
-print('Convert class vector to binary class matrix (for use with categorical_crossentropy)')
-Y_train = np_utils.to_categorical(y_train, nb_classes)
-Y_test = np_utils.to_categorical(y_test, nb_classes)
-print('Y_train shape:', Y_train.shape)
-print('Y_test shape:', Y_test.shape)
+print('Convert class vector to binary class matrix '
+      '(for use with categorical_crossentropy)')
+y_train = keras.utils.to_categorical(y_train, num_classes)
+y_test = keras.utils.to_categorical(y_test, num_classes)
+print('y_train shape:', y_train.shape)
+print('y_test shape:', y_test.shape)
 
 print('Building model...')
 model = Sequential()
 model.add(Dense(512, input_shape=(max_words,)))
 model.add(Activation('relu'))
 model.add(Dropout(0.5))
-model.add(Dense(nb_classes))
+model.add(Dense(num_classes))
 model.add(Activation('softmax'))
 
 model.compile(loss='categorical_crossentropy',
               optimizer='adam',
               metrics=['accuracy'])
 
-history = model.fit(X_train, Y_train,
-                    nb_epoch=nb_epoch, batch_size=batch_size,
+history = model.fit(x_train, y_train,
+                    epochs=epochs, batch_size=batch_size,
                     verbose=1, validation_split=0.1)
-score = model.evaluate(X_test, Y_test,
+score = model.evaluate(x_test, y_test,
                        batch_size=batch_size, verbose=1)
 print('Test score:', score[0])
 print('Test accuracy:', score[1])

examples/stateful_lstm.py

@@ -36,7 +36,7 @@ def gen_cosine_amp(amp=100, period=1000, x0=0, xn=50000, step=1, k=0.0001):
     return cos
 
 
-print('Generating Data')
+print('Generating Data...')
 cos = gen_cosine_amp()
 print('Input shape:', cos.shape)
 
@@ -44,13 +44,13 @@ expected_output = np.zeros((len(cos), 1))
 for i in range(len(cos) - lahead):
     expected_output[i, 0] = np.mean(cos[i + 1:i + lahead + 1])
 
-print('Output shape')
-print(expected_output.shape)
+print('Output shape:', expected_output.shape)
 
-print('Creating Model')
+print('Creating Model...')
 model = Sequential()
 model.add(LSTM(50,
-               batch_input_shape=(batch_size, tsteps, 1),
+               input_shape=(tsteps, 1),
+               batch_size=batch_size,
                return_sequences=True,
                stateful=True))
 model.add(LSTM(50,
@@ -66,7 +66,7 @@ for i in range(epochs):
               expected_output,
               batch_size=batch_size,
               verbose=1,
-              nb_epoch=1,
+              epochs=1,
               shuffle=False)
     model.reset_states()
 

examples/variational_autoencoder.py

@@ -9,14 +9,14 @@ from scipy.stats import norm
 from keras.layers import Input, Dense, Lambda
 from keras.models import Model
 from keras import backend as K
-from keras import objectives
+from keras import metrics
 from keras.datasets import mnist
 
 batch_size = 100
 original_dim = 784
 latent_dim = 2
 intermediate_dim = 256
-nb_epoch = 50
+epochs = 50
 epsilon_std = 1.0
 
 x = Input(batch_shape=(batch_size, original_dim))
@@ -28,7 +28,7 @@ z_log_var = Dense(latent_dim)(h)
 def sampling(args):
     z_mean, z_log_var = args
     epsilon = K.random_normal(shape=(batch_size, latent_dim), mean=0.,
-                              std=epsilon_std)
+                              stddev=epsilon_std)
     return z_mean + K.exp(z_log_var / 2) * epsilon
 
 # note that "output_shape" isn't necessary with the TensorFlow backend
@@ -42,7 +42,7 @@ x_decoded_mean = decoder_mean(h_decoded)
 
 
 def vae_loss(x, x_decoded_mean):
-    xent_loss = original_dim * objectives.binary_crossentropy(x, x_decoded_mean)
+    xent_loss = original_dim * metrics.binary_crossentropy(x, x_decoded_mean)
     kl_loss = - 0.5 * K.sum(1 + z_log_var - K.square(z_mean) - K.exp(z_log_var), axis=-1)
     return xent_loss + kl_loss
 
@@ -59,7 +59,7 @@ x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))
 
 vae.fit(x_train, x_train,
         shuffle=True,
-        nb_epoch=nb_epoch,
+        epochs=epochs,
         batch_size=batch_size,
         validation_data=(x_test, x_test))
 

examples/variational_autoencoder_deconv.py

@@ -8,40 +8,45 @@ import matplotlib.pyplot as plt
 from scipy.stats import norm
 
 from keras.layers import Input, Dense, Lambda, Flatten, Reshape
-from keras.layers import Convolution2D, Deconvolution2D
+from keras.layers import Conv2D, Conv2DTranspose
 from keras.models import Model
 from keras import backend as K
-from keras import objectives
+from keras import metrics
 from keras.datasets import mnist
 
 # input image dimensions
 img_rows, img_cols, img_chns = 28, 28, 1
 # number of convolutional filters to use
-nb_filters = 64
+filters = 64
 # convolution kernel size
-nb_conv = 3
+num_conv = 3
 
 batch_size = 100
-if K.image_dim_ordering() == 'th':
+if K.image_data_format() == 'channels_first':
     original_img_size = (img_chns, img_rows, img_cols)
 else:
     original_img_size = (img_rows, img_cols, img_chns)
 latent_dim = 2
 intermediate_dim = 128
 epsilon_std = 1.0
-nb_epoch = 5
+epochs = 5
 
 x = Input(batch_shape=(batch_size,) + original_img_size)
-conv_1 = Convolution2D(img_chns, 2, 2, border_mode='same', activation='relu')(x)
-conv_2 = Convolution2D(nb_filters, 2, 2,
-                       border_mode='same', activation='relu',
-                       subsample=(2, 2))(conv_1)
-conv_3 = Convolution2D(nb_filters, nb_conv, nb_conv,
-                       border_mode='same', activation='relu',
-                       subsample=(1, 1))(conv_2)
-conv_4 = Convolution2D(nb_filters, nb_conv, nb_conv,
-                       border_mode='same', activation='relu',
-                       subsample=(1, 1))(conv_3)
+conv_1 = Conv2D(img_chns,
+                kernel_size=(2, 2),
+                padding='same', activation='relu')(x)
+conv_2 = Conv2D(filters,
+                kernel_size=(2, 2),
+                padding='same', activation='relu',
+                strides=(2, 2))(conv_1)
+conv_3 = Conv2D(filters,
+                kernel_size=num_conv,
+                padding='same', activation='relu',
+                strides=1)(conv_2)
+conv_4 = Conv2D(filters,
+                kernel_size=num_conv,
+                padding='same', activation='relu',
+                strides=1)(conv_3)
 flat = Flatten()(conv_4)
 hidden = Dense(intermediate_dim, activation='relu')(flat)
 
@@ -52,7 +57,7 @@ z_log_var = Dense(latent_dim)(hidden)
 def sampling(args):
     z_mean, z_log_var = args
     epsilon = K.random_normal(shape=(batch_size, latent_dim),
-                              mean=0., std=epsilon_std)
+                              mean=0., stddev=epsilon_std)
     return z_mean + K.exp(z_log_var) * epsilon
 
 # note that "output_shape" isn't necessary with the TensorFlow backend
@@ -61,36 +66,36 @@ z = Lambda(sampling, output_shape=(latent_dim,))([z_mean, z_log_var])
 
 # we instantiate these layers separately so as to reuse them later
 decoder_hid = Dense(intermediate_dim, activation='relu')
-decoder_upsample = Dense(nb_filters * 14 * 14, activation='relu')
+decoder_upsample = Dense(filters * 14 * 14, activation='relu')
 
-if K.image_dim_ordering() == 'th':
-    output_shape = (batch_size, nb_filters, 14, 14)
+if K.image_data_format() == 'channels_first':
+    output_shape = (batch_size, filters, 14, 14)
 else:
-    output_shape = (batch_size, 14, 14, nb_filters)
+    output_shape = (batch_size, 14, 14, filters)
 
 decoder_reshape = Reshape(output_shape[1:])
-decoder_deconv_1 = Deconvolution2D(nb_filters, nb_conv, nb_conv,
-                                   output_shape,
-                                   border_mode='same',
-                                   subsample=(1, 1),
+decoder_deconv_1 = Conv2DTranspose(filters,
+                                   kernel_size=num_conv,
+                                   padding='same',
+                                   strides=1,
                                    activation='relu')
-decoder_deconv_2 = Deconvolution2D(nb_filters, nb_conv, nb_conv,
-                                   output_shape,
-                                   border_mode='same',
-                                   subsample=(1, 1),
+decoder_deconv_2 = Conv2DTranspose(filters, num_conv,
+                                   padding='same',
+                                   strides=1,
                                    activation='relu')
-if K.image_dim_ordering() == 'th':
-    output_shape = (batch_size, nb_filters, 29, 29)
+if K.image_data_format() == 'channels_first':
+    output_shape = (batch_size, filters, 29, 29)
 else:
-    output_shape = (batch_size, 29, 29, nb_filters)
-decoder_deconv_3_upsamp = Deconvolution2D(nb_filters, 2, 2,
-                                          output_shape,
-                                          border_mode='valid',
-                                          subsample=(2, 2),
+    output_shape = (batch_size, 29, 29, filters)
+decoder_deconv_3_upsamp = Conv2DTranspose(filters,
+                                          kernel_size=(3, 3),
+                                          strides=(2, 2),
+                                          padding='valid',
                                           activation='relu')
-decoder_mean_squash = Convolution2D(img_chns, 2, 2,
-                                    border_mode='valid',
-                                    activation='sigmoid')
+decoder_mean_squash = Conv2D(img_chns,
+                             kernel_size=2,
+                             padding='valid',
+                             activation='sigmoid')
 
 hid_decoded = decoder_hid(z)
 up_decoded = decoder_upsample(hid_decoded)
@@ -100,12 +105,13 @@ deconv_2_decoded = decoder_deconv_2(deconv_1_decoded)
 x_decoded_relu = decoder_deconv_3_upsamp(deconv_2_decoded)
 x_decoded_mean_squash = decoder_mean_squash(x_decoded_relu)
 
+
 def vae_loss(x, x_decoded_mean):
     # NOTE: binary_crossentropy expects a batch_size by dim
     # for x and x_decoded_mean, so we MUST flatten these!
     x = K.flatten(x)
     x_decoded_mean = K.flatten(x_decoded_mean)
-    xent_loss = img_rows * img_cols * objectives.binary_crossentropy(x, x_decoded_mean)
+    xent_loss = img_rows * img_cols * metrics.binary_crossentropy(x, x_decoded_mean)
     kl_loss = - 0.5 * K.mean(1 + z_log_var - K.square(z_mean) - K.exp(z_log_var), axis=-1)
     return xent_loss + kl_loss
 
@@ -125,7 +131,7 @@ print('x_train.shape:', x_train.shape)
 
 vae.fit(x_train, x_train,
         shuffle=True,
-        nb_epoch=nb_epoch,
+        epochs=epochs,
         batch_size=batch_size,
         validation_data=(x_test, x_test))
 

keras/__init__.py

@@ -1,4 +1,7 @@
 from __future__ import absolute_import
+
+from . import activations
+from . import applications
 from . import backend
 from . import datasets
 from . import engine
@@ -8,11 +11,11 @@ from . import utils
 from . import wrappers
 from . import callbacks
 from . import constraints
-from . import initializations
+from . import initializers
 from . import metrics
 from . import models
-from . import objectives
+from . import losses
 from . import optimizers
 from . import regularizers
 
-__version__ = '1.2.0'
+__version__ = '2.0.0'

keras/activations.py

@@ -1,6 +1,7 @@
 from __future__ import absolute_import
+import six
 from . import backend as K
-from .utils.generic_utils import get_from_module
+from .utils.generic_utils import deserialize_keras_object
 
 
 def softmax(x):
@@ -49,7 +50,25 @@ def linear(x):
     return x
 
 
+def serialize(activation):
+    return activation.__name__
+
+
+def deserialize(name, custom_objects=None):
+    return deserialize_keras_object(name,
+                                    module_objects=globals(),
+                                    custom_objects=custom_objects,
+                                    printable_module_name='activation function')
+
+
 def get(identifier):
     if identifier is None:
         return linear
-    return get_from_module(identifier, globals(), 'activation function')
+    if isinstance(identifier, six.string_types):
+        identifier = str(identifier)
+        return deserialize(identifier)
+    elif callable(identifier):
+        return identifier
+    else:
+        raise ValueError('Could not interpret '
+                         'activation function identifier:', identifier)

keras/applications/audio_conv_utils.py

@@ -1,86 +0,0 @@
-import numpy as np
-from .. import backend as K
-
-
-TAGS = ['rock', 'pop', 'alternative', 'indie', 'electronic',
-        'female vocalists', 'dance', '00s', 'alternative rock', 'jazz',
-        'beautiful', 'metal', 'chillout', 'male vocalists',
-        'classic rock', 'soul', 'indie rock', 'Mellow', 'electronica',
-        '80s', 'folk', '90s', 'chill', 'instrumental', 'punk',
-        'oldies', 'blues', 'hard rock', 'ambient', 'acoustic',
-        'experimental', 'female vocalist', 'guitar', 'Hip-Hop',
-        '70s', 'party', 'country', 'easy listening',
-        'sexy', 'catchy', 'funk', 'electro', 'heavy metal',
-        'Progressive rock', '60s', 'rnb', 'indie pop',
-        'sad', 'House', 'happy']
-
-
-def librosa_exists():
-    try:
-        __import__('librosa')
-    except ImportError:
-        return False
-    else:
-        return True
-
-
-def preprocess_input(audio_path, dim_ordering='default'):
-    '''Reads an audio file and outputs a Mel-spectrogram.
-    '''
-    if dim_ordering == 'default':
-        dim_ordering = K.image_dim_ordering()
-    assert dim_ordering in {'tf', 'th'}
-
-    if librosa_exists():
-        import librosa
-    else:
-        raise RuntimeError('Librosa is required to process audio files.\n' +
-                           'Install it via `pip install librosa` \nor visit ' +
-                           'http://librosa.github.io/librosa/ for details.')
-
-    # mel-spectrogram parameters
-    SR = 12000
-    N_FFT = 512
-    N_MELS = 96
-    HOP_LEN = 256
-    DURA = 29.12
-
-    src, sr = librosa.load(audio_path, sr=SR)
-    n_sample = src.shape[0]
-    n_sample_wanted = int(DURA * SR)
-
-    # trim the signal at the center
-    if n_sample < n_sample_wanted:  # if too short
-        src = np.hstack((src, np.zeros((int(DURA * SR) - n_sample,))))
-    elif n_sample > n_sample_wanted:  # if too long
-        src = src[(n_sample - n_sample_wanted) / 2:
-                  (n_sample + n_sample_wanted) / 2]
-
-    logam = librosa.logamplitude
-    melgram = librosa.feature.melspectrogram
-    x = logam(melgram(y=src, sr=SR, hop_length=HOP_LEN,
-                      n_fft=N_FFT, n_mels=N_MELS) ** 2,
-              ref_power=1.0)
-
-    if dim_ordering == 'th':
-        x = np.expand_dims(x, axis=0)
-    elif dim_ordering == 'tf':
-        x = np.expand_dims(x, axis=3)
-    return x
-
-
-def decode_predictions(preds, top_n=5):
-    '''Decode the output of a music tagger model.
-
-    # Arguments
-        preds: 2-dimensional numpy array
-        top_n: integer in [0, 50], number of items to show
-
-    '''
-    assert len(preds.shape) == 2 and preds.shape[1] == 50
-    results = []
-    for pred in preds:
-        result = zip(TAGS, pred)
-        result = sorted(result, key=lambda x: x[1], reverse=True)
-        results.append(result[:top_n])
-    return results

keras/applications/imagenet_utils.py

@@ -1,4 +1,3 @@
-import numpy as np
 import json
 
 from ..utils.data_utils import get_file
@@ -8,12 +7,21 @@ CLASS_INDEX = None
 CLASS_INDEX_PATH = 'https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json'
 
 
-def preprocess_input(x, dim_ordering='default'):
-    if dim_ordering == 'default':
-        dim_ordering = K.image_dim_ordering()
-    assert dim_ordering in {'tf', 'th'}
+def preprocess_input(x, data_format=None):
+    """Preprocesses a tensor encoding a batch of images.
 
-    if dim_ordering == 'th':
+    # Arguments
+        x: input Numpy tensor, 4D.
+        data_format: data format of the image tensor.
+
+    # Returns
+        Preprocessed tensor.
+    """
+    if data_format is None:
+        data_format = K.image_data_format()
+    assert data_format in {'channels_last', 'channels_first'}
+
+    if data_format == 'channels_first':
         # 'RGB'->'BGR'
         x = x[:, ::-1, :, :]
         # Zero-center by mean pixel
@@ -31,6 +39,21 @@ def preprocess_input(x, dim_ordering='default'):
 
 
 def decode_predictions(preds, top=5):
+    """Decodes the prediction of an ImageNet model.
+
+    # Arguments
+        preds: Numpy tensor encoding a batch of predictions.
+        top: integer, how many top-guesses to return.
+
+    # Returns
+        A list of lists of top class prediction tuples
+        `(class_name, class_description, score)`.
+        One list of tuples per sample in batch input.
+
+    # Raises
+        ValueError: in case of invalid shape of the `pred` array
+            (must be 2D).
+    """
     global CLASS_INDEX
     if len(preds.shape) != 2 or preds.shape[1] != 1000:
         raise ValueError('`decode_predictions` expects '
@@ -51,8 +74,29 @@ def decode_predictions(preds, top=5):
     return results
 
 
-def _obtain_input_shape(input_shape, default_size, min_size, dim_ordering, include_top):
-    if dim_ordering == 'th':
+def _obtain_input_shape(input_shape,
+                        default_size,
+                        min_size,
+                        data_format,
+                        include_top):
+    """Internal utility to compute/validate an ImageNet model's input shape.
+
+    # Arguments
+        input_shape: either None (will return the default network input shape),
+            or a user-provided shape to be validated.
+        default_size: default input width/height for the model.
+        min_size: minimum input width/height accepted by the model.
+        data_format: image data format to use.
+        include_top: whether the model is expected to
+            be linked to a classifier via a Flatten layer.
+
+    # Returns
+        An integer shape tuple (may include None entries).
+
+    # Raises
+        ValueError: in case of invalid argument values.
+    """
+    if data_format == 'channels_first':
         default_shape = (3, default_size, default_size)
     else:
         default_shape = (default_size, default_size, 3)
@@ -63,7 +107,7 @@ def _obtain_input_shape(input_shape, default_size, min_size, dim_ordering, inclu
                                  '`input_shape` should be ' + str(default_shape) + '.')
         input_shape = default_shape
     else:
-        if dim_ordering == 'th':
+        if data_format == 'channels_first':
             if input_shape is not None:
                 if len(input_shape) != 3:
                     raise ValueError('`input_shape` must be a tuple of three integers.')

keras/applications/inception_v3.py

@@ -1,79 +1,104 @@
 # -*- coding: utf-8 -*-
-'''Inception V3 model for Keras.
+"""Inception V3 model for Keras.
 
-Note that the ImageNet weights provided are from a model that had not fully converged.
-Inception v3 should be able to reach 6.9% top-5 error, but our model
-only gets to 7.8% (same as a fully-converged ResNet 50).
-For comparison, VGG16 only gets to 9.9%, quite a bit worse.
+Note that the input image format for this model is different than for
+the VGG16 and ResNet models (299x299 instead of 224x224),
+and that the input preprocessing function is also different (same as Xception).
 
-Also, do note that the input image format for this model is different than for
-the VGG16 and ResNet models (299x299 instead of 224x224), and that the input preprocessing function
-is also different (same as Xception).
-
-# Reference:
+# Reference
 
 - [Rethinking the Inception Architecture for Computer Vision](http://arxiv.org/abs/1512.00567)
 
-'''
+"""
 from __future__ import print_function
 from __future__ import absolute_import
 
 import warnings
 
 from ..models import Model
-from ..layers import Flatten, Dense, Input, BatchNormalization, merge
-from ..layers import Convolution2D, MaxPooling2D, AveragePooling2D
+from .. import layers
+from ..layers import Activation
+from ..layers import Dense
+from ..layers import Input
+from ..layers import BatchNormalization
+from ..layers import Conv2D
+from ..layers import MaxPooling2D
+from ..layers import AveragePooling2D
+from ..layers import GlobalAveragePooling2D
+from ..layers import GlobalMaxPooling2D
 from ..engine.topology import get_source_inputs
 from ..utils.layer_utils import convert_all_kernels_in_model
 from ..utils.data_utils import get_file
 from .. import backend as K
-from .imagenet_utils import decode_predictions, _obtain_input_shape
+from .imagenet_utils import decode_predictions
+from .imagenet_utils import _obtain_input_shape
 
 
-TH_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/inception_v3_weights_th_dim_ordering_th_kernels.h5'
-TF_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/inception_v3_weights_tf_dim_ordering_tf_kernels.h5'
-TH_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/inception_v3_weights_th_dim_ordering_th_kernels_notop.h5'
-TF_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5'
+WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.5/inception_v3_weights_tf_dim_ordering_tf_kernels.h5'
+WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.5/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5'
 
 
-def conv2d_bn(x, nb_filter, nb_row, nb_col,
-              border_mode='same', subsample=(1, 1),
+def conv2d_bn(x,
+              filters,
+              num_row,
+              num_col,
+              padding='same',
+              strides=(1, 1),
               name=None):
-    '''Utility function to apply conv + BN.
-    '''
+    """Utility function to apply conv + BN.
+
+    # Arguments
+        x: input tensor.
+        filters: filters in `Conv2D`.
+        num_row: height of the convolution kernel.
+        num_col: width of the convolution kernel.
+        padding: padding mode in `Conv2D`.
+        strides: strides in `Conv2D`.
+        name: name of the ops; will become `name + '_conv'`
+            for the convolution and `name + '_bn'` for the
+            batch norm layer.
+
+    # Returns
+        Output tensor after applying `Conv2D` and `BatchNormalization`.
+    """
     if name is not None:
         bn_name = name + '_bn'
         conv_name = name + '_conv'
     else:
         bn_name = None
         conv_name = None
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         bn_axis = 1
     else:
         bn_axis = 3
-    x = Convolution2D(nb_filter, nb_row, nb_col,
-                      subsample=subsample,
-                      activation='relu',
-                      border_mode=border_mode,
-                      name=conv_name)(x)
-    x = BatchNormalization(axis=bn_axis, name=bn_name)(x)
+    x = Conv2D(
+        filters, (num_row, num_col),
+        strides=strides,
+        padding=padding,
+        use_bias=False,
+        name=conv_name)(x)
+    x = BatchNormalization(axis=bn_axis, scale=False, name=bn_name)(x)
+    x = Activation('relu', name=name)(x)
     return x
 
 
-def InceptionV3(include_top=True, weights='imagenet',
-                input_tensor=None, input_shape=None):
-    '''Instantiate the Inception v3 architecture,
-    optionally loading weights pre-trained
+def InceptionV3(include_top=True,
+                weights='imagenet',
+                input_tensor=None,
+                input_shape=None,
+                pooling=None,
+                classes=1000):
+    """Instantiates the Inception v3 architecture.
+
+    Optionally loads weights pre-trained
     on ImageNet. Note that when using TensorFlow,
     for best performance you should set
-    `image_dim_ordering="tf"` in your Keras config
+    `image_data_format="channels_last"` in your Keras config
     at ~/.keras/keras.json.
-
     The model and the weights are compatible with both
-    TensorFlow and Theano. The dimension ordering
+    TensorFlow and Theano. The data format
     convention used by the model is the one
     specified in your Keras config file.
-
     Note that the default input image size for this model is 299x299.
 
     # Arguments
@@ -83,80 +108,133 @@ def InceptionV3(include_top=True, weights='imagenet',
             or "imagenet" (pre-training on ImageNet).
         input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)
             to use as image input for the model.
-        inputs_shape: optional shape tuple, only to be specified
+        input_shape: optional shape tuple, only to be specified
             if `include_top` is False (otherwise the input shape
-            has to be `(299, 299, 3)` (with `tf` dim ordering)
-            or `(3, 299, 299)` (with `th` dim ordering).
+            has to be `(299, 299, 3)` (with `channels_last` data format)
+            or `(3, 299, 299)` (with `channels_first` data format).
             It should have exactly 3 inputs channels,
             and width and height should be no smaller than 139.
             E.g. `(150, 150, 3)` would be one valid value.
+        pooling: Optional pooling mode for feature extraction
+            when `include_top` is `False`.
+            - `None` means that the output of the model will be
+                the 4D tensor output of the
+                last convolutional layer.
+            - `avg` means that global average pooling
+                will be applied to the output of the
+                last convolutional layer, and thus
+                the output of the model will be a 2D tensor.
+            - `max` means that global max pooling will
+                be applied.
+        classes: optional number of classes to classify images
+            into, only to be specified if `include_top` is True, and
+            if no `weights` argument is specified.
 
     # Returns
         A Keras model instance.
-    '''
+
+    # Raises
+        ValueError: in case of invalid argument for `weights`,
+            or invalid input shape.
+    """
     if weights not in {'imagenet', None}:
         raise ValueError('The `weights` argument should be either '
                          '`None` (random initialization) or `imagenet` '
                          '(pre-training on ImageNet).')
+
+    if weights == 'imagenet' and include_top and classes != 1000:
+        raise ValueError('If using `weights` as imagenet with `include_top`'
+                         ' as true, `classes` should be 1000')
+
     # Determine proper input shape
-    input_shape = _obtain_input_shape(input_shape,
-                                      default_size=299,
-                                      min_size=139,
-                                      dim_ordering=K.image_dim_ordering(),
-                                      include_top=include_top)
+    input_shape = _obtain_input_shape(
+        input_shape,
+        default_size=299,
+        min_size=139,
+        data_format=K.image_data_format(),
+        include_top=include_top)
 
     if input_tensor is None:
         img_input = Input(shape=input_shape)
     else:
-        if not K.is_keras_tensor(input_tensor):
-            img_input = Input(tensor=input_tensor, shape=input_shape)
-        else:
-            img_input = input_tensor
+        img_input = Input(tensor=input_tensor, shape=input_shape)
 
-    if K.image_dim_ordering() == 'th':
+    if K.image_data_format() == 'channels_first':
         channel_axis = 1
     else:
         channel_axis = 3
 
-    x = conv2d_bn(img_input, 32, 3, 3, subsample=(2, 2), border_mode='valid')
-    x = conv2d_bn(x, 32, 3, 3, border_mode='valid')
+    x = conv2d_bn(img_input, 32, 3, 3, strides=(2, 2), padding='valid')
+    x = conv2d_bn(x, 32, 3, 3, padding='valid')
     x = conv2d_bn(x, 64, 3, 3)
     x = MaxPooling2D((3, 3), strides=(2, 2))(x)
 
-    x = conv2d_bn(x, 80, 1, 1, border_mode='valid')
-    x = conv2d_bn(x, 192, 3, 3, border_mode='valid')
+    x = conv2d_bn(x, 80, 1, 1, padding='valid')
+    x = conv2d_bn(x, 192, 3, 3, padding='valid')
     x = MaxPooling2D((3, 3), strides=(2, 2))(x)
 
     # mixed 0, 1, 2: 35 x 35 x 256
-    for i in range(3):
-        branch1x1 = conv2d_bn(x, 64, 1, 1)
+    branch1x1 = conv2d_bn(x, 64, 1, 1)
 
-        branch5x5 = conv2d_bn(x, 48, 1, 1)
-        branch5x5 = conv2d_bn(branch5x5, 64, 5, 5)
-
-        branch3x3dbl = conv2d_bn(x, 64, 1, 1)
-        branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
-        branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
-
-        branch_pool = AveragePooling2D(
-            (3, 3), strides=(1, 1), border_mode='same')(x)
-        branch_pool = conv2d_bn(branch_pool, 32, 1, 1)
-        x = merge([branch1x1, branch5x5, branch3x3dbl, branch_pool],
-                  mode='concat', concat_axis=channel_axis,
-                  name='mixed' + str(i))
-
-    # mixed 3: 17 x 17 x 768
-    branch3x3 = conv2d_bn(x, 384, 3, 3, subsample=(2, 2), border_mode='valid')
+    branch5x5 = conv2d_bn(x, 48, 1, 1)
+    branch5x5 = conv2d_bn(branch5x5, 64, 5, 5)
 
     branch3x3dbl = conv2d_bn(x, 64, 1, 1)
     branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
-    branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3,
-                             subsample=(2, 2), border_mode='valid')
+    branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
+
+    branch_pool = AveragePooling2D((3, 3), strides=(1, 1), padding='same')(x)
+    branch_pool = conv2d_bn(branch_pool, 32, 1, 1)
+    x = layers.concatenate(
+        [branch1x1, branch5x5, branch3x3dbl, branch_pool],
+        axis=channel_axis,
+        name='mixed0')
+
+    # mixed 1: 35 x 35 x 256
+    branch1x1 = conv2d_bn(x, 64, 1, 1)
+
+    branch5x5 = conv2d_bn(x, 48, 1, 1)
+    branch5x5 = conv2d_bn(branch5x5, 64, 5, 5)
+
+    branch3x3dbl = conv2d_bn(x, 64, 1, 1)
+    branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
+    branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
+
+    branch_pool = AveragePooling2D((3, 3), strides=(1, 1), padding='same')(x)
+    branch_pool = conv2d_bn(branch_pool, 64, 1, 1)
+    x = layers.concatenate(
+        [branch1x1, branch5x5, branch3x3dbl, branch_pool],
+        axis=channel_axis,
+        name='mixed1')
+
+    # mixed 2: 35 x 35 x 256
+    branch1x1 = conv2d_bn(x, 64, 1, 1)
+
+    branch5x5 = conv2d_bn(x, 48, 1, 1)
+    branch5x5 = conv2d_bn(branch5x5, 64, 5, 5)
+
+    branch3x3dbl = conv2d_bn(x, 64, 1, 1)
+    branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
+    branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
+
+    branch_pool = AveragePooling2D((3, 3), strides=(1, 1), padding='same')(x)
+    branch_pool = conv2d_bn(branch_pool, 64, 1, 1)
+    x = layers.concatenate(
+        [branch1x1, branch5x5, branch3x3dbl, branch_pool],
+        axis=channel_axis,
+        name='mixed2')
+
+    # mixed 3: 17 x 17 x 768
+    branch3x3 = conv2d_bn(x, 384, 3, 3, strides=(2, 2), padding='valid')
+
+    branch3x3dbl = conv2d_bn(x, 64, 1, 1)
+    branch3x3dbl = conv2d_bn(branch3x3dbl, 96, 3, 3)
+    branch3x3dbl = conv2d_bn(
+        branch3x3dbl, 96, 3, 3, strides=(2, 2), padding='valid')
 
     branch_pool = MaxPooling2D((3, 3), strides=(2, 2))(x)
-    x = merge([branch3x3, branch3x3dbl, branch_pool],
-              mode='concat', concat_axis=channel_axis,
-              name='mixed3')
+    x = layers.concatenate(
+        [branch3x3, branch3x3dbl, branch_pool], axis=channel_axis, name='mixed3')
 
     # mixed 4: 17 x 17 x 768
     branch1x1 = conv2d_bn(x, 192, 1, 1)
@@ -171,11 +249,12 @@ def InceptionV3(include_top=True, weights='imagenet',
     branch7x7dbl = conv2d_bn(branch7x7dbl, 128, 7, 1)
     branch7x7dbl = conv2d_bn(branch7x7dbl, 192, 1, 7)
 
-    branch_pool = AveragePooling2D((3, 3), strides=(1, 1), border_mode='same')(x)
+    branch_pool = AveragePooling2D((3, 3), strides=(1, 1), padding='same')(x)
     branch_pool = conv2d_bn(branch_pool, 192, 1, 1)
-    x = merge([branch1x1, branch7x7, branch7x7dbl, branch_pool],
-              mode='concat', concat_axis=channel_axis,
-              name='mixed4')
+    x = layers.concatenate(
+        [branch1x1, branch7x7, branch7x7dbl, branch_pool],
+        axis=channel_axis,
+        name='mixed4')
 
     # mixed 5, 6: 17 x 17 x 768
     for i in range(2):
@@ -192,11 +271,12 @@ def InceptionV3(include_top=True, weights='imagenet',
         branch7x7dbl = conv2d_bn(branch7x7dbl, 192, 1, 7)
 
         branch_pool = AveragePooling2D(
-            (3, 3), strides=(1, 1), border_mode='same')(x)
+            (3, 3), strides=(1, 1), padding='same')(x)
         branch_pool = conv2d_bn(branch_pool, 192, 1, 1)
-        x = merge([branch1x1, branch7x7, branch7x7dbl, branch_pool],
-                  mode='concat', concat_axis=channel_axis,
-                  name='mixed' + str(5 + i))
+        x = layers.concatenate(
+            [branch1x1, branch7x7, branch7x7dbl, branch_pool],
+            axis=channel_axis,
+            name='mixed' + str(5 + i))
 
     # mixed 7: 17 x 17 x 768
     branch1x1 = conv2d_bn(x, 192, 1, 1)
@@ -205,33 +285,33 @@ def InceptionV3(include_top=True, weights='imagenet',
     branch7x7 = conv2d_bn(branch7x7, 192, 1, 7)
     branch7x7 = conv2d_bn(branch7x7, 192, 7, 1)
 
-    branch7x7dbl = conv2d_bn(x, 160, 1, 1)
+    branch7x7dbl = conv2d_bn(x, 192, 1, 1)
     branch7x7dbl = conv2d_bn(branch7x7dbl, 192, 7, 1)
     branch7x7dbl = conv2d_bn(branch7x7dbl, 192, 1, 7)
     branch7x7dbl = conv2d_bn(branch7x7dbl, 192, 7, 1)
     branch7x7dbl = conv2d_bn(branch7x7dbl, 192, 1, 7)
 
-    branch_pool = AveragePooling2D((3, 3), strides=(1, 1), border_mode='same')(x)
+    branch_pool = AveragePooling2D((3, 3), strides=(1, 1), padding='same')(x)
     branch_pool = conv2d_bn(branch_pool, 192, 1, 1)
-    x = merge([branch1x1, branch7x7, branch7x7dbl, branch_pool],
-              mode='concat', concat_axis=channel_axis,
-              name='mixed7')
+    x = layers.concatenate(
+        [branch1x1, branch7x7, branch7x7dbl, branch_pool],
+        axis=channel_axis,
+        name='mixed7')
 
     # mixed 8: 8 x 8 x 1280
     branch3x3 = conv2d_bn(x, 192, 1, 1)
     branch3x3 = conv2d_bn(branch3x3, 320, 3, 3,
-                          subsample=(2, 2), border_mode='valid')
+                          strides=(2, 2), padding='valid')
 
     branch7x7x3 = conv2d_bn(x, 192, 1, 1)
     branch7x7x3 = conv2d_bn(branch7x7x3, 192, 1, 7)
     branch7x7x3 = conv2d_bn(branch7x7x3, 192, 7, 1)
-    branch7x7x3 = conv2d_bn(branch7x7x3, 192, 3, 3,
-                            subsample=(2, 2), border_mode='valid')
+    branch7x7x3 = conv2d_bn(
+        branch7x7x3, 192, 3, 3, strides=(2, 2), padding='valid')
 
-    branch_pool = AveragePooling2D((3, 3), strides=(2, 2))(x)
-    x = merge([branch3x3, branch7x7x3, branch_pool],
-              mode='concat', concat_axis=channel_axis,
-              name='mixed8')
+    branch_pool = MaxPooling2D((3, 3), strides=(2, 2))(x)
+    x = layers.concatenate(
+        [branch3x3, branch7x7x3, branch_pool], axis=channel_axis, name='mixed8')
 
     # mixed 9: 8 x 8 x 2048
     for i in range(2):
@@ -240,29 +320,32 @@ def InceptionV3(include_top=True, weights='imagenet',
         branch3x3 = conv2d_bn(x, 384, 1, 1)
         branch3x3_1 = conv2d_bn(branch3x3, 384, 1, 3)
         branch3x3_2 = conv2d_bn(branch3x3, 384, 3, 1)
-        branch3x3 = merge([branch3x3_1, branch3x3_2],
-                          mode='concat', concat_axis=channel_axis,
-                          name='mixed9_' + str(i))
+        branch3x3 = layers.concatenate(
+            [branch3x3_1, branch3x3_2], axis=channel_axis, name='mixed9_' + str(i))
 
         branch3x3dbl = conv2d_bn(x, 448, 1, 1)
         branch3x3dbl = conv2d_bn(branch3x3dbl, 384, 3, 3)
         branch3x3dbl_1 = conv2d_bn(branch3x3dbl, 384, 1, 3)
         branch3x3dbl_2 = conv2d_bn(branch3x3dbl, 384, 3, 1)
-        branch3x3dbl = merge([branch3x3dbl_1, branch3x3dbl_2],
-                             mode='concat', concat_axis=channel_axis)
+        branch3x3dbl = layers.concatenate(
+            [branch3x3dbl_1, branch3x3dbl_2], axis=channel_axis)
 
         branch_pool = AveragePooling2D(
-            (3, 3), strides=(1, 1), border_mode='same')(x)
+            (3, 3), strides=(1, 1), padding='same')(x)
         branch_pool = conv2d_bn(branch_pool, 192, 1, 1)
-        x = merge([branch1x1, branch3x3, branch3x3dbl, branch_pool],
-                  mode='concat', concat_axis=channel_axis,
-                  name='mixed' + str(9 + i))
-
+        x = layers.concatenate(
+            [branch1x1, branch3x3, branch3x3dbl, branch_pool],
+            axis=channel_axis,
+            name='mixed' + str(9 + i))
     if include_top:
         # Classification block
-        x = AveragePooling2D((8, 8), strides=(8, 8), name='avg_pool')(x)
-        x = Flatten(name='flatten')(x)
-        x = Dense(1000, activation='softmax', name='predictions')(x)
+        x = GlobalAveragePooling2D(name='avg_pool')(x)
+        x = Dense(classes, activation='softmax', name='predictions')(x)
+    else:
+        if pooling == 'avg':
+            x = GlobalAveragePooling2D()(x)
+        elif pooling == 'max':
+            x = GlobalMaxPooling2D()(x)
 
     # Ensure that the model takes into account
     # any potential predecessors of `input_tensor`.
@@ -275,42 +358,31 @@ def InceptionV3(include_top=True, weights='imagenet',
 
     # load weights
     if weights == 'imagenet':
-        if K.image_dim_ordering() == 'th':
-            if include_top:
-                weights_path = get_file('inception_v3_weights_th_dim_ordering_th_kernels.h5',
-                                        TH_WEIGHTS_PATH,
-                                        cache_subdir='models',
-                                        md5_hash='b3baf3070cc4bf476d43a2ea61b0ca5f')
-            else:
-                weights_path = get_file('inception_v3_weights_th_dim_ordering_th_kernels_notop.h5',
-                                        TH_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models',
-                                        md5_hash='79aaa90ab4372b4593ba3df64e142f05')
-            model.load_weights(weights_path)
+        if K.image_data_format() == 'channels_first':
             if K.backend() == 'tensorflow':
                 warnings.warn('You are using the TensorFlow backend, yet you '
                               'are using the Theano '
-                              'image dimension ordering convention '
-                              '(`image_dim_ordering="th"`). '
+                              'image data format convention '
+                              '(`image_data_format="channels_first"`). '
                               'For best performance, set '
-                              '`image_dim_ordering="tf"` in '
+                              '`image_data_format="channels_last"` in '
                               'your Keras config '
                               'at ~/.keras/keras.json.')
-                convert_all_kernels_in_model(model)
+        if include_top:
+            weights_path = get_file(
+                'inception_v3_weights_tf_dim_ordering_tf_kernels.h5',
+                WEIGHTS_PATH,
+                cache_subdir='models',
+                md5_hash='9a0d58056eeedaa3f26cb7ebd46da564')
         else:
-            if include_top:
-                weights_path = get_file('inception_v3_weights_tf_dim_ordering_tf_kernels.h5',
-                                        TF_WEIGHTS_PATH,
-                                        cache_subdir='models',
-                                        md5_hash='fe114b3ff2ea4bf891e9353d1bbfb32f')
-            else:
-                weights_path = get_file('inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5',
-                                        TF_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models',
-                                        md5_hash='2f3609166de1d967d1a481094754f691')
-            model.load_weights(weights_path)
-            if K.backend() == 'theano':
-                convert_all_kernels_in_model(model)
+            weights_path = get_file(
+                'inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5',
+                WEIGHTS_PATH_NO_TOP,
+                cache_subdir='models',
+                md5_hash='bcbd6486424b2319ff4ef7d526e38f63')
+        model.load_weights(weights_path)
+        if K.backend() == 'theano':
+            convert_all_kernels_in_model(model)
     return model
 
 

keras/applications/music_tagger_crnn.py

@@ -1,155 +0,0 @@
-# -*- coding: utf-8 -*-
-'''MusicTaggerCRNN model for Keras.
-
-# Reference:
-
-- [Music-auto_tagging-keras](https://github.com/keunwoochoi/music-auto_tagging-keras)
-
-'''
-from __future__ import print_function
-from __future__ import absolute_import
-
-from .. import backend as K
-from ..layers import Input, Dense
-from ..models import Model
-from ..layers import Dense, Dropout, Reshape, Permute
-from ..layers.convolutional import Convolution2D
-from ..layers.convolutional import MaxPooling2D, ZeroPadding2D
-from ..layers.normalization import BatchNormalization
-from ..layers.advanced_activations import ELU
-from ..layers.recurrent import GRU
-from ..engine.topology import get_source_inputs
-from ..utils.data_utils import get_file
-from ..utils.layer_utils import convert_all_kernels_in_model
-from .audio_conv_utils import decode_predictions, preprocess_input
-
-TH_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.3/music_tagger_crnn_weights_tf_kernels_th_dim_ordering.h5'
-TF_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.3/music_tagger_crnn_weights_tf_kernels_tf_dim_ordering.h5'
-
-
-def MusicTaggerCRNN(weights='msd', input_tensor=None,
-                    include_top=True):
-    '''Instantiate the MusicTaggerCRNN architecture,
-    optionally loading weights pre-trained
-    on Million Song Dataset. Note that when using TensorFlow,
-    for best performance you should set
-    `image_dim_ordering="tf"` in your Keras config
-    at ~/.keras/keras.json.
-
-    The model and the weights are compatible with both
-    TensorFlow and Theano. The dimension ordering
-    convention used by the model is the one
-    specified in your Keras config file.
-
-    For preparing mel-spectrogram input, see
-    `audio_conv_utils.py` in [applications](https://github.com/fchollet/keras/tree/master/keras/applications).
-    You will need to install [Librosa](http://librosa.github.io/librosa/)
-    to use it.
-
-    # Arguments
-        weights: one of `None` (random initialization)
-            or "msd" (pre-training on ImageNet).
-        input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)
-            to use as image input for the model.
-        include_top: whether to include the 1 fully-connected
-            layer (output layer) at the top of the network.
-            If False, the network outputs 32-dim features.
-
-
-    # Returns
-        A Keras model instance.
-    '''
-    if weights not in {'msd', None}:
-        raise ValueError('The `weights` argument should be either '
-                         '`None` (random initialization) or `msd` '
-                         '(pre-training on Million Song Dataset).')
-
-    # Determine proper input shape
-    if K.image_dim_ordering() == 'th':
-        input_shape = (1, 96, 1366)
-    else:
-        input_shape = (96, 1366, 1)
-
-    if input_tensor is None:
-        melgram_input = Input(shape=input_shape)
-    else:
-        if not K.is_keras_tensor(input_tensor):
-            melgram_input = Input(tensor=input_tensor, shape=input_shape)
-        else:
-            melgram_input = input_tensor
-
-    # Determine input axis
-    if K.image_dim_ordering() == 'th':
-        channel_axis = 1
-        freq_axis = 2
-        time_axis = 3
-    else:
-        channel_axis = 3
-        freq_axis = 1
-        time_axis = 2
-
-    # Input block
-    x = ZeroPadding2D(padding=(0, 37))(melgram_input)
-    x = BatchNormalization(axis=time_axis, name='bn_0_freq')(x)
-
-    # Conv block 1
-    x = Convolution2D(64, 3, 3, border_mode='same', name='conv1')(x)
-    x = BatchNormalization(axis=channel_axis, mode=0, name='bn1')(x)
-    x = ELU()(x)
-    x = MaxPooling2D(pool_size=(2, 2), strides=(2, 2), name='pool1')(x)
-
-    # Conv block 2
-    x = Convolution2D(128, 3, 3, border_mode='same', name='conv2')(x)
-    x = BatchNormalization(axis=channel_axis, mode=0, name='bn2')(x)
-    x = ELU()(x)
-    x = MaxPooling2D(pool_size=(3, 3), strides=(3, 3), name='pool2')(x)
-
-    # Conv block 3
-    x = Convolution2D(128, 3, 3, border_mode='same', name='conv3')(x)
-    x = BatchNormalization(axis=channel_axis, mode=0, name='bn3')(x)
-    x = ELU()(x)
-    x = MaxPooling2D(pool_size=(4, 4), strides=(4, 4), name='pool3')(x)
-
-    # Conv block 4
-    x = Convolution2D(128, 3, 3, border_mode='same', name='conv4')(x)
-    x = BatchNormalization(axis=channel_axis, mode=0, name='bn4')(x)
-    x = ELU()(x)
-    x = MaxPooling2D(pool_size=(4, 4), strides=(4, 4), name='pool4')(x)
-
-    # reshaping
-    if K.image_dim_ordering() == 'th':
-        x = Permute((3, 1, 2))(x)
-    x = Reshape((15, 128))(x)
-
-    # GRU block 1, 2, output
-    x = GRU(32, return_sequences=True, name='gru1')(x)
-    x = GRU(32, return_sequences=False, name='gru2')(x)
-
-    if include_top:
-        x = Dense(50, activation='sigmoid', name='output')(x)
-
-    # Ensure that the model takes into account
-    # any potential predecessors of `input_tensor`.
-    if input_tensor is not None:
-        inputs = get_source_inputs(input_tensor)
-    else:
-        inputs = melgram_input
-    # Create model.
-    model = Model(inputs, x, name='music_tagger_crnn')
-
-    if weights is None:
-        return model
-    else:
-        # Load weights
-        if K.image_dim_ordering() == 'tf':
-            weights_path = get_file('music_tagger_crnn_weights_tf_kernels_tf_dim_ordering.h5',
-                                    TF_WEIGHTS_PATH,
-                                    cache_subdir='models')
-        else:
-            weights_path = get_file('music_tagger_crnn_weights_tf_kernels_th_dim_ordering.h5',
-                                    TH_WEIGHTS_PATH,
-                                    cache_subdir='models')
-        model.load_weights(weights_path, by_name=True)
-        if K.backend() == 'theano':
-            convert_all_kernels_in_model(model)
-        return model

keras/applications/resnet50.py

@@ -1,124 +1,141 @@
 # -*- coding: utf-8 -*-
-'''ResNet50 model for Keras.
+"""ResNet50 model for Keras.
 
 # Reference:
 
 - [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385)
 
 Adapted from code contributed by BigMoyan.
-'''
+"""
 from __future__ import print_function
 from __future__ import absolute_import
 
 import warnings
 
-from ..layers import merge, Input
-from ..layers import Dense, Activation, Flatten
-from ..layers import Convolution2D, MaxPooling2D, ZeroPadding2D, AveragePooling2D
+from ..layers import Input
+from .. import layers
+from ..layers import Dense
+from ..layers import Activation
+from ..layers import Flatten
+from ..layers import Conv2D
+from ..layers import MaxPooling2D
+from ..layers import ZeroPadding2D
+from ..layers import AveragePooling2D
+from ..layers import GlobalAveragePooling2D
+from ..layers import GlobalMaxPooling2D
 from ..layers import BatchNormalization
 from ..models import Model
 from .. import backend as K
 from ..engine.topology import get_source_inputs
-from ..utils.layer_utils import convert_all_kernels_in_model
+from ..utils import layer_utils
 from ..utils.data_utils import get_file
-from .imagenet_utils import decode_predictions, preprocess_input, _obtain_input_shape
+from .imagenet_utils import decode_predictions
+from .imagenet_utils import preprocess_input
+from .imagenet_utils import _obtain_input_shape
 
 
-TH_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_th_dim_ordering_th_kernels.h5'
-TF_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5'
-TH_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_th_dim_ordering_th_kernels_notop.h5'
-TF_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'
+WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5'
+WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'
 
 
 def identity_block(input_tensor, kernel_size, filters, stage, block):
-    '''The identity_block is the block that has no conv layer at shortcut
+    """The identity block is the block that has no conv layer at shortcut.
 
     # Arguments
         input_tensor: input tensor
         kernel_size: defualt 3, the kernel size of middle conv layer at main path
-        filters: list of integers, the nb_filters of 3 conv layer at main path
+        filters: list of integers, the filterss of 3 conv layer at main path
         stage: integer, current stage label, used for generating layer names
         block: 'a','b'..., current block label, used for generating layer names
-    '''
-    nb_filter1, nb_filter2, nb_filter3 = filters
-    if K.image_dim_ordering() == 'tf':
+
+    # Returns
+        Output tensor for the block.
+    """
+    filters1, filters2, filters3 = filters
+    if K.image_data_format() == 'channels_last':
         bn_axis = 3
     else:
         bn_axis = 1
     conv_name_base = 'res' + str(stage) + block + '_branch'
     bn_name_base = 'bn' + str(stage) + block + '_branch'
 
-    x = Convolution2D(nb_filter1, 1, 1, name=conv_name_base + '2a')(input_tensor)
+    x = Conv2D(filters1, (1, 1), name=conv_name_base + '2a')(input_tensor)
     x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2a')(x)
     x = Activation('relu')(x)
 
-    x = Convolution2D(nb_filter2, kernel_size, kernel_size,
-                      border_mode='same', name=conv_name_base + '2b')(x)
+    x = Conv2D(filters2, kernel_size,
+               padding='same', name=conv_name_base + '2b')(x)
     x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2b')(x)
     x = Activation('relu')(x)
 
-    x = Convolution2D(nb_filter3, 1, 1, name=conv_name_base + '2c')(x)
+    x = Conv2D(filters3, (1, 1), name=conv_name_base + '2c')(x)
     x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2c')(x)
 
-    x = merge([x, input_tensor], mode='sum')
+    x = layers.add([x, input_tensor])
     x = Activation('relu')(x)
     return x
 
 
 def conv_block(input_tensor, kernel_size, filters, stage, block, strides=(2, 2)):
-    '''conv_block is the block that has a conv layer at shortcut
+    """conv_block is the block that has a conv layer at shortcut
 
     # Arguments
         input_tensor: input tensor
         kernel_size: defualt 3, the kernel size of middle conv layer at main path
-        filters: list of integers, the nb_filters of 3 conv layer at main path
+        filters: list of integers, the filterss of 3 conv layer at main path
         stage: integer, current stage label, used for generating layer names
         block: 'a','b'..., current block label, used for generating layer names
 
-    Note that from stage 3, the first conv layer at main path is with subsample=(2,2)
-    And the shortcut should have subsample=(2,2) as well
-    '''
-    nb_filter1, nb_filter2, nb_filter3 = filters
-    if K.image_dim_ordering() == 'tf':
+    # Returns
+        Output tensor for the block.
+
+    Note that from stage 3, the first conv layer at main path is with strides=(2,2)
+    And the shortcut should have strides=(2,2) as well
+    """
+    filters1, filters2, filters3 = filters
+    if K.image_data_format() == 'channels_last':
         bn_axis = 3
     else:
         bn_axis = 1
     conv_name_base = 'res' + str(stage) + block + '_branch'
     bn_name_base = 'bn' + str(stage) + block + '_branch'
 
-    x = Convolution2D(nb_filter1, 1, 1, subsample=strides,
-                      name=conv_name_base + '2a')(input_tensor)
+    x = Conv2D(filters1, (1, 1), strides=strides,
+               name=conv_name_base + '2a')(input_tensor)
     x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2a')(x)
     x = Activation('relu')(x)
 
-    x = Convolution2D(nb_filter2, kernel_size, kernel_size, border_mode='same',
-                      name=conv_name_base + '2b')(x)
+    x = Conv2D(filters2, kernel_size, padding='same',
+               name=conv_name_base + '2b')(x)
     x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2b')(x)
     x = Activation('relu')(x)
 
-    x = Convolution2D(nb_filter3, 1, 1, name=conv_name_base + '2c')(x)
+    x = Conv2D(filters3, (1, 1), name=conv_name_base + '2c')(x)
     x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2c')(x)
 
-    shortcut = Convolution2D(nb_filter3, 1, 1, subsample=strides,
-                             name=conv_name_base + '1')(input_tensor)
+    shortcut = Conv2D(filters3, (1, 1), strides=strides,
+                      name=conv_name_base + '1')(input_tensor)
     shortcut = BatchNormalization(axis=bn_axis, name=bn_name_base + '1')(shortcut)
 
-    x = merge([x, shortcut], mode='sum')
+    x = layers.add([x, shortcut])
     x = Activation('relu')(x)
     return x
 
 
 def ResNet50(include_top=True, weights='imagenet',
-             input_tensor=None, input_shape=None):
-    '''Instantiate the ResNet50 architecture,
-    optionally loading weights pre-trained
+             input_tensor=None, input_shape=None,
+             pooling=None,
+             classes=1000):
+    """Instantiates the ResNet50 architecture.
+
+    Optionally loads weights pre-trained
     on ImageNet. Note that when using TensorFlow,
     for best performance you should set
-    `image_dim_ordering="tf"` in your Keras config
+    `image_data_format="channels_last"` in your Keras config
     at ~/.keras/keras.json.
 
     The model and the weights are compatible with both
-    TensorFlow and Theano. The dimension ordering
+    TensorFlow and Theano. The data format
     convention used by the model is the one
     specified in your Keras config file.
 
@@ -129,26 +146,49 @@ def ResNet50(include_top=True, weights='imagenet',
             or "imagenet" (pre-training on ImageNet).
         input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)
             to use as image input for the model.
-        inputs_shape: optional shape tuple, only to be specified
+        input_shape: optional shape tuple, only to be specified
             if `include_top` is False (otherwise the input shape
-            has to be `(224, 224, 3)` (with `tf` dim ordering)
-            or `(3, 224, 244)` (with `th` dim ordering).
+            has to be `(224, 224, 3)` (with `channels_last` data format)
+            or `(3, 224, 244)` (with `channels_first` data format).
             It should have exactly 3 inputs channels,
             and width and height should be no smaller than 197.
             E.g. `(200, 200, 3)` would be one valid value.
+        pooling: Optional pooling mode for feature extraction
+            when `include_top` is `False`.
+            - `None` means that the output of the model will be
+                the 4D tensor output of the
+                last convolutional layer.
+            - `avg` means that global average pooling
+                will be applied to the output of the
+                last convolutional layer, and thus
+                the output of the model will be a 2D tensor.
+            - `max` means that global max pooling will
+                be applied.
+        classes: optional number of classes to classify images
+            into, only to be specified if `include_top` is True, and
+            if no `weights` argument is specified.
 
     # Returns
         A Keras model instance.
-    '''
+
+    # Raises
+        ValueError: in case of invalid argument for `weights`,
+            or invalid input shape.
+    """
     if weights not in {'imagenet', None}:
         raise ValueError('The `weights` argument should be either '
                          '`None` (random initialization) or `imagenet` '
                          '(pre-training on ImageNet).')
+
+    if weights == 'imagenet' and include_top and classes != 1000:
+        raise ValueError('If using `weights` as imagenet with `include_top`'
+                         ' as true, `classes` should be 1000')
+
     # Determine proper input shape
     input_shape = _obtain_input_shape(input_shape,
                                       default_size=224,
                                       min_size=197,
-                                      dim_ordering=K.image_dim_ordering(),
+                                      data_format=K.image_data_format(),
                                       include_top=include_top)
 
     if input_tensor is None:
@@ -158,13 +198,13 @@ def ResNet50(include_top=True, weights='imagenet',
             img_input = Input(tensor=input_tensor, shape=input_shape)
         else:
             img_input = input_tensor
-    if K.image_dim_ordering() == 'tf':
+    if K.image_data_format() == 'channels_last':
         bn_axis = 3
     else:
         bn_axis = 1
 
     x = ZeroPadding2D((3, 3))(img_input)
-    x = Convolution2D(64, 7, 7, subsample=(2, 2), name='conv1')(x)
+    x = Conv2D(64, (7, 7), strides=(2, 2), name='conv1')(x)
     x = BatchNormalization(axis=bn_axis, name='bn_conv1')(x)
     x = Activation('relu')(x)
     x = MaxPooling2D((3, 3), strides=(2, 2))(x)
@@ -193,7 +233,12 @@ def ResNet50(include_top=True, weights='imagenet',
 
     if include_top:
         x = Flatten()(x)
-        x = Dense(1000, activation='softmax', name='fc1000')(x)
+        x = Dense(classes, activation='softmax', name='fc1000')(x)
+    else:
+        if pooling == 'avg':
+            x = GlobalAveragePooling2D()(x)
+        elif pooling == 'max':
+            x = GlobalMaxPooling2D()(x)
 
     # Ensure that the model takes into account
     # any potential predecessors of `input_tensor`.
@@ -206,40 +251,34 @@ def ResNet50(include_top=True, weights='imagenet',
 
     # load weights
     if weights == 'imagenet':
-        if K.image_dim_ordering() == 'th':
+        if include_top:
+            weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels.h5',
+                                    WEIGHTS_PATH,
+                                    cache_subdir='models',
+                                    md5_hash='a7b3fe01876f51b976af0dea6bc144eb')
+        else:
+            weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5',
+                                    WEIGHTS_PATH_NO_TOP,
+                                    cache_subdir='models',
+                                    md5_hash='a268eb855778b3df3c7506639542a6af')
+        model.load_weights(weights_path)
+        if K.backend() == 'theano':
+            layer_utils.convert_all_kernels_in_model(model)
+
+        if K.image_data_format() == 'channels_first':
             if include_top:
-                weights_path = get_file('resnet50_weights_th_dim_ordering_th_kernels.h5',
-                                        TH_WEIGHTS_PATH,
-                                        cache_subdir='models',
-                                        md5_hash='1c1f8f5b0c8ee28fe9d950625a230e1c')
-            else:
-                weights_path = get_file('resnet50_weights_th_dim_ordering_th_kernels_notop.h5',
-                                        TH_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models',
-                                        md5_hash='f64f049c92468c9affcd44b0976cdafe')
-            model.load_weights(weights_path)
+                maxpool = model.get_layer(name='avg_pool')
+                shape = maxpool.output_shape[1:]
+                dense = model.get_layer(name='fc1000')
+                layer_utils.convert_dense_weights_data_format(dense, shape, 'channels_first')
+
             if K.backend() == 'tensorflow':
                 warnings.warn('You are using the TensorFlow backend, yet you '
                               'are using the Theano '
-                              'image dimension ordering convention '
-                              '(`image_dim_ordering="th"`). '
+                              'image data format convention '
+                              '(`image_data_format="channels_first"`). '
                               'For best performance, set '
-                              '`image_dim_ordering="tf"` in '
+                              '`image_data_format="channels_last"` in '
                               'your Keras config '
                               'at ~/.keras/keras.json.')
-                convert_all_kernels_in_model(model)
-        else:
-            if include_top:
-                weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels.h5',
-                                        TF_WEIGHTS_PATH,
-                                        cache_subdir='models',
-                                        md5_hash='a7b3fe01876f51b976af0dea6bc144eb')
-            else:
-                weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5',
-                                        TF_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models',
-                                        md5_hash='a268eb855778b3df3c7506639542a6af')
-            model.load_weights(weights_path)
-            if K.backend() == 'theano':
-                convert_all_kernels_in_model(model)
     return model

keras/applications/vgg16.py

@@ -1,43 +1,51 @@
 # -*- coding: utf-8 -*-
-'''VGG16 model for Keras.
+"""VGG16 model for Keras.
 
-# Reference:
+# Reference
 
 - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556)
 
-'''
+"""
 from __future__ import print_function
 from __future__ import absolute_import
 
 import warnings
 
 from ..models import Model
-from ..layers import Flatten, Dense, Input
-from ..layers import Convolution2D, MaxPooling2D
+from ..layers import Flatten
+from ..layers import Dense
+from ..layers import Input
+from ..layers import Conv2D
+from ..layers import MaxPooling2D
+from ..layers import GlobalAveragePooling2D
+from ..layers import GlobalMaxPooling2D
 from ..engine.topology import get_source_inputs
-from ..utils.layer_utils import convert_all_kernels_in_model
+from ..utils import layer_utils
 from ..utils.data_utils import get_file
 from .. import backend as K
-from .imagenet_utils import decode_predictions, preprocess_input, _obtain_input_shape
+from .imagenet_utils import decode_predictions
+from .imagenet_utils import preprocess_input
+from .imagenet_utils import _obtain_input_shape
 
 
-TH_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_th_dim_ordering_th_kernels.h5'
-TF_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels.h5'
-TH_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_th_dim_ordering_th_kernels_notop.h5'
-TF_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5'
+WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels.h5'
+WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5'
 
 
 def VGG16(include_top=True, weights='imagenet',
-          input_tensor=None, input_shape=None):
-    '''Instantiate the VGG16 architecture,
-    optionally loading weights pre-trained
+          input_tensor=None, input_shape=None,
+          pooling=None,
+          classes=1000):
+    """Instantiates the VGG16 architecture.
+
+    Optionally loads weights pre-trained
     on ImageNet. Note that when using TensorFlow,
     for best performance you should set
-    `image_dim_ordering="tf"` in your Keras config
+    `image_data_format="channels_last"` in your Keras config
     at ~/.keras/keras.json.
 
     The model and the weights are compatible with both
-    TensorFlow and Theano. The dimension ordering
+    TensorFlow and Theano. The data format
     convention used by the model is the one
     specified in your Keras config file.
 
@@ -48,26 +56,48 @@ def VGG16(include_top=True, weights='imagenet',
             or "imagenet" (pre-training on ImageNet).
         input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)
             to use as image input for the model.
-        inputs_shape: optional shape tuple, only to be specified
+        input_shape: optional shape tuple, only to be specified
             if `include_top` is False (otherwise the input shape
-            has to be `(224, 224, 3)` (with `tf` dim ordering)
-            or `(3, 224, 244)` (with `th` dim ordering).
+            has to be `(224, 224, 3)` (with `channels_last` data format)
+            or `(3, 224, 244)` (with `channels_first` data format).
             It should have exactly 3 inputs channels,
             and width and height should be no smaller than 48.
             E.g. `(200, 200, 3)` would be one valid value.
+        pooling: Optional pooling mode for feature extraction
+            when `include_top` is `False`.
+            - `None` means that the output of the model will be
+                the 4D tensor output of the
+                last convolutional layer.
+            - `avg` means that global average pooling
+                will be applied to the output of the
+                last convolutional layer, and thus
+                the output of the model will be a 2D tensor.
+            - `max` means that global max pooling will
+                be applied.
+        classes: optional number of classes to classify images
+            into, only to be specified if `include_top` is True, and
+            if no `weights` argument is specified.
 
     # Returns
         A Keras model instance.
-    '''
+
+    # Raises
+        ValueError: in case of invalid argument for `weights`,
+            or invalid input shape.
+    """
     if weights not in {'imagenet', None}:
         raise ValueError('The `weights` argument should be either '
                          '`None` (random initialization) or `imagenet` '
                          '(pre-training on ImageNet).')
+
+    if weights == 'imagenet' and include_top and classes != 1000:
+        raise ValueError('If using `weights` as imagenet with `include_top`'
+                         ' as true, `classes` should be 1000')
     # Determine proper input shape
     input_shape = _obtain_input_shape(input_shape,
                                       default_size=224,
                                       min_size=48,
-                                      dim_ordering=K.image_dim_ordering(),
+                                      data_format=K.image_data_format(),
                                       include_top=include_top)
 
     if input_tensor is None:
@@ -78,31 +108,31 @@ def VGG16(include_top=True, weights='imagenet',
         else:
             img_input = input_tensor
     # Block 1
-    x = Convolution2D(64, 3, 3, activation='relu', border_mode='same', name='block1_conv1')(img_input)
-    x = Convolution2D(64, 3, 3, activation='relu', border_mode='same', name='block1_conv2')(x)
+    x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1')(img_input)
+    x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)
 
     # Block 2
-    x = Convolution2D(128, 3, 3, activation='relu', border_mode='same', name='block2_conv1')(x)
-    x = Convolution2D(128, 3, 3, activation='relu', border_mode='same', name='block2_conv2')(x)
+    x = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv1')(x)
+    x = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv2')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool')(x)
 
     # Block 3
-    x = Convolution2D(256, 3, 3, activation='relu', border_mode='same', name='block3_conv1')(x)
-    x = Convolution2D(256, 3, 3, activation='relu', border_mode='same', name='block3_conv2')(x)
-    x = Convolution2D(256, 3, 3, activation='relu', border_mode='same', name='block3_conv3')(x)
+    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv1')(x)
+    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv2')(x)
+    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv3')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool')(x)
 
     # Block 4
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block4_conv1')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block4_conv2')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block4_conv3')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv1')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv2')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv3')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool')(x)
 
     # Block 5
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block5_conv1')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block5_conv2')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block5_conv3')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv2')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv3')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool')(x)
 
     if include_top:
@@ -110,7 +140,12 @@ def VGG16(include_top=True, weights='imagenet',
         x = Flatten(name='flatten')(x)
         x = Dense(4096, activation='relu', name='fc1')(x)
         x = Dense(4096, activation='relu', name='fc2')(x)
-        x = Dense(1000, activation='softmax', name='predictions')(x)
+        x = Dense(classes, activation='softmax', name='predictions')(x)
+    else:
+        if pooling == 'avg':
+            x = GlobalAveragePooling2D()(x)
+        elif pooling == 'max':
+            x = GlobalMaxPooling2D()(x)
 
     # Ensure that the model takes into account
     # any potential predecessors of `input_tensor`.
@@ -123,36 +158,32 @@ def VGG16(include_top=True, weights='imagenet',
 
     # load weights
     if weights == 'imagenet':
-        if K.image_dim_ordering() == 'th':
+        if include_top:
+            weights_path = get_file('vgg16_weights_tf_dim_ordering_tf_kernels.h5',
+                                    WEIGHTS_PATH,
+                                    cache_subdir='models')
+        else:
+            weights_path = get_file('vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5',
+                                    WEIGHTS_PATH_NO_TOP,
+                                    cache_subdir='models')
+        model.load_weights(weights_path)
+        if K.backend() == 'theano':
+            layer_utils.convert_all_kernels_in_model(model)
+
+        if K.image_data_format() == 'channels_first':
             if include_top:
-                weights_path = get_file('vgg16_weights_th_dim_ordering_th_kernels.h5',
-                                        TH_WEIGHTS_PATH,
-                                        cache_subdir='models')
-            else:
-                weights_path = get_file('vgg16_weights_th_dim_ordering_th_kernels_notop.h5',
-                                        TH_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models')
-            model.load_weights(weights_path)
+                maxpool = model.get_layer(name='block5_pool')
+                shape = maxpool.output_shape[1:]
+                dense = model.get_layer(name='fc1')
+                layer_utils.convert_dense_weights_data_format(dense, shape, 'channels_first')
+
             if K.backend() == 'tensorflow':
                 warnings.warn('You are using the TensorFlow backend, yet you '
                               'are using the Theano '
-                              'image dimension ordering convention '
-                              '(`image_dim_ordering="th"`). '
+                              'image data format convention '
+                              '(`image_data_format="channels_first"`). '
                               'For best performance, set '
-                              '`image_dim_ordering="tf"` in '
+                              '`image_data_format="channels_last"` in '
                               'your Keras config '
                               'at ~/.keras/keras.json.')
-                convert_all_kernels_in_model(model)
-        else:
-            if include_top:
-                weights_path = get_file('vgg16_weights_tf_dim_ordering_tf_kernels.h5',
-                                        TF_WEIGHTS_PATH,
-                                        cache_subdir='models')
-            else:
-                weights_path = get_file('vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5',
-                                        TF_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models')
-            model.load_weights(weights_path)
-            if K.backend() == 'theano':
-                convert_all_kernels_in_model(model)
     return model

keras/applications/vgg19.py

@@ -1,43 +1,51 @@
 # -*- coding: utf-8 -*-
-'''VGG19 model for Keras.
+"""VGG19 model for Keras.
 
-# Reference:
+# Reference
 
 - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556)
 
-'''
+"""
 from __future__ import print_function
 from __future__ import absolute_import
 
 import warnings
 
 from ..models import Model
-from ..layers import Flatten, Dense, Input
-from ..layers import Convolution2D, MaxPooling2D
+from ..layers import Flatten
+from ..layers import Dense
+from ..layers import Input
+from ..layers import Conv2D
+from ..layers import MaxPooling2D
+from ..layers import GlobalAveragePooling2D
+from ..layers import GlobalMaxPooling2D
 from ..engine.topology import get_source_inputs
-from ..utils.layer_utils import convert_all_kernels_in_model
+from ..utils import layer_utils
 from ..utils.data_utils import get_file
 from .. import backend as K
-from .imagenet_utils import decode_predictions, preprocess_input, _obtain_input_shape
+from .imagenet_utils import decode_predictions
+from .imagenet_utils import preprocess_input
+from .imagenet_utils import _obtain_input_shape
 
 
-TH_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg19_weights_th_dim_ordering_th_kernels.h5'
-TF_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg19_weights_tf_dim_ordering_tf_kernels.h5'
-TH_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg19_weights_th_dim_ordering_th_kernels_notop.h5'
-TF_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5'
+WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg19_weights_tf_dim_ordering_tf_kernels.h5'
+WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5'
 
 
 def VGG19(include_top=True, weights='imagenet',
-          input_tensor=None, input_shape=None):
-    '''Instantiate the VGG19 architecture,
-    optionally loading weights pre-trained
+          input_tensor=None, input_shape=None,
+          pooling=None,
+          classes=1000):
+    """Instantiates the VGG19 architecture.
+
+    Optionally loads weights pre-trained
     on ImageNet. Note that when using TensorFlow,
     for best performance you should set
-    `image_dim_ordering="tf"` in your Keras config
+    `image_data_format="channels_last"` in your Keras config
     at ~/.keras/keras.json.
 
     The model and the weights are compatible with both
-    TensorFlow and Theano. The dimension ordering
+    TensorFlow and Theano. The data format
     convention used by the model is the one
     specified in your Keras config file.
 
@@ -48,26 +56,48 @@ def VGG19(include_top=True, weights='imagenet',
             or "imagenet" (pre-training on ImageNet).
         input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)
             to use as image input for the model.
-        inputs_shape: optional shape tuple, only to be specified
+        input_shape: optional shape tuple, only to be specified
             if `include_top` is False (otherwise the input shape
-            has to be `(224, 224, 3)` (with `tf` dim ordering)
-            or `(3, 224, 244)` (with `th` dim ordering).
+            has to be `(224, 224, 3)` (with `channels_last` data format)
+            or `(3, 224, 244)` (with `channels_first` data format).
             It should have exactly 3 inputs channels,
             and width and height should be no smaller than 48.
             E.g. `(200, 200, 3)` would be one valid value.
+        pooling: Optional pooling mode for feature extraction
+            when `include_top` is `False`.
+            - `None` means that the output of the model will be
+                the 4D tensor output of the
+                last convolutional layer.
+            - `avg` means that global average pooling
+                will be applied to the output of the
+                last convolutional layer, and thus
+                the output of the model will be a 2D tensor.
+            - `max` means that global max pooling will
+                be applied.
+        classes: optional number of classes to classify images
+            into, only to be specified if `include_top` is True, and
+            if no `weights` argument is specified.
 
     # Returns
         A Keras model instance.
-    '''
+
+    # Raises
+        ValueError: in case of invalid argument for `weights`,
+            or invalid input shape.
+    """
     if weights not in {'imagenet', None}:
         raise ValueError('The `weights` argument should be either '
                          '`None` (random initialization) or `imagenet` '
                          '(pre-training on ImageNet).')
+
+    if weights == 'imagenet' and include_top and classes != 1000:
+        raise ValueError('If using `weights` as imagenet with `include_top`'
+                         ' as true, `classes` should be 1000')
     # Determine proper input shape
     input_shape = _obtain_input_shape(input_shape,
                                       default_size=224,
                                       min_size=48,
-                                      dim_ordering=K.image_dim_ordering(),
+                                      data_format=K.image_data_format(),
                                       include_top=include_top)
 
     if input_tensor is None:
@@ -78,34 +108,34 @@ def VGG19(include_top=True, weights='imagenet',
         else:
             img_input = input_tensor
     # Block 1
-    x = Convolution2D(64, 3, 3, activation='relu', border_mode='same', name='block1_conv1')(img_input)
-    x = Convolution2D(64, 3, 3, activation='relu', border_mode='same', name='block1_conv2')(x)
+    x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1')(img_input)
+    x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)
 
     # Block 2
-    x = Convolution2D(128, 3, 3, activation='relu', border_mode='same', name='block2_conv1')(x)
-    x = Convolution2D(128, 3, 3, activation='relu', border_mode='same', name='block2_conv2')(x)
+    x = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv1')(x)
+    x = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv2')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool')(x)
 
     # Block 3
-    x = Convolution2D(256, 3, 3, activation='relu', border_mode='same', name='block3_conv1')(x)
-    x = Convolution2D(256, 3, 3, activation='relu', border_mode='same', name='block3_conv2')(x)
-    x = Convolution2D(256, 3, 3, activation='relu', border_mode='same', name='block3_conv3')(x)
-    x = Convolution2D(256, 3, 3, activation='relu', border_mode='same', name='block3_conv4')(x)
+    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv1')(x)
+    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv2')(x)
+    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv3')(x)
+    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv4')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool')(x)
 
     # Block 4
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block4_conv1')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block4_conv2')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block4_conv3')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block4_conv4')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv1')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv2')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv3')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv4')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool')(x)
 
     # Block 5
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block5_conv1')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block5_conv2')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block5_conv3')(x)
-    x = Convolution2D(512, 3, 3, activation='relu', border_mode='same', name='block5_conv4')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv2')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv3')(x)
+    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv4')(x)
     x = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool')(x)
 
     if include_top:
@@ -113,7 +143,12 @@ def VGG19(include_top=True, weights='imagenet',
         x = Flatten(name='flatten')(x)
         x = Dense(4096, activation='relu', name='fc1')(x)
         x = Dense(4096, activation='relu', name='fc2')(x)
-        x = Dense(1000, activation='softmax', name='predictions')(x)
+        x = Dense(classes, activation='softmax', name='predictions')(x)
+    else:
+        if pooling == 'avg':
+            x = GlobalAveragePooling2D()(x)
+        elif pooling == 'max':
+            x = GlobalMaxPooling2D()(x)
 
     # Ensure that the model takes into account
     # any potential predecessors of `input_tensor`.
@@ -126,36 +161,32 @@ def VGG19(include_top=True, weights='imagenet',
 
     # load weights
     if weights == 'imagenet':
-        if K.image_dim_ordering() == 'th':
+        if include_top:
+            weights_path = get_file('vgg19_weights_tf_dim_ordering_tf_kernels.h5',
+                                    WEIGHTS_PATH,
+                                    cache_subdir='models')
+        else:
+            weights_path = get_file('vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5',
+                                    WEIGHTS_PATH_NO_TOP,
+                                    cache_subdir='models')
+        model.load_weights(weights_path)
+        if K.backend() == 'theano':
+            layer_utils.convert_all_kernels_in_model(model)
+
+        if K.image_data_format() == 'channels_first':
             if include_top:
-                weights_path = get_file('vgg19_weights_th_dim_ordering_th_kernels.h5',
-                                        TH_WEIGHTS_PATH,
-                                        cache_subdir='models')
-            else:
-                weights_path = get_file('vgg19_weights_th_dim_ordering_th_kernels_notop.h5',
-                                        TH_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models')
-            model.load_weights(weights_path)
+                maxpool = model.get_layer(name='block5_pool')
+                shape = maxpool.output_shape[1:]
+                dense = model.get_layer(name='fc1')
+                layer_utils.convert_dense_weights_data_format(dense, shape, 'channels_first')
+
             if K.backend() == 'tensorflow':
                 warnings.warn('You are using the TensorFlow backend, yet you '
                               'are using the Theano '
-                              'image dimension ordering convention '
-                              '(`image_dim_ordering="th"`). '
+                              'image data format convention '
+                              '(`image_data_format="channels_first"`). '
                               'For best performance, set '
-                              '`image_dim_ordering="tf"` in '
+                              '`image_data_format="channels_last"` in '
                               'your Keras config '
                               'at ~/.keras/keras.json.')
-                convert_all_kernels_in_model(model)
-        else:
-            if include_top:
-                weights_path = get_file('vgg19_weights_tf_dim_ordering_tf_kernels.h5',
-                                        TF_WEIGHTS_PATH,
-                                        cache_subdir='models')
-            else:
-                weights_path = get_file('vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5',
-                                        TF_WEIGHTS_PATH_NO_TOP,
-                                        cache_subdir='models')
-            model.load_weights(weights_path)
-            if K.backend() == 'theano':
-                convert_all_kernels_in_model(model)
     return model

keras/applications/xception.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-'''Xception V1 model for Keras.
+"""Xception V1 model for Keras.
 
 On ImageNet, this model gets to a top-1 validation accuracy of 0.790
 and a top-5 validation accuracy of 0.945.
@@ -12,23 +12,32 @@ is also different (same as Inception V3).
 Also do note that this model is only available for the TensorFlow backend,
 due to its reliance on `SeparableConvolution` layers.
 
-# Reference:
+# Reference
 
 - [Xception: Deep Learning with Depthwise Separable Convolutions](https://arxiv.org/abs/1610.02357)
 
-'''
+"""
 from __future__ import print_function
 from __future__ import absolute_import
 
 import warnings
 
 from ..models import Model
-from ..layers import Dense, Input, BatchNormalization, Activation, merge
-from ..layers import Conv2D, SeparableConv2D, MaxPooling2D, GlobalAveragePooling2D
+from .. import layers
+from ..layers import Dense
+from ..layers import Input
+from ..layers import BatchNormalization
+from ..layers import Activation
+from ..layers import Conv2D
+from ..layers import SeparableConv2D
+from ..layers import MaxPooling2D
+from ..layers import GlobalAveragePooling2D
+from ..layers import GlobalMaxPooling2D
 from ..engine.topology import get_source_inputs
 from ..utils.data_utils import get_file
 from .. import backend as K
-from .imagenet_utils import decode_predictions, _obtain_input_shape
+from .imagenet_utils import decode_predictions
+from .imagenet_utils import _obtain_input_shape
 
 
 TF_WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.4/xception_weights_tf_dim_ordering_tf_kernels.h5'
@@ -36,13 +45,16 @@ TF_WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/relea
 
 
 def Xception(include_top=True, weights='imagenet',
-             input_tensor=None, input_shape=None):
-    '''Instantiate the Xception architecture,
-    optionally loading weights pre-trained
+             input_tensor=None, input_shape=None,
+             pooling=None,
+             classes=1000):
+    """Instantiates the Xception architecture.
+
+    Optionally loads weights pre-trained
     on ImageNet. This model is available for TensorFlow only,
     and can only be used with inputs following the TensorFlow
-    dimension ordering `(width, height, channels)`.
-    You should set `image_dim_ordering="tf"` in your Keras config
+    data format `(width, height, channels)`.
+    You should set `image_data_format="channels_last"` in your Keras config
     located at ~/.keras/keras.json.
 
     Note that the default input image size for this model is 299x299.
@@ -54,43 +66,68 @@ def Xception(include_top=True, weights='imagenet',
             or "imagenet" (pre-training on ImageNet).
         input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)
             to use as image input for the model.
-        inputs_shape: optional shape tuple, only to be specified
+        input_shape: optional shape tuple, only to be specified
             if `include_top` is False (otherwise the input shape
             has to be `(299, 299, 3)`.
             It should have exactly 3 inputs channels,
             and width and height should be no smaller than 71.
             E.g. `(150, 150, 3)` would be one valid value.
+        pooling: Optional pooling mode for feature extraction
+            when `include_top` is `False`.
+            - `None` means that the output of the model will be
+                the 4D tensor output of the
+                last convolutional layer.
+            - `avg` means that global average pooling
+                will be applied to the output of the
+                last convolutional layer, and thus
+                the output of the model will be a 2D tensor.
+            - `max` means that global max pooling will
+                be applied.
+        classes: optional number of classes to classify images
+            into, only to be specified if `include_top` is True, and
+            if no `weights` argument is specified.
 
     # Returns
         A Keras model instance.
-    '''
+
+    # Raises
+        ValueError: in case of invalid argument for `weights`,
+            or invalid input shape.
+        RuntimeError: If attempting to run this model with a
+            backend that does not support separable convolutions.
+    """
     if weights not in {'imagenet', None}:
         raise ValueError('The `weights` argument should be either '
                          '`None` (random initialization) or `imagenet` '
                          '(pre-training on ImageNet).')
+
+    if weights == 'imagenet' and include_top and classes != 1000:
+        raise ValueError('If using `weights` as imagenet with `include_top`'
+                         ' as true, `classes` should be 1000')
+
     if K.backend() != 'tensorflow':
         raise RuntimeError('The Xception model is only available with '
                            'the TensorFlow backend.')
-    if K.image_dim_ordering() != 'tf':
+    if K.image_data_format() != 'channels_last':
         warnings.warn('The Xception model is only available for the '
-                      'input dimension ordering "tf" '
+                      'input data format "channels_last" '
                       '(width, height, channels). '
                       'However your settings specify the default '
-                      'dimension ordering "th" (channels, width, height). '
-                      'You should set `image_dim_ordering="tf"` in your Keras '
+                      'data format "channels_first" (channels, width, height). '
+                      'You should set `image_data_format="channels_last"` in your Keras '
                       'config located at ~/.keras/keras.json. '
                       'The model being returned right now will expect inputs '
-                      'to follow the "tf" dimension ordering.')
-        K.set_image_dim_ordering('tf')
-        old_dim_ordering = 'th'
+                      'to follow the "channels_last" data format.')
+        K.set_image_data_format('channels_last')
+        old_data_format = 'channels_first'
     else:
-        old_dim_ordering = None
+        old_data_format = None
 
     # Determine proper input shape
     input_shape = _obtain_input_shape(input_shape,
                                       default_size=299,
                                       min_size=71,
-                                      dim_ordering=K.image_dim_ordering(),
+                                      data_format=K.image_data_format(),
                                       include_top=include_top)
 
     if input_tensor is None:
@@ -101,95 +138,100 @@ def Xception(include_top=True, weights='imagenet',
         else:
             img_input = input_tensor
 
-    x = Conv2D(32, 3, 3, subsample=(2, 2), bias=False, name='block1_conv1')(img_input)
+    x = Conv2D(32, (3, 3), strides=(2, 2), use_bias=False, name='block1_conv1')(img_input)
     x = BatchNormalization(name='block1_conv1_bn')(x)
     x = Activation('relu', name='block1_conv1_act')(x)
-    x = Conv2D(64, 3, 3, bias=False, name='block1_conv2')(x)
+    x = Conv2D(64, (3, 3), use_bias=False, name='block1_conv2')(x)
     x = BatchNormalization(name='block1_conv2_bn')(x)
     x = Activation('relu', name='block1_conv2_act')(x)
 
-    residual = Conv2D(128, 1, 1, subsample=(2, 2),
-                      border_mode='same', bias=False)(x)
+    residual = Conv2D(128, (1, 1), strides=(2, 2),
+                      padding='same', use_bias=False)(x)
     residual = BatchNormalization()(residual)
 
-    x = SeparableConv2D(128, 3, 3, border_mode='same', bias=False, name='block2_sepconv1')(x)
+    x = SeparableConv2D(128, (3, 3), padding='same', use_bias=False, name='block2_sepconv1')(x)
     x = BatchNormalization(name='block2_sepconv1_bn')(x)
     x = Activation('relu', name='block2_sepconv2_act')(x)
-    x = SeparableConv2D(128, 3, 3, border_mode='same', bias=False, name='block2_sepconv2')(x)
+    x = SeparableConv2D(128, (3, 3), padding='same', use_bias=False, name='block2_sepconv2')(x)
     x = BatchNormalization(name='block2_sepconv2_bn')(x)
 
-    x = MaxPooling2D((3, 3), strides=(2, 2), border_mode='same', name='block2_pool')(x)
-    x = merge([x, residual], mode='sum')
+    x = MaxPooling2D((3, 3), strides=(2, 2), padding='same', name='block2_pool')(x)
+    x = layers.add([x, residual])
 
-    residual = Conv2D(256, 1, 1, subsample=(2, 2),
-                      border_mode='same', bias=False)(x)
+    residual = Conv2D(256, (1, 1), strides=(2, 2),
+                      padding='same', use_bias=False)(x)
     residual = BatchNormalization()(residual)
 
     x = Activation('relu', name='block3_sepconv1_act')(x)
-    x = SeparableConv2D(256, 3, 3, border_mode='same', bias=False, name='block3_sepconv1')(x)
+    x = SeparableConv2D(256, (3, 3), padding='same', use_bias=False, name='block3_sepconv1')(x)
     x = BatchNormalization(name='block3_sepconv1_bn')(x)
     x = Activation('relu', name='block3_sepconv2_act')(x)
-    x = SeparableConv2D(256, 3, 3, border_mode='same', bias=False, name='block3_sepconv2')(x)
+    x = SeparableConv2D(256, (3, 3), padding='same', use_bias=False, name='block3_sepconv2')(x)
     x = BatchNormalization(name='block3_sepconv2_bn')(x)
 
-    x = MaxPooling2D((3, 3), strides=(2, 2), border_mode='same', name='block3_pool')(x)
-    x = merge([x, residual], mode='sum')
+    x = MaxPooling2D((3, 3), strides=(2, 2), padding='same', name='block3_pool')(x)
+    x = layers.add([x, residual])
 
-    residual = Conv2D(728, 1, 1, subsample=(2, 2),
-                      border_mode='same', bias=False)(x)
+    residual = Conv2D(728, (1, 1), strides=(2, 2),
+                      padding='same', use_bias=False)(x)
     residual = BatchNormalization()(residual)
 
     x = Activation('relu', name='block4_sepconv1_act')(x)
-    x = SeparableConv2D(728, 3, 3, border_mode='same', bias=False, name='block4_sepconv1')(x)
+    x = SeparableConv2D(728, (3, 3), padding='same', use_bias=False, name='block4_sepconv1')(x)
     x = BatchNormalization(name='block4_sepconv1_bn')(x)
     x = Activation('relu', name='block4_sepconv2_act')(x)
-    x = SeparableConv2D(728, 3, 3, border_mode='same', bias=False, name='block4_sepconv2')(x)
+    x = SeparableConv2D(728, (3, 3), padding='same', use_bias=False, name='block4_sepconv2')(x)
     x = BatchNormalization(name='block4_sepconv2_bn')(x)
 
-    x = MaxPooling2D((3, 3), strides=(2, 2), border_mode='same', name='block4_pool')(x)
-    x = merge([x, residual], mode='sum')
+    x = MaxPooling2D((3, 3), strides=(2, 2), padding='same', name='block4_pool')(x)
+    x = layers.add([x, residual])
 
     for i in range(8):
         residual = x
         prefix = 'block' + str(i + 5)
 
         x = Activation('relu', name=prefix + '_sepconv1_act')(x)
-        x = SeparableConv2D(728, 3, 3, border_mode='same', bias=False, name=prefix + '_sepconv1')(x)
+        x = SeparableConv2D(728, (3, 3), padding='same', use_bias=False, name=prefix + '_sepconv1')(x)
         x = BatchNormalization(name=prefix + '_sepconv1_bn')(x)
         x = Activation('relu', name=prefix + '_sepconv2_act')(x)
-        x = SeparableConv2D(728, 3, 3, border_mode='same', bias=False, name=prefix + '_sepconv2')(x)
+        x = SeparableConv2D(728, (3, 3), padding='same', use_bias=False, name=prefix + '_sepconv2')(x)
         x = BatchNormalization(name=prefix + '_sepconv2_bn')(x)
         x = Activation('relu', name=prefix + '_sepconv3_act')(x)
-        x = SeparableConv2D(728, 3, 3, border_mode='same', bias=False, name=prefix + '_sepconv3')(x)
+        x = SeparableConv2D(728, (3, 3), padding='same', use_bias=False, name=prefix + '_sepconv3')(x)
         x = BatchNormalization(name=prefix + '_sepconv3_bn')(x)
 
-        x = merge([x, residual], mode='sum')
+        x = layers.add([x, residual])
 
-    residual = Conv2D(1024, 1, 1, subsample=(2, 2),
-                      border_mode='same', bias=False)(x)
+    residual = Conv2D(1024, (1, 1), strides=(2, 2),
+                      padding='same', use_bias=False)(x)
     residual = BatchNormalization()(residual)
 
     x = Activation('relu', name='block13_sepconv1_act')(x)
-    x = SeparableConv2D(728, 3, 3, border_mode='same', bias=False, name='block13_sepconv1')(x)
+    x = SeparableConv2D(728, (3, 3), padding='same', use_bias=False, name='block13_sepconv1')(x)
     x = BatchNormalization(name='block13_sepconv1_bn')(x)
     x = Activation('relu', name='block13_sepconv2_act')(x)
-    x = SeparableConv2D(1024, 3, 3, border_mode='same', bias=False, name='block13_sepconv2')(x)
+    x = SeparableConv2D(1024, (3, 3), padding='same', use_bias=False, name='block13_sepconv2')(x)
     x = BatchNormalization(name='block13_sepconv2_bn')(x)
 
-    x = MaxPooling2D((3, 3), strides=(2, 2), border_mode='same', name='block13_pool')(x)
-    x = merge([x, residual], mode='sum')
+    x = MaxPooling2D((3, 3), strides=(2, 2), padding='same', name='block13_pool')(x)
+    x = layers.add([x, residual])
 
-    x = SeparableConv2D(1536, 3, 3, border_mode='same', bias=False, name='block14_sepconv1')(x)
+    x = SeparableConv2D(1536, (3, 3), padding='same', use_bias=False, name='block14_sepconv1')(x)
     x = BatchNormalization(name='block14_sepconv1_bn')(x)
     x = Activation('relu', name='block14_sepconv1_act')(x)
 
-    x = SeparableConv2D(2048, 3, 3, border_mode='same', bias=False, name='block14_sepconv2')(x)
+    x = SeparableConv2D(2048, (3, 3), padding='same', use_bias=False, name='block14_sepconv2')(x)
     x = BatchNormalization(name='block14_sepconv2_bn')(x)
     x = Activation('relu', name='block14_sepconv2_act')(x)
 
     if include_top:
         x = GlobalAveragePooling2D(name='avg_pool')(x)
-        x = Dense(1000, activation='softmax', name='predictions')(x)
+        x = Dense(classes, activation='softmax', name='predictions')(x)
+    else:
+        if pooling == 'avg':
+            x = GlobalAveragePooling2D()(x)
+        elif pooling == 'max':
+            x = GlobalMaxPooling2D()(x)
 
     # Ensure that the model takes into account
     # any potential predecessors of `input_tensor`.
@@ -212,8 +254,8 @@ def Xception(include_top=True, weights='imagenet',
                                     cache_subdir='models')
         model.load_weights(weights_path)
 
-    if old_dim_ordering:
-        K.set_image_dim_ordering(old_dim_ordering)
+    if old_data_format:
+        K.set_image_data_format(old_data_format)
     return model
 
 

keras/backend/__init__.py

@@ -7,13 +7,10 @@ from .common import epsilon
 from .common import floatx
 from .common import set_epsilon
 from .common import set_floatx
-from .common import get_uid
 from .common import cast_to_floatx
-from .common import image_dim_ordering
-from .common import set_image_dim_ordering
+from .common import image_data_format
+from .common import set_image_data_format
 from .common import is_keras_tensor
-from .common import legacy_weight_ordering
-from .common import set_legacy_weight_ordering
 
 _keras_base_dir = os.path.expanduser('~')
 if not os.access(_keras_base_dir, os.W_OK):
@@ -35,13 +32,13 @@ if os.path.exists(_config_path):
     assert isinstance(_epsilon, float)
     _backend = _config.get('backend', _BACKEND)
     assert _backend in {'theano', 'tensorflow'}
-    _image_dim_ordering = _config.get('image_dim_ordering',
-                                      image_dim_ordering())
-    assert _image_dim_ordering in {'tf', 'th'}
+    _image_data_format = _config.get('image_data_format',
+                                     image_data_format())
+    assert _image_data_format in {'channels_last', 'channels_first'}
 
     set_floatx(_floatx)
     set_epsilon(_epsilon)
-    set_image_dim_ordering(_image_dim_ordering)
+    set_image_data_format(_image_data_format)
     _BACKEND = _backend
 
 # save config file
@@ -49,7 +46,7 @@ if not os.path.exists(_config_path):
     _config = {'floatx': floatx(),
                'epsilon': epsilon(),
                'backend': _BACKEND,
-               'image_dim_ordering': image_dim_ordering()}
+               'image_data_format': image_data_format()}
     with open(_config_path, 'w') as f:
         f.write(json.dumps(_config, indent=4))
 
@@ -70,7 +67,7 @@ else:
 
 
 def backend():
-    '''Publicly accessible method
+    """Publicly accessible method
     for determining the current backend.
-    '''
+    """
     return _BACKEND

keras/backend/common.py

@@ -1,17 +1,13 @@
 import numpy as np
 
-from collections import defaultdict
-
 # the type of float to use throughout the session.
 _FLOATX = 'float32'
 _EPSILON = 10e-8
-_UID_PREFIXES = defaultdict(int)
-_IMAGE_DIM_ORDERING = 'tf'
-_LEGACY_WEIGHT_ORDERING = False
+_IMAGE_DATA_FORMAT = 'channels_last'
 
 
 def epsilon():
-    '''Returns the value of the fuzz
+    """Returns the value of the fuzz
     factor used in numeric expressions.
 
     # Returns
@@ -22,12 +18,12 @@ def epsilon():
         >>> keras.backend.epsilon()
         1e-08
     ```
-    '''
+    """
     return _EPSILON
 
 
 def set_epsilon(e):
-    '''Sets the value of the fuzz
+    """Sets the value of the fuzz
     factor used in numeric expressions.
 
     # Arguments
@@ -42,13 +38,13 @@ def set_epsilon(e):
         >>> K.epsilon()
         1e-05
     ```
-    '''
+    """
     global _EPSILON
     _EPSILON = e
 
 
 def floatx():
-    '''Returns the default float type, as a string
+    """Returns the default float type, as a string
     (e.g. 'float16', 'float32', 'float64').
 
     # Returns
@@ -59,12 +55,12 @@ def floatx():
         >>> keras.backend.floatx()
         'float32'
     ```
-    '''
+    """
     return _FLOATX
 
 
 def set_floatx(floatx):
-    '''Sets the default float type.
+    """Sets the default float type.
 
     # Arguments
         String: 'float16', 'float32', or 'float64'.
@@ -78,7 +74,7 @@ def set_floatx(floatx):
         >>> K.floatx()
         'float16'
     ```
-    '''
+    """
     global _FLOATX
     if floatx not in {'float16', 'float32', 'float64'}:
         raise ValueError('Unknown floatx type: ' + str(floatx))
@@ -86,7 +82,7 @@ def set_floatx(floatx):
 
 
 def cast_to_floatx(x):
-    '''Cast a Numpy array to the default Keras float type.
+    """Cast a Numpy array to the default Keras float type.
 
     # Arguments
         x: Numpy array.
@@ -108,78 +104,50 @@ def cast_to_floatx(x):
         >>> new_arr.dtype
         dtype('float32')
     ```
-    '''
+    """
     return np.asarray(x, dtype=_FLOATX)
 
 
-def image_dim_ordering():
-    '''Returns the default image dimension ordering
-    convention ('th' or 'tf').
+def image_data_format():
+    """Returns the default image data format
+    convention ('channels_first' or 'channels_last').
 
     # Returns
-        A string, either `'th'` or `'tf'`
+        A string, either `'channels_first'` or `'channels_last'`
 
     # Example
     ```python
-        >>> keras.backend.image_dim_ordering()
-        'th'
+        >>> keras.backend.image_data_format()
+        'channels_first'
     ```
-    '''
-    return _IMAGE_DIM_ORDERING
+    """
+    return _IMAGE_DATA_FORMAT
 
 
-def set_image_dim_ordering(dim_ordering):
-    '''Sets the value of the image dimension
-    ordering convention ('th' or 'tf').
+def set_image_data_format(data_format):
+    """Sets the value of the data format convention.
 
     # Arguments
-        dim_ordering: string. `'th'` or `'tf'`.
+        data_format: string. `'channels_first'` or `'channels_last'`.
 
     # Example
     ```python
         >>> from keras import backend as K
-        >>> K.image_dim_ordering()
-        'th'
-        >>> K.set_image_dim_ordering('tf')
-        >>> K.image_dim_ordering()
-        'tf'
+        >>> K.image_data_format()
+        'channels_first'
+        >>> K.set_image_data_format('channels_last')
+        >>> K.image_data_format()
+        'channels_last'
     ```
-    '''
-    global _IMAGE_DIM_ORDERING
-    if dim_ordering not in {'tf', 'th'}:
-        raise ValueError('Unknown dim_ordering:', dim_ordering)
-    _IMAGE_DIM_ORDERING = str(dim_ordering)
-
-
-def get_uid(prefix=''):
-    '''Provides a unique UID given a string prefix.
-
-    # Arguments
-        prefix: string.
-
-    # Returns
-        An integer.
-
-    # Example
-    ```
-        >>> keras.backend.get_uid('dense')
-        >>> 1
-        >>> keras.backend.get_uid('dense')
-        >>> 2
-    ```
-
-    '''
-    _UID_PREFIXES[prefix] += 1
-    return _UID_PREFIXES[prefix]
-
-
-def reset_uids():
-    global _UID_PREFIXES
-    _UID_PREFIXES = defaultdict(int)
+    """
+    global _IMAGE_DATA_FORMAT
+    if data_format not in {'channels_last', 'channels_first'}:
+        raise ValueError('Unknown data_format:', data_format)
+    _IMAGE_DATA_FORMAT = str(data_format)
 
 
 def is_keras_tensor(x):
-    '''Returns whether `x` is a Keras tensor.
+    """Returns whether `x` is a Keras tensor.
 
     # Arguments
         x: a potential tensor.
@@ -200,18 +168,45 @@ def is_keras_tensor(x):
         >>> K.is_keras_tensor(keras_placeholder)  # A placeholder is a Tensor.
         True
     ```
-    '''
+    """
     if hasattr(x, '_keras_shape'):
         return True
     else:
         return False
 
 
-def set_legacy_weight_ordering(value):
-    global _LEGACY_WEIGHT_ORDERING
-    assert value in {True, False}
-    _LEGACY_WEIGHT_ORDERING = value
+# Legacy methods
+
+def set_image_dim_ordering(dim_ordering):
+    """Sets the value of the image data format.
+
+    # Arguments
+        data_format: string. `'channels_first'` or `'channels_last'`.
+
+    # Example
+    ```python
+        >>> from keras import backend as K
+        >>> K.image_data_format()
+        'channels_first'
+        >>> K.set_image_data_format('channels_last')
+        >>> K.image_data_format()
+        'channels_last'
+    ```
+    """
+    global _IMAGE_DATA_FORMAT
+    if dim_ordering not in {'tf', 'th'}:
+        raise ValueError('Unknown dim_ordering:', dim_ordering)
+    if dim_ordering == 'th':
+        data_format = 'channels_first'
+    else:
+        data_format = 'channels_last'
+    _IMAGE_DATA_FORMAT = data_format
 
 
-def legacy_weight_ordering():
-    return _LEGACY_WEIGHT_ORDERING
+def image_dim_ordering():
+    """Legacy getter for data format.
+    """
+    if _IMAGE_DATA_FORMAT == 'channels_first':
+        return 'th'
+    else:
+        return 'tf'

keras/callbacks.py

@@ -9,53 +9,100 @@ import time
 import json
 import warnings
 
-from collections import deque, OrderedDict, Iterable
+from collections import deque
+from collections import OrderedDict
+from collections import Iterable
 from .utils.generic_utils import Progbar
-from keras import backend as K
-from pkg_resources import parse_version
+from . import backend as K
+
+try:
+    import requests
+except ImportError:
+    requests = None
+
+if K.backend() == 'tensorflow':
+    import tensorflow as tf
 
 
 class CallbackList(object):
-    def __init__(self, callbacks=[], queue_length=10):
+    """Container abstracting a list of callbacks.
+
+    # Arguments
+        callbacks: List of `Callback` instances.
+        queue_length: Queue length for keeping
+            running statistics over callback execution time.
+    """
+
+    def __init__(self, callbacks=None, queue_length=10):
+        callbacks = callbacks or []
         self.callbacks = [c for c in callbacks]
         self.queue_length = queue_length
 
     def append(self, callback):
         self.callbacks.append(callback)
 
-    def _set_params(self, params):
+    def set_params(self, params):
         for callback in self.callbacks:
-            callback._set_params(params)
+            callback.set_params(params)
 
-    def _set_model(self, model):
+    def set_model(self, model):
         for callback in self.callbacks:
-            callback._set_model(model)
+            callback.set_model(model)
 
-    def on_epoch_begin(self, epoch, logs={}):
+    def on_epoch_begin(self, epoch, logs=None):
+        """Called at the start of an epoch.
+
+        # Arguments
+            epoch: integer, index of epoch.
+            logs: dictionary of logs.
+        """
+        logs = logs or {}
         for callback in self.callbacks:
             callback.on_epoch_begin(epoch, logs)
         self._delta_t_batch = 0.
         self._delta_ts_batch_begin = deque([], maxlen=self.queue_length)
         self._delta_ts_batch_end = deque([], maxlen=self.queue_length)
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
+        """Called at the end of an epoch.
+
+        # Arguments
+            epoch: integer, index of epoch.
+            logs: dictionary of logs.
+        """
+        logs = logs or {}
         for callback in self.callbacks:
             callback.on_epoch_end(epoch, logs)
 
-    def on_batch_begin(self, batch, logs={}):
+    def on_batch_begin(self, batch, logs=None):
+        """Called right before processing a batch.
+
+        # Arguments
+            batch: integer, index of batch within the current epoch.
+            logs: dictionary of logs.
+        """
+        logs = logs or {}
         t_before_callbacks = time.time()
         for callback in self.callbacks:
             callback.on_batch_begin(batch, logs)
         self._delta_ts_batch_begin.append(time.time() - t_before_callbacks)
         delta_t_median = np.median(self._delta_ts_batch_begin)
-        if self._delta_t_batch > 0. and delta_t_median > 0.95 * \
-           self._delta_t_batch and delta_t_median > 0.1:
+        if (self._delta_t_batch > 0. and
+           delta_t_median > 0.95 * self._delta_t_batch and
+           delta_t_median > 0.1):
             warnings.warn('Method on_batch_begin() is slow compared '
                           'to the batch update (%f). Check your callbacks.'
                           % delta_t_median)
         self._t_enter_batch = time.time()
 
-    def on_batch_end(self, batch, logs={}):
+    def on_batch_end(self, batch, logs=None):
+        """Called at the end of a batch.
+
+        # Arguments
+            batch: integer, index of batch within the current epoch.
+            logs: dictionary of logs.
+        """
+        logs = logs or {}
         if not hasattr(self, '_t_enter_batch'):
             self._t_enter_batch = time.time()
         self._delta_t_batch = time.time() - self._t_enter_batch
@@ -64,22 +111,38 @@ class CallbackList(object):
             callback.on_batch_end(batch, logs)
         self._delta_ts_batch_end.append(time.time() - t_before_callbacks)
         delta_t_median = np.median(self._delta_ts_batch_end)
-        if self._delta_t_batch > 0. and (delta_t_median > 0.95 * self._delta_t_batch and delta_t_median > 0.1):
+        if (self._delta_t_batch > 0. and
+           (delta_t_median > 0.95 * self._delta_t_batch and delta_t_median > 0.1)):
             warnings.warn('Method on_batch_end() is slow compared '
                           'to the batch update (%f). Check your callbacks.'
                           % delta_t_median)
 
-    def on_train_begin(self, logs={}):
+    def on_train_begin(self, logs=None):
+        """Called at the beginning of training.
+
+        # Arguments
+            logs: dictionary of logs.
+        """
+        logs = logs or {}
         for callback in self.callbacks:
             callback.on_train_begin(logs)
 
-    def on_train_end(self, logs={}):
+    def on_train_end(self, logs=None):
+        """Called at the end of training.
+
+        # Arguments
+            logs: dictionary of logs.
+        """
+        logs = logs or {}
         for callback in self.callbacks:
             callback.on_train_end(logs)
 
+    def __iter__(self):
+        return iter(self.callbacks)
+
 
 class Callback(object):
-    '''Abstract base class used to build new callbacks.
+    """Abstract base class used to build new callbacks.
 
     # Properties
         params: dict. Training parameters
@@ -103,47 +166,48 @@ class Callback(object):
             the number of samples in the current batch.
         on_batch_end: logs include `loss`, and optionally `acc`
             (if accuracy monitoring is enabled).
-    '''
-    def __init__(self):
-        pass
+    """
 
-    def _set_params(self, params):
+    def __init__(self):
+        self.validation_data = None
+
+    def set_params(self, params):
         self.params = params
 
-    def _set_model(self, model):
+    def set_model(self, model):
         self.model = model
 
-    def on_epoch_begin(self, epoch, logs={}):
+    def on_epoch_begin(self, epoch, logs=None):
         pass
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
         pass
 
-    def on_batch_begin(self, batch, logs={}):
+    def on_batch_begin(self, batch, logs=None):
         pass
 
-    def on_batch_end(self, batch, logs={}):
+    def on_batch_end(self, batch, logs=None):
         pass
 
-    def on_train_begin(self, logs={}):
+    def on_train_begin(self, logs=None):
         pass
 
-    def on_train_end(self, logs={}):
+    def on_train_end(self, logs=None):
         pass
 
 
 class BaseLogger(Callback):
-    '''Callback that accumulates epoch averages of
-    the metrics being monitored.
+    """Callback that accumulates epoch averages of metrics.
 
-    This callback is automatically applied to
-    every Keras model.
-    '''
-    def on_epoch_begin(self, epoch, logs={}):
+    This callback is automatically applied to every Keras model.
+    """
+
+    def on_epoch_begin(self, epoch, logs=None):
         self.seen = 0
         self.totals = {}
 
-    def on_batch_end(self, batch, logs={}):
+    def on_batch_end(self, batch, logs=None):
+        logs = logs or {}
         batch_size = logs.get('size', 0)
         self.seen += batch_size
 
@@ -153,45 +217,74 @@ class BaseLogger(Callback):
             else:
                 self.totals[k] = v * batch_size
 
-    def on_epoch_end(self, epoch, logs={}):
-        for k in self.params['metrics']:
-            if k in self.totals:
-                # make value available to next callbacks
-                logs[k] = self.totals[k] / self.seen
+    def on_epoch_end(self, epoch, logs=None):
+        if logs is not None:
+            for k in self.params['metrics']:
+                if k in self.totals:
+                    # Make value available to next callbacks.
+                    logs[k] = self.totals[k] / self.seen
 
 
 class ProgbarLogger(Callback):
-    '''Callback that prints metrics to stdout.
-    '''
-    def on_train_begin(self, logs={}):
-        self.verbose = self.params['verbose']
-        self.nb_epoch = self.params['nb_epoch']
+    """Callback that prints metrics to stdout.
 
-    def on_epoch_begin(self, epoch, logs={}):
+    # Arguments
+        count_mode: One of "steps" or "samples".
+            Whether the progress bar should
+            count samples seens or steps (batches) seen.
+
+    # Raises
+        ValueError: In case of invalid `count_mode`.
+    """
+
+    def __init__(self, count_mode='samples'):
+        super(ProgbarLogger, self).__init__()
+        if count_mode == 'samples':
+            self.use_steps = False
+        elif count_mode == 'steps':
+            self.use_steps = True
+        else:
+            raise ValueError('Unknown `count_mode`: ' + str(count_mode))
+
+    def on_train_begin(self, logs=None):
+        self.verbose = self.params['verbose']
+        self.epochs = self.params['epochs']
+
+    def on_epoch_begin(self, epoch, logs=None):
         if self.verbose:
-            print('Epoch %d/%d' % (epoch + 1, self.nb_epoch))
-            self.progbar = Progbar(target=self.params['nb_sample'],
+            print('Epoch %d/%d' % (epoch + 1, self.epochs))
+            if self.use_steps:
+                target = self.params['steps']
+            else:
+                target = self.params['samples']
+            self.target = target
+            self.progbar = Progbar(target=self.target,
                                    verbose=self.verbose)
         self.seen = 0
 
-    def on_batch_begin(self, batch, logs={}):
-        if self.seen < self.params['nb_sample']:
+    def on_batch_begin(self, batch, logs=None):
+        if self.seen < self.target:
             self.log_values = []
 
-    def on_batch_end(self, batch, logs={}):
+    def on_batch_end(self, batch, logs=None):
+        logs = logs or {}
         batch_size = logs.get('size', 0)
-        self.seen += batch_size
+        if self.use_steps:
+            self.seen += 1
+        else:
+            self.seen += batch_size
 
         for k in self.params['metrics']:
             if k in logs:
                 self.log_values.append((k, logs[k]))
 
-        # skip progbar update for the last batch;
-        # will be handled by on_epoch_end
-        if self.verbose and self.seen < self.params['nb_sample']:
+        # Skip progbar update for the last batch;
+        # will be handled by on_epoch_end.
+        if self.verbose and self.seen < self.target:
             self.progbar.update(self.seen, self.log_values)
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
         for k in self.params['metrics']:
             if k in logs:
                 self.log_values.append((k, logs[k]))
@@ -200,33 +293,34 @@ class ProgbarLogger(Callback):
 
 
 class History(Callback):
-    '''Callback that records events
-    into a `History` object.
+    """Callback that records events into a `History` object.
 
     This callback is automatically applied to
     every Keras model. The `History` object
     gets returned by the `fit` method of models.
-    '''
-    def on_train_begin(self, logs={}):
+    """
+
+    def on_train_begin(self, logs=None):
         self.epoch = []
         self.history = {}
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
         self.epoch.append(epoch)
         for k, v in logs.items():
             self.history.setdefault(k, []).append(v)
 
 
 class ModelCheckpoint(Callback):
-    '''Save the model after every epoch.
+    """Save the model after every epoch.
 
     `filepath` can contain named formatting options,
     which will be filled the value of `epoch` and
     keys in `logs` (passed in `on_epoch_end`).
 
     For example: if `filepath` is `weights.{epoch:02d}-{val_loss:.2f}.hdf5`,
-    then multiple files will be save with the epoch number and
-    the validation loss.
+    then the model checkpoints will be saved with the epoch number and
+    the validation loss in the filename.
 
     # Arguments
         filepath: string, path to save the model file.
@@ -247,8 +341,8 @@ class ModelCheckpoint(Callback):
             saved (`model.save_weights(filepath)`), else the full model
             is saved (`model.save(filepath)`).
         period: Interval (number of epochs) between checkpoints.
+    """
 
-    '''
     def __init__(self, filepath, monitor='val_loss', verbose=0,
                  save_best_only=False, save_weights_only=False,
                  mode='auto', period=1):
@@ -274,14 +368,15 @@ class ModelCheckpoint(Callback):
             self.monitor_op = np.greater
             self.best = -np.Inf
         else:
-            if 'acc' in self.monitor:
+            if 'acc' in self.monitor or self.monitor.startswith('fmeasure'):
                 self.monitor_op = np.greater
                 self.best = -np.Inf
             else:
                 self.monitor_op = np.less
                 self.best = np.Inf
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
         self.epochs_since_last_save += 1
         if self.epochs_since_last_save >= self.period:
             self.epochs_since_last_save = 0
@@ -317,7 +412,7 @@ class ModelCheckpoint(Callback):
 
 
 class EarlyStopping(Callback):
-    '''Stop training when a monitored quantity has stopped improving.
+    """Stop training when a monitored quantity has stopped improving.
 
     # Arguments
         monitor: quantity to be monitored.
@@ -335,8 +430,10 @@ class EarlyStopping(Callback):
             monitored has stopped increasing; in `auto`
             mode, the direction is automatically inferred
             from the name of the monitored quantity.
-    '''
-    def __init__(self, monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto'):
+    """
+
+    def __init__(self, monitor='val_loss',
+                 min_delta=0, patience=0, verbose=0, mode='auto'):
         super(EarlyStopping, self).__init__()
 
         self.monitor = monitor
@@ -357,7 +454,7 @@ class EarlyStopping(Callback):
         elif mode == 'max':
             self.monitor_op = np.greater
         else:
-            if 'acc' in self.monitor:
+            if 'acc' in self.monitor or self.monitor.startswith('fmeasure'):
                 self.monitor_op = np.greater
             else:
                 self.monitor_op = np.less
@@ -367,11 +464,11 @@ class EarlyStopping(Callback):
         else:
             self.min_delta *= -1
 
-    def on_train_begin(self, logs={}):
-        self.wait = 0       # Allow instances to be re-used
+    def on_train_begin(self, logs=None):
+        self.wait = 0  # Allow instances to be re-used
         self.best = np.Inf if self.monitor_op == np.less else -np.Inf
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
         current = logs.get(self.monitor)
         if current is None:
             warnings.warn('Early stopping requires %s available!' %
@@ -386,37 +483,48 @@ class EarlyStopping(Callback):
                 self.model.stop_training = True
             self.wait += 1
 
-    def on_train_end(self, logs={}):
+    def on_train_end(self, logs=None):
         if self.stopped_epoch > 0 and self.verbose > 0:
             print('Epoch %05d: early stopping' % (self.stopped_epoch))
 
 
 class RemoteMonitor(Callback):
-    '''Callback used to stream events to a server.
+    """Callback used to stream events to a server.
 
     Requires the `requests` library.
+    Events are sent to `root + '/publish/epoch/end/'` by default. Calls are
+    HTTP POST, with a `data` argument which is a
+    JSON-encoded dictionary of event data.
 
     # Arguments
-        root: root url to which the events will be sent (at the end
-            of every epoch). Events are sent to
-            `root + '/publish/epoch/end/'` by default. Calls are
-            HTTP POST, with a `data` argument which is a
-            JSON-encoded dictionary of event data.
-    '''
+        root: String; root url of the target server.
+        path: String; path relative to `root` to which the events will be sent.
+        field: String; JSON field under which the data will be stored.
+        headers: Dictionary; optional custom HTTP headers.
+            Defaults to:
+            `{'Accept': 'application/json',
+              'Content-Type': 'application/json'}`
+    """
 
     def __init__(self,
                  root='http://localhost:9000',
                  path='/publish/epoch/end/',
                  field='data',
-                 headers={'Accept': 'application/json', 'Content-Type': 'application/json'}):
+                 headers=None):
         super(RemoteMonitor, self).__init__()
+        if headers is None:
+            headers = {'Accept': 'application/json',
+                       'Content-Type': 'application/json'}
         self.root = root
         self.path = path
         self.field = field
         self.headers = headers
 
-    def on_epoch_end(self, epoch, logs={}):
-        import requests
+    def on_epoch_end(self, epoch, logs=None):
+        if requests is None:
+            raise ImportError('RemoteMonitor requires '
+                              'the `requests` library.')
+        logs = logs or {}
         send = {}
         send['epoch'] = epoch
         for k, v in logs.items():
@@ -425,37 +533,36 @@ class RemoteMonitor(Callback):
             requests.post(self.root + self.path,
                           {self.field: json.dumps(send)},
                           headers=self.headers)
-        except:
-            print('Warning: could not reach RemoteMonitor '
-                  'root server at ' + str(self.root))
+        except requests.exceptions.RequestException:
+            warnings.warn('Warning: could not reach RemoteMonitor '
+                          'root server at ' + str(self.root))
 
 
 class LearningRateScheduler(Callback):
-    '''Learning rate scheduler.
+    """Learning rate scheduler.
 
     # Arguments
         schedule: a function that takes an epoch index as input
             (integer, indexed from 0) and returns a new
             learning rate as output (float).
-    '''
+    """
+
     def __init__(self, schedule):
         super(LearningRateScheduler, self).__init__()
         self.schedule = schedule
 
-    def on_epoch_begin(self, epoch, logs={}):
-        assert hasattr(self.model.optimizer, 'lr'), \
-            'Optimizer must have a "lr" attribute.'
+    def on_epoch_begin(self, epoch, logs=None):
+        if not hasattr(self.model.optimizer, 'lr'):
+            raise ValueError('Optimizer must have a "lr" attribute.')
         lr = self.schedule(epoch)
-
         if not isinstance(lr, (float, np.float32, np.float64)):
             raise ValueError('The output of the "schedule" function '
                              'should be float.')
-
         K.set_value(self.model.optimizer.lr, lr)
 
 
 class TensorBoard(Callback):
-    ''' Tensorboard basic visualizations.
+    """Tensorboard basic visualizations.
 
     This callback writes a log for TensorBoard, which allows
     you to visualize dynamic graphs of your training and test
@@ -481,11 +588,14 @@ class TensorBoard(Callback):
         write_graph: whether to visualize the graph in Tensorboard.
             The log file can become quite large when
             write_graph is set to True.
-    '''
+    """
 
-    def __init__(self, log_dir='./logs', histogram_freq=0, write_graph=True, write_images=False):
+    def __init__(self, log_dir='./logs',
+                 histogram_freq=0,
+                 write_graph=True,
+                 write_images=False):
         super(TensorBoard, self).__init__()
-        if K._BACKEND != 'tensorflow':
+        if K.backend() != 'tensorflow':
             raise RuntimeError('TensorBoard callback only works '
                                'with the TensorFlow backend.')
         self.log_dir = log_dir
@@ -494,68 +604,48 @@ class TensorBoard(Callback):
         self.write_graph = write_graph
         self.write_images = write_images
 
-    def _set_model(self, model):
-        import tensorflow as tf
-        import keras.backend.tensorflow_backend as KTF
-
+    def set_model(self, model):
         self.model = model
-        self.sess = KTF.get_session()
+        self.sess = K.get_session()
         if self.histogram_freq and self.merged is None:
             for layer in self.model.layers:
 
                 for weight in layer.weights:
-                    tf.histogram_summary(weight.name, weight)
-
+                    tf.summary.histogram(weight.name, weight)
                     if self.write_images:
                         w_img = tf.squeeze(weight)
-
                         shape = w_img.get_shape()
                         if len(shape) > 1 and shape[0] > shape[1]:
                             w_img = tf.transpose(w_img)
-
                         if len(shape) == 1:
                             w_img = tf.expand_dims(w_img, 0)
-
                         w_img = tf.expand_dims(tf.expand_dims(w_img, 0), -1)
-
-                        tf.image_summary(weight.name, w_img)
+                        tf.summary.image(weight.name, w_img)
 
                 if hasattr(layer, 'output'):
-                    tf.histogram_summary('{}_out'.format(layer.name),
+                    tf.summary.histogram('{}_out'.format(layer.name),
                                          layer.output)
-        if parse_version(tf.__version__) >= parse_version('0.12.0'):
-            self.merged = tf.summary.merge_all()
-        else:
-            self.merged = tf.merge_all_summaries()
+        self.merged = tf.summary.merge_all()
+
         if self.write_graph:
-            if parse_version(tf.__version__) >= parse_version('0.12.0'):
-                self.writer = tf.summary.FileWriter(self.log_dir,
-                                                    self.sess.graph)
-            elif parse_version(tf.__version__) >= parse_version('0.8.0'):
-                self.writer = tf.train.SummaryWriter(self.log_dir,
-                                                     self.sess.graph)
-            else:
-                self.writer = tf.train.SummaryWriter(self.log_dir,
-                                                     self.sess.graph_def)
+            self.writer = tf.summary.FileWriter(self.log_dir,
+                                                self.sess.graph)
         else:
-            if parse_version(tf.__version__) >= parse_version('0.12.0'):
-                self.writer = tf.summary.FileWriter(self.log_dir)
-            else:
-                self.writer = tf.train.SummaryWriter(self.log_dir)
+            self.writer = tf.summary.FileWriter(self.log_dir)
 
-    def on_epoch_end(self, epoch, logs={}):
-        import tensorflow as tf
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
 
-        if self.model.validation_data and self.histogram_freq:
+        if self.validation_data and self.histogram_freq:
             if epoch % self.histogram_freq == 0:
                 # TODO: implement batched calls to sess.run
                 # (current call will likely go OOM on GPU)
                 if self.model.uses_learning_phase:
                     cut_v_data = len(self.model.inputs)
-                    val_data = self.model.validation_data[:cut_v_data] + [0]
+                    val_data = self.validation_data[:cut_v_data] + [0]
                     tensors = self.model.inputs + [K.learning_phase()]
                 else:
-                    val_data = self.model.validation_data
+                    val_data = self.validation_data
                     tensors = self.model.inputs
                 feed_dict = dict(zip(tensors, val_data))
                 result = self.sess.run([self.merged], feed_dict=feed_dict)
@@ -577,7 +667,7 @@ class TensorBoard(Callback):
 
 
 class ReduceLROnPlateau(Callback):
-    '''Reduce learning rate when a metric has stopped improving.
+    """Reduce learning rate when a metric has stopped improving.
 
     Models often benefit from reducing the learning rate by a factor
     of 2-10 once learning stagnates. This callback monitors a
@@ -610,15 +700,16 @@ class ReduceLROnPlateau(Callback):
         cooldown: number of epochs to wait before resuming
             normal operation after lr has been reduced.
         min_lr: lower bound on the learning rate.
-    '''
+    """
 
     def __init__(self, monitor='val_loss', factor=0.1, patience=10,
                  verbose=0, mode='auto', epsilon=1e-4, cooldown=0, min_lr=0):
-        super(Callback, self).__init__()
+        super(ReduceLROnPlateau, self).__init__()
 
         self.monitor = monitor
         if factor >= 1.0:
-            raise ValueError('ReduceLROnPlateau does not support a factor >= 1.0.')
+            raise ValueError('ReduceLROnPlateau '
+                             'does not support a factor >= 1.0.')
         self.factor = factor
         self.min_lr = min_lr
         self.epsilon = epsilon
@@ -630,14 +721,18 @@ class ReduceLROnPlateau(Callback):
         self.best = 0
         self.mode = mode
         self.monitor_op = None
-        self.reset()
+        self._reset()
 
-    def reset(self):
+    def _reset(self):
+        """Resets wait counter and cooldown counter.
+        """
         if self.mode not in ['auto', 'min', 'max']:
             warnings.warn('Learning Rate Plateau Reducing mode %s is unknown, '
-                          'fallback to auto mode.' % (self.mode), RuntimeWarning)
+                          'fallback to auto mode.' % (self.mode),
+                          RuntimeWarning)
             self.mode = 'auto'
-        if self.mode == 'min' or (self.mode == 'auto' and 'acc' not in self.monitor):
+        if (self.mode == 'min' or
+           (self.mode == 'auto' and 'acc' not in self.monitor)):
             self.monitor_op = lambda a, b: np.less(a, b - self.epsilon)
             self.best = np.Inf
         else:
@@ -647,10 +742,11 @@ class ReduceLROnPlateau(Callback):
         self.wait = 0
         self.lr_epsilon = self.min_lr * 1e-4
 
-    def on_train_begin(self, logs={}):
-        self.reset()
+    def on_train_begin(self, logs=None):
+        self._reset()
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
         logs['lr'] = K.get_value(self.model.optimizer.lr)
         current = logs.get(self.monitor)
         if current is None:
@@ -682,7 +778,8 @@ class ReduceLROnPlateau(Callback):
 
 
 class CSVLogger(Callback):
-    '''Callback that streams epoch results to a csv file.
+    """Callback that streams epoch results to a csv file.
+
     Supports all values that can be represented as a string,
     including 1D iterables such as np.ndarray.
 
@@ -697,7 +794,7 @@ class CSVLogger(Callback):
         separator: string used to separate elements in the csv file.
         append: True: append if file exists (useful for continuing
             training). False: overwrite existing file,
-    '''
+    """
 
     def __init__(self, filename, separator=',', append=False):
         self.sep = separator
@@ -708,26 +805,33 @@ class CSVLogger(Callback):
         self.append_header = True
         super(CSVLogger, self).__init__()
 
-    def on_train_begin(self, logs={}):
+    def on_train_begin(self, logs=None):
         if self.append:
             if os.path.exists(self.filename):
                 with open(self.filename) as f:
-                    self.append_header = len(f.readline()) == 0
+                    self.append_header = not bool(len(f.readline()))
             self.csv_file = open(self.filename, 'a')
         else:
             self.csv_file = open(self.filename, 'w')
 
-    def on_epoch_end(self, epoch, logs={}):
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
+
         def handle_value(k):
             is_zero_dim_ndarray = isinstance(k, np.ndarray) and k.ndim == 0
             if isinstance(k, Iterable) and not is_zero_dim_ndarray:
-                return '"[%s]"' % (', '.join(map(lambda x: str(x), k)))
+                return '"[%s]"' % (', '.join(map(str, k)))
             else:
                 return k
 
         if not self.writer:
             self.keys = sorted(logs.keys())
-            self.writer = csv.DictWriter(self.csv_file, fieldnames=['epoch'] + self.keys)
+
+            class CustomDialect(csv.excel):
+                delimiter = self.sep
+
+            self.writer = csv.DictWriter(self.csv_file,
+                                         fieldnames=['epoch'] + self.keys, dialect=CustomDialect)
             if self.append_header:
                 self.writer.writeheader()
 
@@ -736,8 +840,9 @@ class CSVLogger(Callback):
         self.writer.writerow(row_dict)
         self.csv_file.flush()
 
-    def on_train_end(self, logs={}):
+    def on_train_end(self, logs=None):
         self.csv_file.close()
+        self.writer = None
 
 
 class LambdaCallback(Callback):
@@ -746,9 +851,12 @@ class LambdaCallback(Callback):
     This callback is constructed with anonymous functions that will be called
     at the appropriate time. Note that the callbacks expects positional
     arguments, as:
-     - `on_epoch_begin` and `on_epoch_end` expect two positional arguments: `epoch`, `logs`
-     - `on_batch_begin` and `on_batch_end` expect two positional arguments: `batch`, `logs`
-     - `on_train_begin` and `on_train_end` expect one positional argument: `logs`
+     - `on_epoch_begin` and `on_epoch_end` expect two positional arguments:
+        `epoch`, `logs`
+     - `on_batch_begin` and `on_batch_end` expect two positional arguments:
+        `batch`, `logs`
+     - `on_train_begin` and `on_train_end` expect one positional argument:
+        `logs`
 
     # Arguments
         on_epoch_begin: called at the beginning of every epoch.
@@ -761,20 +869,27 @@ class LambdaCallback(Callback):
     # Example
         ```python
         # Print the batch number at the beginning of every batch.
-        batch_print_callback = LambdaCallback(on_batch_begin=lambda batch, logs: print(batch))
+        batch_print_callback = LambdaCallback(
+            on_batch_begin=lambda batch,logs: print(batch))
 
         # Plot the loss after every epoch.
         import numpy as np
         import matplotlib.pyplot as plt
-        plot_loss_callback = LambdaCallback(on_epoch_end=lambda epoch, logs: plt.plot(np.arange(epoch), logs['loss']))
+        plot_loss_callback = LambdaCallback(
+            on_epoch_end=lambda epoch, logs: plt.plot(np.arange(epoch),
+                                                      logs['loss']))
 
         # Terminate some processes after having finished model training.
         processes = ...
-        cleanup_callback = LambdaCallback(on_train_end=lambda logs: [p.terminate() for p in processes if p.is_alive()])
+        cleanup_callback = LambdaCallback(
+            on_train_end=lambda logs: [
+                p.terminate() for p in processes if p.is_alive()])
 
-        model.fit(..., callbacks=[batch_print_callback, plot_loss_callback, cleanup_callback])
+        model.fit(...,
+                  callbacks=[batch_print_callback,
+                             plot_loss_callback,
+                             cleanup_callback])
         ```
-
     """
 
     def __init__(self,
@@ -785,11 +900,29 @@ class LambdaCallback(Callback):
                  on_train_begin=None,
                  on_train_end=None,
                  **kwargs):
-        super(Callback, self).__init__()
+        super(LambdaCallback, self).__init__()
         self.__dict__.update(kwargs)
-        self.on_epoch_begin = on_epoch_begin if on_epoch_begin else lambda epoch, logs: None
-        self.on_epoch_end = on_epoch_end if on_epoch_end else lambda epoch, logs: None
-        self.on_batch_begin = on_batch_begin if on_batch_begin else lambda batch, logs: None
-        self.on_batch_end = on_batch_end if on_batch_end else lambda batch, logs: None
-        self.on_train_begin = on_train_begin if on_train_begin else lambda logs: None
-        self.on_train_end = on_train_end if on_train_end else lambda logs: None
+        if on_epoch_begin is not None:
+            self.on_epoch_begin = on_epoch_begin
+        else:
+            self.on_epoch_begin = lambda epoch, logs: None
+        if on_epoch_end is not None:
+            self.on_epoch_end = on_epoch_end
+        else:
+            self.on_epoch_end = lambda epoch, logs: None
+        if on_batch_begin is not None:
+            self.on_batch_begin = on_batch_begin
+        else:
+            self.on_batch_begin = lambda batch, logs: None
+        if on_batch_end is not None:
+            self.on_batch_end = on_batch_end
+        else:
+            self.on_batch_end = lambda batch, logs: None
+        if on_train_begin is not None:
+            self.on_train_begin = on_train_begin
+        else:
+            self.on_train_begin = lambda logs: None
+        if on_train_end is not None:
+            self.on_train_end = on_train_end
+        else:
+            self.on_train_end = lambda logs: None

keras/constraints.py

@@ -1,93 +1,180 @@
 from __future__ import absolute_import
+import six
 from . import backend as K
+from .utils.generic_utils import serialize_keras_object
+from .utils.generic_utils import deserialize_keras_object
 
 
 class Constraint(object):
-    def __call__(self, p):
-        return p
+
+    def __call__(self, w):
+        return w
 
     def get_config(self):
-        return {'name': self.__class__.__name__}
+        return {}
 
 
 class MaxNorm(Constraint):
-    '''Constrain the weights incident to each hidden unit to have a norm less than or equal to a desired value.
+    """MaxNorm weight constraint.
+
+    Constrains the weights incident to each hidden unit
+    to have a norm less than or equal to a desired value.
 
     # Arguments
         m: the maximum norm for the incoming weights.
-        axis: integer, axis along which to calculate weight norms. For instance,
-            in a `Dense` layer the weight matrix has shape (input_dim, output_dim),
-            set `axis` to `0` to constrain each weight vector of length (input_dim).
-            In a `MaxoutDense` layer the weight tensor has shape (nb_feature, input_dim, output_dim),
-            set `axis` to `1` to constrain each weight vector of length (input_dim),
-            i.e. constrain the filters incident to the `max` operation.
-            In a `Convolution2D` layer with the Theano backend, the weight tensor
-            has shape (nb_filter, stack_size, nb_row, nb_col), set `axis` to `[1,2,3]`
-            to constrain the weights of each filter tensor of size (stack_size, nb_row, nb_col).
-            In a `Convolution2D` layer with the TensorFlow backend, the weight tensor
-            has shape (nb_row, nb_col, stack_size, nb_filter), set `axis` to `[0,1,2]`
-            to constrain the weights of each filter tensor of size (nb_row, nb_col, stack_size).
+        axis: integer, axis along which to calculate weight norms.
+            For instance, in a `Dense` layer the weight matrix
+            has shape `(input_dim, output_dim)`,
+            set `axis` to `0` to constrain each weight vector
+            of length `(input_dim,)`.
+            In a `Convolution2D` layer with `data_format="channels_last"`,
+            the weight tensor has shape
+            `(rows, cols, input_depth, output_depth)`,
+            set `axis` to `[0, 1, 2]`
+            to constrain the weights of each filter tensor of size
+            `(rows, cols, input_depth)`.
 
     # References
         - [Dropout: A Simple Way to Prevent Neural Networks from Overfitting Srivastava, Hinton, et al. 2014](http://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf)
-    '''
-    def __init__(self, m=2, axis=0):
-        self.m = m
+    """
+
+    def __init__(self, max_value=2, axis=0):
+        self.max_value = max_value
         self.axis = axis
 
-    def __call__(self, p):
-        norms = K.sqrt(K.sum(K.square(p), axis=self.axis, keepdims=True))
-        desired = K.clip(norms, 0, self.m)
-        p = p * (desired / (K.epsilon() + norms))
-        return p
+    def __call__(self, w):
+        norms = K.sqrt(K.sum(K.square(w), axis=self.axis, keepdims=True))
+        desired = K.clip(norms, 0, self.max_value)
+        w *= (desired / (K.epsilon() + norms))
+        return w
 
     def get_config(self):
-        return {'name': self.__class__.__name__,
-                'm': self.m,
+        return {'max_value': self.max_value,
                 'axis': self.axis}
 
 
 class NonNeg(Constraint):
-    '''Constrain the weights to be non-negative.
-    '''
-    def __call__(self, p):
-        p *= K.cast(p >= 0., K.floatx())
-        return p
+    """Constrains the weights to be non-negative.
+    """
+
+    def __call__(self, w):
+        w *= K.cast(w >= 0., K.floatx())
+        return w
 
 
 class UnitNorm(Constraint):
-    '''Constrain the weights incident to each hidden unit to have unit norm.
+    """Constrains the weights incident to each hidden unit to have unit norm.
 
     # Arguments
-        axis: integer, axis along which to calculate weight norms. For instance,
-            in a `Dense` layer the weight matrix has shape (input_dim, output_dim),
-            set `axis` to `0` to constrain each weight vector of length (input_dim).
-            In a `MaxoutDense` layer the weight tensor has shape (nb_feature, input_dim, output_dim),
-            set `axis` to `1` to constrain each weight vector of length (input_dim),
-            i.e. constrain the filters incident to the `max` operation.
-            In a `Convolution2D` layer with the Theano backend, the weight tensor
-            has shape (nb_filter, stack_size, nb_row, nb_col), set `axis` to `[1,2,3]`
-            to constrain the weights of each filter tensor of size (stack_size, nb_row, nb_col).
-            In a `Convolution2D` layer with the TensorFlow backend, the weight tensor
-            has shape (nb_row, nb_col, stack_size, nb_filter), set `axis` to `[0,1,2]`
-            to constrain the weights of each filter tensor of size (nb_row, nb_col, stack_size).
-    '''
+        axis: integer, axis along which to calculate weight norms.
+            For instance, in a `Dense` layer the weight matrix
+            has shape `(input_dim, output_dim)`,
+            set `axis` to `0` to constrain each weight vector
+            of length `(input_dim,)`.
+            In a `Convolution2D` layer with `data_format="channels_last"`,
+            the weight tensor has shape
+            `(rows, cols, input_depth, output_depth)`,
+            set `axis` to `[0, 1, 2]`
+            to constrain the weights of each filter tensor of size
+            `(rows, cols, input_depth)`.
+    """
+
     def __init__(self, axis=0):
         self.axis = axis
 
-    def __call__(self, p):
-        return p / (K.epsilon() + K.sqrt(K.sum(K.square(p), axis=self.axis, keepdims=True)))
+    def __call__(self, w):
+        return w / (K.epsilon() + K.sqrt(K.sum(K.square(w),
+                                               axis=self.axis,
+                                               keepdims=True)))
 
     def get_config(self):
-        return {'name': self.__class__.__name__,
+        return {'axis': self.axis}
+
+
+class MinMaxNorm(Constraint):
+    """MinMaxNorm weight constraint.
+
+    Constrains the weights incident to each hidden unit
+    to have the norm between a lower bound and an upper bound.
+
+    # Arguments
+        min_value: the minimum norm for the incoming weights.
+        max_value: the maximum norm for the incoming weights.
+        rate: rate for enforcing the constraint: weights will be
+            rescaled to yield
+            `(1 - rate) * norm + rate * norm.clip(min_value, max_value)`.
+            Effectively, this means that rate=1.0 stands for strict
+            enforcement of the constraint, while rate<1.0 means that
+            weights will be rescaled at each step to slowly move
+            towards a value inside the desired interval.
+        axis: integer, axis along which to calculate weight norms.
+            For instance, in a `Dense` layer the weight matrix
+            has shape `(input_dim, output_dim)`,
+            set `axis` to `0` to constrain each weight vector
+            of length `(input_dim,)`.
+            In a `Convolution2D` layer with `dim_ordering="tf"`,
+            the weight tensor has shape
+            `(rows, cols, input_depth, output_depth)`,
+            set `axis` to `[0, 1, 2]`
+            to constrain the weights of each filter tensor of size
+            `(rows, cols, input_depth)`.
+    """
+
+    def __init__(self, min_value=0.0, max_value=1.0, rate=1.0, axis=0):
+        self.min_value = min_value
+        self.max_value = max_value
+        self.rate = rate
+        self.axis = axis
+
+    def __call__(self, w):
+        norms = K.sqrt(K.sum(K.square(w), axis=self.axis, keepdims=True))
+        desired = (self.rate * K.clip(norms, self.min_value, self.max_value) +
+                   (1 - self.rate) * norms)
+        w *= (desired / (K.epsilon() + norms))
+        return w
+
+    def get_config(self):
+        return {'min_value': self.min_value,
+                'max_value': self.max_value,
+                'rate': self.rate,
                 'axis': self.axis}
 
 
-maxnorm = MaxNorm
-nonneg = NonNeg
-unitnorm = UnitNorm
+# Aliases.
 
-from .utils.generic_utils import get_from_module
-def get(identifier, kwargs=None):
-    return get_from_module(identifier, globals(), 'constraint',
-                           instantiate=True, kwargs=kwargs)
+max_norm = MaxNorm
+non_neg = NonNeg
+unit_norm = UnitNorm
+min_max_norm = MinMaxNorm
+
+
+# Legacy aliases.
+maxnorm = max_norm
+nonneg = non_neg
+unitnorm = unit_norm
+
+
+def serialize(constraint):
+    return serialize_keras_object(constraint)
+
+
+def deserialize(config, custom_objects=None):
+    return deserialize_keras_object(config,
+                                    module_objects=globals(),
+                                    custom_objects=custom_objects,
+                                    printable_module_name='constraint')
+
+
+def get(identifier):
+    if identifier is None:
+        return None
+    if isinstance(identifier, dict):
+        return deserialize(identifier)
+    elif isinstance(identifier, six.string_types):
+        config = {'class_name': str(identifier), 'config': {}}
+        return deserialize(config)
+    elif callable(identifier):
+        return identifier
+    else:
+        raise ValueError('Could not interpret constraint identifier:',
+                         identifier)

keras/datasets/__init__.py

@@ -0,0 +1,8 @@
+from __future__ import absolute_import
+
+from . import mnist
+from . import imdb
+from . import reuters
+from . import cifar10
+from . import cifar100
+from . import boston_housing

keras/datasets/boston_housing.py

@@ -0,0 +1,34 @@
+from ..utils.data_utils import get_file
+import numpy as np
+
+
+def load_data(path='boston_housing.npz', seed=113, test_split=0.2):
+    """Loads the Boston Housing dataset.
+
+    # Arguments
+        path: path where to cache the dataset locally
+            (relative to ~/.keras/datasets).
+        seed: Random seed for shuffling the data
+            before computing the test split.
+        test_split: fraction of the data to reserve as test set.
+
+    # Returns
+        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
+    """
+    assert 0 <= test_split < 1
+    path = get_file(path, origin='https://s3.amazonaws.com/keras-datasets/boston_housing.npz')
+    f = np.load(path)
+    x = f['x']
+    y = f['y']
+    f.close()
+
+    np.random.seed(seed)
+    np.random.shuffle(x)
+    np.random.seed(seed)
+    np.random.shuffle(y)
+
+    x_train = np.array(x[:int(len(x) * (1 - test_split))])
+    y_train = np.array(y[:int(len(x) * (1 - test_split))])
+    x_test = np.array(x[int(len(x) * (1 - test_split)):])
+    y_test = np.array(y[int(len(x) * (1 - test_split)):])
+    return (x_train, y_train), (x_test, y_test)

keras/datasets/cifar.py

@@ -5,18 +5,28 @@ from six.moves import cPickle
 
 
 def load_batch(fpath, label_key='labels'):
+    """Internal utility for parsing CIFAR data.
+
+    # Arguments
+        fpath: path the file to parse.
+        label_key: key for label data in the retrieve
+            dictionary.
+
+    # Returns
+        A tuple `(data, labels)`.
+    """
     f = open(fpath, 'rb')
     if sys.version_info < (3,):
         d = cPickle.load(f)
     else:
-        d = cPickle.load(f, encoding="bytes")
+        d = cPickle.load(f, encoding='bytes')
         # decode utf8
         d_decoded = {}
         for k, v in d.items():
-            d_decoded[k.decode("utf8")] = v
+            d_decoded[k.decode('utf8')] = v
         d = d_decoded
     f.close()
-    data = d["data"]
+    data = d['data']
     labels = d[label_key]
 
     data = data.reshape(data.shape[0], 3, 32, 32)

keras/datasets/cifar10.py

@@ -7,29 +7,34 @@ import os
 
 
 def load_data():
-    dirname = "cifar-10-batches-py"
-    origin = "http://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
+    """Loads CIFAR10 dataset.
+
+    # Returns
+        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
+    """
+    dirname = 'cifar-10-batches-py'
+    origin = 'http://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz'
     path = get_file(dirname, origin=origin, untar=True)
 
-    nb_train_samples = 50000
+    num_train_samples = 50000
 
-    X_train = np.zeros((nb_train_samples, 3, 32, 32), dtype="uint8")
-    y_train = np.zeros((nb_train_samples,), dtype="uint8")
+    x_train = np.zeros((num_train_samples, 3, 32, 32), dtype='uint8')
+    y_train = np.zeros((num_train_samples,), dtype='uint8')
 
     for i in range(1, 6):
         fpath = os.path.join(path, 'data_batch_' + str(i))
         data, labels = load_batch(fpath)
-        X_train[(i - 1) * 10000: i * 10000, :, :, :] = data
+        x_train[(i - 1) * 10000: i * 10000, :, :, :] = data
         y_train[(i - 1) * 10000: i * 10000] = labels
 
     fpath = os.path.join(path, 'test_batch')
-    X_test, y_test = load_batch(fpath)
+    x_test, y_test = load_batch(fpath)
 
     y_train = np.reshape(y_train, (len(y_train), 1))
     y_test = np.reshape(y_test, (len(y_test), 1))
 
-    if K.image_dim_ordering() == 'tf':
-        X_train = X_train.transpose(0, 2, 3, 1)
-        X_test = X_test.transpose(0, 2, 3, 1)
+    if K.image_data_format() == 'channels_last':
+        x_train = x_train.transpose(0, 2, 3, 1)
+        x_test = x_test.transpose(0, 2, 3, 1)
 
-    return (X_train, y_train), (X_test, y_test)
+    return (x_train, y_train), (x_test, y_test)

keras/datasets/cifar100.py

@@ -7,6 +7,17 @@ import os
 
 
 def load_data(label_mode='fine'):
+    """Loads CIFAR100 dataset.
+
+    # Arguments
+        label_mode: one of "fine", "coarse".
+
+    # Returns
+        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
+
+    # Raises
+        ValueError: in case of invalid `label_mode`.
+    """
     if label_mode not in ['fine', 'coarse']:
         raise ValueError('label_mode must be one of "fine" "coarse".')
 
@@ -15,16 +26,16 @@ def load_data(label_mode='fine'):
     path = get_file(dirname, origin=origin, untar=True)
 
     fpath = os.path.join(path, 'train')
-    X_train, y_train = load_batch(fpath, label_key=label_mode + '_labels')
+    x_train, y_train = load_batch(fpath, label_key=label_mode + '_labels')
 
     fpath = os.path.join(path, 'test')
-    X_test, y_test = load_batch(fpath, label_key=label_mode + '_labels')
+    x_test, y_test = load_batch(fpath, label_key=label_mode + '_labels')
 
     y_train = np.reshape(y_train, (len(y_train), 1))
     y_test = np.reshape(y_test, (len(y_test), 1))
 
-    if K.image_dim_ordering() == 'tf':
-        X_train = X_train.transpose(0, 2, 3, 1)
-        X_test = X_test.transpose(0, 2, 3, 1)
+    if K.image_data_format() == 'channels_last':
+        x_train = x_train.transpose(0, 2, 3, 1)
+        x_test = x_test.transpose(0, 2, 3, 1)
 
-    return (X_train, y_train), (X_test, y_test)
+    return (x_train, y_train), (x_test, y_test)

keras/datasets/data_utils.py

@@ -1,4 +0,0 @@
-from ..utils.data_utils import *
-import warnings
-
-warnings.warn('data_utils has been moved to keras.utils.data_utils.')

keras/datasets/imdb.py

@@ -1,20 +1,19 @@
 from __future__ import absolute_import
-from six.moves import cPickle
-import gzip
 from ..utils.data_utils import get_file
 from six.moves import zip
 import numpy as np
-import sys
+import json
+import warnings
 
 
-def load_data(path='imdb_full.pkl', nb_words=None, skip_top=0,
+def load_data(path='imdb.npz', num_words=None, skip_top=0,
               maxlen=None, seed=113,
-              start_char=1, oov_char=2, index_from=3):
-    '''Loads IMDB dataset.
+              start_char=1, oov_char=2, index_from=3, **kwargs):
+    """Loads the IMDB dataset.
 
     # Arguments
-        path: where to store the data (in `/.keras/dataset`)
-        nb_words: max number of words to include. Words are ranked
+        path: where to cache the data (relative to `~/.keras/dataset`).
+        num_words: max number of words to include. Words are ranked
             by how often they occur (in the training set) and only
             the most frequent words are kept
         skip_top: skip the top N most frequently occuring words
@@ -23,26 +22,38 @@ def load_data(path='imdb_full.pkl', nb_words=None, skip_top=0,
         seed: random seed for sample shuffling.
         start_char: The start of a sequence will be marked with this character.
             Set to 1 because 0 is usually the padding character.
-        oov_char: words that were cut out because of the `nb_words`
+        oov_char: words that were cut out because of the `num_words`
             or `skip_top` limit will be replaced with this character.
         index_from: index actual words with this index and higher.
 
+    # Returns
+        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
+
+    # Raises
+        ValueError: in case `maxlen` is so low
+            that no input sequence could be kept.
+
     Note that the 'out of vocabulary' character is only used for
     words that were present in the training set but are not included
-    because they're not making the `nb_words` cut here.
-    Words that were not seen in the trining set but are in the test set
+    because they're not making the `num_words` cut here.
+    Words that were not seen in the training set but are in the test set
     have simply been skipped.
-    '''
+    """
+    # Legacy support
+    if 'nb_words' in kwargs:
+        warnings.warn('The `nb_words` argument in `load_data` '
+                      'has been renamed `num_words`.')
+        num_words = kwargs.pop('nb_words')
+    if kwargs:
+        raise TypeError('Unrecognized keyword arguments: ' + str(kwargs))
+
     path = get_file(path,
-                    origin='https://s3.amazonaws.com/text-datasets/imdb_full.pkl',
-                    md5_hash='d091312047c43cf9e4e38fef92437263')
-
-    if path.endswith('.gz'):
-        f = gzip.open(path, 'rb')
-    else:
-        f = open(path, 'rb')
-
-    (x_train, labels_train), (x_test, labels_test) = cPickle.load(f)
+                    origin='https://s3.amazonaws.com/text-datasets/imdb.npz')
+    f = np.load(path)
+    x_train = f['x_train']
+    labels_train = f['y_train']
+    x_test = f['x_test']
+    labels_test = f['y_test']
     f.close()
 
     np.random.seed(seed)
@@ -55,63 +66,66 @@ def load_data(path='imdb_full.pkl', nb_words=None, skip_top=0,
     np.random.seed(seed * 2)
     np.random.shuffle(labels_test)
 
-    X = x_train + x_test
-    labels = labels_train + labels_test
+    xs = np.concatenate([x_train, x_test])
+    labels = np.concatenate([labels_train, labels_test])
 
     if start_char is not None:
-        X = [[start_char] + [w + index_from for w in x] for x in X]
+        xs = [[start_char] + [w + index_from for w in x] for x in xs]
     elif index_from:
-        X = [[w + index_from for w in x] for x in X]
+        xs = [[w + index_from for w in x] for x in xs]
 
     if maxlen:
-        new_X = []
+        new_xs = []
         new_labels = []
-        for x, y in zip(X, labels):
+        for x, y in zip(xs, labels):
             if len(x) < maxlen:
-                new_X.append(x)
+                new_xs.append(x)
                 new_labels.append(y)
-        X = new_X
+        xs = new_xs
         labels = new_labels
-    if not X:
+    if not xs:
         raise ValueError('After filtering for sequences shorter than maxlen=' +
                          str(maxlen) + ', no sequence was kept. '
                          'Increase maxlen.')
-    if not nb_words:
-        nb_words = max([max(x) for x in X])
+    if not num_words:
+        num_words = max([max(x) for x in xs])
 
     # by convention, use 2 as OOV word
-    # reserve 'index_from' (=3 by default) characters: 0 (padding), 1 (start), 2 (OOV)
+    # reserve 'index_from' (=3 by default) characters:
+    # 0 (padding), 1 (start), 2 (OOV)
     if oov_char is not None:
-        X = [[oov_char if (w >= nb_words or w < skip_top) else w for w in x] for x in X]
+        xs = [[oov_char if (w >= num_words or w < skip_top) else w for w in x] for x in xs]
     else:
-        nX = []
-        for x in X:
+        new_xs = []
+        for x in xs:
             nx = []
             for w in x:
-                if (w >= nb_words or w < skip_top):
+                if w >= num_words or w < skip_top:
                     nx.append(w)
-            nX.append(nx)
-        X = nX
+            new_xs.append(nx)
+        xs = new_xs
 
-    X_train = np.array(X[:len(x_train)])
+    x_train = np.array(xs[:len(x_train)])
     y_train = np.array(labels[:len(x_train)])
 
-    X_test = np.array(X[len(x_train):])
+    x_test = np.array(xs[len(x_train):])
     y_test = np.array(labels[len(x_train):])
 
-    return (X_train, y_train), (X_test, y_test)
+    return (x_train, y_train), (x_test, y_test)
 
 
-def get_word_index(path='imdb_word_index.pkl'):
+def get_word_index(path='imdb_word_index.json'):
+    """Retrieves the dictionary mapping word indices back to words.
+
+    # Arguments
+        path: where to cache the data (relative to `~/.keras/dataset`).
+
+    # Returns
+        The word index dictionary.
+    """
     path = get_file(path,
-                    origin='https://s3.amazonaws.com/text-datasets/imdb_word_index.pkl',
-                    md5_hash='72d94b01291be4ff843198d3b0e1e4d7')
-    f = open(path, 'rb')
-
-    if sys.version_info < (3,):
-        data = cPickle.load(f)
-    else:
-        data = cPickle.load(f, encoding='latin1')
-
+                    origin='https://s3.amazonaws.com/text-datasets/imdb_word_index.json')
+    f = open(path)
+    data = json.load(f)
     f.close()
     return data

keras/datasets/mnist.py

@@ -1,21 +1,22 @@
-import gzip
 from ..utils.data_utils import get_file
-from six.moves import cPickle
-import sys
+import numpy as np
 
 
-def load_data(path='mnist.pkl.gz'):
-    path = get_file(path, origin='https://s3.amazonaws.com/img-datasets/mnist.pkl.gz')
+def load_data(path='mnist.npz'):
+    """Loads the MNIST dataset.
 
-    if path.endswith('.gz'):
-        f = gzip.open(path, 'rb')
-    else:
-        f = open(path, 'rb')
-
-    if sys.version_info < (3,):
-        data = cPickle.load(f)
-    else:
-        data = cPickle.load(f, encoding='bytes')
+    # Arguments
+        path: path where to cache the dataset locally
+            (relative to ~/.keras/datasets).
 
+    # Returns
+        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
+    """
+    path = get_file(path, origin='https://s3.amazonaws.com/img-datasets/mnist.npz')
+    f = np.load(path)
+    x_train = f['x_train']
+    y_train = f['y_train']
+    x_test = f['x_test']
+    y_test = f['y_test']
     f.close()
-    return data  # (X_train, y_train), (X_test, y_test)
+    return (x_train, y_train), (x_test, y_test)

keras/datasets/reuters.py

@@ -1,20 +1,20 @@
 # -*- coding: utf-8 -*-
 from __future__ import absolute_import
 from ..utils.data_utils import get_file
-from six.moves import cPickle
 from six.moves import zip
 import numpy as np
-import sys
+import json
+import warnings
 
 
-def load_data(path='reuters.pkl', nb_words=None, skip_top=0,
+def load_data(path='reuters.npz', num_words=None, skip_top=0,
               maxlen=None, test_split=0.2, seed=113,
-              start_char=1, oov_char=2, index_from=3):
-    '''Loads the Reuters newswire classification dataset.
+              start_char=1, oov_char=2, index_from=3, **kwargs):
+    """Loads the Reuters newswire classification dataset.
 
     # Arguments
-        path: where to store the data (in `/.keras/dataset`)
-        nb_words: max number of words to include. Words are ranked
+        path: where to cache the data (relative to `~/.keras/dataset`).
+        num_words: max number of words to include. Words are ranked
             by how often they occur (in the training set) and only
             the most frequent words are kept
         skip_top: skip the top N most frequently occuring words
@@ -24,76 +24,91 @@ def load_data(path='reuters.pkl', nb_words=None, skip_top=0,
         seed: random seed for sample shuffling.
         start_char: The start of a sequence will be marked with this character.
             Set to 1 because 0 is usually the padding character.
-        oov_char: words that were cut out because of the `nb_words`
+        oov_char: words that were cut out because of the `num_words`
             or `skip_top` limit will be replaced with this character.
         index_from: index actual words with this index and higher.
 
+    # Returns
+        Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
+
     Note that the 'out of vocabulary' character is only used for
     words that were present in the training set but are not included
-    because they're not making the `nb_words` cut here.
-    Words that were not seen in the trining set but are in the test set
+    because they're not making the `num_words` cut here.
+    Words that were not seen in the training set but are in the test set
     have simply been skipped.
-    '''
+    """
+    # Legacy support
+    if 'nb_words' in kwargs:
+        warnings.warn('The `nb_words` argument in `load_data` '
+                      'has been renamed `num_words`.')
+        num_words = kwargs.pop('nb_words')
+    if kwargs:
+        raise TypeError('Unrecognized keyword arguments: ' + str(kwargs))
 
-    path = get_file(path, origin='https://s3.amazonaws.com/text-datasets/reuters.pkl')
-    f = open(path, 'rb')
-    X, labels = cPickle.load(f)
-    f.close()
+    path = get_file(path, origin='https://s3.amazonaws.com/text-datasets/reuters.npz')
+    npzfile = np.load(path)
+    xs = npzfile['x']
+    labels = npzfile['y']
+    npzfile.close()
 
     np.random.seed(seed)
-    np.random.shuffle(X)
+    np.random.shuffle(xs)
     np.random.seed(seed)
     np.random.shuffle(labels)
 
     if start_char is not None:
-        X = [[start_char] + [w + index_from for w in x] for x in X]
+        xs = [[start_char] + [w + index_from for w in x] for x in xs]
     elif index_from:
-        X = [[w + index_from for w in x] for x in X]
+        xs = [[w + index_from for w in x] for x in xs]
 
     if maxlen:
-        new_X = []
+        new_xs = []
         new_labels = []
-        for x, y in zip(X, labels):
+        for x, y in zip(xs, labels):
             if len(x) < maxlen:
-                new_X.append(x)
+                new_xs.append(x)
                 new_labels.append(y)
-        X = new_X
+        xs = new_xs
         labels = new_labels
 
-    if not nb_words:
-        nb_words = max([max(x) for x in X])
+    if not num_words:
+        num_words = max([max(x) for x in xs])
 
     # by convention, use 2 as OOV word
-    # reserve 'index_from' (=3 by default) characters: 0 (padding), 1 (start), 2 (OOV)
+    # reserve 'index_from' (=3 by default) characters:
+    # 0 (padding), 1 (start), 2 (OOV)
     if oov_char is not None:
-        X = [[oov_char if (w >= nb_words or w < skip_top) else w for w in x] for x in X]
+        xs = [[oov_char if (w >= num_words or w < skip_top) else w for w in x] for x in xs]
     else:
-        nX = []
-        for x in X:
+        new_xs = []
+        for x in xs:
             nx = []
             for w in x:
-                if (w >= nb_words or w < skip_top):
+                if w >= num_words or w < skip_top:
                     nx.append(w)
-            nX.append(nx)
-        X = nX
+            new_xs.append(nx)
+        xs = new_xs
 
-    X_train = X[:int(len(X) * (1 - test_split))]
-    y_train = labels[:int(len(X) * (1 - test_split))]
+    x_train = np.array(xs[:int(len(xs) * (1 - test_split))])
+    y_train = np.array(labels[:int(len(xs) * (1 - test_split))])
 
-    X_test = X[int(len(X) * (1 - test_split)):]
-    y_test = labels[int(len(X) * (1 - test_split)):]
+    x_test = np.array(xs[int(len(xs) * (1 - test_split)):])
+    y_test = np.array(labels[int(len(xs) * (1 - test_split)):])
 
-    return (X_train, y_train), (X_test, y_test)
+    return (x_train, y_train), (x_test, y_test)
 
 
-def get_word_index(path='reuters_word_index.pkl'):
-    path = get_file(path, origin='https://s3.amazonaws.com/text-datasets/reuters_word_index.pkl')
-    f = open(path, 'rb')
+def get_word_index(path='reuters_word_index.json'):
+    """Retrieves the dictionary mapping word indices back to words.
 
-    if sys.version_info < (3,):
-        data = cPickle.load(f)
-    else:
-        data = cPickle.load(f, encoding='latin1')
+    # Arguments
+        path: where to cache the data (relative to `~/.keras/dataset`).
 
+    # Returns
+        The word index dictionary.
+    """
+    path = get_file(path, origin='https://s3.amazonaws.com/text-datasets/reuters_word_index.json')
+    f = open(path)
+    data = json.load(f)
     f.close()
     return data

keras/engine/__init__.py

@@ -4,7 +4,5 @@ from .topology import InputSpec
 from .topology import Input
 from .topology import InputLayer
 from .topology import Layer
-from .topology import Merge
-from .topology import merge
 from .topology import get_source_inputs
 from .training import Model

keras/initializations.py

@@ -1,107 +0,0 @@
-from __future__ import absolute_import
-import numpy as np
-from . import backend as K
-from .utils.generic_utils import get_from_module
-
-
-def get_fans(shape, dim_ordering='th'):
-    if len(shape) == 2:
-        fan_in = shape[0]
-        fan_out = shape[1]
-    elif len(shape) == 4 or len(shape) == 5:
-        # assuming convolution kernels (2D or 3D).
-        # TH kernel shape: (depth, input_depth, ...)
-        # TF kernel shape: (..., input_depth, depth)
-        if dim_ordering == 'th':
-            receptive_field_size = np.prod(shape[2:])
-            fan_in = shape[1] * receptive_field_size
-            fan_out = shape[0] * receptive_field_size
-        elif dim_ordering == 'tf':
-            receptive_field_size = np.prod(shape[:2])
-            fan_in = shape[-2] * receptive_field_size
-            fan_out = shape[-1] * receptive_field_size
-        else:
-            raise ValueError('Invalid dim_ordering: ' + dim_ordering)
-    else:
-        # no specific assumptions
-        fan_in = np.sqrt(np.prod(shape))
-        fan_out = np.sqrt(np.prod(shape))
-    return fan_in, fan_out
-
-
-def uniform(shape, scale=0.05, name=None):
-    return K.random_uniform_variable(shape, -scale, scale, name=name)
-
-
-def normal(shape, scale=0.05, name=None):
-    return K.random_normal_variable(shape, 0.0, scale, name=name)
-
-
-def lecun_uniform(shape, name=None, dim_ordering='th'):
-    ''' Reference: LeCun 98, Efficient Backprop
-        http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf
-    '''
-    fan_in, fan_out = get_fans(shape, dim_ordering=dim_ordering)
-    scale = np.sqrt(3. / fan_in)
-    return uniform(shape, scale, name=name)
-
-
-def glorot_normal(shape, name=None, dim_ordering='th'):
-    ''' Reference: Glorot & Bengio, AISTATS 2010
-    '''
-    fan_in, fan_out = get_fans(shape, dim_ordering=dim_ordering)
-    s = np.sqrt(2. / (fan_in + fan_out))
-    return normal(shape, s, name=name)
-
-
-def glorot_uniform(shape, name=None, dim_ordering='th'):
-    fan_in, fan_out = get_fans(shape, dim_ordering=dim_ordering)
-    s = np.sqrt(6. / (fan_in + fan_out))
-    return uniform(shape, s, name=name)
-
-
-def he_normal(shape, name=None, dim_ordering='th'):
-    ''' Reference:  He et al., http://arxiv.org/abs/1502.01852
-    '''
-    fan_in, fan_out = get_fans(shape, dim_ordering=dim_ordering)
-    s = np.sqrt(2. / fan_in)
-    return normal(shape, s, name=name)
-
-
-def he_uniform(shape, name=None, dim_ordering='th'):
-    fan_in, fan_out = get_fans(shape, dim_ordering=dim_ordering)
-    s = np.sqrt(6. / fan_in)
-    return uniform(shape, s, name=name)
-
-
-def orthogonal(shape, scale=1.1, name=None):
-    ''' From Lasagne. Reference: Saxe et al., http://arxiv.org/abs/1312.6120
-    '''
-    flat_shape = (shape[0], np.prod(shape[1:]))
-    a = np.random.normal(0.0, 1.0, flat_shape)
-    u, _, v = np.linalg.svd(a, full_matrices=False)
-    # pick the one with the correct shape
-    q = u if u.shape == flat_shape else v
-    q = q.reshape(shape)
-    return K.variable(scale * q[:shape[0], :shape[1]], name=name)
-
-
-def identity(shape, scale=1, name=None):
-    if len(shape) != 2 or shape[0] != shape[1]:
-        raise ValueError('Identity matrix initialization can only be used '
-                         'for 2D square matrices.')
-    else:
-        return K.variable(scale * np.identity(shape[0]), name=name)
-
-
-def zero(shape, name=None):
-    return K.zeros(shape, name=name)
-
-
-def one(shape, name=None):
-    return K.ones(shape, name=name)
-
-
-def get(identifier, **kwargs):
-    return get_from_module(identifier, globals(),
-                           'initialization', kwargs=kwargs)

keras/initializers.py

@@ -0,0 +1,468 @@
+from __future__ import absolute_import
+import numpy as np
+import six
+from . import backend as K
+from .utils.generic_utils import serialize_keras_object
+from .utils.generic_utils import deserialize_keras_object
+
+
+class Initializer(object):
+    """Initializer base class: all initializers inherit from this class.
+    """
+
+    def __call__(self, shape, dtype=None):
+        raise NotImplementedError
+
+    def get_config(self):
+        return {}
+
+    @classmethod
+    def from_config(cls, config):
+        return cls(**config)
+
+
+class Zeros(Initializer):
+    """Initializer that generates tensors initialized to 0."""
+
+    def __call__(self, shape, dtype=None):
+        return K.constant(0, shape=shape, dtype=dtype)
+
+
+class Ones(Initializer):
+    """Initializer that generates tensors initialized to 1."""
+
+    def __call__(self, shape, dtype=None):
+        return K.constant(1, shape=shape, dtype=dtype)
+
+
+class Constant(Initializer):
+    """Initializer that generates tensors initialized to a constant value.
+
+    # Arguments
+        value: float; the value of the generator tensors.
+    """
+
+    def __init__(self, value=0):
+        self.value = value
+
+    def __call__(self, shape, dtype=None):
+        return K.constant(self.value, shape=shape, dtype=dtype)
+
+    def get_config(self):
+        return {'value': self.value}
+
+
+class RandomNormal(Initializer):
+    """Initializer that generates tensors with a normal distribution.
+
+    # Arguments
+        mean: a python scalar or a scalar tensor. Mean of the random values
+          to generate.
+        stddev: a python scalar or a scalar tensor. Standard deviation of the
+          random values to generate.
+        seed: A Python integer. Used to seed the random generator.
+    """
+
+    def __init__(self, mean=0., stddev=0.05, seed=None):
+        self.mean = mean
+        self.stddev = stddev
+        self.seed = seed
+
+    def __call__(self, shape, dtype=None):
+        return K.random_normal(shape, self.mean, self.stddev,
+                               dtype=dtype, seed=self.seed)
+
+    def get_config(self):
+        return {
+            'mean': self.mean,
+            'stddev': self.stddev,
+            'seed': self.seed
+        }
+
+
+class RandomUniform(Initializer):
+    """Initializer that generates tensors with a uniform distribution.
+
+    # Arguments
+        minval: A python scalar or a scalar tensor. Lower bound of the range
+          of random values to generate.
+        maxval: A python scalar or a scalar tensor. Upper bound of the range
+          of random values to generate.  Defaults to 1 for float types.
+        seed: A Python integer. Used to seed the random generator.
+    """
+
+    def __init__(self, minval=-0.05, maxval=0.05, seed=None):
+        self.minval = minval
+        self.maxval = maxval
+        self.seed = seed
+
+    def __call__(self, shape, dtype=None):
+        return K.random_uniform(shape, self.minval, self.maxval,
+                                dtype=dtype, seed=self.seed)
+
+    def get_config(self):
+        return {
+            'minval': self.minval,
+            'maxval': self.maxval,
+            'seed': self.seed,
+        }
+
+
+class TruncatedNormal(Initializer):
+    """Initializer that generates a truncated normal distribution.
+
+    These values are similar to values from a `random_normal_initializer`
+    except that values more than two standard deviations from the mean
+    are discarded and re-drawn. This is the recommended initializer for
+    neural network weights and filters.
+
+    # Arguments
+        mean: a python scalar or a scalar tensor. Mean of the random values
+          to generate.
+        stddev: a python scalar or a scalar tensor. Standard deviation of the
+          random values to generate.
+        seed: A Python integer. Used to seed the random generator.
+    """
+
+    def __init__(self, mean=0., stddev=0.05, seed=None):
+        self.mean = mean
+        self.stddev = stddev
+        self.seed = seed
+
+    def __call__(self, shape, dtype=None):
+        return K.truncated_normal(shape, self.mean, self.stddev,
+                                  dtype=dtype, seed=self.seed)
+
+    def get_config(self):
+        return {
+            'mean': self.mean,
+            'stddev': self.stddev,
+            'seed': self.seed
+        }
+
+
+class VarianceScaling(Initializer):
+    """Initializer capable of adapting its scale to the shape of weights.
+
+    With `distribution="normal"`, samples are drawn from a truncated normal
+    distribution centered on zero, with `stddev = sqrt(scale / n)` where n is:
+        - number of input units in the weight tensor, if mode = "fan_in"
+        - number of output units, if mode = "fan_out"
+        - average of the numbers of input and output units, if mode = "fan_avg"
+
+    With `distribution="uniform"`,
+    samples are drawn from a uniform distribution
+    within [-limit, limit], with `limit = sqrt(3 * scale / n)`.
+
+    # Arguments
+        scale: Scaling factor (positive float).
+        mode: One of "fan_in", "fan_out", "fan_avg".
+        distribution: Random distribution to use. One of "normal", "uniform".
+        seed: A Python integer. Used to seed the random generator.
+
+    # Raises
+        ValueError: In case of an invalid value for the "scale", mode" or
+          "distribution" arguments.
+    """
+
+    def __init__(self, scale=1.0,
+                 mode='fan_in',
+                 distribution='normal',
+                 seed=None):
+        if scale <= 0.:
+            raise ValueError('`scale` must be a positive float. Got:', scale)
+        mode = mode.lower()
+        if mode not in {'fan_in', 'fan_out', 'fan_avg'}:
+            raise ValueError('Invalid `mode` argument: '
+                             'expected on of {"fan_in", "fan_out", "fan_avg"} '
+                             'but got', mode)
+        distribution = distribution.lower()
+        if distribution not in {'normal', 'uniform'}:
+            raise ValueError('Invalid `distribution` argument: '
+                             'expected one of {"normal", "uniform"} '
+                             'but got', distribution)
+        self.scale = scale
+        self.mode = mode
+        self.distribution = distribution
+        self.seed = seed
+
+    def __call__(self, shape, dtype=None):
+        fan_in, fan_out = _compute_fans(shape)
+        scale = self.scale
+        if self.mode == 'fan_in':
+            scale /= max(1., fan_in)
+        elif self.mode == 'fan_out':
+            scale /= max(1., fan_out)
+        else:
+            scale /= max(1., float(fan_in + fan_out) / 2)
+        if self.distribution == 'normal':
+            stddev = np.sqrt(scale)
+            return K.truncated_normal(shape, 0., stddev,
+                                      dtype=dtype, seed=self.seed)
+        else:
+            limit = np.sqrt(3. * scale)
+            return K.random_uniform(shape, -limit, limit,
+                                    dtype=dtype, seed=self.seed)
+
+    def get_config(self):
+        return {
+            'scale': self.scale,
+            'mode': self.mode,
+            'distribution': self.distribution,
+            'seed': self.seed
+        }
+
+
+class Orthogonal(Initializer):
+    """Initializer that generates a random orthogonal matrix.
+
+    # Arguments
+        gain: Multiplicative factor to apply to the orthogonal matrix.
+        seed: A Python integer. Used to seed the random generator.
+
+    # References
+        Saxe et al., http://arxiv.org/abs/1312.6120
+    """
+
+    def __init__(self, gain=1., seed=None):
+        self.gain = gain
+        self.seed = seed
+
+    def __call__(self, shape, dtype=None):
+        num_rows = 1
+        for dim in shape[:-1]:
+            num_rows *= dim
+        num_cols = shape[-1]
+        flat_shape = (num_rows, num_cols)
+        if self.seed is not None:
+            np.random.seed(self.seed)
+        a = np.random.normal(0.0, 1.0, flat_shape)
+        u, _, v = np.linalg.svd(a, full_matrices=False)
+        # Pick the one with the correct shape.
+        q = u if u.shape == flat_shape else v
+        q = q.reshape(shape)
+        return self.gain * q[:shape[0], :shape[1]]
+
+    def get_config(self):
+        return {
+            'gain': self.gain,
+            'seed': self.seed
+        }
+
+
+class Identity(Initializer):
+    """Initializer that generates the identity matrix.
+
+    Only use for square 2D matrices.
+
+    # Arguments
+        gain: Multiplicative factor to apply to the identity matrix.
+    """
+
+    def __init__(self, gain=1.):
+        self.gain = gain
+
+    def __call__(self, shape, dtype=None):
+        if len(shape) != 2 or shape[0] != shape[1]:
+            raise ValueError('Identity matrix initializer can only be used '
+                             'for 2D square matrices.')
+        else:
+            return self.gain * np.identity(shape[0])
+
+    def get_config(self):
+        return {
+            'gain': self.gain
+        }
+
+
+def lecun_uniform(seed=None):
+    """LeCun uniform initializer.
+
+    It draws samples from a uniform distribution within [-limit, limit]
+    where `limit` is `sqrt(3 / fan_in)`
+    where `fan_in` is the number of input units in the weight tensor.
+
+    # Arguments
+        seed: A Python integer. Used to seed the random generator.
+
+    # Returns
+        An initializer.
+
+    # References
+        LeCun 98, Efficient Backprop,
+        http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf
+    """
+    return VarianceScaling(scale=1.,
+                           mode='fan_in',
+                           distribution='uniform',
+                           seed=seed)
+
+
+def glorot_normal(seed=None):
+    """Glorot normal initializer, also called Xavier normal initializer.
+
+    It draws samples from a truncated normal distribution centered on 0
+    with `stddev = sqrt(2 / (fan_in + fan_out))`
+    where `fan_in` is the number of input units in the weight tensor
+    and `fan_out` is the number of output units in the weight tensor.
+
+    # Arguments
+        seed: A Python integer. Used to seed the random generator.
+
+    # Returns
+        An initializer.
+
+    # References
+        Glorot & Bengio, AISTATS 2010
+        http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf
+    """
+    return VarianceScaling(scale=1.,
+                           mode='fan_avg',
+                           distribution='normal',
+                           seed=seed)
+
+
+def glorot_uniform(seed=None):
+    """Glorot uniform initializer, also called Xavier uniform initializer.
+
+    It draws samples from a uniform distribution within [-limit, limit]
+    where `limit` is `sqrt(6 / (fan_in + fan_out))`
+    where `fan_in` is the number of input units in the weight tensor
+    and `fan_out` is the number of output units in the weight tensor.
+
+    # Arguments
+        seed: A Python integer. Used to seed the random generator.
+
+    # Returns
+        An initializer.
+
+    # References
+        Glorot & Bengio, AISTATS 2010
+        http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf
+    """
+    return VarianceScaling(scale=1.,
+                           mode='fan_avg',
+                           distribution='uniform',
+                           seed=seed)
+
+
+def he_normal(seed=None):
+    """He normal initializer.
+
+    It draws samples from a truncated normal distribution centered on 0
+    with `stddev = sqrt(2 / fan_in)`
+    where `fan_in` is the number of input units in the weight tensor.
+
+    # Arguments
+        seed: A Python integer. Used to seed the random generator.
+
+    # Returns
+        An initializer.
+
+    # References
+        He et al., http://arxiv.org/abs/1502.01852
+    """
+    return VarianceScaling(scale=2.,
+                           mode='fan_in',
+                           distribution='normal',
+                           seed=seed)
+
+
+def he_uniform(seed=None):
+    """He uniform variance scaling initializer.
+
+    It draws samples from a uniform distribution within [-limit, limit]
+    where `limit` is `sqrt(6 / fan_in)`
+    where `fan_in` is the number of input units in the weight tensor.
+
+    # Arguments
+        seed: A Python integer. Used to seed the random generator.
+
+    # Returns
+        An initializer.
+
+    # References
+        He et al., http://arxiv.org/abs/1502.01852
+    """
+    return VarianceScaling(scale=2.,
+                           mode='fan_in',
+                           distribution='uniform',
+                           seed=seed)
+
+
+# Compatibility aliases
+
+zero = zeros = Zeros
+one = ones = Ones
+constant = Constant
+uniform = random_uniform = RandomUniform
+normal = random_normal = RandomNormal
+truncated_normal = TruncatedNormal
+identity = Identity
+orthogonal = Orthogonal
+
+# Utility functions
+
+
+def _compute_fans(shape, data_format='channels_last'):
+    """Computes the number of input and output units for a weight shape.
+
+    # Arguments
+        shape: Integer shape tuple.
+        data_format: Image data format to use for convolution kernels.
+            Note that all kernels in Keras are standardized on the
+            `channels_last` ordering (even when inputs are set
+            to `channels_first`).
+
+    # Returns
+        A tuple of scalars, `(fan_in, fan_out)`.
+
+    # Raises
+        ValueError: in case of invalid `data_format` argument.
+    """
+    if len(shape) == 2:
+        fan_in = shape[0]
+        fan_out = shape[1]
+    elif len(shape) in {3, 4, 5}:
+        # Assuming convolution kernels (1D, 2D or 3D).
+        # TH kernel shape: (depth, input_depth, ...)
+        # TF kernel shape: (..., input_depth, depth)
+        if data_format == 'channels_first':
+            receptive_field_size = np.prod(shape[2:])
+            fan_in = shape[1] * receptive_field_size
+            fan_out = shape[0] * receptive_field_size
+        elif data_format == 'channels_last':
+            receptive_field_size = np.prod(shape[:2])
+            fan_in = shape[-2] * receptive_field_size
+            fan_out = shape[-1] * receptive_field_size
+        else:
+            raise ValueError('Invalid data_format: ' + data_format)
+    else:
+        # No specific assumptions.
+        fan_in = np.sqrt(np.prod(shape))
+        fan_out = np.sqrt(np.prod(shape))
+    return fan_in, fan_out
+
+
+def serialize(initializer):
+    return serialize_keras_object(initializer)
+
+
+def deserialize(config, custom_objects=None):
+    return deserialize_keras_object(config,
+                                    module_objects=globals(),
+                                    custom_objects=custom_objects,
+                                    printable_module_name='initializer')
+
+
+def get(identifier):
+    if isinstance(identifier, dict):
+        return deserialize(identifier)
+    elif isinstance(identifier, six.string_types):
+        config = {'class_name': str(identifier), 'config': {}}
+        return deserialize(config)
+    elif callable(identifier):
+        return identifier
+    else:
+        raise ValueError('Could not interpret initializer identifier:',
+                         identifier)

keras/layers/__init__.py

@@ -1,5 +1,11 @@
 from __future__ import absolute_import
-from ..engine import Layer, Input, InputLayer, Merge, merge, InputSpec
+
+from ..utils.generic_utils import deserialize_keras_object
+from ..engine import Layer
+from ..engine import Input
+from ..engine import InputLayer
+from ..engine import InputSpec
+from .merge import *
 from .core import *
 from .convolutional import *
 from .pooling import *
@@ -11,3 +17,30 @@ from .noise import *
 from .advanced_activations import *
 from .wrappers import *
 from .convolutional_recurrent import *
+from ..legacy.layers import *
+
+
+def serialize(layer):
+    return {'class_name': layer.__class__.__name__,
+            'config': layer.get_config()}
+
+
+def deserialize(config, custom_objects=None):
+    """Instantiate a layer from a config dictionary.
+
+    # Arguments
+        config: dict of the form {'class_name': str, 'config': dict}
+        custom_objects: dict mapping class names (or function names)
+            of custom (non-Keras) objects to class/functions
+
+    # Returns
+        Layer instance (may be Model, Sequential, Layer...)
+    """
+    from .. import models
+    globs = globals()  # All layers.
+    globs['Model'] = models.Model
+    globs['Sequential'] = models.Sequential
+    return deserialize_keras_object(config,
+                                    module_objects=globs,
+                                    custom_objects=custom_objects,
+                                    printable_module_name='layer')

keras/layers/advanced_activations.py

@@ -1,12 +1,19 @@
-from .. import initializations
+# -*- coding: utf-8 -*-
+from __future__ import absolute_import
+
+from .. import initializers
+from .. import regularizers
+from .. import constraints
 from ..engine import Layer
+from ..engine import InputSpec
 from .. import backend as K
-import numpy as np
+from ..legacy import interfaces
 
 
 class LeakyReLU(Layer):
-    '''Special version of a Rectified Linear Unit
-    that allows a small gradient when the unit is not active:
+    """Leaky version of a Rectified Linear Unit.
+
+    It allows a small gradient when the unit is not active:
     `f(x) = alpha * x for x < 0`,
     `f(x) = x for x >= 0`.
 
@@ -20,14 +27,18 @@ class LeakyReLU(Layer):
 
     # Arguments
         alpha: float >= 0. Negative slope coefficient.
-    '''
-    def __init__(self, alpha=0.3, **kwargs):
-        self.supports_masking = True
-        self.alpha = alpha
-        super(LeakyReLU, self).__init__(**kwargs)
 
-    def call(self, x, mask=None):
-        return K.relu(x, alpha=self.alpha)
+    # References
+        - [Rectifier Nonlinearities Improve Neural Network Acoustic Models](https://web.stanford.edu/~awni/papers/relu_hybrid_icml2013_final.pdf)
+    """
+
+    def __init__(self, alpha=0.3, **kwargs):
+        super(LeakyReLU, self).__init__(**kwargs)
+        self.supports_masking = True
+        self.alpha = K.cast_to_floatx(alpha)
+
+    def call(self, inputs):
+        return K.relu(inputs, alpha=self.alpha)
 
     def get_config(self):
         config = {'alpha': self.alpha}
@@ -36,10 +47,12 @@ class LeakyReLU(Layer):
 
 
 class PReLU(Layer):
-    '''Parametric Rectified Linear Unit:
-    `f(x) = alphas * x for x < 0`,
+    """Parametric Rectified Linear Unit.
+
+    It follows:
+    `f(x) = alpha * x for x < 0`,
     `f(x) = x for x >= 0`,
-    where `alphas` is a learned array with the same shape as x.
+    where `alpha` is a learned array with the same shape as x.
 
     # Input shape
         Arbitrary. Use the keyword argument `input_shape`
@@ -50,8 +63,9 @@ class PReLU(Layer):
         Same shape as the input.
 
     # Arguments
-        init: initialization function for the weights.
-        weights: initial weights, as a list of a single Numpy array.
+        alpha_initializer: initializer function for the weights.
+        alpha_regularizer: regularizer for the weights.
+        alpha_constraint: constraint for the weights.
         shared_axes: the axes along which to share learnable
             parameters for the activation function.
             For example, if the incoming feature maps
@@ -62,50 +76,72 @@ class PReLU(Layer):
             set `shared_axes=[1, 2]`.
 
     # References
-        - [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](http://arxiv.org/pdf/1502.01852v1.pdf)
-    '''
-    def __init__(self, init='zero', weights=None, shared_axes=None, **kwargs):
+        - [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://arxiv.org/abs/1502.01852)
+    """
+
+    @interfaces.legacy_prelu_support
+    def __init__(self, alpha_initializer='zeros',
+                 alpha_regularizer=None,
+                 alpha_constraint=None,
+                 shared_axes=None,
+                 **kwargs):
+        super(PReLU, self).__init__(**kwargs)
         self.supports_masking = True
-        self.init = initializations.get(init)
-        self.initial_weights = weights
-        if type(shared_axes) is not list and type(shared_axes) is not tuple:
+        self.alpha_initializer = initializers.get(alpha_initializer)
+        self.alpha_regularizer = regularizers.get(alpha_regularizer)
+        self.alpha_constraint = constraints.get(alpha_constraint)
+        if shared_axes is None:
+            self.shared_axes = None
+        elif not isinstance(shared_axes, (list, tuple)):
             self.shared_axes = [shared_axes]
         else:
             self.shared_axes = list(shared_axes)
-        super(PReLU, self).__init__(**kwargs)
 
     def build(self, input_shape):
         param_shape = list(input_shape[1:])
         self.param_broadcast = [False] * len(param_shape)
-        if self.shared_axes[0] is not None:
+        if self.shared_axes is not None:
             for i in self.shared_axes:
-                param_shape[i] = 1
-                self.param_broadcast[i] = True
+                param_shape[i - 1] = 1
+                self.param_broadcast[i - 1] = True
+        self.alpha = self.add_weight(param_shape,
+                                     name='alpha',
+                                     initializer=self.alpha_initializer,
+                                     regularizer=self.alpha_regularizer,
+                                     constraint=self.alpha_constraint)
+        # Set input spec
+        axes = {}
+        if self.shared_axes:
+            for i in range(1, len(input_shape)):
+                if i not in self.shared_axes:
+                    axes[i] = input_shape[i]
+        self.input_spec = InputSpec(ndim=len(input_shape), axes=axes)
+        self.built = True
 
-        self.alphas = self.init(param_shape,
-                                name='{}_alphas'.format(self.name))
-        self.trainable_weights = [self.alphas]
-
-        if self.initial_weights is not None:
-            self.set_weights(self.initial_weights)
-            del self.initial_weights
-
-    def call(self, x, mask=None):
-        pos = K.relu(x)
+    def call(self, inputs, mask=None):
+        pos = K.relu(inputs)
         if K.backend() == 'theano':
-            neg = K.pattern_broadcast(self.alphas, self.param_broadcast) * (x - abs(x)) * 0.5
+            neg = (K.pattern_broadcast(self.alpha, self.param_broadcast) *
+                   (inputs - K.abs(inputs)) * 0.5)
         else:
-            neg = self.alphas * (x - abs(x)) * 0.5
+            neg = -self.alpha * K.relu(-inputs)
         return pos + neg
 
     def get_config(self):
-        config = {'init': self.init.__name__}
+        config = {
+            'alpha_initializer': initializers.serialize(self.alpha_initializer),
+            'alpha_regularizer': regularizers.serialize(self.alpha_regularizer),
+            'alpha_constraint': constraints.serialize(self.alpha_constraint),
+            'shared_axes': self.shared_axes
+        }
         base_config = super(PReLU, self).get_config()
         return dict(list(base_config.items()) + list(config.items()))
 
 
 class ELU(Layer):
-    '''Exponential Linear Unit:
+    """Exponential Linear Unit.
+
+    It follows:
     `f(x) =  alpha * (exp(x) - 1.) for x < 0`,
     `f(x) = x for x >= 0`.
 
@@ -121,15 +157,16 @@ class ELU(Layer):
         alpha: scale for the negative factor.
 
     # References
-        - [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](http://arxiv.org/pdf/1511.07289v1.pdf)
-    '''
+        - [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://arxiv.org/abs/1511.07289v1)
+    """
+
     def __init__(self, alpha=1.0, **kwargs):
+        super(ELU, self).__init__(**kwargs)
         self.supports_masking = True
         self.alpha = K.cast_to_floatx(alpha)
-        super(ELU, self).__init__(**kwargs)
 
-    def call(self, x, mask=None):
-        return K.elu(x, self.alpha)
+    def call(self, inputs):
+        return K.elu(inputs, self.alpha)
 
     def get_config(self):
         config = {'alpha': float(self.alpha)}
@@ -137,80 +174,11 @@ class ELU(Layer):
         return dict(list(base_config.items()) + list(config.items()))
 
 
-class ParametricSoftplus(Layer):
-    '''Parametric Softplus:
-    `alpha * log(1 + exp(beta * x))`
-
-    # Input shape
-        Arbitrary. Use the keyword argument `input_shape`
-        (tuple of integers, does not include the samples axis)
-        when using this layer as the first layer in a model.
-
-    # Output shape
-        Same shape as the input.
-
-    # Arguments
-        alpha_init: float. Initial value of the alpha weights.
-        beta_init: float. Initial values of the beta weights.
-        weights: initial weights, as a list of 2 numpy arrays.
-        shared_axes: the axes along which to share learnable
-            parameters for the activation function.
-            For example, if the incoming feature maps
-            are from a 2D convolution
-            with output shape `(batch, height, width, channels)`,
-            and you wish to share parameters across space
-            so that each filter only has one set of parameters,
-            set `shared_axes=[1, 2]`.
-
-    # References
-        - [Inferring Nonlinear Neuronal Computation Based on Physiologically Plausible Inputs](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003143)
-    '''
-    def __init__(self, alpha_init=0.2, beta_init=5.0,
-                 weights=None, shared_axes=None, **kwargs):
-        self.supports_masking = True
-        self.alpha_init = K.cast_to_floatx(alpha_init)
-        self.beta_init = K.cast_to_floatx(beta_init)
-        self.initial_weights = weights
-        if type(shared_axes) is not list and type(shared_axes) is not tuple:
-            self.shared_axes = [shared_axes]
-        else:
-            self.shared_axes = list(shared_axes)
-        super(ParametricSoftplus, self).__init__(**kwargs)
-
-    def build(self, input_shape):
-        param_shape = list(input_shape[1:])
-        self.param_broadcast = [False] * len(param_shape)
-        if self.shared_axes[0] is not None:
-            for i in self.shared_axes:
-                param_shape[i] = 1
-                self.param_broadcast[i] = True
-
-        self.alphas = K.variable(self.alpha_init * np.ones(param_shape),
-                                 name='{}_alphas'.format(self.name))
-        self.betas = K.variable(self.beta_init * np.ones(param_shape),
-                                name='{}_betas'.format(self.name))
-        self.trainable_weights = [self.alphas, self.betas]
-
-        if self.initial_weights is not None:
-            self.set_weights(self.initial_weights)
-            del self.initial_weights
-
-    def call(self, x, mask=None):
-        if K.backend() == 'theano':
-            return K.softplus(K.pattern_broadcast(self.betas, self.param_broadcast) * x) * K.pattern_broadcast(self.alphas, self.param_broadcast)
-        else:
-            return K.softplus(self.betas * x) * self.alphas
-
-    def get_config(self):
-        config = {'alpha_init': float(self.alpha_init),
-                  'beta_init': float(self.beta_init)}
-        base_config = super(ParametricSoftplus, self).get_config()
-        return dict(list(base_config.items()) + list(config.items()))
-
-
 class ThresholdedReLU(Layer):
-    '''Thresholded Rectified Linear Unit:
-    `f(x) = x for x > theta`
+    """Thresholded Rectified Linear Unit.
+
+    It follows:
+    `f(x) = x for x > theta`,
     `f(x) = 0 otherwise`.
 
     # Input shape
@@ -225,111 +193,18 @@ class ThresholdedReLU(Layer):
         theta: float >= 0. Threshold location of activation.
 
     # References
-        - [Zero-Bias Autoencoders and the Benefits of Co-Adapting Features](http://arxiv.org/pdf/1402.3337.pdf)
-    '''
+        - [Zero-Bias Autoencoders and the Benefits of Co-Adapting Features](http://arxiv.org/abs/1402.3337)
+    """
+
     def __init__(self, theta=1.0, **kwargs):
+        super(ThresholdedReLU, self).__init__(**kwargs)
         self.supports_masking = True
         self.theta = K.cast_to_floatx(theta)
-        super(ThresholdedReLU, self).__init__(**kwargs)
 
-    def call(self, x, mask=None):
-        return x * K.cast(x > self.theta, K.floatx())
+    def call(self, inputs, mask=None):
+        return inputs * K.cast(inputs > self.theta, K.floatx())
 
     def get_config(self):
         config = {'theta': float(self.theta)}
         base_config = super(ThresholdedReLU, self).get_config()
         return dict(list(base_config.items()) + list(config.items()))
-
-
-class SReLU(Layer):
-    '''S-shaped Rectified Linear Unit.
-
-    # Input shape
-        Arbitrary. Use the keyword argument `input_shape`
-        (tuple of integers, does not include the samples axis)
-        when using this layer as the first layer in a model.
-
-    # Output shape
-        Same shape as the input.
-
-    # Arguments
-        t_left_init: initialization function for the left part intercept
-        a_left_init: initialization function for the left part slope
-        t_right_init: initialization function for the right part intercept
-        a_right_init: initialization function for the right part slope
-        shared_axes: the axes along which to share learnable
-            parameters for the activation function.
-            For example, if the incoming feature maps
-            are from a 2D convolution
-            with output shape `(batch, height, width, channels)`,
-            and you wish to share parameters across space
-            so that each filter only has one set of parameters,
-            set `shared_axes=[1, 2]`.
-
-    # References
-        - [Deep Learning with S-shaped Rectified Linear Activation Units](http://arxiv.org/abs/1512.07030)
-    '''
-    def __init__(self, t_left_init='zero', a_left_init='glorot_uniform',
-                 t_right_init='glorot_uniform', a_right_init='one', shared_axes=None, **kwargs):
-        self.supports_masking = True
-        self.t_left_init = t_left_init
-        self.a_left_init = a_left_init
-        self.t_right_init = t_right_init
-        self.a_right_init = a_right_init
-        if type(shared_axes) is not list and type(shared_axes) is not tuple:
-            self.shared_axes = [shared_axes]
-        else:
-            self.shared_axes = list(shared_axes)
-        super(SReLU, self).__init__(**kwargs)
-
-    def build(self, input_shape):
-        param_shape = list(input_shape[1:])
-        self.param_broadcast = [False] * len(param_shape)
-        if self.shared_axes[0] is not None:
-            for i in self.shared_axes:
-                param_shape[i] = 1
-                self.param_broadcast[i] = True
-
-        t_left_init = initializations.get(self.t_left_init)
-        a_left_init = initializations.get(self.a_left_init)
-        t_right_init = initializations.get(self.t_right_init)
-        a_right_init = initializations.get(self.a_right_init)
-
-        self.t_left = t_left_init(param_shape,
-                                  name='{}_t_left'.format(self.name))
-        self.a_left = a_left_init(param_shape,
-                                  name='{}_a_left'.format(self.name))
-        self.t_right = t_right_init(param_shape,
-                                    name='{}_t_right'.format(self.name))
-        self.a_right = a_right_init(param_shape,
-                                    name='{}_a_right'.format(self.name))
-        # ensure the the right part is always to the right of the left
-        self.t_right_actual = self.t_left + abs(self.t_right)
-        self.trainable_weights = [self.t_left, self.a_left,
-                                  self.t_right, self.a_right]
-
-    def call(self, x, mask=None):
-        if K.backend() == 'theano':
-            t_left = K.pattern_broadcast(self.t_left, self.param_broadcast)
-            a_left = K.pattern_broadcast(self.a_left, self.param_broadcast)
-            a_right = K.pattern_broadcast(self.a_right, self.param_broadcast)
-            t_right_actual = K.pattern_broadcast(self.t_right_actual, self.param_broadcast)
-        else:
-            t_left = self.t_left
-            a_left = self.a_left
-            a_right = self.a_right
-            t_right_actual = self.t_right_actual
-
-        Y_left_and_center = t_left + K.relu(x - t_left,
-                                            a_left,
-                                            t_right_actual - t_left)
-        Y_right = K.relu(x - t_right_actual) * a_right
-        return Y_left_and_center + Y_right
-
-    def get_config(self):
-        config = {'t_left_init': self.t_left_init,
-                  'a_left_init': self.a_left_init,
-                  't_right_init': self.t_right_init,
-                  'a_right_init': self.a_right_init}
-        base_config = super(SReLU, self).get_config()
-        return dict(list(base_config.items()) + list(config.items()))

keras/layers/convolutional_recurrent.py

@@ -1,32 +1,48 @@
+# -*- coding: utf-8 -*-
+from __future__ import absolute_import
+
 from .. import backend as K
-from .. import activations, initializations, regularizers
+from .. import activations
+from .. import initializers
+from .. import regularizers
+from .. import constraints
+from .recurrent import Recurrent
 
 import numpy as np
-from ..engine import Layer, InputSpec
-from ..utils.np_utils import conv_output_length
-import warnings
+from ..engine import InputSpec
+from ..utils import conv_utils
+from ..legacy import interfaces
 
 
-class ConvRecurrent2D(Layer):
-    '''Abstract base class for convolutional recurrent layers.
+class ConvRecurrent2D(Recurrent):
+    """Abstract base class for convolutional recurrent layers.
+
     Do not use in a model -- it's not a functional layer!
 
-    ConvLSTM2D
-    follow the specifications of this class and accept
-    the keyword arguments listed below.
-
-    # Input shape
-        5D tensor with shape `(nb_samples, timesteps, channels, rows, cols)`.
-
-    # Output shape
-        - if `return_sequences`: 5D tensor with shape
-            `(nb_samples, timesteps, channels, rows, cols)`.
-        - else, 4D tensor with shape `(nb_samples, channels, rows, cols)`.
-
     # Arguments
-        weights: list of numpy arrays to set as initial weights.
-            The list should have 3 elements, of shapes:
-            `[(input_dim, nb_filter), (nb_filter, nb_filter), (nb_filter,)]`.
+        filters: Integer, the dimensionality of the output space
+            (i.e. the number output of filters in the convolution).
+        kernel_size: An integer or tuple/list of n integers, specifying the
+            dimensions of the convolution window.
+        strides: An integer or tuple/list of n integers,
+            specifying the strides of the convolution.
+            Specifying any stride value != 1 is incompatible with specifying
+            any `dilation_rate` value != 1.
+        padding: One of `"valid"` or `"same"` (case-insensitive).
+        data_format: A string,
+            one of `channels_last` (default) or `channels_first`.
+            The ordering of the dimensions in the inputs.
+            `channels_last` corresponds to inputs with shape
+            `(batch, time, ..., channels)`
+            while `channels_first` corresponds to
+            inputs with shape `(batch, time, channels, ...)`.
+            It defaults to the `image_data_format` value found in your
+            Keras config file at `~/.keras/keras.json`.
+            If you never set it, then it will be "channels_last".
+        dilation_rate: An integer or tuple/list of n integers, specifying
+            the dilation rate to use for dilated convolution.
+            Currently, specifying any `dilation_rate` value != 1 is
+            incompatible with specifying any `strides` value != 1.
         return_sequences: Boolean. Whether to return the last output
             in the output sequence, or the full sequence.
         go_backwards: Boolean (default False).
@@ -34,11 +50,14 @@ class ConvRecurrent2D(Layer):
         stateful: Boolean (default False). If True, the last state
             for each sample at index i in a batch will be used as initial
             state for the sample of index i in the following batch.
-        nb_filter: Number of convolution filters to use.
-        nb_row: Number of rows in the convolution kernel.
-        nb_col: Number of columns in the convolution kernel.
-            is required when using this layer as the first layer in a model.
-        input_shape: input_shape
+
+    # Input shape
+        5D tensor with shape `(num_samples, timesteps, channels, rows, cols)`.
+
+    # Output shape
+        - if `return_sequences`: 5D tensor with shape
+            `(num_samples, timesteps, channels, rows, cols)`.
+        - else, 4D tensor with shape `(num_samples, channels, rows, cols)`.
 
     # Masking
         This layer supports masking for input data with a variable number
@@ -47,15 +66,6 @@ class ConvRecurrent2D(Layer):
         set to `True`.
         **Note:** for the time being, masking is only supported with Theano.
 
-    # TensorFlow warning
-        For the time being, when using the TensorFlow backend,
-        the number of timesteps used must be specified in your model.
-        Make sure to pass an `input_length` int argument to your
-        recurrent layer (if it comes first in your model),
-        or to pass a complete `input_shape` argument to the first layer
-        in your model otherwise.
-
-
     # Note on using statefulness in RNNs
         You can set RNN layers to be 'stateful', which means that the states
         computed for the samples in one batch will be reused as initial states
@@ -73,308 +83,343 @@ class ConvRecurrent2D(Layer):
 
         To reset the states of your model, call `.reset_states()` on either
         a specific layer, or on your entire model.
-    '''
+    """
 
-    def __init__(self, weights=None, nb_row=None, nb_col=None, nb_filter=None,
-                 return_sequences=False, go_backwards=False, stateful=False,
-                 dim_ordering=None, **kwargs):
+    def __init__(self, filters,
+                 kernel_size,
+                 strides=(1, 1),
+                 padding='valid',
+                 data_format=None,
+                 dilation_rate=(1, 1),
+                 return_sequences=False,
+                 go_backwards=False,
+                 stateful=False,
+                 **kwargs):
+        super(ConvRecurrent2D, self).__init__(**kwargs)
+        self.filters = filters
+        self.kernel_size = conv_utils.normalize_tuple(kernel_size, 2, 'kernel_size')
+        self.strides = conv_utils.normalize_tuple(strides, 2, 'strides')
+        self.padding = conv_utils.normalize_padding(padding)
+        self.data_format = conv_utils.normalize_data_format(data_format)
+        self.dilation_rate = conv_utils.normalize_tuple(dilation_rate, 2, 'dilation_rate')
         self.return_sequences = return_sequences
         self.go_backwards = go_backwards
         self.stateful = stateful
-        self.initial_weights = weights
-        self.nb_row = nb_row
-        self.nb_col = nb_col
-        self.nb_filter = nb_filter
-        self.dim_ordering = dim_ordering
-        self.input_spec = [InputSpec(ndim=5)]
+        self.input_spec = InputSpec(ndim=5)
 
-        super(ConvRecurrent2D, self).__init__(**kwargs)
-
-    def compute_mask(self, input, mask):
-        if self.return_sequences:
-            return mask
-        else:
-            return None
-
-    def get_output_shape_for(self, input_shape):
-
-        if self.dim_ordering == 'th':
+    def compute_output_shape(self, input_shape):
+        if self.data_format == 'channels_first':
             rows = input_shape[3]
             cols = input_shape[4]
-        elif self.dim_ordering == 'tf':
+        elif self.data_format == 'channels_last':
             rows = input_shape[2]
             cols = input_shape[3]
-        else:
-            raise ValueError('Invalid dim_ordering:', self.dim_ordering)
-
-        rows = conv_output_length(rows, self.nb_row,
-                                  self.border_mode, self.subsample[0])
-        cols = conv_output_length(cols, self.nb_col,
-                                  self.border_mode, self.subsample[1])
-
+        rows = conv_utils.conv_output_length(rows,
+                                             self.kernel_size[0],
+                                             padding=self.padding,
+                                             stride=self.strides[0],
+                                             dilation=self.dilation_rate[0])
+        cols = conv_utils.conv_output_length(cols,
+                                             self.kernel_size[1],
+                                             padding=self.padding,
+                                             stride=self.strides[1],
+                                             dilation=self.dilation_rate[1])
         if self.return_sequences:
-            if self.dim_ordering == 'th':
+            if self.data_format == 'channels_first':
                 return (input_shape[0], input_shape[1],
-                        self.nb_filter, rows, cols)
-            elif self.dim_ordering == 'tf':
+                        self.filters, rows, cols)
+            elif self.data_format == 'channels_last':
                 return (input_shape[0], input_shape[1],
-                        rows, cols, self.nb_filter)
+                        rows, cols, self.filters)
         else:
-            if self.dim_ordering == 'th':
-                return (input_shape[0], self.nb_filter, rows, cols)
-            elif self.dim_ordering == 'tf':
-                return (input_shape[0], rows, cols, self.nb_filter)
-
-    def step(self, x, states):
-        raise NotImplementedError
-
-    def get_constants(self, X, train=False):
-        return None
-
-    def get_initial_states(self, X):
-        # (samples, timesteps, row, col, filter)
-        initial_state = K.zeros_like(X)
-        # (samples,row, col, filter)
-        initial_state = K.sum(initial_state, axis=1)
-        initial_state = self.conv_step(initial_state, K.zeros(self.W_shape),
-                                       border_mode=self.border_mode)
-
-        initial_states = [initial_state for _ in range(2)]
-        return initial_states
-
-    def preprocess_input(self, x):
-        return x
-
-    def call(self, x, mask=None):
-        assert K.ndim(x) == 5
-        input_shape = self.input_spec[0].shape
-        unroll = False
-
-        if self.stateful:
-            initial_states = self.states
-        else:
-            initial_states = self.get_initial_states(x)
-
-        constants = self.get_constants(x)
-        preprocessed_input = self.preprocess_input(x)
-
-        last_output, outputs, states = K.rnn(self.step, preprocessed_input,
-                                             initial_states,
-                                             go_backwards=self.go_backwards,
-                                             mask=mask,
-                                             constants=constants,
-                                             unroll=unroll,
-                                             input_length=input_shape[1])
-        if self.stateful:
-            self.updates = []
-            for i in range(len(states)):
-                self.updates.append((self.states[i], states[i]))
-
-        if self.return_sequences:
-            return outputs
-        else:
-            return last_output
+            if self.data_format == 'channels_first':
+                return (input_shape[0], self.filters, rows, cols)
+            elif self.data_format == 'channels_last':
+                return (input_shape[0], rows, cols, self.filters)
 
     def get_config(self):
-        config = {'return_sequences': self.return_sequences,
+        config = {'filters': self.filters,
+                  'kernel_size': self.kernel_size,
+                  'strides': self.strides,
+                  'padding': self.padding,
+                  'data_format': self.data_format,
+                  'dilation_rate': self.dilation_rate,
+                  'return_sequences': self.return_sequences,
                   'go_backwards': self.go_backwards,
                   'stateful': self.stateful}
-        if self.stateful:
-            config['batch_input_shape'] = self.input_spec[0].shape
-
         base_config = super(ConvRecurrent2D, self).get_config()
         return dict(list(base_config.items()) + list(config.items()))
 
 
 class ConvLSTM2D(ConvRecurrent2D):
-    '''Convolutional LSTM.
+    """Convolutional LSTM.
+
+    It is similar to an LSTM layer, but the input transformations
+    and recurrent transformations are both convolutional.
+
+    # Arguments
+        filters: Integer, the dimensionality of the output space
+            (i.e. the number output of filters in the convolution).
+        kernel_size: An integer or tuple/list of n integers, specifying the
+            dimensions of the convolution window.
+        strides: An integer or tuple/list of n integers,
+            specifying the strides of the convolution.
+            Specifying any stride value != 1 is incompatible with specifying
+            any `dilation_rate` value != 1.
+        padding: One of `"valid"` or `"same"` (case-insensitive).
+        data_format: A string,
+            one of `channels_last` (default) or `channels_first`.
+            The ordering of the dimensions in the inputs.
+            `channels_last` corresponds to inputs with shape
+            `(batch, time, ..., channels)`
+            while `channels_first` corresponds to
+            inputs with shape `(batch, time, channels, ...)`.
+            It defaults to the `image_data_format` value found in your
+            Keras config file at `~/.keras/keras.json`.
+            If you never set it, then it will be "channels_last".
+        dilation_rate: An integer or tuple/list of n integers, specifying
+            the dilation rate to use for dilated convolution.
+            Currently, specifying any `dilation_rate` value != 1 is
+            incompatible with specifying any `strides` value != 1.
+        activation: Activation function to use
+            (see [activations](../activations.md)).
+            If you don't specify anything, no activation is applied
+            (ie. "linear" activation: `a(x) = x`).
+        recurrent_activation: Activation function to use
+            for the recurrent step
+            (see [activations](../activations.md)).
+        use_bias: Boolean, whether the layer uses a bias vector.
+        kernel_initializer: Initializer for the `kernel` weights matrix,
+            used for the linear transformation of the inputs.
+            (see [initializers](../initializers.md)).
+        recurrent_initializer: Initializer for the `recurrent_kernel`
+            weights matrix,
+            used for the linear transformation of the recurrent state.
+            (see [initializers](../initializers.md)).
+        bias_initializer: Initializer for the bias vector
+            (see [initializers](../initializers.md)).
+        unit_forget_bias: Boolean.
+            If True, add 1 to the bias of the forget gate at initialization.
+            Use in combination with `bias_initializer="zeros"`.
+            This is recommended in [Jozefowicz et al.](http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf)
+        kernel_regularizer: Regularizer function applied to
+            the `kernel` weights matrix
+            (see [regularizer](../regularizers.md)).
+        recurrent_regularizer: Regularizer function applied to
+            the `recurrent_kernel` weights matrix
+            (see [regularizer](../regularizers.md)).
+        bias_regularizer: Regularizer function applied to the bias vector
+            (see [regularizer](../regularizers.md)).
+        activity_regularizer: Regularizer function applied to
+            the output of the layer (its "activation").
+            (see [regularizer](../regularizers.md)).
+        kernel_constraint: Constraint function applied to
+            the `kernel` weights matrix
+            (see [constraints](../constraints.md)).
+        recurrent_constraint: Constraint function applied to
+            the `recurrent_kernel` weights matrix
+            (see [constraints](../constraints.md)).
+        bias_constraint: Constraint function applied to the bias vector
+            (see [constraints](../constraints.md)).
+        return_sequences: Boolean. Whether to return the last output
+            in the output sequence, or the full sequence.
+        go_backwards: Boolean (default False).
+            If True, rocess the input sequence backwards.
+        stateful: Boolean (default False). If True, the last state
+            for each sample at index i in a batch will be used as initial
+            state for the sample of index i in the following batch.
+        dropout: Float between 0 and 1.
+            Fraction of the units to drop for
+            the linear transformation of the inputs.
+        recurrent_dropout: Float between 0 and 1.
+            Fraction of the units to drop for
+            the linear transformation of the recurrent state.
 
     # Input shape
-        - if dim_ordering='th'
+        - if data_format='channels_first'
             5D tensor with shape:
             `(samples,time, channels, rows, cols)`
-        - if dim_ordering='tf'
+        - if data_format='channels_last'
             5D tensor with shape:
             `(samples,time, rows, cols, channels)`
 
      # Output shape
         - if `return_sequences`
-             - if dim_ordering='th'
+             - if data_format='channels_first'
                 5D tensor with shape:
-                `(samples, time, nb_filter, output_row, output_col)`
-             - if dim_ordering='tf'
+                `(samples, time, filters, output_row, output_col)`
+             - if data_format='channels_last'
                 5D tensor with shape:
-                `(samples, time, output_row, output_col, nb_filter)`
+                `(samples, time, output_row, output_col, filters)`
         - else
-            - if dim_ordering ='th'
+            - if data_format ='channels_first'
                 4D tensor with shape:
-                `(samples, nb_filter, output_row, output_col)`
-            - if dim_ordering='tf'
+                `(samples, filters, output_row, output_col)`
+            - if data_format='channels_last'
                 4D tensor with shape:
-                `(samples, output_row, output_col, nb_filter)`
+                `(samples, output_row, output_col, filters)`
+            where o_row and o_col depend on the shape of the filter and
+            the padding
 
-        where o_row and o_col depend on the shape of the filter and
-        the border_mode
-
-        # Arguments
-            nb_filter: Number of convolution filters to use.
-            nb_row: Number of rows in the convolution kernel.
-            nb_col: Number of columns in the convolution kernel.
-            border_mode: 'valid' or 'same'.
-            subsample: tuple of length 2. Factor by which to subsample output.
-                Also called strides elsewhere.
-            dim_ordering: 'tf' if the feature are at the last dimension or 'th'
-            stateful : Boolean (default False). If True, the last state
-                for each sample at index i in a batch will be used as initial
-                state for the sample of index i in the following batch.
-            init: weight initialization function.
-                Can be the name of an existing function (str),
-                or a Theano function
-                (see: [initializations](../initializations.md)).
-            inner_init: initialization function of the inner cells.
-            forget_bias_init: initialization function for the bias of the
-            forget gate.
-                [Jozefowicz et al.](http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf)
-                recommend initializing with ones.
-            activation: activation function.
-                Can be the name of an existing function (str),
-                or a Theano function (see: [activations](../activations.md)).
-            inner_activation: activation function for the inner cells.
+    # Raises
+        ValueError: in case of invalid constructor arguments.
 
     # References
         - [Convolutional LSTM Network: A Machine Learning Approach for
-        Precipitation Nowcasting](http://arxiv.org/pdf/1506.04214v1.pdf)
+        Precipitation Nowcasting](http://arxiv.org/abs/1506.04214v1)
         The current implementation does not include the feedback loop on the
         cells output
-    '''
-    def __init__(self, nb_filter, nb_row, nb_col,
-                 init='glorot_uniform', inner_init='orthogonal',
-                 forget_bias_init='one', activation='tanh',
-                 inner_activation='hard_sigmoid',
-                 dim_ordering='default',
-                 border_mode='valid', subsample=(1, 1),
-                 W_regularizer=None, U_regularizer=None, b_regularizer=None,
-                 dropout_W=0., dropout_U=0., **kwargs):
+    """
 
-        if dim_ordering == 'default':
-            dim_ordering = K.image_dim_ordering()
-        if dim_ordering not in {'tf', 'th'}:
-            raise ValueError('dim_ordering must be in {tf,th}', dim_ordering)
-        self.nb_filter = nb_filter
-        self.nb_row = nb_row
-        self.nb_col = nb_col
-        self.init = initializations.get(init)
-        self.inner_init = initializations.get(inner_init)
-        self.forget_bias_init = initializations.get(forget_bias_init)
+    @interfaces.legacy_convlstm2d_support
+    def __init__(self, filters,
+                 kernel_size,
+                 strides=(1, 1),
+                 padding='valid',
+                 data_format=None,
+                 dilation_rate=(1, 1),
+                 activation='tanh',
+                 recurrent_activation='hard_sigmoid',
+                 use_bias=True,
+                 kernel_initializer='glorot_uniform',
+                 recurrent_initializer='orthogonal',
+                 bias_initializer='zeros',
+                 unit_forget_bias=True,
+                 kernel_regularizer=None,
+                 recurrent_regularizer=None,
+                 bias_regularizer=None,
+                 activity_regularizer=None,
+                 kernel_constraint=None,
+                 recurrent_constraint=None,
+                 bias_constraint=None,
+                 return_sequences=False,
+                 go_backwards=False,
+                 stateful=False,
+                 dropout=0.,
+                 recurrent_dropout=0.,
+                 **kwargs):
+        super(ConvLSTM2D, self).__init__(filters,
+                                         kernel_size,
+                                         strides=strides,
+                                         padding=padding,
+                                         data_format=data_format,
+                                         dilation_rate=dilation_rate,
+                                         return_sequences=return_sequences,
+                                         go_backwards=go_backwards,
+                                         stateful=stateful,
+                                         **kwargs)
         self.activation = activations.get(activation)
-        self.inner_activation = activations.get(inner_activation)
-        self.border_mode = border_mode
-        self.subsample = subsample
+        self.recurrent_activation = activations.get(recurrent_activation)
+        self.use_bias = use_bias
 
-        if dim_ordering == 'th':
-            warnings.warn('Be carefull if used with convolution3D layers:\n'
-                          'th in convolution 3D corresponds to '
-                          '(samples, channels, conv_dim1, conv_dim2,'
-                          'conv_dim3)\n'
-                          'while for this network it corresponds to: '
-                          '(samples, time, channels, rows, cols)')
-        self.dim_ordering = dim_ordering
+        self.kernel_initializer = initializers.get(kernel_initializer)
+        self.recurrent_initializer = initializers.get(recurrent_initializer)
+        self.bias_initializer = initializers.get(bias_initializer)
+        self.unit_forget_bias = unit_forget_bias
 
-        kwargs['nb_filter'] = nb_filter
-        kwargs['nb_row'] = nb_row
-        kwargs['nb_col'] = nb_col
-        kwargs['dim_ordering'] = dim_ordering
+        self.kernel_regularizer = regularizers.get(kernel_regularizer)
+        self.recurrent_regularizer = regularizers.get(recurrent_regularizer)
+        self.bias_regularizer = regularizers.get(bias_regularizer)
+        self.activity_regularizer = regularizers.get(activity_regularizer)
 
-        self.W_regularizer = regularizers.get(W_regularizer)
-        self.U_regularizer = regularizers.get(U_regularizer)
-        self.b_regularizer = regularizers.get(b_regularizer)
-        self.dropout_W, self.dropout_U = dropout_W, dropout_U
-        if self.dropout_W or self.dropout_U:
-            self.uses_learning_phase = True
+        self.kernel_constraint = constraints.get(kernel_constraint)
+        self.recurrent_constraint = constraints.get(recurrent_constraint)
+        self.bias_constraint = constraints.get(bias_constraint)
 
-        super(ConvLSTM2D, self).__init__(**kwargs)
+        self.dropout = min(1., max(0., dropout))
+        self.recurrent_dropout = min(1., max(0., recurrent_dropout))
 
     def build(self, input_shape):
-        self.input_spec = [InputSpec(shape=input_shape)]
-
-        if self.dim_ordering == 'th':
-            stack_size = input_shape[2]
-            self.W_shape = (self.nb_filter, stack_size,
-                            self.nb_row, self.nb_col)
-        elif self.dim_ordering == 'tf':
-            stack_size = input_shape[4]
-            self.W_shape = (self.nb_row, self.nb_col,
-                            stack_size, self.nb_filter)
-        else:
-            raise ValueError('Invalid dim_ordering:', self.dim_ordering)
-
-        if self.dim_ordering == 'th':
-            self.W_shape1 = (self.nb_filter, self.nb_filter,
-                             self.nb_row, self.nb_col)
-        elif self.dim_ordering == 'tf':
-            self.W_shape1 = (self.nb_row, self.nb_col,
-                             self.nb_filter, self.nb_filter)
+        # TODO: better handling of input spec
+        self.input_spec = InputSpec(shape=input_shape)
 
         if self.stateful:
             self.reset_states()
         else:
-            # initial states: 2 all-zero tensor of shape (nb_filter)
-            self.states = [None, None, None, None]
+            # initial states: 2 all-zero tensor of shape (filters)
+            self.states = [None, None]
 
-        self.W_i = self.init(self.W_shape, name='{}_W_i'.format(self.name))
-        self.U_i = self.inner_init(self.W_shape1,
-                                   name='{}_U_i'.format(self.name))
-        self.b_i = K.zeros((self.nb_filter,), name='{}_b_i'.format(self.name))
+        if self.data_format == 'channels_first':
+            channel_axis = 1
+        else:
+            channel_axis = -1
+        if input_shape[channel_axis] is None:
+            raise ValueError('The channel dimension of the inputs '
+                             'should be defined. Found `None`.')
+        input_dim = input_shape[channel_axis]
+        kernel_shape = self.kernel_size + (input_dim, self.filters * 4)
+        self.kernel_shape = kernel_shape
+        recurrent_kernel_shape = self.kernel_size + (self.filters, self.filters * 4)
 
-        self.W_f = self.init(self.W_shape, name='{}_W_f'.format(self.name))
-        self.U_f = self.inner_init(self.W_shape1,
-                                   name='{}_U_f'.format(self.name))
-        self.b_f = self.forget_bias_init((self.nb_filter,),
-                                         name='{}_b_f'.format(self.name))
+        self.kernel = self.add_weight(kernel_shape,
+                                      initializer=self.kernel_initializer,
+                                      name='kernel',
+                                      regularizer=self.kernel_regularizer,
+                                      constraint=self.kernel_constraint)
+        self.recurrent_kernel = self.add_weight(
+            recurrent_kernel_shape,
+            initializer=self.recurrent_initializer,
+            name='recurrent_kernel',
+            regularizer=self.recurrent_regularizer,
+            constraint=self.recurrent_constraint)
+        if self.use_bias:
+            self.bias = self.add_weight((self.filters * 4,),
+                                        initializer=self.bias_initializer,
+                                        name='bias',
+                                        regularizer=self.bias_regularizer,
+                                        constraint=self.bias_constraint)
+            if self.unit_forget_bias:
+                bias_value = np.zeros((self.filters * 4,))
+                bias_value[self.filters: self.filters * 2] = 1.
+                K.set_value(self.bias, bias_value)
+        else:
+            self.bias = None
 
-        self.W_c = self.init(self.W_shape, name='{}_W_c'.format(self.name))
-        self.U_c = self.inner_init(self.W_shape1,
-                                   name='{}_U_c'.format(self.name))
-        self.b_c = K.zeros((self.nb_filter,), name='{}_b_c'.format(self.name))
+        self.kernel_i = self.kernel[:, :, :, :self.filters]
+        self.recurrent_kernel_i = self.recurrent_kernel[:, :, :, :self.filters]
+        self.kernel_f = self.kernel[:, :, :, self.filters: self.filters * 2]
+        self.recurrent_kernel_f = self.recurrent_kernel[:, :, :, self.filters: self.filters * 2]
+        self.kernel_c = self.kernel[:, :, :, self.filters * 2: self.filters * 3]
+        self.recurrent_kernel_c = self.recurrent_kernel[:, :, :, self.filters * 2: self.filters * 3]
+        self.kernel_o = self.kernel[:, :, :, self.filters * 3:]
+        self.recurrent_kernel_o = self.recurrent_kernel[:, :, :, self.filters * 3:]
 
-        self.W_o = self.init(self.W_shape, name='{}_W_o'.format(self.name))
-        self.U_o = self.inner_init(self.W_shape1,
-                                   name='{}_U_o'.format(self.name))
-        self.b_o = K.zeros((self.nb_filter,), name='{}_b_o'.format(self.name))
-
-        self.trainable_weights = [self.W_i, self.U_i, self.b_i,
-                                  self.W_c, self.U_c, self.b_c,
-                                  self.W_f, self.U_f, self.b_f,
-                                  self.W_o, self.U_o, self.b_o]
-
-        self.W = K.concatenate([self.W_i, self.W_f, self.W_c, self.W_o])
-        self.U = K.concatenate([self.U_i, self.U_f, self.U_c, self.U_o])
-        self.b = K.concatenate([self.b_i, self.b_f, self.b_c, self.b_o])
-
-        self.regularizers = []
-        if self.W_regularizer:
-            self.W_regularizer.set_param(self.W)
-            self.regularizers.append(self.W_regularizer)
-        if self.U_regularizer:
-            self.U_regularizer.set_param(self.U)
-            self.regularizers.append(self.U_regularizer)
-        if self.b_regularizer:
-            self.b_regularizer.set_param(self.b)
-            self.regularizers.append(self.b_regularizer)
-
-        if self.initial_weights is not None:
-            self.set_weights(self.initial_weights)
-            del self.initial_weights
+        if self.use_bias:
+            self.bias_i = self.bias[:self.filters]
+            self.bias_f = self.bias[self.filters: self.filters * 2]
+            self.bias_c = self.bias[self.filters * 2: self.filters * 3]
+            self.bias_o = self.bias[self.filters * 3:]
+        else:
+            self.bias_i = None
+            self.bias_f = None
+            self.bias_c = None
+            self.bias_o = None
         self.built = True
 
+    def get_initial_states(self, inputs):
+        # (samples, timesteps, rows, cols, filters)
+        initial_state = K.zeros_like(inputs)
+        # (samples, rows, cols, filters)
+        initial_state = K.sum(initial_state, axis=1)
+        shape = list(self.kernel_shape)
+        shape[-1] = self.filters
+        initial_state = self.input_conv(initial_state,
+                                        K.zeros(tuple(shape)),
+                                        padding=self.padding)
+
+        initial_states = [initial_state for _ in range(2)]
+        return initial_states
+
     def reset_states(self):
-        assert self.stateful, 'Layer must be stateful.'
-        input_shape = self.input_spec[0].shape
-        output_shape = self.get_output_shape_for(input_shape)
+        if not self.stateful:
+            raise RuntimeError('Layer must be stateful.')
+        input_shape = self.input_spec.shape
+        output_shape = self.compute_output_shape(input_shape)
         if not input_shape[0]:
-            raise ValueError('If a RNN is stateful, a complete ' +
-                             'input_shape must be provided ' +
-                             '(including batch size).')
+            raise ValueError('If a RNN is stateful, a complete '
+                             'input_shape must be provided '
+                             '(including batch size). '
+                             'Got input shape: ' + str(input_shape))
 
         if self.return_sequences:
             out_row, out_col, out_filter = output_shape[2:]
@@ -394,117 +439,105 @@ class ConvLSTM2D(ConvRecurrent2D):
                            K.zeros((input_shape[0],
                                     out_row, out_col, out_filter))]
 
-    def conv_step(self, x, W, b=None, border_mode='valid'):
-        input_shape = self.input_spec[0].shape
-
-        conv_out = K.conv2d(x, W, strides=self.subsample,
-                            border_mode=border_mode,
-                            dim_ordering=self.dim_ordering,
-                            image_shape=(input_shape[0],
-                                         input_shape[2],
-                                         input_shape[3],
-                                         input_shape[4]),
-                            filter_shape=self.W_shape)
-        if b:
-            if self.dim_ordering == 'th':
-                conv_out = conv_out + K.reshape(b, (1, self.nb_filter, 1, 1))
-            elif self.dim_ordering == 'tf':
-                conv_out = conv_out + K.reshape(b, (1, 1, 1, self.nb_filter))
-            else:
-                raise ValueError('Invalid dim_ordering:', self.dim_ordering)
-
-        return conv_out
-
-    def conv_step_hidden(self, x, W, border_mode='valid'):
-        # This new function was defined because the
-        # image shape must be hardcoded
-        input_shape = self.input_spec[0].shape
-        output_shape = self.get_output_shape_for(input_shape)
-        if self.return_sequences:
-            out_row, out_col, out_filter = output_shape[2:]
-        else:
-            out_row, out_col, out_filter = output_shape[1:]
-
-        conv_out = K.conv2d(x, W, strides=(1, 1),
-                            border_mode=border_mode,
-                            dim_ordering=self.dim_ordering,
-                            image_shape=(input_shape[0],
-                                         out_row, out_col,
-                                         out_filter),
-                            filter_shape=self.W_shape1)
-
-        return conv_out
-
-    def step(self, x, states):
-        assert len(states) == 4
-        h_tm1 = states[0]
-        c_tm1 = states[1]
-        B_U = states[2]
-        B_W = states[3]
-
-        x_i = self.conv_step(x * B_W[0], self.W_i, self.b_i,
-                             border_mode=self.border_mode)
-        x_f = self.conv_step(x * B_W[1], self.W_f, self.b_f,
-                             border_mode=self.border_mode)
-        x_c = self.conv_step(x * B_W[2], self.W_c, self.b_c,
-                             border_mode=self.border_mode)
-        x_o = self.conv_step(x * B_W[3], self.W_o, self.b_o,
-                             border_mode=self.border_mode)
-
-        # U : from nb_filter to nb_filter
-        # Same because must be stable in the output space
-        h_i = self.conv_step_hidden(h_tm1 * B_U[0], self.U_i,
-                                    border_mode='same')
-        h_f = self.conv_step_hidden(h_tm1 * B_U[1], self.U_f,
-                                    border_mode='same')
-        h_c = self.conv_step_hidden(h_tm1 * B_U[2], self.U_c,
-                                    border_mode='same')
-        h_o = self.conv_step_hidden(h_tm1 * B_U[3], self.U_o,
-                                    border_mode='same')
-
-        i = self.inner_activation(x_i + h_i)
-        f = self.inner_activation(x_f + h_f)
-        c = f * c_tm1 + i * self.activation(x_c + h_c)
-        o = self.inner_activation(x_o + h_o)
-        h = o * self.activation(c)
-
-        return h, [h, c]
-
-    def get_constants(self, x):
+    def get_constants(self, inputs, training=None):
         constants = []
-        if 0 < self.dropout_U < 1:
-            ones = K.zeros_like(x)
+        if self.implementation == 0 and 0 < self.dropout < 1:
+            ones = K.zeros_like(inputs)
             ones = K.sum(ones, axis=1)
-            ones = self.conv_step(ones, K.zeros(self.W_shape),
-                                  border_mode=self.border_mode)
-            ones = ones + 1
-            B_U = [K.in_train_phase(K.dropout(ones, self.dropout_U), ones)
-                   for _ in range(4)]
-            constants.append(B_U)
+            ones += 1
+
+            def dropped_inputs():
+                return K.dropout(ones, self.dropout)
+
+            dp_mask = [K.in_train_phase(dropped_inputs,
+                                        ones,
+                                        training=training) for _ in range(4)]
+            constants.append(dp_mask)
         else:
             constants.append([K.cast_to_floatx(1.) for _ in range(4)])
 
-        if 0 < self.dropout_W < 1:
-            ones = K.zeros_like(x)
+        if 0 < self.recurrent_dropout < 1:
+            shape = list(self.kernel_shape)
+            shape[-1] = self.filters
+            ones = K.zeros_like(inputs)
             ones = K.sum(ones, axis=1)
-            ones = ones + 1
-            B_W = [K.in_train_phase(K.dropout(ones, self.dropout_W), ones)
-                   for _ in range(4)]
-            constants.append(B_W)
+            ones = self.input_conv(ones, K.zeros(shape),
+                                   padding=self.padding)
+            ones += 1.
+
+            def dropped_inputs():
+                return K.dropout(ones, self.recurrent_dropout)
+            rec_dp_mask = [K.in_train_phase(dropped_inputs,
+                                            ones,
+                                            training=training) for _ in range(4)]
+            constants.append(rec_dp_mask)
         else:
             constants.append([K.cast_to_floatx(1.) for _ in range(4)])
         return constants
 
+    def input_conv(self, x, w, b=None, padding='valid'):
+        conv_out = K.conv2d(x, w, strides=self.strides,
+                            padding=padding,
+                            data_format=self.data_format,
+                            dilation_rate=self.dilation_rate)
+        if b is not None:
+            conv_out = K.bias_add(conv_out, b,
+                                  data_format=self.data_format)
+        return conv_out
+
+    def reccurent_conv(self, x, w):
+        conv_out = K.conv2d(x, w, strides=(1, 1),
+                            padding='same',
+                            data_format=self.data_format)
+        return conv_out
+
+    def step(self, inputs, states):
+        assert len(states) == 4
+        h_tm1 = states[0]
+        c_tm1 = states[1]
+        dp_mask = states[2]
+        rec_dp_mask = states[3]
+
+        x_i = self.input_conv(inputs * dp_mask[0], self.kernel_i, self.bias_i,
+                              padding=self.padding)
+        x_f = self.input_conv(inputs * dp_mask[1], self.kernel_f, self.bias_f,
+                              padding=self.padding)
+        x_c = self.input_conv(inputs * dp_mask[2], self.kernel_c, self.bias_c,
+                              padding=self.padding)
+        x_o = self.input_conv(inputs * dp_mask[3], self.kernel_o, self.bias_o,
+                              padding=self.padding)
+        h_i = self.reccurent_conv(h_tm1 * rec_dp_mask[0],
+                                  self.recurrent_kernel_i)
+        h_f = self.reccurent_conv(h_tm1 * rec_dp_mask[1],
+                                  self.recurrent_kernel_f)
+        h_c = self.reccurent_conv(h_tm1 * rec_dp_mask[2],
+                                  self.recurrent_kernel_c)
+        h_o = self.reccurent_conv(h_tm1 * rec_dp_mask[3],
+                                  self.recurrent_kernel_o)
+
+        i = self.recurrent_activation(x_i + h_i)
+        f = self.recurrent_activation(x_f + h_f)
+        c = f * c_tm1 + i * self.activation(x_c + h_c)
+        o = self.recurrent_activation(x_o + h_o)
+        h = o * self.activation(c)
+        return h, [h, c]
+
     def get_config(self):
-        config = {'nb_filter': self.nb_filter,
-                  'nb_row': self.nb_row,
-                  'nb_col': self.nb_col,
-                  'init': self.init.__name__,
-                  'inner_init': self.inner_init.__name__,
-                  'forget_bias_init': self.forget_bias_init.__name__,
-                  'activation': self.activation.__name__,
-                  'dim_ordering': self.dim_ordering,
-                  'border_mode': self.border_mode,
-                  'inner_activation': self.inner_activation.__name__}
+        config = {'activation': activations.serialize(self.activation),
+                  'recurrent_activation': activations.serialize(self.recurrent_activation),
+                  'use_bias': self.use_bias,
+                  'kernel_initializer': initializers.serialize(self.kernel_initializer),
+                  'recurrent_initializer': initializers.serialize(self.recurrent_initializer),
+                  'bias_initializer': initializers.serialize(self.bias_initializer),
+                  'unit_forget_bias': self.unit_forget_bias,
+                  'kernel_regularizer': regularizers.serialize(self.kernel_regularizer),
+                  'recurrent_regularizer': regularizers.serialize(self.recurrent_regularizer),
+                  'bias_regularizer': regularizers.serialize(self.bias_regularizer),
+                  'activity_regularizer': regularizers.serialize(self.activity_regularizer),
+                  'kernel_constraint': constraints.serialize(self.kernel_constraint),
+                  'recurrent_constraint': constraints.serialize(self.recurrent_constraint),
+                  'bias_constraint': constraints.serialize(self.bias_constraint),
+                  'dropout': self.dropout,
+                  'recurrent_dropout': self.recurrent_dropout}
         base_config = super(ConvLSTM2D, self).get_config()
         return dict(list(base_config.items()) + list(config.items()))