{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "nbsphinx": "hidden"
   },
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "import os\n",
    "os.sys.path.insert(0, '/home/schirrmr/braindecode/code/braindecode/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using the Experiment Class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Braindecode provides a convenience `Experiment` class, which removes the necessity to write your own training loop. It expects a training, a validation and a test set and trains as follows:\n",
    "\n",
    "1. Train on training set until a given stop criterion is fulfilled\n",
    "2. Reset to the best epoch, i.e. reset parameters of the model and the optimizer to the state at the best epoch (\"best\" according to a given criterion) \n",
    "3. Continue training on the combined training + validation set until the loss on the validation set is as low as it was on the best epoch for the training set. (or until the ConvNet was trained twice as many epochs as the best epoch to prevent infinite training)\n",
    "\n",
    "<div class='alert alert-warning'>\n",
    "\n",
    "It is not necessary to use the Experiment class to use the remaning functionality of Braindecode. Feel free to ignore it :)\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Removing orphaned offset at the beginning of the file.\n",
      "179 events found\n",
      "Events id: [1 2 3]\n",
      "90 matching events found\n",
      "Loading data for 90 events and 497 original time points ...\n",
      "0 bad epochs dropped\n"
     ]
    }
   ],
   "source": [
    "import mne\n",
    "from mne.io import concatenate_raws\n",
    "\n",
    "# 5,6,7,10,13,14 are codes for executed and imagined hands/feet\n",
    "subject_id = 1\n",
    "event_codes = [5,6,9,10,13,14]\n",
    "\n",
    "# This will download the files if you don't have them yet,\n",
    "# and then return the paths to the files.\n",
    "physionet_paths = mne.datasets.eegbci.load_data(subject_id, event_codes)\n",
    "\n",
    "# Load each of the files\n",
    "parts = [mne.io.read_raw_edf(path, preload=True,stim_channel='auto', verbose='WARNING')\n",
    "         for path in physionet_paths]\n",
    "\n",
    "# Concatenate them\n",
    "raw = concatenate_raws(parts)\n",
    "\n",
    "# Find the events in this dataset\n",
    "events = mne.find_events(raw, shortest_event=0, stim_channel='STI 014')\n",
    "\n",
    "# Use only EEG channels\n",
    "eeg_channel_inds = mne.pick_types(raw.info, meg=False, eeg=True, stim=False, eog=False,\n",
    "                   exclude='bads')\n",
    "\n",
    "# Extract trials, only using EEG channels\n",
    "epoched = mne.Epochs(raw, events, dict(hands=2, feet=3), tmin=1, tmax=4.1, proj=False, picks=eeg_channel_inds,\n",
    "                baseline=None, preload=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert data to Braindecode Format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from braindecode.datautil.signal_target import SignalAndTarget\n",
    "from braindecode.datautil.splitters import split_into_two_sets\n",
    "# Convert data from volt to millivolt\n",
    "# Pytorch expects float32 for input and int64 for labels.\n",
    "X = (epoched.get_data() * 1e6).astype(np.float32)\n",
    "y = (epoched.events[:,2] - 2).astype(np.int64) #2,3 -> 0,1\n",
    "\n",
    "train_set = SignalAndTarget(X[:60], y=y[:60])\n",
    "test_set = SignalAndTarget(X[60:], y=y[60:])\n",
    "\n",
    "train_set, valid_set = split_into_two_sets(train_set, first_set_fraction=0.8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from braindecode.models.shallow_fbcsp import ShallowFBCSPNet\n",
    "from torch import nn\n",
    "from braindecode.torch_ext.util import set_random_seeds\n",
    "from braindecode.models.util import to_dense_prediction_model\n",
    "\n",
    "# Set if you want to use GPU\n",
    "# You can also use torch.cuda.is_available() to determine if cuda is available on your machine.\n",
    "cuda = False\n",
    "set_random_seeds(seed=20170629, cuda=cuda)\n",
    "\n",
    "# This will determine how many crops are processed in parallel\n",
    "input_time_length = 450\n",
    "n_classes = 2\n",
    "in_chans = train_set.X.shape[1]\n",
    "# final_conv_length determines the size of the receptive field of the ConvNet\n",
    "model = ShallowFBCSPNet(in_chans=in_chans, n_classes=n_classes, input_time_length=input_time_length,\n",
    "                        final_conv_length=12).create_network()\n",
    "to_dense_prediction_model(model)\n",
    "\n",
    "if cuda:\n",
    "    model.cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch import optim\n",
    "\n",
    "optimizer = optim.Adam(model.parameters())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "187 predictions per input/trial\n"
     ]
    }
   ],
   "source": [
    "from braindecode.torch_ext.util import np_to_var\n",
    "# determine output size\n",
    "test_input = np_to_var(np.ones((2, in_chans, input_time_length, 1), dtype=np.float32))\n",
    "if cuda:\n",
    "    test_input = test_input.cuda()\n",
    "out = model(test_input)\n",
    "n_preds_per_input = out.cpu().data.numpy().shape[2]\n",
    "print(\"{:d} predictions per input/trial\".format(n_preds_per_input))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to setup everything for the experiment: Iterator, loss function, monitors and stop criterion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from braindecode.experiments.experiment import Experiment\n",
    "from braindecode.datautil.iterators import CropsFromTrialsIterator\n",
    "from braindecode.experiments.monitors import RuntimeMonitor, LossMonitor, CroppedTrialMisclassMonitor, MisclassMonitor\n",
    "from braindecode.experiments.stopcriteria import MaxEpochs\n",
    "import torch.nn.functional as F\n",
    "import torch as th\n",
    "from braindecode.torch_ext.modules import Expression\n",
    "# Iterator is used to iterate over datasets both for training\n",
    "# and evaluation\n",
    "iterator = CropsFromTrialsIterator(batch_size=32,input_time_length=input_time_length,\n",
    "                                  n_preds_per_input=n_preds_per_input)\n",
    "\n",
    "# Loss function takes predictions as they come out of the network and the targets\n",
    "# and returns a loss\n",
    "loss_function = lambda preds, targets: F.nll_loss(th.mean(preds, dim=2, keepdim=False), targets)\n",
    "\n",
    "# Could be used to apply some constraint on the models, then should be object\n",
    "# with apply method that accepts a module\n",
    "model_constraint = None\n",
    "# Monitors log the training progress\n",
    "monitors = [LossMonitor(), MisclassMonitor(col_suffix='sample_misclass'),\n",
    "            CroppedTrialMisclassMonitor(input_time_length), RuntimeMonitor(),]\n",
    "# Stop criterion determines when the first stop happens\n",
    "stop_criterion = MaxEpochs(20)\n",
    "exp = Experiment(model, train_set, valid_set, test_set, iterator, loss_function, optimizer, model_constraint,\n",
    "          monitors, stop_criterion, remember_best_column='valid_misclass',\n",
    "          run_after_early_stop=True, batch_modifier=None, cuda=cuda)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-10-18 14:35:26,018 INFO : Run until first stop...\n",
      "2017-10-18 14:35:27,899 INFO : Epoch 0\n",
      "2017-10-18 14:35:27,904 INFO : train_loss                0.86930\n",
      "2017-10-18 14:35:27,908 INFO : valid_loss                0.74838\n",
      "2017-10-18 14:35:27,909 INFO : test_loss                 0.69756\n",
      "2017-10-18 14:35:27,910 INFO : train_sample_misclass     0.53894\n",
      "2017-10-18 14:35:27,912 INFO : valid_sample_misclass     0.47103\n",
      "2017-10-18 14:35:27,913 INFO : test_sample_misclass      0.44251\n",
      "2017-10-18 14:35:27,914 INFO : train_misclass            0.60417\n",
      "2017-10-18 14:35:27,915 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:35:27,916 INFO : test_misclass             0.40000\n",
      "2017-10-18 14:35:27,917 INFO : runtime                   0.00000\n",
      "2017-10-18 14:35:27,919 INFO : \n",
      "2017-10-18 14:35:27,922 INFO : New best valid_misclass: 0.500000\n",
      "2017-10-18 14:35:27,924 INFO : \n",
      "2017-10-18 14:35:30,830 INFO : Time only for training updates: 2.87s\n",
      "2017-10-18 14:35:32,663 INFO : Epoch 1\n",
      "2017-10-18 14:35:32,667 INFO : train_loss                2.33626\n",
      "2017-10-18 14:35:32,671 INFO : valid_loss                2.31771\n",
      "2017-10-18 14:35:32,674 INFO : test_loss                 2.14077\n",
      "2017-10-18 14:35:32,677 INFO : train_sample_misclass     0.48279\n",
      "2017-10-18 14:35:32,681 INFO : valid_sample_misclass     0.50000\n",
      "2017-10-18 14:35:32,684 INFO : test_sample_misclass      0.46667\n",
      "2017-10-18 14:35:32,687 INFO : train_misclass            0.50000\n",
      "2017-10-18 14:35:32,688 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:35:32,690 INFO : test_misclass             0.46667\n",
      "2017-10-18 14:35:32,691 INFO : runtime                   4.81331\n",
      "2017-10-18 14:35:32,694 INFO : \n",
      "2017-10-18 14:35:32,697 INFO : New best valid_misclass: 0.500000\n",
      "2017-10-18 14:35:32,698 INFO : \n",
      "2017-10-18 14:35:35,105 INFO : Time only for training updates: 2.39s\n",
      "2017-10-18 14:35:37,198 INFO : Epoch 2\n",
      "2017-10-18 14:35:37,203 INFO : train_loss                0.59815\n",
      "2017-10-18 14:35:37,206 INFO : valid_loss                0.78503\n",
      "2017-10-18 14:35:37,208 INFO : test_loss                 0.70060\n",
      "2017-10-18 14:35:37,209 INFO : train_sample_misclass     0.33918\n",
      "2017-10-18 14:35:37,210 INFO : valid_sample_misclass     0.47995\n",
      "2017-10-18 14:35:37,212 INFO : test_sample_misclass      0.41996\n",
      "2017-10-18 14:35:37,213 INFO : train_misclass            0.22917\n",
      "2017-10-18 14:35:37,214 INFO : valid_misclass            0.41667\n",
      "2017-10-18 14:35:37,215 INFO : test_misclass             0.43333\n",
      "2017-10-18 14:35:37,217 INFO : runtime                   4.29139\n",
      "2017-10-18 14:35:37,218 INFO : \n",
      "2017-10-18 14:35:37,221 INFO : New best valid_misclass: 0.416667\n",
      "2017-10-18 14:35:37,222 INFO : \n",
      "2017-10-18 14:35:39,655 INFO : Time only for training updates: 2.42s\n",
      "2017-10-18 14:35:41,575 INFO : Epoch 3\n",
      "2017-10-18 14:35:41,577 INFO : train_loss                0.53122\n",
      "2017-10-18 14:35:41,578 INFO : valid_loss                0.87284\n",
      "2017-10-18 14:35:41,579 INFO : test_loss                 0.69774\n",
      "2017-10-18 14:35:41,580 INFO : train_sample_misclass     0.24588\n",
      "2017-10-18 14:35:41,581 INFO : valid_sample_misclass     0.55548\n",
      "2017-10-18 14:35:41,582 INFO : test_sample_misclass      0.42709\n",
      "2017-10-18 14:35:41,583 INFO : train_misclass            0.22917\n",
      "2017-10-18 14:35:41,585 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:35:41,586 INFO : test_misclass             0.36667\n",
      "2017-10-18 14:35:41,587 INFO : runtime                   4.53691\n",
      "2017-10-18 14:35:41,588 INFO : \n",
      "2017-10-18 14:35:44,007 INFO : Time only for training updates: 2.40s\n",
      "2017-10-18 14:35:45,935 INFO : Epoch 4\n",
      "2017-10-18 14:35:45,939 INFO : train_loss                0.51574\n",
      "2017-10-18 14:35:45,943 INFO : valid_loss                1.10377\n",
      "2017-10-18 14:35:45,946 INFO : test_loss                 0.80095\n",
      "2017-10-18 14:35:45,949 INFO : train_sample_misclass     0.25869\n",
      "2017-10-18 14:35:45,952 INFO : valid_sample_misclass     0.60094\n",
      "2017-10-18 14:35:45,956 INFO : test_sample_misclass      0.42727\n",
      "2017-10-18 14:35:45,959 INFO : train_misclass            0.27083\n",
      "2017-10-18 14:35:45,960 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:35:45,961 INFO : test_misclass             0.46667\n",
      "2017-10-18 14:35:45,966 INFO : runtime                   4.35103\n",
      "2017-10-18 14:35:45,967 INFO : \n",
      "2017-10-18 14:35:48,389 INFO : Time only for training updates: 2.40s\n",
      "2017-10-18 14:35:50,406 INFO : Epoch 5\n",
      "2017-10-18 14:35:50,417 INFO : train_loss                0.52613\n",
      "2017-10-18 14:35:50,419 INFO : valid_loss                1.22322\n",
      "2017-10-18 14:35:50,421 INFO : test_loss                 0.86285\n",
      "2017-10-18 14:35:50,423 INFO : train_sample_misclass     0.27635\n",
      "2017-10-18 14:35:50,425 INFO : valid_sample_misclass     0.58445\n",
      "2017-10-18 14:35:50,426 INFO : test_sample_misclass      0.43841\n",
      "2017-10-18 14:35:50,428 INFO : train_misclass            0.33333\n",
      "2017-10-18 14:35:50,430 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:35:50,432 INFO : test_misclass             0.46667\n",
      "2017-10-18 14:35:50,433 INFO : runtime                   4.38171\n",
      "2017-10-18 14:35:50,435 INFO : \n",
      "2017-10-18 14:35:53,064 INFO : Time only for training updates: 2.61s\n",
      "2017-10-18 14:35:55,069 INFO : Epoch 6\n",
      "2017-10-18 14:35:55,074 INFO : train_loss                0.54906\n",
      "2017-10-18 14:35:55,077 INFO : valid_loss                1.23573\n",
      "2017-10-18 14:35:55,080 INFO : test_loss                 0.92210\n",
      "2017-10-18 14:35:55,082 INFO : train_sample_misclass     0.28816\n",
      "2017-10-18 14:35:55,083 INFO : valid_sample_misclass     0.53810\n",
      "2017-10-18 14:35:55,084 INFO : test_sample_misclass      0.45205\n",
      "2017-10-18 14:35:55,085 INFO : train_misclass            0.33333\n",
      "2017-10-18 14:35:55,086 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:35:55,087 INFO : test_misclass             0.46667\n",
      "2017-10-18 14:35:55,089 INFO : runtime                   4.67268\n",
      "2017-10-18 14:35:55,090 INFO : \n",
      "2017-10-18 14:35:57,534 INFO : Time only for training updates: 2.42s\n",
      "2017-10-18 14:35:59,286 INFO : Epoch 7\n",
      "2017-10-18 14:35:59,287 INFO : train_loss                0.59627\n",
      "2017-10-18 14:35:59,289 INFO : valid_loss                1.16602\n",
      "2017-10-18 14:35:59,290 INFO : test_loss                 0.95655\n",
      "2017-10-18 14:35:59,291 INFO : train_sample_misclass     0.32253\n",
      "2017-10-18 14:35:59,292 INFO : valid_sample_misclass     0.53053\n",
      "2017-10-18 14:35:59,293 INFO : test_sample_misclass      0.46346\n",
      "2017-10-18 14:35:59,294 INFO : train_misclass            0.37500\n",
      "2017-10-18 14:35:59,295 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:35:59,296 INFO : test_misclass             0.46667\n",
      "2017-10-18 14:35:59,297 INFO : runtime                   4.47253\n",
      "2017-10-18 14:35:59,298 INFO : \n",
      "2017-10-18 14:36:01,422 INFO : Time only for training updates: 2.11s\n",
      "2017-10-18 14:36:02,798 INFO : Epoch 8\n",
      "2017-10-18 14:36:02,800 INFO : train_loss                0.40527\n",
      "2017-10-18 14:36:02,801 INFO : valid_loss                0.95821\n",
      "2017-10-18 14:36:02,802 INFO : test_loss                 0.79267\n",
      "2017-10-18 14:36:02,802 INFO : train_sample_misclass     0.19112\n",
      "2017-10-18 14:36:02,803 INFO : valid_sample_misclass     0.52607\n",
      "2017-10-18 14:36:02,804 INFO : test_sample_misclass      0.43351\n",
      "2017-10-18 14:36:02,805 INFO : train_misclass            0.16667\n",
      "2017-10-18 14:36:02,806 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:36:02,807 INFO : test_misclass             0.40000\n",
      "2017-10-18 14:36:02,808 INFO : runtime                   3.88520\n",
      "2017-10-18 14:36:02,809 INFO : \n",
      "2017-10-18 14:36:04,816 INFO : Time only for training updates: 2.00s\n",
      "2017-10-18 14:36:06,168 INFO : Epoch 9\n",
      "2017-10-18 14:36:06,169 INFO : train_loss                0.30290\n",
      "2017-10-18 14:36:06,171 INFO : valid_loss                0.89595\n",
      "2017-10-18 14:36:06,172 INFO : test_loss                 0.75205\n",
      "2017-10-18 14:36:06,172 INFO : train_sample_misclass     0.10968\n",
      "2017-10-18 14:36:06,173 INFO : valid_sample_misclass     0.56684\n",
      "2017-10-18 14:36:06,174 INFO : test_sample_misclass      0.43119\n",
      "2017-10-18 14:36:06,175 INFO : train_misclass            0.08333\n",
      "2017-10-18 14:36:06,176 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:36:06,177 INFO : test_misclass             0.36667\n",
      "2017-10-18 14:36:06,178 INFO : runtime                   3.39363\n",
      "2017-10-18 14:36:06,179 INFO : \n",
      "2017-10-18 14:36:08,220 INFO : Time only for training updates: 2.03s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-10-18 14:36:09,672 INFO : Epoch 10\n",
      "2017-10-18 14:36:09,674 INFO : train_loss                0.25623\n",
      "2017-10-18 14:36:09,675 INFO : valid_loss                0.89555\n",
      "2017-10-18 14:36:09,676 INFO : test_loss                 0.75208\n",
      "2017-10-18 14:36:09,678 INFO : train_sample_misclass     0.06088\n",
      "2017-10-18 14:36:09,679 INFO : valid_sample_misclass     0.55637\n",
      "2017-10-18 14:36:09,680 INFO : test_sample_misclass      0.41765\n",
      "2017-10-18 14:36:09,681 INFO : train_misclass            0.02083\n",
      "2017-10-18 14:36:09,682 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:36:09,683 INFO : test_misclass             0.33333\n",
      "2017-10-18 14:36:09,684 INFO : runtime                   3.40419\n",
      "2017-10-18 14:36:09,685 INFO : \n",
      "2017-10-18 14:36:11,601 INFO : Time only for training updates: 1.90s\n",
      "2017-10-18 14:36:12,948 INFO : Epoch 11\n",
      "2017-10-18 14:36:12,950 INFO : train_loss                0.24021\n",
      "2017-10-18 14:36:12,951 INFO : valid_loss                0.90580\n",
      "2017-10-18 14:36:12,953 INFO : test_loss                 0.75648\n",
      "2017-10-18 14:36:12,954 INFO : train_sample_misclass     0.06205\n",
      "2017-10-18 14:36:12,956 INFO : valid_sample_misclass     0.54256\n",
      "2017-10-18 14:36:12,957 INFO : test_sample_misclass      0.39804\n",
      "2017-10-18 14:36:12,958 INFO : train_misclass            0.02083\n",
      "2017-10-18 14:36:12,960 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:36:12,961 INFO : test_misclass             0.33333\n",
      "2017-10-18 14:36:12,963 INFO : runtime                   3.38414\n",
      "2017-10-18 14:36:12,964 INFO : \n",
      "2017-10-18 14:36:14,862 INFO : Time only for training updates: 1.88s\n",
      "2017-10-18 14:36:16,237 INFO : Epoch 12\n",
      "2017-10-18 14:36:16,238 INFO : train_loss                0.20491\n",
      "2017-10-18 14:36:16,240 INFO : valid_loss                0.90257\n",
      "2017-10-18 14:36:16,241 INFO : test_loss                 0.75600\n",
      "2017-10-18 14:36:16,242 INFO : train_sample_misclass     0.05041\n",
      "2017-10-18 14:36:16,244 INFO : valid_sample_misclass     0.54100\n",
      "2017-10-18 14:36:16,245 INFO : test_sample_misclass      0.39501\n",
      "2017-10-18 14:36:16,247 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:16,248 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:36:16,250 INFO : test_misclass             0.33333\n",
      "2017-10-18 14:36:16,251 INFO : runtime                   3.26028\n",
      "2017-10-18 14:36:16,253 INFO : \n",
      "2017-10-18 14:36:18,224 INFO : Time only for training updates: 1.95s\n",
      "2017-10-18 14:36:19,583 INFO : Epoch 13\n",
      "2017-10-18 14:36:19,584 INFO : train_loss                0.17244\n",
      "2017-10-18 14:36:19,585 INFO : valid_loss                0.90562\n",
      "2017-10-18 14:36:19,588 INFO : test_loss                 0.76539\n",
      "2017-10-18 14:36:19,590 INFO : train_sample_misclass     0.03793\n",
      "2017-10-18 14:36:19,592 INFO : valid_sample_misclass     0.56194\n",
      "2017-10-18 14:36:19,594 INFO : test_sample_misclass      0.38752\n",
      "2017-10-18 14:36:19,595 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:19,596 INFO : valid_misclass            0.66667\n",
      "2017-10-18 14:36:19,600 INFO : test_misclass             0.36667\n",
      "2017-10-18 14:36:19,601 INFO : runtime                   3.35971\n",
      "2017-10-18 14:36:19,602 INFO : \n",
      "2017-10-18 14:36:21,657 INFO : Time only for training updates: 2.04s\n",
      "2017-10-18 14:36:23,308 INFO : Epoch 14\n",
      "2017-10-18 14:36:23,310 INFO : train_loss                0.15207\n",
      "2017-10-18 14:36:23,311 INFO : valid_loss                0.90600\n",
      "2017-10-18 14:36:23,312 INFO : test_loss                 0.76428\n",
      "2017-10-18 14:36:23,313 INFO : train_sample_misclass     0.02969\n",
      "2017-10-18 14:36:23,315 INFO : valid_sample_misclass     0.54924\n",
      "2017-10-18 14:36:23,316 INFO : test_sample_misclass      0.36551\n",
      "2017-10-18 14:36:23,317 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:23,318 INFO : valid_misclass            0.58333\n",
      "2017-10-18 14:36:23,319 INFO : test_misclass             0.36667\n",
      "2017-10-18 14:36:23,320 INFO : runtime                   3.43644\n",
      "2017-10-18 14:36:23,322 INFO : \n",
      "2017-10-18 14:36:25,291 INFO : Time only for training updates: 1.95s\n",
      "2017-10-18 14:36:26,659 INFO : Epoch 15\n",
      "2017-10-18 14:36:26,660 INFO : train_loss                0.14998\n",
      "2017-10-18 14:36:26,662 INFO : valid_loss                0.96410\n",
      "2017-10-18 14:36:26,663 INFO : test_loss                 0.81256\n",
      "2017-10-18 14:36:26,665 INFO : train_sample_misclass     0.04027\n",
      "2017-10-18 14:36:26,666 INFO : valid_sample_misclass     0.48061\n",
      "2017-10-18 14:36:26,668 INFO : test_sample_misclass      0.35811\n",
      "2017-10-18 14:36:26,669 INFO : train_misclass            0.02083\n",
      "2017-10-18 14:36:26,671 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:36:26,672 INFO : test_misclass             0.33333\n",
      "2017-10-18 14:36:26,674 INFO : runtime                   3.63001\n",
      "2017-10-18 14:36:26,675 INFO : \n",
      "2017-10-18 14:36:28,567 INFO : Time only for training updates: 1.88s\n",
      "2017-10-18 14:36:29,934 INFO : Epoch 16\n",
      "2017-10-18 14:36:29,935 INFO : train_loss                0.10915\n",
      "2017-10-18 14:36:29,936 INFO : valid_loss                0.91489\n",
      "2017-10-18 14:36:29,937 INFO : test_loss                 0.80608\n",
      "2017-10-18 14:36:29,938 INFO : train_sample_misclass     0.01588\n",
      "2017-10-18 14:36:29,939 INFO : valid_sample_misclass     0.54234\n",
      "2017-10-18 14:36:29,940 INFO : test_sample_misclass      0.34537\n",
      "2017-10-18 14:36:29,941 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:29,942 INFO : valid_misclass            0.33333\n",
      "2017-10-18 14:36:29,943 INFO : test_misclass             0.33333\n",
      "2017-10-18 14:36:29,943 INFO : runtime                   3.27961\n",
      "2017-10-18 14:36:29,944 INFO : \n",
      "2017-10-18 14:36:29,946 INFO : New best valid_misclass: 0.333333\n",
      "2017-10-18 14:36:29,947 INFO : \n",
      "2017-10-18 14:36:31,906 INFO : Time only for training updates: 1.95s\n",
      "2017-10-18 14:36:33,248 INFO : Epoch 17\n",
      "2017-10-18 14:36:33,249 INFO : train_loss                0.15003\n",
      "2017-10-18 14:36:33,250 INFO : valid_loss                0.91323\n",
      "2017-10-18 14:36:33,251 INFO : test_loss                 0.81002\n",
      "2017-10-18 14:36:33,252 INFO : train_sample_misclass     0.05403\n",
      "2017-10-18 14:36:33,253 INFO : valid_sample_misclass     0.51337\n",
      "2017-10-18 14:36:33,254 INFO : test_sample_misclass      0.34403\n",
      "2017-10-18 14:36:33,255 INFO : train_misclass            0.04167\n",
      "2017-10-18 14:36:33,256 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:36:33,256 INFO : test_misclass             0.23333\n",
      "2017-10-18 14:36:33,257 INFO : runtime                   3.33830\n",
      "2017-10-18 14:36:33,258 INFO : \n",
      "2017-10-18 14:36:35,205 INFO : Time only for training updates: 1.94s\n",
      "2017-10-18 14:36:36,576 INFO : Epoch 18\n",
      "2017-10-18 14:36:36,577 INFO : train_loss                0.09315\n",
      "2017-10-18 14:36:36,578 INFO : valid_loss                0.89064\n",
      "2017-10-18 14:36:36,580 INFO : test_loss                 0.77360\n",
      "2017-10-18 14:36:36,581 INFO : train_sample_misclass     0.01025\n",
      "2017-10-18 14:36:36,583 INFO : valid_sample_misclass     0.49666\n",
      "2017-10-18 14:36:36,584 INFO : test_sample_misclass      0.30873\n",
      "2017-10-18 14:36:36,586 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:36,587 INFO : valid_misclass            0.33333\n",
      "2017-10-18 14:36:36,588 INFO : test_misclass             0.33333\n",
      "2017-10-18 14:36:36,589 INFO : runtime                   3.30138\n",
      "2017-10-18 14:36:36,590 INFO : \n",
      "2017-10-18 14:36:36,593 INFO : New best valid_misclass: 0.333333\n",
      "2017-10-18 14:36:36,594 INFO : \n",
      "2017-10-18 14:36:38,535 INFO : Time only for training updates: 1.93s\n",
      "2017-10-18 14:36:39,895 INFO : Epoch 19\n",
      "2017-10-18 14:36:39,896 INFO : train_loss                0.09408\n",
      "2017-10-18 14:36:39,897 INFO : valid_loss                0.96349\n",
      "2017-10-18 14:36:39,898 INFO : test_loss                 0.84831\n",
      "2017-10-18 14:36:39,899 INFO : train_sample_misclass     0.02078\n",
      "2017-10-18 14:36:39,900 INFO : valid_sample_misclass     0.49042\n",
      "2017-10-18 14:36:39,901 INFO : test_sample_misclass      0.33137\n",
      "2017-10-18 14:36:39,901 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:39,902 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:36:39,903 INFO : test_misclass             0.30000\n",
      "2017-10-18 14:36:39,904 INFO : runtime                   3.32518\n",
      "2017-10-18 14:36:39,905 INFO : \n",
      "2017-10-18 14:36:41,863 INFO : Time only for training updates: 1.95s\n",
      "2017-10-18 14:36:43,294 INFO : Epoch 20\n",
      "2017-10-18 14:36:43,295 INFO : train_loss                0.06822\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-10-18 14:36:43,296 INFO : valid_loss                1.05598\n",
      "2017-10-18 14:36:43,297 INFO : test_loss                 0.88760\n",
      "2017-10-18 14:36:43,298 INFO : train_sample_misclass     0.00847\n",
      "2017-10-18 14:36:43,300 INFO : valid_sample_misclass     0.51448\n",
      "2017-10-18 14:36:43,301 INFO : test_sample_misclass      0.33066\n",
      "2017-10-18 14:36:43,302 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:43,303 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:36:43,305 INFO : test_misclass             0.36667\n",
      "2017-10-18 14:36:43,306 INFO : runtime                   3.33038\n",
      "2017-10-18 14:36:43,307 INFO : \n",
      "2017-10-18 14:36:43,308 INFO : Setup for second stop...\n",
      "2017-10-18 14:36:43,314 INFO : Train loss to reach 0.09315\n",
      "2017-10-18 14:36:43,315 INFO : Run until second stop...\n",
      "2017-10-18 14:36:44,773 INFO : Epoch 19\n",
      "2017-10-18 14:36:44,774 INFO : train_loss                0.25265\n",
      "2017-10-18 14:36:44,776 INFO : valid_loss                0.89064\n",
      "2017-10-18 14:36:44,777 INFO : test_loss                 0.77360\n",
      "2017-10-18 14:36:44,778 INFO : train_sample_misclass     0.10753\n",
      "2017-10-18 14:36:44,779 INFO : valid_sample_misclass     0.49666\n",
      "2017-10-18 14:36:44,780 INFO : test_sample_misclass      0.30873\n",
      "2017-10-18 14:36:44,781 INFO : train_misclass            0.06667\n",
      "2017-10-18 14:36:44,782 INFO : valid_misclass            0.33333\n",
      "2017-10-18 14:36:44,783 INFO : test_misclass             0.33333\n",
      "2017-10-18 14:36:44,784 INFO : runtime                   1.45227\n",
      "2017-10-18 14:36:44,785 INFO : \n",
      "2017-10-18 14:36:47,428 INFO : Time only for training updates: 2.62s\n",
      "2017-10-18 14:36:49,022 INFO : Epoch 20\n",
      "2017-10-18 14:36:49,024 INFO : train_loss                0.21080\n",
      "2017-10-18 14:36:49,025 INFO : valid_loss                0.68103\n",
      "2017-10-18 14:36:49,026 INFO : test_loss                 0.73509\n",
      "2017-10-18 14:36:49,028 INFO : train_sample_misclass     0.08324\n",
      "2017-10-18 14:36:49,028 INFO : valid_sample_misclass     0.34915\n",
      "2017-10-18 14:36:49,029 INFO : test_sample_misclass      0.31025\n",
      "2017-10-18 14:36:49,030 INFO : train_misclass            0.10000\n",
      "2017-10-18 14:36:49,031 INFO : valid_misclass            0.50000\n",
      "2017-10-18 14:36:49,032 INFO : test_misclass             0.26667\n",
      "2017-10-18 14:36:49,033 INFO : runtime                   4.11350\n",
      "2017-10-18 14:36:49,033 INFO : \n",
      "2017-10-18 14:36:51,467 INFO : Time only for training updates: 2.42s\n",
      "2017-10-18 14:36:53,069 INFO : Epoch 21\n",
      "2017-10-18 14:36:53,071 INFO : train_loss                0.18816\n",
      "2017-10-18 14:36:53,072 INFO : valid_loss                0.46174\n",
      "2017-10-18 14:36:53,073 INFO : test_loss                 0.66965\n",
      "2017-10-18 14:36:53,074 INFO : train_sample_misclass     0.07406\n",
      "2017-10-18 14:36:53,075 INFO : valid_sample_misclass     0.24643\n",
      "2017-10-18 14:36:53,076 INFO : test_sample_misclass      0.28066\n",
      "2017-10-18 14:36:53,077 INFO : train_misclass            0.05000\n",
      "2017-10-18 14:36:53,077 INFO : valid_misclass            0.16667\n",
      "2017-10-18 14:36:53,078 INFO : test_misclass             0.26667\n",
      "2017-10-18 14:36:53,079 INFO : runtime                   4.03852\n",
      "2017-10-18 14:36:53,080 INFO : \n",
      "2017-10-18 14:36:55,529 INFO : Time only for training updates: 2.44s\n",
      "2017-10-18 14:36:57,074 INFO : Epoch 22\n",
      "2017-10-18 14:36:57,076 INFO : train_loss                0.13663\n",
      "2017-10-18 14:36:57,077 INFO : valid_loss                0.28806\n",
      "2017-10-18 14:36:57,078 INFO : test_loss                 0.63336\n",
      "2017-10-18 14:36:57,079 INFO : train_sample_misclass     0.02567\n",
      "2017-10-18 14:36:57,080 INFO : valid_sample_misclass     0.06796\n",
      "2017-10-18 14:36:57,081 INFO : test_sample_misclass      0.25677\n",
      "2017-10-18 14:36:57,082 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:36:57,083 INFO : valid_misclass            0.00000\n",
      "2017-10-18 14:36:57,084 INFO : test_misclass             0.23333\n",
      "2017-10-18 14:36:57,085 INFO : runtime                   4.06202\n",
      "2017-10-18 14:36:57,086 INFO : \n",
      "2017-10-18 14:36:59,625 INFO : Time only for training updates: 2.53s\n",
      "2017-10-18 14:37:01,160 INFO : Epoch 23\n",
      "2017-10-18 14:37:01,161 INFO : train_loss                0.18469\n",
      "2017-10-18 14:37:01,162 INFO : valid_loss                0.33505\n",
      "2017-10-18 14:37:01,163 INFO : test_loss                 0.74569\n",
      "2017-10-18 14:37:01,164 INFO : train_sample_misclass     0.07469\n",
      "2017-10-18 14:37:01,165 INFO : valid_sample_misclass     0.15575\n",
      "2017-10-18 14:37:01,166 INFO : test_sample_misclass      0.29652\n",
      "2017-10-18 14:37:01,167 INFO : train_misclass            0.05000\n",
      "2017-10-18 14:37:01,168 INFO : valid_misclass            0.08333\n",
      "2017-10-18 14:37:01,168 INFO : test_misclass             0.26667\n",
      "2017-10-18 14:37:01,169 INFO : runtime                   4.09383\n",
      "2017-10-18 14:37:01,170 INFO : \n",
      "2017-10-18 14:37:03,695 INFO : Time only for training updates: 2.51s\n",
      "2017-10-18 14:37:05,297 INFO : Epoch 24\n",
      "2017-10-18 14:37:05,299 INFO : train_loss                0.12348\n",
      "2017-10-18 14:37:05,300 INFO : valid_loss                0.28360\n",
      "2017-10-18 14:37:05,301 INFO : test_loss                 0.78044\n",
      "2017-10-18 14:37:05,302 INFO : train_sample_misclass     0.03632\n",
      "2017-10-18 14:37:05,303 INFO : valid_sample_misclass     0.10495\n",
      "2017-10-18 14:37:05,304 INFO : test_sample_misclass      0.28529\n",
      "2017-10-18 14:37:05,305 INFO : train_misclass            0.03333\n",
      "2017-10-18 14:37:05,306 INFO : valid_misclass            0.08333\n",
      "2017-10-18 14:37:05,307 INFO : test_misclass             0.26667\n",
      "2017-10-18 14:37:05,308 INFO : runtime                   4.06927\n",
      "2017-10-18 14:37:05,310 INFO : \n",
      "2017-10-18 14:37:07,772 INFO : Time only for training updates: 2.44s\n",
      "2017-10-18 14:37:09,352 INFO : Epoch 25\n",
      "2017-10-18 14:37:09,354 INFO : train_loss                0.14714\n",
      "2017-10-18 14:37:09,355 INFO : valid_loss                0.19252\n",
      "2017-10-18 14:37:09,356 INFO : test_loss                 0.77621\n",
      "2017-10-18 14:37:09,357 INFO : train_sample_misclass     0.04635\n",
      "2017-10-18 14:37:09,359 INFO : valid_sample_misclass     0.05660\n",
      "2017-10-18 14:37:09,360 INFO : test_sample_misclass      0.26863\n",
      "2017-10-18 14:37:09,362 INFO : train_misclass            0.05000\n",
      "2017-10-18 14:37:09,363 INFO : valid_misclass            0.08333\n",
      "2017-10-18 14:37:09,364 INFO : test_misclass             0.23333\n",
      "2017-10-18 14:37:09,366 INFO : runtime                   4.08017\n",
      "2017-10-18 14:37:09,367 INFO : \n",
      "2017-10-18 14:37:11,754 INFO : Time only for training updates: 2.37s\n",
      "2017-10-18 14:37:13,257 INFO : Epoch 26\n",
      "2017-10-18 14:37:13,259 INFO : train_loss                0.33872\n",
      "2017-10-18 14:37:13,260 INFO : valid_loss                0.33738\n",
      "2017-10-18 14:37:13,262 INFO : test_loss                 0.75610\n",
      "2017-10-18 14:37:13,264 INFO : train_sample_misclass     0.16430\n",
      "2017-10-18 14:37:13,265 INFO : valid_sample_misclass     0.16689\n",
      "2017-10-18 14:37:13,266 INFO : test_sample_misclass      0.27763\n",
      "2017-10-18 14:37:13,267 INFO : train_misclass            0.11667\n",
      "2017-10-18 14:37:13,269 INFO : valid_misclass            0.16667\n",
      "2017-10-18 14:37:13,270 INFO : test_misclass             0.26667\n",
      "2017-10-18 14:37:13,271 INFO : runtime                   3.98591\n",
      "2017-10-18 14:37:13,272 INFO : \n",
      "2017-10-18 14:37:15,774 INFO : Time only for training updates: 2.48s\n",
      "2017-10-18 14:37:17,314 INFO : Epoch 27\n",
      "2017-10-18 14:37:17,315 INFO : train_loss                0.12509\n",
      "2017-10-18 14:37:17,316 INFO : valid_loss                0.14952\n",
      "2017-10-18 14:37:17,317 INFO : test_loss                 0.60954\n",
      "2017-10-18 14:37:17,318 INFO : train_sample_misclass     0.03111\n",
      "2017-10-18 14:37:17,318 INFO : valid_sample_misclass     0.02718\n",
      "2017-10-18 14:37:17,319 INFO : test_sample_misclass      0.23351\n",
      "2017-10-18 14:37:17,320 INFO : train_misclass            0.01667\n",
      "2017-10-18 14:37:17,321 INFO : valid_misclass            0.00000\n",
      "2017-10-18 14:37:17,321 INFO : test_misclass             0.16667\n",
      "2017-10-18 14:37:17,322 INFO : runtime                   4.01630\n",
      "2017-10-18 14:37:17,323 INFO : \n",
      "2017-10-18 14:37:19,768 INFO : Time only for training updates: 2.43s\n",
      "2017-10-18 14:37:21,331 INFO : Epoch 28\n",
      "2017-10-18 14:37:21,333 INFO : train_loss                0.09141\n",
      "2017-10-18 14:37:21,334 INFO : valid_loss                0.15032\n",
      "2017-10-18 14:37:21,335 INFO : test_loss                 0.64890\n",
      "2017-10-18 14:37:21,336 INFO : train_sample_misclass     0.01239\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-10-18 14:37:21,337 INFO : valid_sample_misclass     0.03387\n",
      "2017-10-18 14:37:21,338 INFO : test_sample_misclass      0.25856\n",
      "2017-10-18 14:37:21,339 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:37:21,339 INFO : valid_misclass            0.00000\n",
      "2017-10-18 14:37:21,340 INFO : test_misclass             0.26667\n",
      "2017-10-18 14:37:21,341 INFO : runtime                   3.99343\n",
      "2017-10-18 14:37:21,342 INFO : \n",
      "2017-10-18 14:37:23,868 INFO : Time only for training updates: 2.51s\n",
      "2017-10-18 14:37:25,405 INFO : Epoch 29\n",
      "2017-10-18 14:37:25,406 INFO : train_loss                0.12664\n",
      "2017-10-18 14:37:25,407 INFO : valid_loss                0.21718\n",
      "2017-10-18 14:37:25,408 INFO : test_loss                 0.74796\n",
      "2017-10-18 14:37:25,409 INFO : train_sample_misclass     0.04274\n",
      "2017-10-18 14:37:25,410 INFO : valid_sample_misclass     0.08623\n",
      "2017-10-18 14:37:25,411 INFO : test_sample_misclass      0.26381\n",
      "2017-10-18 14:37:25,411 INFO : train_misclass            0.03333\n",
      "2017-10-18 14:37:25,412 INFO : valid_misclass            0.08333\n",
      "2017-10-18 14:37:25,413 INFO : test_misclass             0.23333\n",
      "2017-10-18 14:37:25,414 INFO : runtime                   4.09971\n",
      "2017-10-18 14:37:25,415 INFO : \n",
      "2017-10-18 14:37:27,883 INFO : Time only for training updates: 2.46s\n",
      "2017-10-18 14:37:29,539 INFO : Epoch 30\n",
      "2017-10-18 14:37:29,541 INFO : train_loss                0.07408\n",
      "2017-10-18 14:37:29,542 INFO : valid_loss                0.15273\n",
      "2017-10-18 14:37:29,543 INFO : test_loss                 0.67137\n",
      "2017-10-18 14:37:29,544 INFO : train_sample_misclass     0.01497\n",
      "2017-10-18 14:37:29,546 INFO : valid_sample_misclass     0.06061\n",
      "2017-10-18 14:37:29,547 INFO : test_sample_misclass      0.23984\n",
      "2017-10-18 14:37:29,548 INFO : train_misclass            0.01667\n",
      "2017-10-18 14:37:29,549 INFO : valid_misclass            0.08333\n",
      "2017-10-18 14:37:29,550 INFO : test_misclass             0.20000\n",
      "2017-10-18 14:37:29,551 INFO : runtime                   4.01253\n",
      "2017-10-18 14:37:29,552 INFO : \n",
      "2017-10-18 14:37:32,114 INFO : Time only for training updates: 2.55s\n",
      "2017-10-18 14:37:33,748 INFO : Epoch 31\n",
      "2017-10-18 14:37:33,750 INFO : train_loss                0.05036\n",
      "2017-10-18 14:37:33,751 INFO : valid_loss                0.09104\n",
      "2017-10-18 14:37:33,752 INFO : test_loss                 0.66771\n",
      "2017-10-18 14:37:33,753 INFO : train_sample_misclass     0.00744\n",
      "2017-10-18 14:37:33,755 INFO : valid_sample_misclass     0.02317\n",
      "2017-10-18 14:37:33,756 INFO : test_sample_misclass      0.20677\n",
      "2017-10-18 14:37:33,757 INFO : train_misclass            0.00000\n",
      "2017-10-18 14:37:33,758 INFO : valid_misclass            0.00000\n",
      "2017-10-18 14:37:33,759 INFO : test_misclass             0.20000\n",
      "2017-10-18 14:37:33,760 INFO : runtime                   4.23093\n",
      "2017-10-18 14:37:33,761 INFO : \n"
     ]
    }
   ],
   "source": [
    "# need to setup python logging before to be able to see anything\n",
    "import logging\n",
    "import sys\n",
    "logging.basicConfig(format='%(asctime)s %(levelname)s : %(message)s',\n",
    "                     level=logging.DEBUG, stream=sys.stdout)\n",
    "exp.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, we arrive at 80.0% accuracy, the training stops after the validation loss decreases below the training loss at the best epoch of 0.09315."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset References\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " This dataset was created and contributed to PhysioNet by the developers of the [BCI2000](http://www.schalklab.org/research/bci2000) instrumentation system, which they used in making these recordings. The system is described in:\n",
    " \n",
    "     Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R. (2004) BCI2000: A General-Purpose Brain-Computer Interface (BCI) System. IEEE TBME 51(6):1034-1043.\n",
    "\n",
    "[PhysioBank](https://physionet.org/physiobank/) is a large and growing archive of well-characterized digital recordings of physiologic signals and related data for use by the biomedical research community and further described in:\n",
    "\n",
    "    Goldberger AL, Amaral LAN, Glass L, Hausdorff JM, Ivanov PCh, Mark RG, Mietus JE, Moody GB, Peng C-K, Stanley HE. (2000) PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals. Circulation 101(23):e215-e220."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
