{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "nbsphinx": "hidden"
   },
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "import os\n",
    "os.sys.path.insert(0, '/home/schirrmr/braindecode/code/braindecode/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Read and Decode BBCI Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial shows how to read and decode BBCI data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup logging to see outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "import sys\n",
    "logging.basicConfig(format='%(asctime)s %(levelname)s : %(message)s',\n",
    "                     level=logging.DEBUG, stream=sys.stdout)\n",
    "log = logging.getLogger()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and preprocess data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First set the filename and the sensors you want to load. If you set\n",
    "\n",
    "```python\n",
    "load_sensor_names=None\n",
    "```\n",
    "\n",
    "or just remove the parameter from the function call, all sensors will be loaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating RawArray with float64 data, n_channels=3, n_times=3451320\n",
      "    Range : 0 ... 3451319 =      0.000 ...  6902.638 secs\n",
      "Ready.\n"
     ]
    }
   ],
   "source": [
    "from braindecode.datasets.bbci import BBCIDataset\n",
    "train_filename = '/home/schirrmr/data/BBCI-without-last-runs/BhNoMoSc1S001R01_ds10_1-12.BBCI.mat'\n",
    "cnt = BBCIDataset(train_filename, load_sensor_names=['C3', 'CPz', 'C4']).load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing on continous data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First remove the stimulus channel, than apply any preprocessing you like. There are some very few directions available from Braindecode, such as resample_cnt. But you can apply any function on the chan x time matrix of the mne raw object (`cnt` in the code) by calling `mne_apply` with two arguments:\n",
    "\n",
    "1. Your function (2d-array-> 2darray), that transforms the channel x timesteps data array\n",
    "2. the Raw data object from mne itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-11-03 17:59:09,513 WARNING : This is not causal, uses future data....\n",
      "2017-11-03 17:59:09,514 INFO : Resampling from 500.000000 to 250.000000 Hz.\n",
      "Creating RawArray with float64 data, n_channels=3, n_times=1725660\n",
      "    Range : 0 ... 1725659 =      0.000 ...  6902.636 secs\n",
      "Ready.\n"
     ]
    }
   ],
   "source": [
    "from braindecode.mne_ext.signalproc import resample_cnt, mne_apply\n",
    "from braindecode.datautil.signalproc import exponential_running_standardize\n",
    "# Remove stimulus channel\n",
    "cnt = cnt.drop_channels(['STI 014'])\n",
    "cnt = resample_cnt(cnt, 250)\n",
    "# mne apply will apply the function to the data (a 2d-numpy-array)\n",
    "# have to transpose data back and forth, since\n",
    "# exponential_running_standardize expects time x chans order\n",
    "# while mne object has chans x time order\n",
    "cnt = mne_apply(lambda a: exponential_running_standardize(\n",
    "    a.T, init_block_size=1000,factor_new=0.001, eps=1e-4).T,\n",
    "    cnt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transform to epoched dataset "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Braindecode supplies the `create_signal_target_from_raw_mne` function, which will transform the mne raw object into a `SignalAndTarget` object for use in Braindecode.\n",
    "`name_to_code` should be an `OrderedDict` that maps class names to either one or a list of marker codes for that class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-11-03 17:59:11,376 INFO : Trial per class:\n",
      "Counter({'Feet': 225, 'Right': 224, 'Rest': 224, 'Left': 224})\n"
     ]
    }
   ],
   "source": [
    "from braindecode.datautil.trial_segment import create_signal_target_from_raw_mne\n",
    "from collections import OrderedDict\n",
    "# can also give lists of marker codes in case a class has multiple marker codes...\n",
    "name_to_code = OrderedDict([('Right', 1), ('Left', 2), ('Rest', 3), ('Feet', 4)])\n",
    "segment_ival_ms = [-500,4000]\n",
    "\n",
    "train_set = create_signal_target_from_raw_mne(cnt, name_to_code, segment_ival_ms)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Same for test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating RawArray with float64 data, n_channels=3, n_times=617090\n",
      "    Range : 0 ... 617089 =      0.000 ...  1234.178 secs\n",
      "Ready.\n",
      "2017-11-03 17:59:12,305 WARNING : This is not causal, uses future data....\n",
      "2017-11-03 17:59:12,306 INFO : Resampling from 500.000000 to 250.000000 Hz.\n",
      "Creating RawArray with float64 data, n_channels=3, n_times=308545\n",
      "    Range : 0 ... 308544 =      0.000 ...  1234.176 secs\n",
      "Ready.\n",
      "2017-11-03 17:59:12,653 INFO : Trial per class:\n",
      "Counter({'Feet': 40, 'Left': 40, 'Rest': 40, 'Right': 40})\n"
     ]
    }
   ],
   "source": [
    "test_filename = '/home/schirrmr/data/BBCI-only-last-runs/BhNoMoSc1S001R13_ds10_1-2BBCI.mat'\n",
    "cnt = BBCIDataset(test_filename, load_sensor_names=['C3', 'CPz', 'C4']).load()\n",
    "# Remove stimulus channel\n",
    "cnt = cnt.drop_channels(['STI 014'])\n",
    "cnt = resample_cnt(cnt, 250)\n",
    "cnt = mne_apply(lambda a: exponential_running_standardize(\n",
    "    a.T, init_block_size=1000,factor_new=0.001, eps=1e-4).T,\n",
    "    cnt)\n",
    "test_set = create_signal_target_from_raw_mne(cnt, name_to_code, segment_ival_ms)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "In case of start and stop markers, provide a `name_to_stop_codes` dictionary (same as for the start codes in this example) as a final argument to `create_signal_target_from_raw_mne`. See [Read and Decode BBCI Data with Start-Stop-Markers Tutorial](BBCI_Data_Start_Stop.html)\n",
    "\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split off a validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from braindecode.datautil.splitters import split_into_two_sets\n",
    "\n",
    "train_set, valid_set = split_into_two_sets(train_set, first_set_fraction=0.8)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from braindecode.models.shallow_fbcsp import ShallowFBCSPNet\n",
    "from torch import nn\n",
    "from braindecode.torch_ext.util import set_random_seeds\n",
    "\n",
    "# Set if you want to use GPU\n",
    "# You can also use torch.cuda.is_available() to determine if cuda is available on your machine.\n",
    "cuda = True\n",
    "set_random_seeds(seed=20170629, cuda=cuda)\n",
    "\n",
    "\n",
    "# This will determine how many crops are processed in parallel\n",
    "input_time_length = train_set.X.shape[2]\n",
    "in_chans = 3\n",
    "n_classes = 4\n",
    "# final_conv_length determines the size of the receptive field of the ConvNet\n",
    "model = ShallowFBCSPNet(in_chans=in_chans, n_classes=n_classes, input_time_length=input_time_length,\n",
    "                        final_conv_length='auto').create_network()\n",
    "\n",
    "if cuda:\n",
    "    model.cuda()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup optimizer and iterator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch import optim\n",
    "import numpy as np\n",
    "\n",
    "optimizer = optim.Adam(model.parameters())\n",
    "\n",
    "\n",
    "from braindecode.datautil.iterators import BalancedBatchSizeIterator\n",
    "iterator = BalancedBatchSizeIterator(batch_size=32)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Monitors, Loss function, Stop Criteria"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from braindecode.experiments.experiment import Experiment\n",
    "from braindecode.experiments.monitors import RuntimeMonitor, LossMonitor, CroppedTrialMisclassMonitor, MisclassMonitor\n",
    "from braindecode.experiments.stopcriteria import MaxEpochs\n",
    "import torch.nn.functional as F\n",
    "import torch as th\n",
    "from braindecode.torch_ext.modules import Expression\n",
    "\n",
    "\n",
    "loss_function = F.nll_loss\n",
    "\n",
    "model_constraint = None\n",
    "monitors = [LossMonitor(), MisclassMonitor(col_suffix='misclass'), \n",
    "            RuntimeMonitor(),]\n",
    "stop_criterion = MaxEpochs(20)\n",
    "exp = Experiment(model, train_set, valid_set, test_set, iterator, loss_function, optimizer, model_constraint,\n",
    "          monitors, stop_criterion, remember_best_column='valid_misclass',\n",
    "          run_after_early_stop=True, batch_modifier=None, cuda=cuda)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-11-03 17:59:15,468 INFO : Run until first stop...\n",
      "2017-11-03 17:59:15,979 INFO : Epoch 0\n",
      "2017-11-03 17:59:15,981 INFO : train_loss                3.82544\n",
      "2017-11-03 17:59:15,982 INFO : valid_loss                3.72206\n",
      "2017-11-03 17:59:15,982 INFO : test_loss                 3.90394\n",
      "2017-11-03 17:59:15,983 INFO : train_misclass            0.76184\n",
      "2017-11-03 17:59:15,984 INFO : valid_misclass            0.76536\n",
      "2017-11-03 17:59:15,985 INFO : test_misclass             0.75625\n",
      "2017-11-03 17:59:15,985 INFO : runtime                   0.00000\n",
      "2017-11-03 17:59:15,986 INFO : \n",
      "2017-11-03 17:59:15,988 INFO : New best valid_misclass: 0.765363\n",
      "2017-11-03 17:59:15,989 INFO : \n",
      "2017-11-03 17:59:16,564 INFO : Time only for training updates: 0.57s\n",
      "2017-11-03 17:59:16,671 INFO : Epoch 1\n",
      "2017-11-03 17:59:16,672 INFO : train_loss                2.65436\n",
      "2017-11-03 17:59:16,673 INFO : valid_loss                2.88156\n",
      "2017-11-03 17:59:16,674 INFO : test_loss                 3.03544\n",
      "2017-11-03 17:59:16,675 INFO : train_misclass            0.72006\n",
      "2017-11-03 17:59:16,675 INFO : valid_misclass            0.73184\n",
      "2017-11-03 17:59:16,676 INFO : test_misclass             0.71250\n",
      "2017-11-03 17:59:16,677 INFO : runtime                   1.09594\n",
      "2017-11-03 17:59:16,678 INFO : \n",
      "2017-11-03 17:59:16,681 INFO : New best valid_misclass: 0.731844\n",
      "2017-11-03 17:59:16,682 INFO : \n",
      "2017-11-03 17:59:17,228 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:17,336 INFO : Epoch 2\n",
      "2017-11-03 17:59:17,338 INFO : train_loss                0.60706\n",
      "2017-11-03 17:59:17,339 INFO : valid_loss                0.61918\n",
      "2017-11-03 17:59:17,339 INFO : test_loss                 0.81560\n",
      "2017-11-03 17:59:17,340 INFO : train_misclass            0.24791\n",
      "2017-11-03 17:59:17,341 INFO : valid_misclass            0.27933\n",
      "2017-11-03 17:59:17,342 INFO : test_misclass             0.33750\n",
      "2017-11-03 17:59:17,343 INFO : runtime                   0.66460\n",
      "2017-11-03 17:59:17,344 INFO : \n",
      "2017-11-03 17:59:17,346 INFO : New best valid_misclass: 0.279330\n",
      "2017-11-03 17:59:17,347 INFO : \n",
      "2017-11-03 17:59:17,893 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:17,998 INFO : Epoch 3\n",
      "2017-11-03 17:59:17,999 INFO : train_loss                0.58638\n",
      "2017-11-03 17:59:18,000 INFO : valid_loss                0.62992\n",
      "2017-11-03 17:59:18,000 INFO : test_loss                 0.85668\n",
      "2017-11-03 17:59:18,001 INFO : train_misclass            0.23677\n",
      "2017-11-03 17:59:18,002 INFO : valid_misclass            0.27374\n",
      "2017-11-03 17:59:18,003 INFO : test_misclass             0.38750\n",
      "2017-11-03 17:59:18,003 INFO : runtime                   0.66507\n",
      "2017-11-03 17:59:18,004 INFO : \n",
      "2017-11-03 17:59:18,006 INFO : New best valid_misclass: 0.273743\n",
      "2017-11-03 17:59:18,007 INFO : \n",
      "2017-11-03 17:59:18,552 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:18,664 INFO : Epoch 4\n",
      "2017-11-03 17:59:18,666 INFO : train_loss                0.48614\n",
      "2017-11-03 17:59:18,666 INFO : valid_loss                0.56710\n",
      "2017-11-03 17:59:18,667 INFO : test_loss                 0.76815\n",
      "2017-11-03 17:59:18,668 INFO : train_misclass            0.19638\n",
      "2017-11-03 17:59:18,669 INFO : valid_misclass            0.26816\n",
      "2017-11-03 17:59:18,669 INFO : test_misclass             0.31875\n",
      "2017-11-03 17:59:18,670 INFO : runtime                   0.65807\n",
      "2017-11-03 17:59:18,671 INFO : \n",
      "2017-11-03 17:59:18,673 INFO : New best valid_misclass: 0.268156\n",
      "2017-11-03 17:59:18,674 INFO : \n",
      "2017-11-03 17:59:19,219 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:19,327 INFO : Epoch 5\n",
      "2017-11-03 17:59:19,329 INFO : train_loss                0.42408\n",
      "2017-11-03 17:59:19,329 INFO : valid_loss                0.52337\n",
      "2017-11-03 17:59:19,330 INFO : test_loss                 0.79153\n",
      "2017-11-03 17:59:19,331 INFO : train_misclass            0.14485\n",
      "2017-11-03 17:59:19,332 INFO : valid_misclass            0.22905\n",
      "2017-11-03 17:59:19,333 INFO : test_misclass             0.36250\n",
      "2017-11-03 17:59:19,333 INFO : runtime                   0.66796\n",
      "2017-11-03 17:59:19,334 INFO : \n",
      "2017-11-03 17:59:19,336 INFO : New best valid_misclass: 0.229050\n",
      "2017-11-03 17:59:19,337 INFO : \n",
      "2017-11-03 17:59:19,884 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:19,992 INFO : Epoch 6\n",
      "2017-11-03 17:59:19,993 INFO : train_loss                0.35395\n",
      "2017-11-03 17:59:19,994 INFO : valid_loss                0.51115\n",
      "2017-11-03 17:59:19,995 INFO : test_loss                 0.72994\n",
      "2017-11-03 17:59:19,996 INFO : train_misclass            0.11003\n",
      "2017-11-03 17:59:19,996 INFO : valid_misclass            0.20112\n",
      "2017-11-03 17:59:19,997 INFO : test_misclass             0.27500\n",
      "2017-11-03 17:59:19,998 INFO : runtime                   0.66456\n",
      "2017-11-03 17:59:19,999 INFO : \n",
      "2017-11-03 17:59:20,001 INFO : New best valid_misclass: 0.201117\n",
      "2017-11-03 17:59:20,002 INFO : \n",
      "2017-11-03 17:59:20,548 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:20,656 INFO : Epoch 7\n",
      "2017-11-03 17:59:20,657 INFO : train_loss                0.34276\n",
      "2017-11-03 17:59:20,658 INFO : valid_loss                0.57949\n",
      "2017-11-03 17:59:20,659 INFO : test_loss                 0.79834\n",
      "2017-11-03 17:59:20,659 INFO : train_misclass            0.11978\n",
      "2017-11-03 17:59:20,660 INFO : valid_misclass            0.25140\n",
      "2017-11-03 17:59:20,661 INFO : test_misclass             0.32500\n",
      "2017-11-03 17:59:20,662 INFO : runtime                   0.66413\n",
      "2017-11-03 17:59:20,662 INFO : \n",
      "2017-11-03 17:59:21,209 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:21,318 INFO : Epoch 8\n",
      "2017-11-03 17:59:21,319 INFO : train_loss                0.34212\n",
      "2017-11-03 17:59:21,320 INFO : valid_loss                0.45382\n",
      "2017-11-03 17:59:21,321 INFO : test_loss                 0.68899\n",
      "2017-11-03 17:59:21,322 INFO : train_misclass            0.09610\n",
      "2017-11-03 17:59:21,323 INFO : valid_misclass            0.19553\n",
      "2017-11-03 17:59:21,323 INFO : test_misclass             0.27500\n",
      "2017-11-03 17:59:21,324 INFO : runtime                   0.66085\n",
      "2017-11-03 17:59:21,325 INFO : \n",
      "2017-11-03 17:59:21,327 INFO : New best valid_misclass: 0.195531\n",
      "2017-11-03 17:59:21,328 INFO : \n",
      "2017-11-03 17:59:21,874 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:21,981 INFO : Epoch 9\n",
      "2017-11-03 17:59:21,982 INFO : train_loss                0.26619\n",
      "2017-11-03 17:59:21,983 INFO : valid_loss                0.48593\n",
      "2017-11-03 17:59:21,983 INFO : test_loss                 0.70376\n",
      "2017-11-03 17:59:21,984 INFO : train_misclass            0.07242\n",
      "2017-11-03 17:59:21,985 INFO : valid_misclass            0.18436\n",
      "2017-11-03 17:59:21,986 INFO : test_misclass             0.26250\n",
      "2017-11-03 17:59:21,987 INFO : runtime                   0.66527\n",
      "2017-11-03 17:59:21,987 INFO : \n",
      "2017-11-03 17:59:21,990 INFO : New best valid_misclass: 0.184358\n",
      "2017-11-03 17:59:21,990 INFO : \n",
      "2017-11-03 17:59:22,536 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:22,644 INFO : Epoch 10\n",
      "2017-11-03 17:59:22,645 INFO : train_loss                0.26250\n",
      "2017-11-03 17:59:22,646 INFO : valid_loss                0.47718\n",
      "2017-11-03 17:59:22,647 INFO : test_loss                 0.71324\n",
      "2017-11-03 17:59:22,648 INFO : train_misclass            0.08914\n",
      "2017-11-03 17:59:22,649 INFO : valid_misclass            0.18994\n",
      "2017-11-03 17:59:22,649 INFO : test_misclass             0.28125\n",
      "2017-11-03 17:59:22,650 INFO : runtime                   0.66180\n",
      "2017-11-03 17:59:22,651 INFO : \n",
      "2017-11-03 17:59:23,198 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:23,309 INFO : Epoch 11\n",
      "2017-11-03 17:59:23,311 INFO : train_loss                0.21297\n",
      "2017-11-03 17:59:23,312 INFO : valid_loss                0.48684\n",
      "2017-11-03 17:59:23,312 INFO : test_loss                 0.74883\n",
      "2017-11-03 17:59:23,313 INFO : train_misclass            0.06546\n",
      "2017-11-03 17:59:23,314 INFO : valid_misclass            0.20670\n",
      "2017-11-03 17:59:23,315 INFO : test_misclass             0.31250\n",
      "2017-11-03 17:59:23,315 INFO : runtime                   0.66222\n",
      "2017-11-03 17:59:23,316 INFO : \n",
      "2017-11-03 17:59:23,864 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:23,973 INFO : Epoch 12\n",
      "2017-11-03 17:59:23,974 INFO : train_loss                0.21066\n",
      "2017-11-03 17:59:23,975 INFO : valid_loss                0.49149\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-11-03 17:59:23,976 INFO : test_loss                 0.78449\n",
      "2017-11-03 17:59:23,977 INFO : train_misclass            0.05710\n",
      "2017-11-03 17:59:23,977 INFO : valid_misclass            0.18994\n",
      "2017-11-03 17:59:23,978 INFO : test_misclass             0.30000\n",
      "2017-11-03 17:59:23,979 INFO : runtime                   0.66576\n",
      "2017-11-03 17:59:23,980 INFO : \n",
      "2017-11-03 17:59:24,527 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:24,636 INFO : Epoch 13\n",
      "2017-11-03 17:59:24,637 INFO : train_loss                0.22913\n",
      "2017-11-03 17:59:24,638 INFO : valid_loss                0.52561\n",
      "2017-11-03 17:59:24,639 INFO : test_loss                 0.68987\n",
      "2017-11-03 17:59:24,639 INFO : train_misclass            0.06407\n",
      "2017-11-03 17:59:24,640 INFO : valid_misclass            0.18436\n",
      "2017-11-03 17:59:24,641 INFO : test_misclass             0.26250\n",
      "2017-11-03 17:59:24,642 INFO : runtime                   0.66327\n",
      "2017-11-03 17:59:24,642 INFO : \n",
      "2017-11-03 17:59:24,645 INFO : New best valid_misclass: 0.184358\n",
      "2017-11-03 17:59:24,645 INFO : \n",
      "2017-11-03 17:59:25,191 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:25,303 INFO : Epoch 14\n",
      "2017-11-03 17:59:25,304 INFO : train_loss                0.29666\n",
      "2017-11-03 17:59:25,305 INFO : valid_loss                0.62789\n",
      "2017-11-03 17:59:25,306 INFO : test_loss                 0.88607\n",
      "2017-11-03 17:59:25,306 INFO : train_misclass            0.09610\n",
      "2017-11-03 17:59:25,307 INFO : valid_misclass            0.26257\n",
      "2017-11-03 17:59:25,308 INFO : test_misclass             0.31875\n",
      "2017-11-03 17:59:25,309 INFO : runtime                   0.66325\n",
      "2017-11-03 17:59:25,309 INFO : \n",
      "2017-11-03 17:59:25,857 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:25,965 INFO : Epoch 15\n",
      "2017-11-03 17:59:25,966 INFO : train_loss                0.20635\n",
      "2017-11-03 17:59:25,967 INFO : valid_loss                0.51987\n",
      "2017-11-03 17:59:25,968 INFO : test_loss                 0.77714\n",
      "2017-11-03 17:59:25,969 INFO : train_misclass            0.07103\n",
      "2017-11-03 17:59:25,970 INFO : valid_misclass            0.20670\n",
      "2017-11-03 17:59:25,970 INFO : test_misclass             0.31250\n",
      "2017-11-03 17:59:25,971 INFO : runtime                   0.66714\n",
      "2017-11-03 17:59:25,972 INFO : \n",
      "2017-11-03 17:59:26,519 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:26,631 INFO : Epoch 16\n",
      "2017-11-03 17:59:26,632 INFO : train_loss                0.18682\n",
      "2017-11-03 17:59:26,633 INFO : valid_loss                0.47353\n",
      "2017-11-03 17:59:26,634 INFO : test_loss                 0.71371\n",
      "2017-11-03 17:59:26,635 INFO : train_misclass            0.05432\n",
      "2017-11-03 17:59:26,636 INFO : valid_misclass            0.19553\n",
      "2017-11-03 17:59:26,637 INFO : test_misclass             0.26250\n",
      "2017-11-03 17:59:26,637 INFO : runtime                   0.66127\n",
      "2017-11-03 17:59:26,638 INFO : \n",
      "2017-11-03 17:59:27,185 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:27,293 INFO : Epoch 17\n",
      "2017-11-03 17:59:27,295 INFO : train_loss                0.15229\n",
      "2017-11-03 17:59:27,296 INFO : valid_loss                0.51995\n",
      "2017-11-03 17:59:27,296 INFO : test_loss                 0.70597\n",
      "2017-11-03 17:59:27,297 INFO : train_misclass            0.04178\n",
      "2017-11-03 17:59:27,298 INFO : valid_misclass            0.21229\n",
      "2017-11-03 17:59:27,299 INFO : test_misclass             0.30000\n",
      "2017-11-03 17:59:27,300 INFO : runtime                   0.66622\n",
      "2017-11-03 17:59:27,301 INFO : \n",
      "2017-11-03 17:59:27,849 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:27,954 INFO : Epoch 18\n",
      "2017-11-03 17:59:27,955 INFO : train_loss                0.19083\n",
      "2017-11-03 17:59:27,956 INFO : valid_loss                0.50357\n",
      "2017-11-03 17:59:27,957 INFO : test_loss                 0.67131\n",
      "2017-11-03 17:59:27,957 INFO : train_misclass            0.04735\n",
      "2017-11-03 17:59:27,958 INFO : valid_misclass            0.19553\n",
      "2017-11-03 17:59:27,959 INFO : test_misclass             0.26250\n",
      "2017-11-03 17:59:27,960 INFO : runtime                   0.66379\n",
      "2017-11-03 17:59:27,960 INFO : \n",
      "2017-11-03 17:59:28,508 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:28,616 INFO : Epoch 19\n",
      "2017-11-03 17:59:28,617 INFO : train_loss                0.14053\n",
      "2017-11-03 17:59:28,618 INFO : valid_loss                0.53879\n",
      "2017-11-03 17:59:28,619 INFO : test_loss                 0.74517\n",
      "2017-11-03 17:59:28,619 INFO : train_misclass            0.03343\n",
      "2017-11-03 17:59:28,620 INFO : valid_misclass            0.20670\n",
      "2017-11-03 17:59:28,621 INFO : test_misclass             0.25625\n",
      "2017-11-03 17:59:28,622 INFO : runtime                   0.65849\n",
      "2017-11-03 17:59:28,622 INFO : \n",
      "2017-11-03 17:59:29,169 INFO : Time only for training updates: 0.55s\n",
      "2017-11-03 17:59:29,279 INFO : Epoch 20\n",
      "2017-11-03 17:59:29,280 INFO : train_loss                0.22331\n",
      "2017-11-03 17:59:29,281 INFO : valid_loss                0.49784\n",
      "2017-11-03 17:59:29,282 INFO : test_loss                 0.70844\n",
      "2017-11-03 17:59:29,283 INFO : train_misclass            0.06964\n",
      "2017-11-03 17:59:29,283 INFO : valid_misclass            0.16760\n",
      "2017-11-03 17:59:29,284 INFO : test_misclass             0.23750\n",
      "2017-11-03 17:59:29,285 INFO : runtime                   0.66110\n",
      "2017-11-03 17:59:29,286 INFO : \n",
      "2017-11-03 17:59:29,290 INFO : New best valid_misclass: 0.167598\n",
      "2017-11-03 17:59:29,290 INFO : \n",
      "2017-11-03 17:59:29,291 INFO : Setup for second stop...\n",
      "2017-11-03 17:59:29,295 INFO : Train loss to reach 0.22331\n",
      "2017-11-03 17:59:29,296 INFO : Run until second stop...\n",
      "2017-11-03 17:59:29,410 INFO : Epoch 21\n",
      "2017-11-03 17:59:29,412 INFO : train_loss                0.27810\n",
      "2017-11-03 17:59:29,412 INFO : valid_loss                0.49784\n",
      "2017-11-03 17:59:29,413 INFO : test_loss                 0.70844\n",
      "2017-11-03 17:59:29,414 INFO : train_misclass            0.08919\n",
      "2017-11-03 17:59:29,415 INFO : valid_misclass            0.16760\n",
      "2017-11-03 17:59:29,415 INFO : test_misclass             0.23750\n",
      "2017-11-03 17:59:29,416 INFO : runtime                   0.13360\n",
      "2017-11-03 17:59:29,417 INFO : \n",
      "2017-11-03 17:59:29,958 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:30,085 INFO : Epoch 22\n",
      "2017-11-03 17:59:30,087 INFO : train_loss                0.23934\n",
      "2017-11-03 17:59:30,088 INFO : valid_loss                0.45107\n",
      "2017-11-03 17:59:30,088 INFO : test_loss                 0.86186\n",
      "2017-11-03 17:59:30,089 INFO : train_misclass            0.09142\n",
      "2017-11-03 17:59:30,090 INFO : valid_misclass            0.20670\n",
      "2017-11-03 17:59:30,091 INFO : test_misclass             0.28750\n",
      "2017-11-03 17:59:30,091 INFO : runtime                   0.65554\n",
      "2017-11-03 17:59:30,092 INFO : \n",
      "2017-11-03 17:59:30,630 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:30,754 INFO : Epoch 23\n",
      "2017-11-03 17:59:30,755 INFO : train_loss                0.23411\n",
      "2017-11-03 17:59:30,756 INFO : valid_loss                0.33481\n",
      "2017-11-03 17:59:30,757 INFO : test_loss                 0.72723\n",
      "2017-11-03 17:59:30,758 INFO : train_misclass            0.06132\n",
      "2017-11-03 17:59:30,759 INFO : valid_misclass            0.13408\n",
      "2017-11-03 17:59:30,760 INFO : test_misclass             0.31250\n",
      "2017-11-03 17:59:30,761 INFO : runtime                   0.67196\n",
      "2017-11-03 17:59:30,762 INFO : \n",
      "2017-11-03 17:59:31,304 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:31,429 INFO : Epoch 24\n",
      "2017-11-03 17:59:31,431 INFO : train_loss                0.24864\n",
      "2017-11-03 17:59:31,432 INFO : valid_loss                0.30050\n",
      "2017-11-03 17:59:31,432 INFO : test_loss                 0.63026\n",
      "2017-11-03 17:59:31,433 INFO : train_misclass            0.08250\n",
      "2017-11-03 17:59:31,434 INFO : valid_misclass            0.11173\n",
      "2017-11-03 17:59:31,435 INFO : test_misclass             0.23750\n",
      "2017-11-03 17:59:31,435 INFO : runtime                   0.67471\n",
      "2017-11-03 17:59:31,437 INFO : \n",
      "2017-11-03 17:59:31,976 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:32,101 INFO : Epoch 25\n",
      "2017-11-03 17:59:32,102 INFO : train_loss                0.16780\n",
      "2017-11-03 17:59:32,103 INFO : valid_loss                0.26910\n",
      "2017-11-03 17:59:32,104 INFO : test_loss                 0.76967\n",
      "2017-11-03 17:59:32,105 INFO : train_misclass            0.05017\n",
      "2017-11-03 17:59:32,105 INFO : valid_misclass            0.09497\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-11-03 17:59:32,106 INFO : test_misclass             0.29375\n",
      "2017-11-03 17:59:32,107 INFO : runtime                   0.67182\n",
      "2017-11-03 17:59:32,108 INFO : \n",
      "2017-11-03 17:59:32,648 INFO : Time only for training updates: 0.54s\n",
      "2017-11-03 17:59:32,772 INFO : Epoch 26\n",
      "2017-11-03 17:59:32,773 INFO : train_loss                0.13316\n",
      "2017-11-03 17:59:32,774 INFO : valid_loss                0.21726\n",
      "2017-11-03 17:59:32,775 INFO : test_loss                 0.72940\n",
      "2017-11-03 17:59:32,776 INFO : train_misclass            0.04571\n",
      "2017-11-03 17:59:32,776 INFO : valid_misclass            0.08380\n",
      "2017-11-03 17:59:32,777 INFO : test_misclass             0.27500\n",
      "2017-11-03 17:59:32,778 INFO : runtime                   0.67151\n",
      "2017-11-03 17:59:32,779 INFO : \n"
     ]
    }
   ],
   "source": [
    "exp.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We arrive around 26%, exact value depending on stars :))"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
