Merge pull request #1893 from nd-02110114/update-colab

%% Cell type:markdown id: tags:

# Tutorial Part 2: Learning MNIST Digit Classifiers

In the previous tutorial, we learned some basics of how to load data into DeepChem and how to use the basic DeepChem objects to load and manipulate this data. In this tutorial, you'll put the parts together and learn how to train a basic image classification model in DeepChem. You might ask, why are we bothering to learn this material in DeepChem? Part of the reason is that image processing is an increasingly important part of AI for the life sciences. So learning how to train image processing models will be very useful for using some of the more advanced DeepChem features.

The MNIST dataset contains handwritten digits along with their human annotated labels. The learning challenge for this dataset is to train a model that maps the digit image to its true label. MNIST has been a standard benchmark for machine learning for decades at this point.

![MNIST](https://github.com/deepchem/deepchem/blob/master/examples/tutorials/mnist_examples.png?raw=1)

## Colab

This tutorial and the rest in this sequence are designed to be done in Google colab. If you'd like to open this notebook in colab, you can use the following link.

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepchem/deepchem/blob/master/examples/tutorials/02_Learning_MNIST_Digit_Classifiers.ipynb)

## Setup

We recommend running this tutorial on Google colab. You'll need to run the following cell of installation commands on Colab to get your environment set up. If you'd rather run the tutorial locally, make sure you don't run these commands (since they'll download and install a new Anaconda python setup)

%% Cell type:code id: tags:

``` 

%tensorflow_version 1.x

!curl -Lo deepchem_installer.py https://raw.githubusercontent.com/deepchem/deepchem/master/scripts/colab_install.py

import deepchem_installer

%time deepchem_installer.install(version='2.3.0')

```

%% Output

    TensorFlow 1.x selected.

      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                     Dload  Upload   Total   Spent    Left  Speed

    100  2814  100  2814    0     0  28140      0 --:--:-- --:--:-- --:--:-- 28140

    100  3477  100  3477    0     0   9934      0 --:--:-- --:--:-- --:--:--  9934

    add /root/miniconda/lib/python3.6/site-packages to PYTHONPATH

    python version: 3.6.9

    remove current miniconda

    fetching installer from https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh

    done

    installing miniconda to /root/miniconda

    done

    installing deepchem

    done

    /usr/local/lib/python3.6/dist-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.

      warnings.warn(msg, category=FutureWarning)

    WARNING:tensorflow:

    The TensorFlow contrib module will not be included in TensorFlow 2.0.

    For more information, please see:

      * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md

      * https://github.com/tensorflow/addons

      * https://github.com/tensorflow/io (for I/O related ops)

    If you depend on functionality not listed there, please file an issue.

    deepchem-2.3.0 installation finished!

    CPU times: user 2.71 s, sys: 996 ms, total: 3.71 s

    Wall time: 4min 14s

    CPU times: user 2.54 s, sys: 520 ms, total: 3.06 s

    Wall time: 1min 58s

%% Cell type:code id: tags:

``` 

from tensorflow.examples.tutorials.mnist import input_data

```

%% Cell type:code id: tags:

``` 

# TODO: This is deprecated. Let's replace with a DeepChem native loader for maintainability.

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

```

%% Output

    WARNING:tensorflow:From <ipython-input-3-227956e9c9c1>:2: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.

    Instructions for updating:

    Please use alternatives such as official/mnist/dataset.py from tensorflow/models.

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.

    Instructions for updating:

    Please write your own downloading logic.

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/contrib/learn/python/learn/datasets/base.py:252: _internal_retry.<locals>.wrap.<locals>.wrapped_fn (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.

    Instructions for updating:

    Please use urllib or similar directly.

    Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.

    Instructions for updating:

    Please use tf.data to implement this functionality.

    Extracting MNIST_data/train-images-idx3-ubyte.gz

    Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.

    Instructions for updating:

    Please use tf.data to implement this functionality.

    Extracting MNIST_data/train-labels-idx1-ubyte.gz

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.

    Instructions for updating:

    Please use tf.one_hot on tensors.

    Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.

    Extracting MNIST_data/t10k-images-idx3-ubyte.gz

    Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.

    Extracting MNIST_data/t10k-labels-idx1-ubyte.gz

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.

    Instructions for updating:

    Please use alternatives such as official/mnist/dataset.py from tensorflow/models.

%% Cell type:code id: tags:

``` 

import deepchem as dc

import tensorflow as tf

from tensorflow.keras.layers import Reshape, Conv2D, Flatten, Dense, Softmax

```

%% Cell type:code id: tags:

``` 

train = dc.data.NumpyDataset(mnist.train.images, mnist.train.labels)

valid = dc.data.NumpyDataset(mnist.validation.images, mnist.validation.labels)

```

%% Cell type:code id: tags:

``` 

keras_model = tf.keras.Sequential([

    Reshape((28, 28, 1)),

    Conv2D(filters=32, kernel_size=5, activation=tf.nn.relu),

    Conv2D(filters=64, kernel_size=5, activation=tf.nn.relu),

    Flatten(),

    Dense(1024, activation=tf.nn.relu),

    Dense(10),

    Softmax()

])

model = dc.models.KerasModel(keras_model, dc.models.losses.CategoricalCrossEntropy())

```

%% Cell type:code id: tags:

``` 

model.fit(train, nb_epoch=2)

```

%% Output

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:169: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/optimizers.py:76: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:258: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:260: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:200: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.

    Instructions for updating:

    If using Keras pass *_constraint arguments to layers.

    0.0

%% Cell type:code id: tags:

``` 

from sklearn.metrics import roc_curve, auc

import numpy as np

print("Validation")

prediction = np.squeeze(model.predict_on_batch(valid.X))

fpr = dict()

tpr = dict()

roc_auc = dict()

for i in range(10):

    fpr[i], tpr[i], thresh = roc_curve(valid.y[:, i], prediction[:, i])

    roc_auc[i] = auc(fpr[i], tpr[i])

    print("class %s:auc=%s" % (i, roc_auc[i]))

```

%% Output

    Validation

    class 0:auc=0.9999473577030227

    class 1:auc=0.9999211378882008

    class 2:auc=0.9999187049470992

    class 3:auc=0.9999000878057167

    class 4:auc=0.9999732080250338

    class 5:auc=0.9998869625422123

    class 6:auc=0.9999294587087216

    class 7:auc=0.9998189989785494

    class 8:auc=0.9996727967199541

    class 9:auc=0.9998515678426888

    class 0:auc=0.9999482812520925

    class 1:auc=0.9999327470315621

    class 2:auc=0.9999223382455529

    class 3:auc=0.9999378924197698

    class 4:auc=0.999804920932277

    class 5:auc=0.9997608046652174

    class 6:auc=0.9999347825797615

    class 7:auc=0.9997099080694587

    class 8:auc=0.999882187740275

    class 9:auc=0.9996286953889618

%% Cell type:markdown id: tags:

# Congratulations! Time to join the Community!

Congratulations on completing this tutorial notebook! If you enjoyed working through the tutorial, and want to continue working with DeepChem, we encourage you to finish the rest of the tutorials in this series. You can also help the DeepChem community in the following ways:

## Star DeepChem on [GitHub](https://github.com/deepchem/deepchem)

This helps build awareness of the DeepChem project and the tools for open source drug discovery that we're trying to build.

## Join the DeepChem Gitter

The DeepChem [Gitter](https://gitter.im/deepchem/Lobby) hosts a number of scientists, developers, and enthusiasts interested in deep learning for the life sciences. Join the conversation!

%% Cell type:markdown id: tags:

# Tutorial Part 5: Putting Multitask Learning to Work

This notebook walks through the creation of multitask models on MUV [1]. The goal is to demonstrate that multitask methods outperform singletask methods on MUV.

## Colab

This tutorial and the rest in this sequence are designed to be done in Google colab. If you'd like to open this notebook in colab, you can use the following link.

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepchem/deepchem/blob/master/examples/tutorials/05_Putting_Multitask_Learning_to_Work.ipynb)

## Setup

To run DeepChem within Colab, you'll need to run the following cell of installation commands. This will take about 5 minutes to run to completion and install your environment.

%% Cell type:code id: tags:

``` 

%tensorflow_version 1.x

!curl -Lo deepchem_installer.py https://raw.githubusercontent.com/deepchem/deepchem/master/scripts/colab_install.py

import deepchem_installer

%time deepchem_installer.install(version='2.3.0')

```

%% Output

    TensorFlow 1.x selected.

      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                     Dload  Upload   Total   Spent    Left  Speed

    100  2814  100  2814    0     0  65441      0 --:--:-- --:--:-- --:--:-- 65441

    100  3477  100  3477    0     0  14733      0 --:--:-- --:--:-- --:--:-- 14733

    add /root/miniconda/lib/python3.6/site-packages to PYTHONPATH

    python version: 3.6.9

    fetching installer from https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh

    done

    installing miniconda to /root/miniconda

    done

    installing deepchem

    done

    /usr/local/lib/python3.6/dist-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.

      warnings.warn(msg, category=FutureWarning)

    WARNING:tensorflow:

    The TensorFlow contrib module will not be included in TensorFlow 2.0.

    For more information, please see:

      * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md

      * https://github.com/tensorflow/addons

      * https://github.com/tensorflow/io (for I/O related ops)

    If you depend on functionality not listed there, please file an issue.

    deepchem-2.3.0 installation finished!

    CPU times: user 2.52 s, sys: 535 ms, total: 3.05 s

    Wall time: 4min 19s

    CPU times: user 3.1 s, sys: 736 ms, total: 3.84 s

    Wall time: 2min 19s

%% Cell type:markdown id: tags:

The MUV dataset is a challenging benchmark in molecular design that consists of 17 different "targets" where there are only a few "active" compounds per target. The goal of working with this dataset is to make a machine learnign model which achieves high accuracy on held-out compounds at predicting activity. To get started, let's download the MUV dataset for us to play with.

%% Cell type:code id: tags:

``` 

import os

import deepchem as dc

current_dir = os.path.dirname(os.path.realpath("__file__"))

dataset_file = "medium_muv.csv.gz"

full_dataset_file = "muv.csv.gz"

# We use a small version of MUV to make online rendering of notebooks easy. Replace with full_dataset_file

# In order to run the full version of this notebook

dc.utils.download_url("https://s3-us-west-1.amazonaws.com/deepchem.io/datasets/%s" % dataset_file,

                      current_dir)

dataset = dc.utils.save.load_from_disk(dataset_file)

print("Columns of dataset: %s" % str(dataset.columns.values))

print("Number of examples in dataset: %s" % str(dataset.shape[0]))

```

%% Output

    Columns of dataset: ['MUV-466' 'MUV-548' 'MUV-600' 'MUV-644' 'MUV-652' 'MUV-689' 'MUV-692'

     'MUV-712' 'MUV-713' 'MUV-733' 'MUV-737' 'MUV-810' 'MUV-832' 'MUV-846'

     'MUV-852' 'MUV-858' 'MUV-859' 'mol_id' 'smiles']

    Number of examples in dataset: 10000

%% Cell type:markdown id: tags:

Now, let's visualize some compounds from our dataset

%% Cell type:code id: tags:

``` 

from rdkit import Chem

from rdkit.Chem import Draw

from itertools import islice

from IPython.display import Image, display, HTML

def display_images(filenames):

    """Helper to pretty-print images."""

    for filename in filenames:

        display(Image(filename))

def mols_to_pngs(mols, basename="test"):

    """Helper to write RDKit mols to png files."""

    filenames = []

    for i, mol in enumerate(mols):

        filename = "MUV_%s%d.png" % (basename, i)

        Draw.MolToFile(mol, filename)

        filenames.append(filename)

    return filenames

num_to_display = 12

molecules = []

for _, data in islice(dataset.iterrows(), num_to_display):

    molecules.append(Chem.MolFromSmiles(data["smiles"]))

display_images(mols_to_pngs(molecules))

```

%% Output

%% Cell type:markdown id: tags:

There are 17 datasets total in MUV as we mentioned previously. We're going to train a multitask model that attempts to build a joint model to predict activity across all 17 datasets simultaneously. There's some evidence [2] that multitask training creates more robust models.

As fair warning, from my experience, this effect can be quite fragile. Nonetheless, it's a tool worth trying given how easy DeepChem makes it to build these models. To get started towards building our actual model, let's first featurize our data.

%% Cell type:code id: tags:

``` 

MUV_tasks = ['MUV-692', 'MUV-689', 'MUV-846', 'MUV-859', 'MUV-644',

             'MUV-548', 'MUV-852', 'MUV-600', 'MUV-810', 'MUV-712',

             'MUV-737', 'MUV-858', 'MUV-713', 'MUV-733', 'MUV-652',

             'MUV-466', 'MUV-832']

featurizer = dc.feat.CircularFingerprint(size=1024)

loader = dc.data.CSVLoader(

      tasks=MUV_tasks, smiles_field="smiles",

      featurizer=featurizer)

dataset = loader.featurize(dataset_file)

```

%% Output

    Loading raw samples now.

    shard_size: 8192

    About to start loading CSV from medium_muv.csv.gz

    Loading shard 1 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 0 took 26.406 s

    TIMING: featurizing shard 0 took 38.166 s

    Loading shard 2 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    TIMING: featurizing shard 1 took 5.849 s

    TIMING: dataset construction took 32.666 s

    TIMING: featurizing shard 1 took 8.325 s

    TIMING: dataset construction took 46.915 s

    Loading dataset from disk.

%% Cell type:markdown id: tags:

We'll now want to split our dataset into training, validation, and test sets. We're going to do a simple random split using `dc.splits.RandomSplitter`. It's worth noting that this will provide overestimates of real generalizability! For better real world estimates of prospective performance, you'll want to use a harder splitter.

%% Cell type:code id: tags:

``` 

splitter = dc.splits.RandomSplitter(dataset_file)

train_dataset, valid_dataset, test_dataset = splitter.train_valid_test_split(

    dataset)

#NOTE THE RENAMING:

valid_dataset, test_dataset = test_dataset, valid_dataset

```

%% Output

    Computing train/valid/test indices

    TIMING: dataset construction took 0.471 s

    TIMING: dataset construction took 0.529 s

    Loading dataset from disk.

    TIMING: dataset construction took 0.236 s

    TIMING: dataset construction took 0.254 s

    Loading dataset from disk.

    TIMING: dataset construction took 0.230 s

    TIMING: dataset construction took 0.272 s

    Loading dataset from disk.

%% Cell type:markdown id: tags:

Let's now get started building some models!  We'll do some simple hyperparameter searching to build a robust model.

%% Cell type:code id: tags:

``` 

import numpy as np

import numpy.random

params_dict = {"activation": ["relu"],

               "momentum": [.9],

               "batch_size": [50],

               "init": ["glorot_uniform"],

               "data_shape": [train_dataset.get_data_shape()],

               "learning_rate": [1e-3],

               "decay": [1e-6],

               "nb_epoch": [1],

               "nesterov": [False],

               "dropouts": [(.5,)],

               "nb_layers": [1],

               "batchnorm": [False],

               "layer_sizes": [(1000,)],

               "weight_init_stddevs": [(.1,)],

               "bias_init_consts": [(1.,)],

               "penalty": [0.],

              }

n_features = train_dataset.get_data_shape()[0]

def model_builder(model_params, model_dir):

  model = dc.models.MultitaskClassifier(

    len(MUV_tasks), n_features, **model_params)

  return model

metric = dc.metrics.Metric(dc.metrics.roc_auc_score, np.mean)

optimizer = dc.hyper.HyperparamOpt(model_builder)

best_dnn, best_hyperparams, all_results = optimizer.hyperparam_search(

    params_dict, train_dataset, valid_dataset, [], metric)

```

%% Output

    Fitting model 1/1

    hyperparameters: {'activation': 'relu', 'momentum': 0.9, 'batch_size': 50, 'init': 'glorot_uniform', 'data_shape': (1024,), 'learning_rate': 0.001, 'decay': 1e-06, 'nb_epoch': 1, 'nesterov': False, 'dropouts': (0.5,), 'nb_layers': 1, 'batchnorm': False, 'layer_sizes': (1000,), 'weight_init_stddevs': (0.1,), 'bias_init_consts': (1.0,), 'penalty': 0.0}

    WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.

    Instructions for updating:

    If using Keras pass *_constraint arguments to layers.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:169: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/optimizers.py:76: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:258: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:260: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/keras_model.py:237: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/losses.py:108: The name tf.losses.softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.softmax_cross_entropy instead.

    WARNING:tensorflow:From /root/miniconda/lib/python3.6/site-packages/deepchem/models/losses.py:109: The name tf.losses.Reduction is deprecated. Please use tf.compat.v1.losses.Reduction instead.

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

    computed_metrics: [nan, nan, 0.8181818181818181, nan, nan, nan, nan, nan, nan, nan, nan, 0.7116477272727273, nan, nan, 0.7777777777777778, nan, nan]

    Model 1/1, Metric mean-roc_auc_score, Validation set 0: 0.769202

    	best_validation_score so far: 0.769202

    computed_metrics: [nan, nan, nan, 0.3168604651162791, 0.525, nan, 0.7647058823529411, 0.26775147928994086, 0.18300653594771243, nan, nan, nan, 0.5405405405405406, nan, 0.24614197530864193, nan, nan]

    Model 1/1, Metric mean-roc_auc_score, Validation set 0: 0.406287

    	best_validation_score so far: 0.406287

    computed_metrics: [1.0, nan, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

    Best hyperparameters: ('relu', 0.9, 50, 'glorot_uniform', (1024,), 0.001, 1e-06, 1, False, (0.5,), 1, False, (1000,), (0.1,), (1.0,), 0.0)

    train_score: 1.000000

    validation_score: 0.769202

    validation_score: 0.406287

    /root/miniconda/lib/python3.6/site-packages/deepchem/metrics/__init__.py:368: UserWarning: Error calculating metric mean-roc_auc_score: Only one class present in y_true. ROC AUC score is not defined in that case.

      warnings.warn("Error calculating metric %s: %s" % (self.name, e))

%% Cell type:markdown id: tags:

# Congratulations! Time to join the Community!

Congratulations on completing this tutorial notebook! If you enjoyed working through the tutorial, and want to continue working with DeepChem, we encourage you to finish the rest of the tutorials in this series. You can also help the DeepChem community in the following ways:

## Star DeepChem on [GitHub](https://github.com/deepchem/deepchem)

This helps build awareness of the DeepChem project and the tools for open source drug discovery that we're trying to build.

## Join the DeepChem Gitter

The DeepChem [Gitter](https://gitter.im/deepchem/Lobby) hosts a number of scientists, developers, and enthusiasts interested in deep learning for the life sciences. Join the conversation!

# Bibliography

[1] https://pubs.acs.org/doi/10.1021/ci8002649

[2] https://pubs.acs.org/doi/abs/10.1021/acs.jcim.7b00146