Commit 638f32c2 authored by Bharath Ramsundar's avatar Bharath Ramsundar Committed by GitHub
Browse files

Merge pull request #717 from lilleswing/notebook-tests-more-cr 

More ipython notebook tests
parents 0bc47be0 87e2f255

devtools/jenkins/test_notebooks.sh

+8 −7
Original line number Diff line number Diff line
@@ -5,13 +5,14 @@ export python_version=2.7 
bash scripts/install_deepchem_conda.sh $envname

source activate $envname

python setup.py install

conda install jupyter

conda install nbconvert

conda install jupyter_client

conda install ipykernel

conda install matplotlib

pip install nglview

conda install ipywidgets

conda install -y jupyter

conda install -y nbconvert

conda install -y jupyter_client

conda install -y ipykernel

conda install -y matplotlib

yes | pip install nglview

conda install -y ipywidgets

yes | pip uninstall tensorflow



cd examples/notebooks

nosetests --with-timer tests.py --with-xunit --xunit-file=notebook_tests.xml|| true

examples/notebooks/BACE.ipynb

+253 −273

File changed.

Preview size limit exceeded, changes collapsed.

examples/notebooks/Multitask Networks on MUV.ipynb→examples/notebooks/Multitask_Networks_on_MUV.ipynb

+208 −29
Original line number Diff line number Diff line 
%% Cell type:markdown id: tags:



This notebook walks through the creation of multitask models on MUV. The goal is to demonstrate that multitask methods outperform singletask methods on MUV.



%% Cell type:code id: tags:



``` python

%reload_ext autoreload

%autoreload 2

%pdb off

reload = True

```



%% Output



    Automatic pdb calling has been turned OFF



%% Cell type:code id: tags:



``` python

import deepchem as dc



dataset_file= "../../datasets/muv.csv.gz"

dataset = dc.utils.save.load_from_disk(dataset_file)

print("Columns of dataset: %s" % str(dataset.columns.values))

print("Number of examples in dataset: %s" % str(dataset.shape[0]))

```



%% Output



    /home/leswing/anaconda3/envs/deepchem27/lib/python2.7/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.

      "This module will be removed in 0.20.", DeprecationWarning)



    Columns of dataset: ['MUV-466' 'MUV-548' 'MUV-600' 'MUV-644' 'MUV-652' 'MUV-689' 'MUV-692'

     'MUV-712' 'MUV-713' 'MUV-733' 'MUV-737' 'MUV-810' 'MUV-832' 'MUV-846'

     'MUV-852' 'MUV-858' 'MUV-859' 'mol_id' 'smiles']

    Number of examples in dataset: 93127



%% Cell type:markdown id: tags:



Now, let's visualize some compounds from our dataset



%% Cell type:code id: tags:



``` python

from rdkit import Chem

from rdkit.Chem import Draw

from itertools import islice

from IPython.display import Image, display, HTML



def display_images(filenames):

    """Helper to pretty-print images."""

    imagesList=''.join(

        ["<img style='width: 140px; margin: 0px; float: left; border: 1px solid black;' src='%s' />"

         % str(s) for s in sorted(filenames)])

    display(HTML(imagesList))



def mols_to_pngs(mols, basename="test"):

    """Helper to write RDKit mols to png files."""

    filenames = []

    for i, mol in enumerate(mols):

        filename = "%s%d.png" % (basename, i)

        Draw.MolToFile(mol, filename)

        filenames.append(filename)

    return filenames



num_to_display = 12

molecules = []

for _, data in islice(dataset.iterrows(), num_to_display):

    molecules.append(Chem.MolFromSmiles(data["smiles"]))

display_images(mols_to_pngs(molecules))

```



%% Output





%% Cell type:code id: tags:



``` python

MUV_tasks = ['MUV-692', 'MUV-689', 'MUV-846', 'MUV-859', 'MUV-644',

             'MUV-548', 'MUV-852', 'MUV-600', 'MUV-810', 'MUV-712',

             'MUV-737', 'MUV-858', 'MUV-713', 'MUV-733', 'MUV-652',

             'MUV-466', 'MUV-832']



featurizer = dc.feat.CircularFingerprint(size=1024)

loader = dc.data.CSVLoader(

      tasks=MUV_tasks, smiles_field="smiles",

      featurizer=featurizer)

dataset = loader.featurize(dataset_file)

```



%% Output



    Loading raw samples now.

    shard_size: 8192

    About to start loading CSV from ../../datasets/muv.csv.gz

    Loading shard 1 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 0 took 15.158 s

    Loading shard 2 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 1 took 15.454 s

    Loading shard 3 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 2 took 15.119 s

    Loading shard 4 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 3 took 15.337 s

    Loading shard 5 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 4 took 15.104 s

    Loading shard 6 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 5 took 15.349 s

    Loading shard 7 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 6 took 15.181 s

    Loading shard 8 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 7 took 15.062 s

    Loading shard 9 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 8 took 14.960 s

    Loading shard 10 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 9 took 15.355 s

    Loading shard 11 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    Featurizing sample 4000

    Featurizing sample 5000

    Featurizing sample 6000

    Featurizing sample 7000

    Featurizing sample 8000

    TIMING: featurizing shard 10 took 15.209 s

    Loading shard 12 of size 8192.

    Featurizing sample 0

    Featurizing sample 1000

    Featurizing sample 2000

    Featurizing sample 3000

    TIMING: featurizing shard 11 took 5.511 s

    TIMING: dataset construction took 176.355 s

    Loading dataset from disk.



%% Cell type:code id: tags:



``` python

splitter = dc.splits.RandomSplitter(dataset_file)

train_dataset, valid_dataset, test_dataset = splitter.train_valid_test_split(

    dataset)

#NOTE THE RENAMING:

valid_dataset, test_dataset = test_dataset, valid_dataset

```



%% Output



    Computing train/valid/test indices

    TIMING: dataset construction took 3.757 s

    Loading dataset from disk.

    TIMING: dataset construction took 1.717 s

    Loading dataset from disk.

    TIMING: dataset construction took 1.707 s

    Loading dataset from disk.



%% Cell type:code id: tags:



``` python

import numpy as np

import numpy.random



params_dict = {"activation": ["relu"],

               "momentum": [.9],

               "batch_size": [50],

               "init": ["glorot_uniform"],

               "data_shape": [train_dataset.get_data_shape()],

               "learning_rate": [1e-3],

               "decay": [1e-6],

               "nb_epoch": [1],

               "nesterov": [False],

               "dropouts": [(.5,)],

               "nb_layers": [1],

               "batchnorm": [False],

               "layer_sizes": [(1000,)],

               "weight_init_stddevs": [(.1,)],

               "bias_init_consts": [(1.,)],

               "penalty": [0.],

              }





n_features = train_dataset.get_data_shape()[0]

def model_builder(model_params, model_dir):

  model = dc.models.TensorflowMultiTaskClassifier(

    len(MUV_tasks), n_features, **model_params)

  return model



metric = dc.metrics.Metric(dc.metrics.roc_auc_score, np.mean)

optimizer = dc.hyper.HyperparamOpt(model_builder)

best_dnn, best_hyperparams, all_results = optimizer.hyperparam_search(

    params_dict, train_dataset, valid_dataset, [], metric)

```



%% Output



    Fitting model 1/1

    hyperparameters: {'learning_rate': 0.001, 'layer_sizes': (1000,), 'data_shape': (1024,), 'dropouts': (0.5,), 'activation': 'relu', 'decay': 1e-06, 'batch_size': 50, 'penalty': 0.0, 'nesterov': False, 'init': 'glorot_uniform', 'bias_init_consts': (1.0,), 'weight_init_stddevs': (0.1,), 'batchnorm': False, 'nb_layers': 1, 'nb_epoch': 1, 'momentum': 0.9}

    Training for 1 epochs

    On batch 0

    On batch 50

    On batch 100

    On batch 150

    On batch 200

    On batch 250

    On batch 300

    On batch 350

    On batch 400

    On batch 450

    On batch 500

    On batch 550

    On batch 600

    On batch 650

    On batch 700

    On batch 750

    On batch 800

    On batch 850

    On batch 900

    On batch 950

    On batch 1000

    On batch 1050

    On batch 1100

    On batch 1150

    On batch 1200

    On batch 1250

    On batch 1300

    On batch 1350

    On batch 1400

    On batch 1450

    Ending epoch 0: Average loss 0.0512837

    On batch 0

    TIMING: model fitting took 21.154 s True

    computed_metrics: [0.82459677419354838, 0.57987551867219911, 0.87625923438549358, 0.40115904700579524, 0.85100593667546176, 0.84176720285969608, 0.81982616866337787, 0.92716408268733852, 0.90555555555555556, 0.93358568479008941, 0.84312370421561855, 0.5993421052631579, 0.40930420711974114, 0.53205995041694831, 0.99836708033964727, 0.6931528662420382, 0.89704918032786884]

    Model 1/1, Metric mean-roc_auc_score, Validation set 0: 0.760776

    	best_validation_score so far: 0.760776

    computed_metrics: [0.88035724476402444, 0.92499356167911406, 0.98634926369263543, 0.86291516863550766, 0.95042120575284494, 0.97632672092234651, 0.9834740792000799, 0.91723021876461464, 0.9288788241326269, 0.9427229610525234, 0.9281144552776599, 0.94111351509505148, 0.90709998845843109, 0.93500087572450941, 0.95015000422547113, 0.92490461101880572, 0.97988502677604972]

    Best hyperparameters: (1e-06, (1024,), 1, 'relu', (1000,), 50, 0.0, False, 'glorot_uniform', (1.0,), (0.1,), 1, False, (0.5,), 0.001, 0.9)

    train_score: 0.936467

    validation_score: 0.760776

examples/notebooks/mnist.ipynb

0 → 100644
+127 −0
Original line number Diff line number Diff line 
%% Cell type:code id: tags:



``` python

from tensorflow.examples.tutorials.mnist import input_data

```



%% Cell type:code id: tags:



``` python

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

```



%% Output



    Extracting MNIST_data/train-images-idx3-ubyte.gz

    Extracting MNIST_data/train-labels-idx1-ubyte.gz

    Extracting MNIST_data/t10k-images-idx3-ubyte.gz

    Extracting MNIST_data/t10k-labels-idx1-ubyte.gz



%% Cell type:code id: tags:



``` python

import deepchem as dc

import tensorflow as tf

from deepchem.models.tensorgraph.layers import Layer, Input, Reshape, Flatten, Conv2D, Label, Feature

from deepchem.models.tensorgraph.layers import Dense, SoftMaxCrossEntropy, ReduceMean, SoftMax

```



%% Output



    /home/kleswing/miniconda3/envs/j3udBScCYSIoPrpc/lib/python2.7/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.

      "This module will be removed in 0.20.", DeprecationWarning)



%% Cell type:code id: tags:



``` python

train = dc.data.NumpyDataset(mnist.train.images, mnist.train.labels)

valid = dc.data.NumpyDataset(mnist.validation.images, mnist.validation.labels)

```



%% Cell type:code id: tags:



``` python

tg = dc.models.TensorGraph(tensorboard=True, model_dir='/tmp/mnist')

feature = Feature(shape=(None, 784))



# Images are square 28x28 (batch, height, width, channel)

make_image = Reshape(shape=(-1, 28, 28, 1), in_layers=[feature])



conv2d_1 = Conv2D(num_outputs=32, in_layers=[make_image])



conv2d_2 = Conv2D(num_outputs=64, in_layers=[conv2d_1])



flatten = Flatten(in_layers=[conv2d_2])



dense1 = Dense(out_channels=1024, activation_fn=tf.nn.relu, in_layers=[flatten])



dense2 = Dense(out_channels=10, in_layers=[dense1])



label = Label(shape=(None, 10))



smce = SoftMaxCrossEntropy(in_layers=[label, dense2])

loss = ReduceMean(in_layers=[smce])

tg.set_loss(loss)



output = SoftMax(in_layers=[dense2])

tg.add_output(output)

```



%% Cell type:code id: tags:



``` python

tg.fit(train, nb_epoch=10)

tg.save()

```



%% Output



    Ending global_step 999: Average loss 0.0826627

    Ending global_step 1999: Average loss 0.0179067

    Ending global_step 2999: Average loss 0.0104273

    Ending global_step 3999: Average loss 0.00756823

    Ending global_step 4999: Average loss 0.00675081

    Ending global_step 5446: Average loss 0.00677046

    TIMING: model fitting took 91.544 s



%% Cell type:code id: tags:



``` python

from sklearn.metrics import roc_curve, auc

import numpy as np



print("Validation")

prediction = np.squeeze(tg.predict_on_batch(valid.X))



fpr = dict()

tpr = dict()

roc_auc = dict()

for i in range(10):

    fpr[i], tpr[i], thresh = roc_curve(valid.y[:, i], prediction[:, i])

    roc_auc[i] = auc(fpr[i], tpr[i])

    print("class %s:auc=%s" % (i, roc_auc[i]))

```



%% Output



    Validation

    class 0:auc=0.99998060547

    class 1:auc=0.999953963742

    class 2:auc=0.999864659633

    class 3:auc=0.99996714599

    class 4:auc=0.999773524087

    class 5:auc=0.999956097059

    class 6:auc=0.999952972472

    class 7:auc=0.99973442288

    class 8:auc=0.999880756822

    class 9:auc=0.999752015157



%% Cell type:code id: tags:



``` python

```



%% Cell type:code id: tags:



``` python

```

examples/notebooks/quantum_machine_gdb1k.ipynb

+40 −92

File changed.

Preview size limit exceeded, changes collapsed.

CRA Git | Maintained and supported by SUSTech CRA and CCSE