Commit 68fff8cc authored by Franklin Lee's avatar Franklin Lee
Browse files

Merge branch 'master' of https://github.com/deepchem/deepchem

parents 9b00d5a6 f5218230
Loading
Loading
Loading
Loading
+9 −9
Original line number Diff line number Diff line
language: python
python:
- '2.7'
- '3.5'
sudo: required
dist: trusty
install:
- wget http://repo.continuum.io/archive/Anaconda2-2.4.1-Linux-x86_64.sh -O anaconda.sh;
- if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then wget https://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh
  -O anaconda.sh; else wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh
  -O anaconda.sh; fi
- bash anaconda.sh -b -p $HOME/anaconda
- export PATH="$HOME/anaconda/bin:$PATH"
- hash -r
@@ -16,8 +19,9 @@ install:
- conda install -c omnia boost=1.59.0
- conda install -c omnia openbabel
- conda install joblib
- conda install -c omnia theano
- conda install -c omnia keras
- conda install h5py
- pip install keras
- export KERAS_BACKEND=tensorflow
- conda install seaborn
- conda install six
- conda install dill
@@ -31,9 +35,5 @@ install:
script:
- nosetests -v deepchem --nologcapture
after_success:
- echo $TRAVIS_SECURE_ENV_VARS
- source devtools/travis-ci/after_sucess.sh
# AWS access_key and secret key secured through travis secure var api.
env:
  global:
  - secure: FojR+Zrw/XLeDWHaZnwqkBt7DGwOJ6N9X0HJIkkzGsTxQy405qY9bfKMgzivY4UAppqWH2R9kcY/3K2lnFOU9Z2dSf9ZSLVmXjQQK+YhiFiPVQG1bQ8XEhQ51927AZn2wGqzPSFlOI8M4ek2V4a6d72Wg6SOooVbI+A9PtrdXNJlEfonY0QfSakhg+IFPBsN84khbGaHlTRCXvozRTSm3Ubo4jidN54WbO7Ll18qJ183Fgh8SVxYfodDAusZaWU2e/q9lCk6nkJ9bDC9anPbuTXpSlIkH7x7WcrMbLCrvvHJq26YOCjReCDea1/N8ECsarXWfvPjz1Cv59QSghoLxFeRRNCDHWaWTccKgWRAFd60+vSgd8CcoMFsMY0XedO2mSQztLAb2pS2TCoCPMD/5Zip5pkXOxJB5CwPqSuJ41FBPYthzyhG1MPbaQnPDZjtx5TDaVdN5w+Nfc40i9+0rFNisEyAdi2dTcr6bWZ7UlWj7LBDJGVjYPC8lKsaip6dyov3KeLveod0IbY5zCVnqF2rwuID5n/6ejGgytv1caZWa9TvEjzuSJoo8d2uXwyTzQLntardt2ZvtA8ZmwChyaq0pNtUgxQE4wn8hZG19+lAvIc95ZDJuSV7xwuo5XsMDffa3jrjyWhByveyfbHs1znAmAl4w6NJ32ylWr0/+Do=
  - secure: JDldUkMQE7+vlKX2ghTFhGWCWHzJJtC4w8xIzAaKo04zHaBcrHSmhXXwkkPjyGjgcXE6ORKUI/zJnCecErpPbHYVEQUGzyTG9ikQr0pIENTua5j5AHJUt5Y+aWsD2fzNWE1ZT88sxKAqj96rsfRfj7coymkoyLXrvqdd5OVm+sEiup5MLGEutbMrOPXChSXBUiGetLakoMnQ0WznOttWVUb52j5SsCJSOh7baX0eNdEmaGZ+ok4R/j8/c8LTARVOC2AZ6c6BIfJPHk7rPpq4HoeVRIgp2o7vbjosE+eI499OPP1AcFHUm38XJ68mDsw5exu/ONEbFgxzA2hXqVW/TDnWpaJ0hLKxjGg8JnJ63OYWyY2nk7UGQNPwiVj4DiN7zsNkV1+IbaUmiGjbWAxyCa8FBOoXos7DQ7pVtWNvhZtsAzJaCNgvCFgsrxcutxGuN9zkpmL30QNpE6HT/mvmww5MHH5KacBYPLjqmihQCuasT9L6k2+dPspRRfbrgyBZJxffteWumXBThIaqs8BuqG1CxxgkOgvUTSBZyoZtfX3CW8ClEIKT89+bQXQKwcqSqWCu7qsFx7A6ssmzliCzO5iXZhxaPS7rICqRu4HPJ1YNS5bUbwwEmBeMw0ATsxpfo0ityNSJbO0AvXiIr9fcVv0cPJuz3pialFrGDMMPjeY=
+141 −24
Original line number Diff line number Diff line
deepchem
=============

Deep Learning Toolchain for Drug Discovery and Quantum Chemistry

Requirements
------------
# DeepChem

DeepChem aims to provide a high quality open-source toolchain that
democratizes the use of deep-learning in drug discovery, materials science, and quantum
chemistry. DeepChem is a package developed by the [Pande group](https://pande.stanford.edu/) at
Stanford and originally created by [Bharath Ramsundar](http://rbharath.github.io/). 

### Table of contents:

* [Requirements](#requirements)
* [Installation from Source](#installation)
* [FAQ](#faq)
* [Getting Started](#getting-started)
    * [Input Formats](#input-formats)
    * [Data Featurization](#data-featurization)
* [Contributing to DeepChem](#contributing-to-deepchem)
    * [Code Style Guidelines](#code-style-guidelines)
    * [Documentation Style Guidelines](#documentation-style-guidelines)
* [DeepChem Publications](#deepchem-publications)
* [Examples](/examples)
* [About Us](#about-us)

## Requirements
* [openbabel](http://openbabel.org/wiki/Main_Page)
* [pandas](http://pandas.pydata.org/)
* [rdkit](http://www.rdkit.org/docs/Install.html)
@@ -12,7 +28,6 @@ Requirements
* [joblib](https://pypi.python.org/pypi/joblib)
* [sklearn](https://github.com/scikit-learn/scikit-learn.git)
* [numpy](https://store.continuum.io/cshop/anaconda/)
* [theano](http://deeplearning.net/software/theano/)
* [keras](http://keras.io)
* [six](https://pypi.python.org/pypi/six)
* [dill](https://pypi.python.org/pypi/dill)
@@ -20,13 +35,12 @@ Requirements
* [mdtraj](http://mdtraj.org/)
* [tensorflow](https://www.tensorflow.org/)

Linux (64-bit) Installation 
---------------------------
## Installation

Installation from source is the only currently supported format. ```deepchem``` currently supports both Python 2.7 and Python 3.5, but is not supported on any OS'es except 64 bit linux. Please make sure you follow the directions below precisely. While you may already have system versions of some of these packages, there is no guarantee that `deepchem` will work with alternate versions than those specified below.

```deepchem``` currently requires Python 2.7, and is not supported on any platforms except 64 bit linux. Please make sure you follow the directions below precisely. While you may already have system versions of some of these packages, there is no guarantee that `deepchem` will work with alternate versions than those specified below. This is especially true for the `keras` library, where `deepchem` is known not to work with the latest default version.
1. Download the **64-bit** Python 2.7 or Python 3.5 versions of Anaconda for linux [here](https://www.continuum.io/downloads#_unix). 
   
1. Anaconda 2.7
   Download the **64-bit Python 2.7** version of Anaconda for linux [here](https://www.continuum.io/downloads#_unix).  
   Follow the [installation instructions](http://docs.continuum.io/anaconda/install#linux-install)

2. `openbabel`
@@ -54,35 +68,41 @@ Linux (64-bit) Installation
   conda install joblib 
   ```

7. `theano`
7. `keras`
   ```bash
   conda install -c omnia theano
   pip install keras --user
   ```

8. `keras`
   `deepchem` only supports the `tensorflow` backend for keras. To set the backend to `tensorflow`,
   add the following line to your `~/.bashrc`
   ```bash
   conda install -c omnia keras
   export KERAS_BACKEND=tensorflow
   ```
   See [keras docs](https://keras.io/backend/) for more details and alternate methods of setting backend.

9. `six`
8. `six`
   ```bash
   conda install six
   ```
10. `dill`
9. `dill`
    ```bash
    conda install dill
    ```

11. `ipyparallel`
10. `ipyparallel`
    ```bash
    conda install ipyparallel
    ```

12. `mdtraj`
11. `mdtraj`
   ```bash
   conda install -c omnia mdtraj
   ```
   
12. `scikit-learn`
    ```bash
    conda install scikit-learn 
    ```

13. `tensorflow`: Installing `tensorflow` on older versions of Linux (which
    have glibc < 2.17) can be very challenging. For these older Linux versions,
    contact your local sysadmin to work out a custom installation. If your
@@ -91,7 +111,12 @@ Linux (64-bit) Installation
    conda install -c https://conda.anaconda.org/jjhelmus tensorflow
    ```

14. `deepchem`: Clone the `deepchem` github repo:
14. `h5py`:
    ```
    conda install h5py
    ```

15. `deepchem`: Clone the `deepchem` github repo:
    ```bash
    git clone https://github.com/deepchem/deepchem.git
    ```
@@ -100,7 +125,99 @@ Linux (64-bit) Installation
    python setup.py install
    ```

15. If installation has been successful, all tests in test suite should pass:
16. To run test suite, install `nosetests`:
    ```bash
    pip install nose --user
    ```
    Make sure that the correct version of `nosetests` is active by running
    ```bash
    which nosetests 
    ```
    You might need to uninstall a system install of `nosetests` if
    there is a conflict.

17. If installation has been successful, all tests in test suite should pass:
    ```bash
    nosetests -v deepchem --nologcapture 
    ```
    Note that the full test-suite uses up a fair amount of memory. 
    Try running tests for one submodule at a time if memory proves an issue.

## FAQ
1. Question: I'm seeing some failures in my test suite having to do with MKL
   ```Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.```

   Answer: This is a general issue with the newest version of `scikit-learn` enabling MKL by default. This doesn't play well with many linux systems. See BVLC/caffe#3884 for discussions. The following seems to fix the issue
   ```bash
   conda install nomkl numpy scipy scikit-learn numexpr
   conda remove mkl mkl-service
   ```
2. Question: The test suite is core-dumping for me. What's up?
   ```
   [rbharath]$ nosetests -v deepchem --nologcapture
   Illegal instruction (core dumped)
   ```
   
   Answer: This is often due to `openbabel` issues on older linux systems. Open `ipython` and run the following
   ```
   In [1]: import openbabel as ob
   ```
   If you see a core-dump, then it's a sign there's an issue with your `openbabel` install. Try reinstalling `openbabel` from source for your machine.
   
   
## Getting Started
The first step to getting started is looking at the examples in the `examples/` directory. Try running some of these examples on your system and verify that the models train successfully. Afterwards, to apply `deepchem` to a new problem, try starting from one of the existing examples and modifying it step by step to work with your new use-case.

### Input Formats
Accepted input formats for deepchem include csv, pkl.gz, and sdf files. For
example, with a csv input, in order to build models, we expect the
following columns to have entries for each row in the csv file.

1. A column containing SMILES strings [1].
2. A column containing an experimental measurement.
3. (Optional) A column containing a unique compound identifier.

Here's an example of a potential input file. 

|Compound ID    | measured log solubility in mols per litre | smiles         | 
|---------------|-------------------------------------------|----------------| 
| benzothiazole | -1.5                                      | c2ccc1scnc1c2  | 


Here the "smiles" column contains the SMILES string, the "measured log
solubility in mols per litre" contains the experimental measurement and
"Compound ID" contains the unique compound identifier.

[2] Anderson, Eric, Gilman D. Veith, and David Weininger. "SMILES, a line
notation and computerized interpreter for chemical structures." US
Environmental Protection Agency, Environmental Research Laboratory, 1987.

### Data Featurization

Most machine learning algorithms require that input data form vectors.
However, input data for drug-discovery datasets routinely come in the
format of lists of molecules and associated experimental readouts. To
transform lists of molecules into vectors, we need to use the ``deechem``
featurization class ``DataFeaturizer``. Instances of this class must be
passed a ``Featurizer`` object. ``deepchem`` provides a number of
different subclasses of ``Featurizer`` for convenience:

## Contributing to DeepChem

We actively encourage community contributions to DeepChem. The first place to start getting involved is by running our examples locally. Afterwards, we encourage contributors to give a shot to improving our documentation. While we take effort to provide good docs, there's plenty of room for improvement. All docs are hosted on Github, either in this `README.md` file, or in the `docs/` directory.

Once you've got a sense of how the package works, we encourage the use of Github issues to discuss more complex changes,  raise requests for new features or propose changes to the global architecture of DeepChem. Once consensus is reached on the issue, please submit a PR with proposed modifications. All contributed code to DeepChem will be reviewed by a member of the DeepChem team, so please make sure your code style and documentation style match our guidelines!

### Code Style Guidelines
DeepChem broadly follows the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html). In terms of practical changes, the biggest effect is that all code uses 2-space indents instead of 4-space indents. We encourage new contributors to make use of [pylint](https://www.pylint.org/). Aim for a score of at least 8/10 on contributed files.

### Documentation Style Guidelines
DeepChem uses [NumPy style documentation](https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt). Please follow these conventions when documenting code, since we use [Sphinx+Napoleon](http://www.sphinx-doc.org/en/stable/ext/napoleon.html) to automatically generate docs on [deepchem.io](deepchem.io). 

## DeepChem Publications
1. [Computational Modeling of β-secretase 1 (BACE-1) Inhibitors using
Ligand Based
Approaches](http://pubs.acs.org/doi/abs/10.1021/acs.jcim.6b00290)

## About Us
DeepChem is a package by the [Pande group](https://pande.stanford.edu/) at Stanford. DeepChem was originally created by [Bharath Ramsundar](http://rbharath.github.io/), and has grown through the contributions of a number of undergraduate, graduate, and postdoctoral researchers working with the Pande lab.
+510 −218

File changed.

Preview size limit exceeded, changes collapsed.

+27 −9
Original line number Diff line number Diff line
@@ -9,12 +9,12 @@ import os
import numpy as np
import shutil
from deepchem.utils.save import load_from_disk
from deepchem.datasets import Dataset
from deepchem.datasets import DiskDataset
from deepchem.featurizers.featurize import DataLoader
from deepchem.featurizers.fingerprints import CircularFingerprint
from deepchem.transformers import BalancingTransformer

def load_muv(base_dir, reload=True):
def load_muv(base_dir, reload=True, frac_train=.8):
  """Load MUV datasets. Does not do train/test split"""
  # Set some global variables up top
  reload = True
@@ -32,6 +32,8 @@ def load_muv(base_dir, reload=True):
  current_dir = os.path.dirname(os.path.realpath(__file__))
  #Make directories to store the raw and featurized datasets.
  data_dir = os.path.join(base_dir, "dataset")
  train_dir = os.path.join(base_dir, "train_dataset")
  valid_dir = os.path.join(base_dir, "valid_dataset")

  # Load MUV dataset
  print("About to load MUV dataset.")
@@ -44,12 +46,12 @@ def load_muv(base_dir, reload=True):
  # Featurize MUV dataset
  print("About to featurize MUV dataset.")
  featurizer = CircularFingerprint(size=1024)
  all_MUV_tasks = sorted(['MUV-692', 'MUV-689', 'MUV-846', 'MUV-859', 'MUV-644',
  MUV_tasks = sorted(['MUV-692', 'MUV-689', 'MUV-846', 'MUV-859', 'MUV-644',
                      'MUV-548', 'MUV-852', 'MUV-600', 'MUV-810', 'MUV-712',
                      'MUV-737', 'MUV-858', 'MUV-713', 'MUV-733', 'MUV-652',
                      'MUV-466', 'MUV-832'])

  loader = DataLoader(tasks=all_MUV_tasks,
  loader = DataLoader(tasks=MUV_tasks,
                      smiles_field="smiles",
                      featurizer=featurizer,
                      verbosity=verbosity)
@@ -57,7 +59,7 @@ def load_muv(base_dir, reload=True):
    dataset = loader.featurize(dataset_file, data_dir)
    regen = True
  else:
    dataset = Dataset(data_dir, reload=True)
    dataset = DiskDataset(data_dir, reload=True)

  # Initialize transformers 
  transformers = [
@@ -67,4 +69,20 @@ def load_muv(base_dir, reload=True):
    for transformer in transformers:
        transformer.transform(dataset)

  return all_MUV_tasks, dataset, transformers
  X, y, w, ids = (dataset.X, dataset.y, dataset.w, dataset.ids)
  num_tasks = 17
  num_train = frac_train * len(dataset)
  MUV_tasks = MUV_tasks[:num_tasks]
  print("Using following tasks")
  print(MUV_tasks)
  X_train, X_valid = X[:num_train], X[num_train:]
  y_train, y_valid = y[:num_train, :num_tasks], y[num_train:, :num_tasks]
  w_train, w_valid = w[:num_train, :num_tasks], w[num_train:, :num_tasks]
  ids_train, ids_valid = ids[:num_train], ids[num_train:]

  train_dataset = DiskDataset.from_numpy(train_dir, X_train, y_train,
                                     w_train, ids_train, MUV_tasks)
  valid_dataset = DiskDataset.from_numpy(valid_dir, X_valid, y_valid,
                                     w_valid, ids_valid, MUV_tasks)
  
  return MUV_tasks, (train_dataset, valid_dataset), transformers
+4 −2
Original line number Diff line number Diff line
@@ -17,7 +17,8 @@ from deepchem.featurizers.featurize import DataLoader
from deepchem.featurizers.fingerprints import CircularFingerprint
from deepchem.transformers import NormalizationTransformer

def load_nci(base_dir, reload=True, force_transform=False):
def load_nci(base_dir, reload=True, force_transform=False,
             shard_size=1000, num_shards_per_batch=4):
  """Load NCI datasets. Does not do train/test split"""
  # Set some global variables up top
  verbosity = "high"
@@ -70,7 +71,8 @@ def load_nci(base_dir, reload=True, force_transform=False):
                      featurizer=featurizer,
                      verbosity=verbosity)
  if not reload or not os.path.exists(data_dir):
    dataset = loader.featurize(dataset_paths, data_dir)
    dataset = loader.featurize(dataset_paths, data_dir, shard_size=shard_size,
                               num_shards_per_batch=num_shards_per_batch)
    regen = True
  else:
    dataset = Dataset(data_dir, reload=True)
Loading