Loading README.md +37 −20 Original line number Diff line number Diff line Loading @@ -203,28 +203,30 @@ passed a ``Featurizer`` object. DeepChem provides a number of different subclasses of ``Featurizer`` for convenience: ### Performances 1. Classification Index splitting |Dataset |Model |Train score/ROC-AUC|Valid score/ROC-AUC| |-----------|--------------------|-------------------|-------------------| |tox21 |logistic regression |0.903 |0.705 | | |tensorflow(MT-NN) |0.856 |0.763 | | |Multitask network |0.856 |0.763 | | |robust MT-NN |0.857 |0.767 | | |graph convolution |0.872 |0.798 | |muv |logistic regression |0.963 |0.766 | | |tensorflow(MT-NN) |0.904 |0.764 | | |Multitask network |0.904 |0.764 | | |robust MT-NN |0.934 |0.781 | | |graph convolution |0.840 |0.823 | |pcba |logistic regression |0.809 |0.776 | | |tensorflow(MT-NN) |0.826 |0.802 | | |Multitask network |0.826 |0.802 | | |robust MT-NN |0.809 |0.783 | | |graph convolution |0.876 |0.852 | |sider |logistic regression |0.933 |0.620 | | |tensorflow(MT-NN) |0.775 |0.634 | | |Multitask network |0.775 |0.634 | | |robust MT-NN |0.803 |0.632 | | |graph convolution |0.708 |0.594 | |toxcast |logistic regression |0.721 |0.575 | | |tensorflow(MT-NN) |0.830 |0.678 | | |Multitask network |0.830 |0.678 | | |robust MT-NN |0.825 |0.680 | | |graph convolution |0.821 |0.720 | Loading @@ -233,23 +235,23 @@ Random splitting |Dataset |Model |Train score/ROC-AUC|Valid score/ROC-AUC| |-----------|--------------------|-------------------|-------------------| |tox21 |logistic regression |0.903 |0.741 | | |tensorflow(MT-NN) |0.846 |0.812 | | |Multitask network |0.846 |0.812 | | |robust MT-NN |0.844 |0.793 | | |graph convolution |0.872 |0.816 | |muv |logistic regression |0.961 |0.696 | | |tensorflow(MT-NN) |0.895 |0.740 | | |Multitask network |0.895 |0.740 | | |robust MT-NN |0.914 |0.667 | | |graph convolution |0.846 |0.776 | |pcba |logistic regression |0.807 |0.772 | | |tensorflow(MT-NN) |0.811 |0.787 | | |Multitask network |0.811 |0.787 | | |robust MT-NN |0.809 |0.778 | | |graph convolution |0.875 |0.844 | |sider |logistic regression |0.932 |0.628 | | |tensorflow(MT-NN) |0.779 |0.665 | | |Multitask network |0.779 |0.665 | | |robust MT-NN |0.761 |0.621 | | |graph convolution |0.706 |0.638 | |toxcast |logistic regression |0.737 |0.543 | | |tensorflow(MT-NN) |0.831 |0.684 | | |Multitask network |0.831 |0.684 | | |robust MT-NN |0.814 |0.692 | | |graph convolution |0.820 |0.692 | Loading @@ -258,26 +260,37 @@ Scaffold splitting |Dataset |Model |Train score/ROC-AUC|Valid score/ROC-AUC| |-----------|--------------------|-------------------|-------------------| |tox21 |logistic regression |0.900 |0.650 | | |tensorflow(MT-NN) |0.863 |0.703 | | |Multitask network |0.863 |0.703 | | |robust MT-NN |0.861 |0.710 | | |graph convolution |0.885 |0.732 | |muv |logistic regression |0.947 |0.767 | | |tensorflow(MT-NN) |0.899 |0.762 | | |Multitask network |0.899 |0.762 | | |robust MT-NN |0.944 |0.726 | | |graph convolution |0.872 |0.795 | |pcba |logistic regression |0.810 |0.742 | | |tensorflow(MT-NN) |0.814 |0.760 | | |Multitask network |0.814 |0.760 | | |robust MT-NN |0.812 |0.756 | | |graph convolution |0.874 |0.817 | |sider |logistic regression |0.926 |0.592 | | |tensorflow(MT-NN) |0.776 |0.557 | | |Multitask network |0.776 |0.557 | | |robust MT-NN |0.797 |0.560 | | |graph convolution |0.722 |0.583 | |toxcast |logistic regression |0.716 |0.492 | | |tensorflow(MT-NN) |0.828 |0.617 | | |Multitask network |0.828 |0.617 | | |robust MT-NN |0.830 |0.614 | | |graph convolution |0.832 |0.638 | 2. Regression |Dataset |Model |Splitting |Train score/R2|Valid score/R2| |-----------|--------------------|------------|--------------|--------------| |delaney |MT-NN regression |Index |0.773 |0.574 | | |MT-NN regression |Random |0.769 |0.591 | | |MT-NN regression |Scaffold |0.782 |0.426 | |kaggle |MT-NN regression |User-defined|0.748 |0.452 | 3. General features Number of tasks and examples in the datasets |Dataset |N(tasks) |N(samples) | Loading @@ -287,31 +300,35 @@ Number of tasks and examples in the datasets |pcba |128 |439863 | |sider |27 |1427 | |toxcast |617 |8615 | |delaney |1 |1128 | |kaggle |15 |173065 | Time needed for benchmark test(~20h in total) |Dataset |Model |Time(loading)/s |Time(running)/s| |-----------|--------------------|----------------|---------------| |tox21 |logistic regression |30 |60 | | |tensorflow(MT-NN) |30 |60 | | |Multitask network |30 |60 | | |robust MT-NN |30 |90 | | |graph convolution |40 |160 | |muv |logistic regression |600 |450 | | |tensorflow(MT-NN) |600 |400 | | |Multitask network |600 |400 | | |robust MT-NN |600 |550 | | |graph convolution |800 |1800 | |pcba |logistic regression |1800 |10000 | | |tensorflow(MT-NN) |1800 |9000 | | |Multitask network |1800 |9000 | | |robust MT-NN |1800 |14000 | | |graph convolution |2200 |14000 | |sider |logistic regression |15 |80 | | |tensorflow(MT-NN) |15 |75 | | |Multitask network |15 |75 | | |robust MT-NN |15 |150 | | |graph convolution |20 |50 | |toxcast |logistic regression |80 |2600 | | |tensorflow(MT-NN) |80 |2300 | | |Multitask network |80 |2300 | | |robust MT-NN |80 |4000 | | |graph convolution |80 |900 | |delaney |MT-NN regression |10 |40 | |kaggle |MT-NN regression |2200 |3200 | ## Contributing to DeepChem Loading examples/kaggle/kaggle_datasets.py +1 −1 Original line number Diff line number Diff line Loading @@ -31,7 +31,7 @@ def remove_missing_entries(dataset): dataset.set_shard(i, X, y, w, ids) # Set shard size low to avoid memory problems. def load_kaggle(shard_size=10000, num_shards_per_batch=4, def load_kaggle(shard_size=2000, num_shards_per_batch=4, featurizer=None): """Load KAGGLE datasets. Does not do train/test split""" ############################################################## TIMING Loading Loading
README.md +37 −20 Original line number Diff line number Diff line Loading @@ -203,28 +203,30 @@ passed a ``Featurizer`` object. DeepChem provides a number of different subclasses of ``Featurizer`` for convenience: ### Performances 1. Classification Index splitting |Dataset |Model |Train score/ROC-AUC|Valid score/ROC-AUC| |-----------|--------------------|-------------------|-------------------| |tox21 |logistic regression |0.903 |0.705 | | |tensorflow(MT-NN) |0.856 |0.763 | | |Multitask network |0.856 |0.763 | | |robust MT-NN |0.857 |0.767 | | |graph convolution |0.872 |0.798 | |muv |logistic regression |0.963 |0.766 | | |tensorflow(MT-NN) |0.904 |0.764 | | |Multitask network |0.904 |0.764 | | |robust MT-NN |0.934 |0.781 | | |graph convolution |0.840 |0.823 | |pcba |logistic regression |0.809 |0.776 | | |tensorflow(MT-NN) |0.826 |0.802 | | |Multitask network |0.826 |0.802 | | |robust MT-NN |0.809 |0.783 | | |graph convolution |0.876 |0.852 | |sider |logistic regression |0.933 |0.620 | | |tensorflow(MT-NN) |0.775 |0.634 | | |Multitask network |0.775 |0.634 | | |robust MT-NN |0.803 |0.632 | | |graph convolution |0.708 |0.594 | |toxcast |logistic regression |0.721 |0.575 | | |tensorflow(MT-NN) |0.830 |0.678 | | |Multitask network |0.830 |0.678 | | |robust MT-NN |0.825 |0.680 | | |graph convolution |0.821 |0.720 | Loading @@ -233,23 +235,23 @@ Random splitting |Dataset |Model |Train score/ROC-AUC|Valid score/ROC-AUC| |-----------|--------------------|-------------------|-------------------| |tox21 |logistic regression |0.903 |0.741 | | |tensorflow(MT-NN) |0.846 |0.812 | | |Multitask network |0.846 |0.812 | | |robust MT-NN |0.844 |0.793 | | |graph convolution |0.872 |0.816 | |muv |logistic regression |0.961 |0.696 | | |tensorflow(MT-NN) |0.895 |0.740 | | |Multitask network |0.895 |0.740 | | |robust MT-NN |0.914 |0.667 | | |graph convolution |0.846 |0.776 | |pcba |logistic regression |0.807 |0.772 | | |tensorflow(MT-NN) |0.811 |0.787 | | |Multitask network |0.811 |0.787 | | |robust MT-NN |0.809 |0.778 | | |graph convolution |0.875 |0.844 | |sider |logistic regression |0.932 |0.628 | | |tensorflow(MT-NN) |0.779 |0.665 | | |Multitask network |0.779 |0.665 | | |robust MT-NN |0.761 |0.621 | | |graph convolution |0.706 |0.638 | |toxcast |logistic regression |0.737 |0.543 | | |tensorflow(MT-NN) |0.831 |0.684 | | |Multitask network |0.831 |0.684 | | |robust MT-NN |0.814 |0.692 | | |graph convolution |0.820 |0.692 | Loading @@ -258,26 +260,37 @@ Scaffold splitting |Dataset |Model |Train score/ROC-AUC|Valid score/ROC-AUC| |-----------|--------------------|-------------------|-------------------| |tox21 |logistic regression |0.900 |0.650 | | |tensorflow(MT-NN) |0.863 |0.703 | | |Multitask network |0.863 |0.703 | | |robust MT-NN |0.861 |0.710 | | |graph convolution |0.885 |0.732 | |muv |logistic regression |0.947 |0.767 | | |tensorflow(MT-NN) |0.899 |0.762 | | |Multitask network |0.899 |0.762 | | |robust MT-NN |0.944 |0.726 | | |graph convolution |0.872 |0.795 | |pcba |logistic regression |0.810 |0.742 | | |tensorflow(MT-NN) |0.814 |0.760 | | |Multitask network |0.814 |0.760 | | |robust MT-NN |0.812 |0.756 | | |graph convolution |0.874 |0.817 | |sider |logistic regression |0.926 |0.592 | | |tensorflow(MT-NN) |0.776 |0.557 | | |Multitask network |0.776 |0.557 | | |robust MT-NN |0.797 |0.560 | | |graph convolution |0.722 |0.583 | |toxcast |logistic regression |0.716 |0.492 | | |tensorflow(MT-NN) |0.828 |0.617 | | |Multitask network |0.828 |0.617 | | |robust MT-NN |0.830 |0.614 | | |graph convolution |0.832 |0.638 | 2. Regression |Dataset |Model |Splitting |Train score/R2|Valid score/R2| |-----------|--------------------|------------|--------------|--------------| |delaney |MT-NN regression |Index |0.773 |0.574 | | |MT-NN regression |Random |0.769 |0.591 | | |MT-NN regression |Scaffold |0.782 |0.426 | |kaggle |MT-NN regression |User-defined|0.748 |0.452 | 3. General features Number of tasks and examples in the datasets |Dataset |N(tasks) |N(samples) | Loading @@ -287,31 +300,35 @@ Number of tasks and examples in the datasets |pcba |128 |439863 | |sider |27 |1427 | |toxcast |617 |8615 | |delaney |1 |1128 | |kaggle |15 |173065 | Time needed for benchmark test(~20h in total) |Dataset |Model |Time(loading)/s |Time(running)/s| |-----------|--------------------|----------------|---------------| |tox21 |logistic regression |30 |60 | | |tensorflow(MT-NN) |30 |60 | | |Multitask network |30 |60 | | |robust MT-NN |30 |90 | | |graph convolution |40 |160 | |muv |logistic regression |600 |450 | | |tensorflow(MT-NN) |600 |400 | | |Multitask network |600 |400 | | |robust MT-NN |600 |550 | | |graph convolution |800 |1800 | |pcba |logistic regression |1800 |10000 | | |tensorflow(MT-NN) |1800 |9000 | | |Multitask network |1800 |9000 | | |robust MT-NN |1800 |14000 | | |graph convolution |2200 |14000 | |sider |logistic regression |15 |80 | | |tensorflow(MT-NN) |15 |75 | | |Multitask network |15 |75 | | |robust MT-NN |15 |150 | | |graph convolution |20 |50 | |toxcast |logistic regression |80 |2600 | | |tensorflow(MT-NN) |80 |2300 | | |Multitask network |80 |2300 | | |robust MT-NN |80 |4000 | | |graph convolution |80 |900 | |delaney |MT-NN regression |10 |40 | |kaggle |MT-NN regression |2200 |3200 | ## Contributing to DeepChem Loading
examples/kaggle/kaggle_datasets.py +1 −1 Original line number Diff line number Diff line Loading @@ -31,7 +31,7 @@ def remove_missing_entries(dataset): dataset.set_shard(i, X, y, w, ids) # Set shard size low to avoid memory problems. def load_kaggle(shard_size=10000, num_shards_per_batch=4, def load_kaggle(shard_size=2000, num_shards_per_batch=4, featurizer=None): """Load KAGGLE datasets. Does not do train/test split""" ############################################################## TIMING Loading
