add discussion in interpretability section and update molecular design #985
Conversation
|
AppVeyor build 1.0.15 for commit 1cb14cb by @delton137 is now complete. The rendered manuscript from this build is temporarily available for download at: |
|
Thanks for the contributions @delton137. It's unclear how much maintenance activity there actually is to help review and merge this pull request. Recent pull requests struggled to find reviewers with the right domain expertise who could comment. Anyone who wants to help review can please comment here. If we do find reviewers, I recommend splitting these changes into two smaller pull requests, one for drug design and one for interpretability. That will help us discuss domain-specific changes. Keeping our convention of one sentence per line will also help us focus on which parts of the text actually changed. You will also need to exclude the files in the Did you see that |
|
The problem seems to be on my end with docker not working for running Athena. I'll keep an eye on this page. I am happy to contribute more content / copyediting, especially on drug-discovery. It is something I know quite a bit about as I wrote a review on deep learning for molecular design. |
|
Getting this going is in my goals for the first half of 2020. I have one more review I'm trying to get out first. Sorry! |
|
@delton137 I can't commit to it immediately, but I could possibly review your drug discovery edits if no one else is able to. |
|
Hi @delton137 : can you remove the files in the build directory from your PR or enable commits by project maintainers? I'll be happy to go ahead and review this so we can get things moving forward. Thanks! |
|
Hi, I'm not sure how to remove the files that are in /build/ from my PR. Should I create a new PR? Alternatively, can you only accept the relevant files when you accept the commit? |
|
"Allow edits from maintainers." is/was checked. |
|
Interesting - it didn't let me make edits before, but it does now. I'm going to batch the deletions of the /build/files. If you make edits locally, you'll have to pull the branch down again. |
|
OK, thank you for your help and for going forward with the review! |
|
Ok! You should pull from your master before you make the next set of changes. I'll have a few more detailed suggestions. |
|
AppVeyor build 1.0.21 for commit e7f6ca6 by @cgreene is now complete. The rendered manuscript from this build is temporarily available for download at: |
| It explores an estimated 10<sup>60</sup> synthesizable organic molecules with drug-like properties without explicit enumeration [@doi:10.1002/wcms.1104]. | ||
| To test or score structures, algorithms like those discussed earlier are used. | ||
| To test or score structures, physics-based simulation could be used, or machine learning models based on techniques discussed may be used, as they are much more computationally efficient. |
There was a problem hiding this comment.
Can you cite at least one example of each approach + the assertion around computational efficiency? I'm not sure if that part pertains just to prediction or if it also includes prediction.
| Even more interesting is the prospect of performing gradient-based or Bayesian optimization of molecules within this latent space. | ||
| The strategy of constructing simple, continuous features before applying supervised learning techniques is reminiscent of autoencoders trained on high-dimensional EHR data [@tag:BeaulieuJones2016_ehr_encode]. | ||
|
|
||
| In the past few years a large number of techniques for the generative modeling and optimization of molecules with deep learning have been explored, including recursive neural networks, variational autoencoders, generative adversarial networks, and reinforcement learning -- for a review see Elton, et al.[@tag:Elton_molecular_design_review] |
There was a problem hiding this comment.
| In the past few years a large number of techniques for the generative modeling and optimization of molecules with deep learning have been explored, including recursive neural networks, variational autoencoders, generative adversarial networks, and reinforcement learning -- for a review see Elton, et al.[@tag:Elton_molecular_design_review] | |
| In the past few years a large number of techniques for the generative modeling and optimization of molecules with deep learning have been explored, including recursive neural networks, variational autoencoders, generative adversarial networks, and reinforcement learning -- for a review see Elton, et al.[@tag:Elton_molecular_design_review]. |
| Building off the large amount of work that has already gone into text generation,[@arxiv:1308.0850] many generative neural networks for drug design represent chemicals with the simplified molecular-input line-entry system (SMILES), a standard string-based representation with characters that represent atoms, bonds, and rings [@tag:Segler2017_drug_design]. | ||
|
|
||
| The first successful demonstration of a deep learning based approach for molecular optimization occured in 2016 with the development of a SMILES-to-SMILES autoencoder capable of learning a continuous latent feature space for molecules[@tag:Gomezb2016_automatic]. | ||
| In this learned continuous space it is possible to interpolate between molecular structures in a manner that is not possible with discrete |
There was a problem hiding this comment.
Can you make this part one sentence per line so it's easier to see changes?
| Unfortunately, this often leads to overfit, "weird" molecules, which are difficult to synthesize in the lab. | ||
| Current programs have settled on rule-based virtual chemical reactions to generate molecular structures [@doi:10.1021/acs.jmedchem.5b01849]. | ||
| Deep learning models that generate realistic, synthesizable molecules have been proposed as an alternative. | ||
| In contrast to the classical, symbolic approaches, generative models learned from data would not depend on laboriously encoded expert knowledge. |
There was a problem hiding this comment.
Did you mean to get rid of this information? Is it offloaded to the newly cited review?
There was a problem hiding this comment.
Not sure what I was thinking. The first two sentences are both debateable but also summarize general sentiments. I'm restoring the sentences and this will be in my commit later today.
| The initial RNN is then fine-tuned to generate molecules that are likely to be active against a specific target by either continuing training on a small set of positive examples [@tag:Segler2017_drug_design] or adopting reinforcement learning strategies [@tag:Olivecrona2017_drug_design; @arxiv:1611.02796]. | ||
| Both the fine-tuning and reinforcement learning approaches can rediscover known, held-out active molecules. | ||
| The great flexibility of neural networks, and progress in generative models offers many opportunities for deep architectures in *de novo* design (e.g. the adaptation of GANs for molecules). | ||
| These generative models successfully learn the grammar of compound representations, with 94% [@tag:Olivecrona2017_drug_design] or nearly 98% [@tag:Segler2017_drug_design] of generated SMILES corresponding to valid molecular structures. The initial RNN is then fine-tuned to generate molecules that are likely to be active against a specific target by either continuing training on a small set of positive examples [@tag:Segler2017_drug_design] or adopting reinforcement learning strategies [@tag:Olivecrona2017_drug_design; @arxiv:1611.02796]. Both the fine-tuning and reinforcement learning approaches can rediscover known, held-out active molecules. |
There was a problem hiding this comment.
Can you make this one sentence per line so it's easier to review and track changes?
| Methods for adjusting the flexibility of deep learning models include dropout, reduced data projections, and transfer learning (described below). | ||
| One way of understanding such model optimizations is that they incorporate external information to limit model flexibility and thereby improve predictions. | ||
| This balance is formally described as a tradeoff between "bias and variance" | ||
| In particular, many problem-specific optimizations described in this review reflect a recurring universal tradeoff---controlling the flexibility of a model in order to maximize generalizability and prevent overfitting. |
There was a problem hiding this comment.
| In particular, many problem-specific optimizations described in this review reflect a recurring universal tradeoff---controlling the flexibility of a model in order to maximize generalizability and prevent overfitting. | |
| Many problem-specific optimizations described in this review reflect a recurring universal tradeoff---controlling the flexibility of a model in order to maximize generalizability and prevent overfitting. |
| [@url:http://www.deeplearningbook.org/]. | ||
|
|
||
| Although the bias-variance tradeoff is common to all machine learning applications, recent empirical and theoretical observations suggest that deep learning models may have uniquely advantageous generalization properties [@tag:Zhang2017_generalization; @tag:Lin2017_why_dl_works]. | ||
| Nevertheless, additional advances will be needed to establish a coherent theoretical foundation that enables practitioners to better reason about their models from first principles. | ||
| Although the bias-variance tradeoff is is important to take into account in many machine learning tasks, recent empirical and theoretical observations suggest that deep neural networks have uniquely advantageous generalization properties and do not obey the tradeoff as expected [@tag:Belkin2019_PNAS; @tag:Zhang2017_generalization; @tag:Lin2017_why_dl_works]. According to the bias-variance theory, many of the most successful deep neural networks have so many free parameters they should overfit.[@tag:Belkin2019_PNAS] It has been shown that deep neural networks operate in a regime where they can exactly interpolate their training data yet are still able to generalize.[@tag:Belkin2019_PNAS] Thus, poor generalizability can often be remedied by adding more layers and increasing the number of free parameters, in conflict with the classic bias-variance theory. Additional advances will be needed to establish a coherent theoretical foundation that enables practitioners to better reason about their models from first principles. |
There was a problem hiding this comment.
Can you make this one sentence per line? Also citation style is inside the periods.
|
|
||
| As the concept of interpretability is quite broad, many methods described as improving the interpretability of deep learning models take disparate and often complementary approaches. | ||
| Firstly, a model that achieves breakthrough performance may have identified patterns in the data that practitioners in the field would like to understand. |
There was a problem hiding this comment.
One sentence per line here, but I also might note that one of the major hazards of noninterpretable models with high predictive accuracy would be learning patterns in the data that capture the data selection process as opposed to the underlying data generating process (i.e., ascertainment biases, etc).
There was a problem hiding this comment.
I can give more domain-specific comments on the generative molecular models once this is updated to one sentence per line. It's hard to comment on specific papers currently and to see exactly what has changed in the interpretability section.
There is some newer work on generative models for lead optimization we could mention here as a slightly different application. @delton137 were those already covered in your review?
| The great flexibility of neural networks, and progress in generative models offers many opportunities for deep architectures in *de novo* design (e.g. the adaptation of GANs for molecules). | ||
| These generative models successfully learn the grammar of compound representations, with 94% [@tag:Olivecrona2017_drug_design] or nearly 98% [@tag:Segler2017_drug_design] of generated SMILES corresponding to valid molecular structures. The initial RNN is then fine-tuned to generate molecules that are likely to be active against a specific target by either continuing training on a small set of positive examples [@tag:Segler2017_drug_design] or adopting reinforcement learning strategies [@tag:Olivecrona2017_drug_design; @arxiv:1611.02796]. Both the fine-tuning and reinforcement learning approaches can rediscover known, held-out active molecules. | ||
|
|
||
| Reinforcement learning approaches where operations are performed directly on the molecular graph bypass the need to learn the details of SMILES syntax, allowing the model to focus purely on chemistry. Additionally, they seem to require less training data and generate more valid molecules since they are constrained by design only to graph operations which satisfy chemical valiance rules.[@tag:Elton_molecular_design_review] A reinforcement learning agent developed by Zhou et al. demonstrated superior molecular optimization performance on certain easy to compute metrics when compared with other deep learning based approaches such as the Junction Tree VAE, Objective Reinforced Generative Adversarial Network, and Graph Convolutional Policy Network.[@doi:10.1038/s41598-019-47148-x] As another example, Zhavoronkov et al. used generative tensorial reinforcement learning to discover potent inhibitors of discoidin domain receptor 1 (DDR1).[@tag:Zhavoronkov2019_drugs] Their work is unique in that six lead candidates discovered using their approach were synthesized and tested in the lab, with 4/6 achieving some degree of binding to DDR1.[@tag:Zhavoronkov2019_drugs] |
There was a problem hiding this comment.
If we're discussing the DDR1 example, we should also consider the new letter from Walters and Murcko:
Assessing the impact of generative AI on medicinal chemistry
https://doi.org/10.1038/s41587-020-0418-2
In general, should we add a couple more caveats or areas where these generative models still struggle?
There was a problem hiding this comment.
I will have time to respond to your points and make a number of improvements starting on Friday afternoon.
Incidently, there has also back and forth recently about the GENTRL reinforcement learning method, with some skepticism from the phareceutical world. (see the response here and references)
There was a problem hiding this comment.
I read the paper Walters and Murcko and the response paper I linked to, but I'm not sure how much of that needs to go in the review as it gets a bit "in the weeds". The basic critique is that a lot of the molecules being generated are variations on stuff in the training data. They argue for standards such as including the training data used with the paper and a ranking of the closest molecules in the training data, so that novelty can be discerned.
It may be worth discussing the need for metrics / criteria for scoring generative models or deep learning based systems for optimization more generally.. I discussed it in my review article on deep learning for molecular design (section 3.1). If you can't access the Royal Society of Chemistry version, its on the arXiv here. Note the criteria discussed are more for generation (ie ensuring a large, diverse set of "interesting" molecules), rather than optimization for a specific target.
|
@delton137 : did you mean to close this? |
|
Oh - I see this got re-filed as #988 |
Hello,
I have tried to make some changes to the manuscript. Unfortunately, I wasn't able to completely build the new version to pdf.
the build.sh runs until it outputs:
## INFO
Generated manscript stats:
{
"reference_counts": {
"arxiv": 132,
"doi": 425,
"pmcid": 6,
"url": 48,
"total": 611
},
"word_count": 34912
}
Exporting HTML manuscript
Exporting PDF manuscript using Docker + Athena
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
Please let me know if I'm doing something wrong. Perhaps you can accept only the changes I made to the manuscript and not the changes in the /build/ directory. Otherwise, feel free to reject this pull request. You can reach me at daniel.elton@nih.gov.