-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Review guidelines
edenlightning edited this page Aug 4, 2020
·
12 revisions
“Reviews allow for discussion of proposed changes and help ensure that the changes meet the repository’s contributing guidelines and other quality standards.“ (from GitHub documentation)
- Make sure that the PR title summarizes the change (not "fixes bug #2168").
- Make sure there's an issue associated with this PR. No issue associated? Look for the relevant issue or ask the author to create one.
- Make sure PR has the right labels (bugfix, docs, tests, enhancements, etc.).
- Make sure PR description includes:
- A detailed summary of what changed
- The motivation for the change
- We value everyone putting time and effort into PyTorch Lightning, but not every change should be included.
- We encourage everyone to open issues describing what they want to do before going off and doing it.
- Every feature we add takes attention and maintenance, but most importantly- might break something else.
- Discuss the tradeoffs in the PR or associated issue.
- Refactors should always be in separate PRs than changes.
- Every PR should change/fix exactly one thing. Otherwise, ask the author to split to smaller PRs.
- It is usually good practice to clone the branch and try it out yourself.
- Code is read way more often than it is written or modified, so readability is always important. You should aim to understand every changed line. If you don't understand it, it's likely someone else won't either. Ask questions and comments for things you don't understand.
- Make sure error scenarios and edge cases are well handled.
- Would this work for single node? multi node?
- CPU? GPU? TPU?
- Anything with spawn needs to be serializable. Spawn is basically like: a car is driving down a single lane- this is the main process. When spawn is called, that car gets copied into 8 other cars (8 gpus). That car then goes into its own lane. The car is trained on that lane, but the original car isn’t... so consider what happens in those scenarios.
- DDP is like the same analogy but instead of copying the car 8 times, we copy it 7 times and the main car then joins those 7 to train like all.
- Make sure this works with 16 bit precision.
- If you don’t think a variable name accurately represents what’s in it, suggest a better one.
- When making suggestions, mention that explicitly.
- When something has to change in the code, explain what should be changed and more importantly, why it should be changed (it is always useful to include links to external documentation/articles that provide more detailed information on the subject).
- Are there sufficient tests to ensure this change doesn’t break in the future?
- Read the tests and make sure they will break if the code does. The code in the tests is just as important as the code in the product.
- No tests? Ask the author to add them or add them yourself.
If you checked all the above and everything is looking good, you may approve the PR in your review or request changes. As a rule of thumb:
- Approve: if you are confident that the issue or feature is fully addressed and the content can be merged as it is now (no extra conditions in comments).
- Comment: if there are any doubts (in your understanding, in the general direction, etc.) but you want to express your opinion about a specific point.
- Request changes: if you are sure that the actual change will break something or has obvious issues.
Remember that if you approve a PR and it reaches the minimum number of required approvals and given that all tests pass, the branch will be automatically merged into master.
- Be nice and constructive, remember your fist PR and what helped you / made you motivated to continue contributing.
- Use suggestions for simple edits when you know what it should be instead of leaving a long description (you can still add a short justification of your proposal) so the author can simply accept it.