Git and Github
In the first instance, all repositories should be created using the ACCESS-NRI General Repository Template which contains the correct software COPYRIGHT and LICENSE information.
Repositories should always have a README.md providing a description of the repo purpose, content and any useful links / references. The more detailed documentation that can be provided here the better.
Repository names should have unique, all lower-case names with words separated by hyphen/dashes. Ideally repo names should be as semantic as possible. Examples:
repomy-repomy-more-complex-repo
Our GitHub organisation has well over 100 repositories. It is difficult to browse or find repositories, and to identify which is the responsible team:
https://github.com/orgs/ACCESS-NRI/repositories?type=all
As an organisation we are using topic tags on repositories. These are broadly in 3 main categories: subject, purpose and responsible teams.
For examples, all repositories that the Model Release Team are involved with:
All repos related to ocean models
https://github.com/search?q=topic%3Aocean+org%3AACCESS-NRI&type=Repositories
or sea-ice
https://github.com/search?q=topic%3Asea-ice+org%3AACCESS-NRI&type=Repositories
Use -team topic tags to indicate which teams are responsible for repositories. There is not a limit to the number of teams tagged.
This allows for team members and others to find all repositories with which a team is involved. This can help to find repositories and acquaint new team members with the scope of a team's responsibilities.
atmosphere-team
land-team
ocean-team
ice-sheet-team
med-team
model-release-team
software-transformation-team
training-team
hive-docs-team
webops-team
Topic tags help with discovery. Use an existing tag where possible. If appropriate tag is not available create a new one and add to the list below.
Currently used topic tags:
access-esm1p5
access-esm1p6
access-ram3
cable
coupler
unified-model
waves
atmosphere
ocean
biogeochemistry
carbon-cycle
land
sea-ice
working-group
actions
apache2
conda
grafana
deprecated
infrastructure
inputs
library
model
model-configuration
payu
reporting
reproducibility
spack
template
testing
tool
Permanent branches should contain two default branches:
-
main(ormasterfor legacy repositories)- Fully tested code ready for release
-
development- For release-candidate code ready to be tested
To add a new feature to an existing development branch, first create an issue to both describe the new feature and/or problem being solved, and generate a unique issue number for tracking. Feature branches should be as short-lived as possible and reference an issue directly (where possible). Issues should be closed when feature branches are merged or terminated.
A workflow has been developed to enable ACCESS-NRI to be a MOM6 node whilst having flexibility to have releases that differ from the Mom-ocean repository, this workflow may be applicable to other development efforts where there is an interest in contributing to upstream development efforts. Further details of the workflow are here.
All feature branches should be named using the naming convention described above prefixed with a unique issue number.
[issue number]-[short-name-separated-by-hyphens]
Examples:
9-some-issue12-some-other-issue
See here.
For model configuration repositories, where branches in the repository represent different model configurations, use the branch being merged into in the issue branch name.
[issue number]-[target branch name]
Examples:
-
21-dev-025deg_jra55do_ryffor issue 21's changes ofdev-025deg_jra55do_ryfbranch 99-dev-preindustrial+concentrations
All direct Github actions (e.g. repo creation, naming etc) follow the Git style guide above.
- Use the anu.edu.au email address for your commits
- Enable two-factor authentication
- Enable commit signing
Two-factor authentication is an important security measure to protect your account and all ACCESS-NRI Staff must enable it. Outside contributors with write access to ACCESS-NRI Organisation repos are also required to enable 2FA.
There is extensive GitHub documentation covering how to enable 2FA.
Some notes:
- It is a good idea to configure more than one 2FA method in case you lose access
- Both BitWarden and Microsoft Authenticator are TOTP apps that are already in use by many in the organisation
- The GitHub Mobile app is also a convenient 2FA method
- Check that the Pull Request is actually "green-lit" for review before reviewing.
- Checkout the PR branch and inspect entire files as required. e.g. Search for variables that have been removed.
- The task of the reviewer is to review code, not to trigger the merge.
- The creator of the PR shall include details of the testing that was completed.
- The Assignee is responsible for triggering the merge.
- Be clear and concise: Explain the problem or issue in as few words as possible. Be specific and provide all relevant details.
- Short and descriptive title: The title is the most visible part of your issue, you want it short and to the point.
- Provide context: Explain why the problem is important and how it affects you or others. Provide any relevant background information that will help others understand the issue.
- Provide a way to reproduce the issue: Clearly outline the steps needed to reproduce the issue, and provide any relevant logs or error messages. Ideally include a Minimal Complete Verifiable Example (MCVE).
- Include relevant files or code: If the issue is related to a specific file or piece of code, include it in the issue description.
- Provide a desired outcome: Explain what you expect to happen when the issue is resolved.
- Use proper grammar and spelling: Poor grammar and spelling can make it difficult for others to understand your issue, so be sure to proofread your message before submitting it.
- Be respectful: Treat others as you would like to be treated. Be respectful and polite in your interactions with others, even if you are frustrated or upset.
- Use relevant labels: Use relevant labels to help others quickly identify the type of issue you are reporting.
- Provide updates: Keep the issue updated as you continue to work on it, and provide any additional information that may be helpful to others.
By following these guidelines, you can help ensure that your GitHub issue is clear, concise, and easy for others to understand and work on.
Empty issues are an anti-pattern. There should be some descriptive text in the body of the issue. Not only is it good practice, an empty issue sends a message, and it isn't a good one.
Pull requests should always be associated with an issue. The issue is for outlining the problem, discuss approaches and background. The pull request is where the codebase is updated to fix the related issue, including detailed code review.
Signed commits are very important.
Git by itself does not track and validate user action on commits. This means you can easily impersonate someone else and push commits under their name (for more information, refer to Why You Should Sign All Your Git Commits).
-
Generate a new SSH key
If you don't have one already, generate a new SSH key and add your ssh key to the ssh-agent. -
Add your public SSH key to GitHub
If you don't have an SSH already added to GitHub, add the SSH key to your GitHub account. -
Set up signed commits with SSH
To set up signed commits using SSH run the following commands (by substituting the path to your public SSH key):git config --global gpg.format ssh git config --global user.signingkey ~/.ssh/<path-to-your-ssh-key.pub> -
Set up signed commits globally
To set Git to sign all commits globally by default, run:git config --global commit.gpgsign true -
Create an allowed signers file
The allowed signers file is a list of trusted public SSH keys that tell Git who is allowed to sign commits.
To create the file run the following commands (by substituting your email and public SSH key content):Then, to inform Git about this file, use the following command:mkdir -p ~/.config/git cat > ~/.config/git/allowed_signers <<< "<your@email.address> <your-public-ssh-key-content>"git config --global gpg.ssh.allowedSignersFile ~/.config/git/allowed_signers -
Add your public SSH key as a signing key to GitHub
In GitHub's Settings, go to the SSH and GPG keys panel.
Click on "New SSH key" and change the "Key type" to Signing Key.
Copy your public SSH key content in the box and click on "Add SSH key".
You are set! You should now see a Verified green tag next to your commit messages on GitHub.
On Gadi, depending on which ssh-keygen distribution you are using, when trying to issue a signed commit you might get an error similar to the following:
error: ssh-keygen -Y sign is needed for ssh signing (available in openssh version 8.2p1+)
To solve this issue, the recommended option is to set the gpg.ssh.program Git configuration option to an ssh-keygen executable with version >= 8.2p1.
An ssh-keygen executable with version >= 8.2p1 is available, for example, in the vk83 payu environment.
To set the gpg.ssh.program option to the ssh-keygen executable from the vk83 payu environment, run:
git config --global gpg.ssh.program /g/data/vk83/apps/payu/1.1.5/bin/ssh-keygen
After enabling signed commits, you can sign old commits by running:
git rebase --exec 'git commit --amend --no-edit -n -S' -i <GITREF>
with <GITREF> usually being a Git ref to the last signed commit.
For example, to sign all commits within a feature branch (branched off main), run:
git rebase --exec 'git commit --amend --no-edit -n -S' -i main
Instead, to sign the last 3 commits within a branch, run:
git rebase --exec 'git commit --amend --no-edit -n -S' -i HEAD~3
Important
Signing old commits changes the Git history (as all git rebase commands do). Therefore, the new signed commits will have to be force-pushed to origin. Instead of simply using git push --force, we recommend using the safer --force-with-lease:
git push --force-with-lease
It is possible to to run git commands (including signing commits) from Gadi, without having to set new SSH keys for authentication and signing, but using your local SSH keys instead.
To do so, follow the steps below:
Your Gadi Host should look something like:
Host gadi
Hostname gadi.nci.org.au
User <your_username>
IdentityFile <path_to_gadi_ssh_private_key>
ForwardAgent yes
...
This step should only be done once.
Alternatively, for every connection you can run the SSH command using ssh -A ...
- Start an SSH agent by running:
eval "$(ssh-agent -s)" -
Add your Git authentication and signing SSH keys to the agent
To add an SSH key to the agent run:
You will prompted to insert the SSH key passphrase. Repeat the step for each SSH key you wish to add.
ssh-add <path_to_the_ssh_private_key>
This steps should be done for each new local session (for example every time you open a new instance of your terminal app or you reboot your computer).
On Gadi, set the command run by Git when looking for the signing key by running
git config --global gpg.ssh.defaultKeyCommand "ssh-add -L"
Important
ssh-add -L lists all keys added to the ssh-agent (including the forwarded from local ones). This will always work if the agent only has one ssh-key added, but might select a wrong SSH key if multiple ones are present, resulting in your commits not being correctly signed.
To avoid this, the suggestion is to specify a custom command that outputs the SSH key you use for signing.
For example, if by running ssh-add -L your signing key starts with ssh-rsa AAABCDEF... your command could be:
git config --global gpg.ssh.defaultKeyCommand "sh -c 'ssh-add -L | grep \"^ssh-rsa AAABCDEF\"'"