Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mount data and user directories in SRD #1480

Merged
merged 14 commits into from
May 30, 2023
Merged

Mount data and user directories in SRD #1480

merged 14 commits into from
May 30, 2023

Conversation

jemrobinson
Copy link
Member

@jemrobinson jemrobinson commented May 25, 2023 •
edited
Loading

✅ Checklist

  • You have given your pull request a meaningful title (e.g. Enable foobar integration rather than 515 foobar).
  • You are targeting the python-migration branch.
  • Your branch is up-to-date with the python-migration branch (you probably started your branch from develop but it may have changed since then).
  • If-and-only-if your changes are not yet ready to merge, you have marked this pull request as a draft pull request and added '[WIP]' to the title.
  • If-and-only-if you have changed any Powershell code, you have run the code formatter. You can do this with ./tests/AutoFormat_Powershell.ps1 -TargetPath <path to file or directory>.

⤴️ Summary

This is part of the python migration project.

Add NFS mounts for /data, /output, /home and /shared. /data and /output use NFSv3 to mount blob storage while /home and /shared use NFSv4 to mount fileshares.

🌂 Related issues

Closes #1418 #1419

🔬 Tests

Tested on a development SRE

jemrobinson added 14 commits May 25, 2023 09:31
@jemrobinson
🚚 Move secrets into SHM and SRE keyvaults
6f0919d
@jemrobinson
🔧 Add data provider and research user IP addresses to SRE config
4d88924
@jemrobinson
✨ Add method for requesting all IP addresses in an AzureIPv4Range
937f3c4
@jemrobinson
🐛 Update SRE provisioning to account for new structure of SRE outputs
a9b53f9
@jemrobinson
✨ Add data storage accounts and containers to SHM and SRE
ff6fcf5
@jemrobinson
🚚 Rename state component to data
89394e8
@jemrobinson
🐛 Switch user commands to depend on Pulumi stacks rather than config …
eb68a89 
…file
@jemrobinson
🐛 Ensure that Guacamole database connections are correctly updated
6aef887
@jemrobinson
✨ Add SRE private data endpoints
78f5d09
@jemrobinson
🐛 Allow access to Powershell modules from identity server
fcf3358
@jemrobinson
⬆️ Update pulumi version
1d170ee
@jemrobinson
✨ Add dynamic provider for setting blob container ACLs
aedc912
@jemrobinson
✨ Add NFS mounts to SRD
8ea106a
@jemrobinson
👽 Workaround for Azure ACL implementation.
374114b 
When default ACLs are set, this results in all files being owned by 65533 even when ownership is correct on the storage account.
However, when default ACLs are *not* set, any files created outside the NFS mount have permissions 0640 and cannot be read inside the SRE.
This means that we need:
- ingress container: default ACLs set to allow access from Azure Storage Explorer
- egress container: no default ACLs to allow access from inside the SRE
@jemrobinson jemrobinson force-pushed the 1418-sensitive-data branch from a4c53cb to 374114b Compare May 25, 2023 08:32
@jemrobinson jemrobinson merged commit a34326e into alan-turing-institute:python-migration May 30, 2023
@jemrobinson jemrobinson mentioned this pull request Apr 8, 2024
Merged
5 tasks
@jemrobinson jemrobinson deleted the 1418-sensitive-data branch April 19, 2024 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jemrobinson