PEP458: Bootstrap TUF metadata repository for development #15484
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
993d7b7 to
862c04d
Compare
Contributor
|
Amazing: never seen a PR enable code through pure config 👏🏽 |
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
862c04d to
c73fac9
Compare
Contributor
Author
|
Hi @ewdurbin, |
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
c73fac9 to
fced473
Compare
ewdurbin
approved these changes
Mar 11, 2024
ewdurbin
reviewed
Mar 11, 2024
Member
ewdurbin
left a comment
ewdurbin
left a comment
There was a problem hiding this comment.
Overall this is working as expected with two concerns:
- Seems there is some raciness that is causing the rstuff-worker to fail to boot due to the DB not existing, #15549 should address this.
- Both rstuf-api and rstuf-worker containers do not exit cleanly, and instead are killed after a 10s timeout. Is this expected?
Contributor
Author
Thanks, @ewdurbin. It looks like an great solution.
Indeed, these two containers take about 10s to stop, including in my development environment without a Warehouse. Aborting on container exit...
[+] Stopping 5/5
✔ Container repository-service-tuf-worker-repository-service-tuf-worker-1 Stopped 10.2s
✔ Container repository-service-tuf-worker-repository-service-tuf-api-1 Stopped 10.2s
✔ Container repository-service-tuf-worker-redis-1 Stopped 0.2s
✔ Container repository-service-tuf-worker-postgres-1 Stopped 0.2s
✔ Container repository-service-tuf-worker-localstack-1 Stopped 0.9sI will file an issue on the RSTUF project to investigate if we can make it faster. |
1 task
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
54c9790 to
6226bed
Compare
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
6226bed to
341bab0
Compare
Contributor
Author
|
Rebased |
ewdurbin
reviewed
Mar 25, 2024
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
341bab0 to
5196968
Compare
ewdurbin
reviewed
Mar 25, 2024
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
69ef27c to
de1a4e0
Compare
added 10 commits
April 5, 2024 11:12
- RSTUF API v0.12.0b1 - RSTUF Worker v0.14.0b1 Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com>
Include RSTUF keys to be used on development environment. Include two root keys: root1: ecdsa format root2: ed25519 format Include online key 2f685fa7546f1856b123223ab086b3def14c89d24eef18f49c32508c2f60e241 RSA Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com> Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Include in `docker-compose.yml` the configuration for using local online key for development. - Use local volume from `./dev/rstuf/keys/online` as `/keyvault` - Use `/keyvault` as RSTUF Worker setting `RSTUF_ONLINE_KEY_DIR` Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com>
Include the RSTUF command line interface (`repository-service-tuf`). This tool will be used in the development enviroment to perform tasks on the RSTUF service. This tool will be used primary for RSTUF bootstrap. Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com>
Include the RSTUF `bootstrap.json` payload in the `dev/rstuf/` folder. The bootstrap payload has the RSTUF complete ceremony process. It uses the keys available in the `dev/rstuf/keys` This commit also includes the `inittuf` in Makefile. The information in the development docs. - context (PEP 458) - command (bootstrap) - use (try out API) Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com> Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
It exposes the TUF metadata at `http://files:9001:tuf-metadata` in the container network. For developers at `http://localhost:9001/tuf-metadata`. This commit includes the entry on `docker-compose.yml` and the `rstuf-metadata` volume which is attached to the `files` container. Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com>
Fix formatting for `make server` mention in getting-started doc. Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com>
Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com>
Signed-off-by: Kairo Araujo <kairo.araujo@testifysec.com>
kairoaraujo
force-pushed
the
rstuf-bootstrap
branch
from
de1a4e0 to
d2ee527
Compare
Contributor
Author
|
I rebased the PR. |
Member
|
Any progress on the slow stop? I think what's here is good to go but would really like to see that improved sooner than later. |
Contributor
Author
I filed the issue but haven't worked on it yet. |
Member
|
Added 5f5282d for now, which gets them down by force. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is part 2 in a series of PRs to integrate Repository Service for TUF (RSTUF) with Warehouse for PEP 458 adoption.
Previous PR was #15241
This PR adds scripts, configuration and static metadata to bootstrap a PEP 458 - compliant TUF metadata repository for development.
This bootstrapping process also initializes the RSTUF API to accept API requests, and trigger the related TUF metadata updates.
Change details
Add RSTUF dev keys for offline and online signing
These keys are for demo purposes only and must not (!) be used in production. In production we recommend hardware tokens for offline signing and Cloud KMS for online signing.
Configure RSTUF Worker to mount online signing key file 1
Add static RSTUF development bootstrap payload
This payload contains bootstrapping configuration and the development root metadata. The latter defines two root keys and one online key (see above), and is pre-signed.
Add Makefile target to send the payload to the API, to trigger bootstrapping.
Add dev instructions on how to run init and how to interact with the API.
Next PRs will include:
Thanks, @lukpueh, for helping with many parts of this PR.
Footnotes
For the interested reader only: The RSTUF worker will locate the online signing key file by combining ambient configuration from the container environment (see "RSTUF_ONLINE_KEY_DIR") with the exact identifier included in the root metadata (see "x-rstuf-online-key-uri"). The same approach works, when using Cloud KMS, where e.g. access credentials are configured via environment variables, and the particular keyid with the public key in the delegating metadata. ↩