From 950dc60b3177e121548cd3c4e7d7c482f82845b8 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 20 Dec 2018 11:17:20 -0800 Subject: [PATCH 01/40] added TAP 12 draft --- tap12.md | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 tap12.md diff --git a/tap12.md b/tap12.md new file mode 100644 index 00000000..9cff8202 --- /dev/null +++ b/tap12.md @@ -0,0 +1,54 @@ +* TAP: 12 +* Title: Managing TUF Versions +* Version: 1 +* Last-Modified: 19-December-2018 +* Author: Marina Moore, Justin Cappos +* Status: Draft +* Content-Type: text/markdown +* Created: 19-December-2018 + +# Abstract + +This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. This TAP will define a procedure for TUF clients to ensure that they are using a compatible version of the TUF spec before performing updates. + +# Motivation + +Various TAPs, including TAPs 3 and 8 include changes that will make clients using the old version of the spec incompatible with servers using the new version. This TAP defines a procedure to ensure that clients are not missing important features to ensure the security of updates. + +# Rationale + +Breaking changes should only occur during a major release of the TUF spec (1.x.x to 2.x.x). The client should check the major version when the root metadata is downloaded, and if a new version is found update the client before performing any software updates. + +Additionally, this TAP clarifies how TUF version numbers should be determined. For clarity, semantic versioning is used to determine version numbers that can be easily be understood. + +# Specification + +The root metadata already contains the TUF spec-version. The client shall compare the spec-version in the root metadata (server spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described below. + +## Procedure + +If the server spec-version is lower than the client spec-version, the client shall terminate the update and report an error. + +If the major version (the first digit) of the spec-version has been incremented, the client must update before proceeding. This could be an automatic process or an error could be reported, requesting a manual client update. + +If a minor version or patch number of the spec-version has been incremented, the client should report this and may update, but can chose to proceed without further action. + +## Version Number format + +TUF version number shall be determined based on [semantic versioning](https://semver.org/). This format specifies versions in the format MAJOR.MINOR.PATCH. In this format, only major changes are non-backwards compatible. + +# Security Analysis + +There should be minimal security impact. Ensuring that the client is up to date should improve security in the event that a security vulnerability is patched in a release of the spec. + +# Backwards Compatibility + +This TAP is backwards compatible, and should be implemented before any non-backwards compatible TAPs are released. + +# Augmented Reference Implementation + +TODO + +# Copyright + +This document has been placed in the public domain. From cc9f3188787f431bac519d08c96a5b9018a44787 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 7 Jan 2019 11:45:17 -0800 Subject: [PATCH 02/40] clarified client update --- tap12.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tap12.md b/tap12.md index 9cff8202..2541e0bc 100644 --- a/tap12.md +++ b/tap12.md @@ -29,7 +29,7 @@ The root metadata already contains the TUF spec-version. The client shall compar If the server spec-version is lower than the client spec-version, the client shall terminate the update and report an error. -If the major version (the first digit) of the spec-version has been incremented, the client must update before proceeding. This could be an automatic process or an error could be reported, requesting a manual client update. +If the major version (the first digit) of the spec-version has been incremented, the client must update itself to a client supporting the same major version before proceeding. This could be an automatic process or an error could be reported, requesting a manual client update. If a minor version or patch number of the spec-version has been incremented, the client should report this and may update, but can chose to proceed without further action. From 1622a14fcca00d48f901bb2401ed28d1221c7af1 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Tue, 19 Mar 2019 15:10:37 -0400 Subject: [PATCH 03/40] removed tap number for draft tap --- tap12.md => candidate-tuf-versions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename tap12.md => candidate-tuf-versions.md (99%) diff --git a/tap12.md b/candidate-tuf-versions.md similarity index 99% rename from tap12.md rename to candidate-tuf-versions.md index 2541e0bc..86b4b26d 100644 --- a/tap12.md +++ b/candidate-tuf-versions.md @@ -1,4 +1,4 @@ -* TAP: 12 +* TAP: * Title: Managing TUF Versions * Version: 1 * Last-Modified: 19-December-2018 From b3485460d6b71b943d98769ee844b1027ed8f94a Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 2 May 2019 13:59:59 -0400 Subject: [PATCH 04/40] added clarifications about location of version numbers and semantic versioning --- candidate-tuf-versions.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 86b4b26d..3bacdcdd 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,4 +1,4 @@ -* TAP: +* TAP: * Title: Managing TUF Versions * Version: 1 * Last-Modified: 19-December-2018 @@ -9,21 +9,21 @@ # Abstract -This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. This TAP will define a procedure for TUF clients to ensure that they are using a compatible version of the TUF spec before performing updates. +This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. Breaking changes mean that a client and server must implement the changes at the same time in order to continue functioning as expected. This TAP will define a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the metadata they download. # Motivation -Various TAPs, including TAPs 3 and 8 include changes that will make clients using the old version of the spec incompatible with servers using the new version. This TAP defines a procedure to ensure that clients are not missing important features to ensure the security of updates. +Various TAPs, including TAPs 3 and 8 include changes that will make clients using the old version of the spec incompatible with servers using the new version. This TAP defines a procedure to ensure that clients are not missing important features to ensure the security of updates. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. # Rationale -Breaking changes should only occur during a major release of the TUF spec (1.x.x to 2.x.x). The client should check the major version when the root metadata is downloaded, and if a new version is found update the client before performing any software updates. +This TAP clarifies that spec versions should be based on [semantic versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. -Additionally, this TAP clarifies how TUF version numbers should be determined. For clarity, semantic versioning is used to determine version numbers that can be easily be understood. +Breaking changes should only occur during a major release of the TUF spec (1.x.x to 2.x.x). The client should check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. # Specification -The root metadata already contains the TUF spec-version. The client shall compare the spec-version in the root metadata (server spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described below. +The root metadata already contains the TUF spec-version. After downloading the root metadata, the client shall compare the spec-version in the root metadata (server spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described below. ## Procedure @@ -35,7 +35,7 @@ If a minor version or patch number of the spec-version has been incremented, the ## Version Number format -TUF version number shall be determined based on [semantic versioning](https://semver.org/). This format specifies versions in the format MAJOR.MINOR.PATCH. In this format, only major changes are non-backwards compatible. +TUF version numbers shall be determined based on [semantic versioning](https://semver.org/). This specification describes version numbers in the format MAJOR.MINOR.PATCH. In semantic versioning, only major changes are non-backwards compatible. # Security Analysis From a80ab21b95cb1db41798c71f4b140cedbda231e5 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Wed, 22 May 2019 13:13:40 -0400 Subject: [PATCH 05/40] add intermediate root metadata for root metadata format changes --- candidate-tuf-versions.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 3bacdcdd..092f7b8b 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -25,6 +25,8 @@ Breaking changes should only occur during a major release of the TUF spec (1.x.x The root metadata already contains the TUF spec-version. After downloading the root metadata, the client shall compare the spec-version in the root metadata (server spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described below. +To allow for changes to the format of root metadata, an intermediate root metadata file will be used. This intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client updates to the new spec-version, they can download the root metadata file that follows the intermediate one, and continue with the update. This process will only be used for major spec-version updates. + ## Procedure If the server spec-version is lower than the client spec-version, the client shall terminate the update and report an error. From 03f7e59fc2775794cbe1abb08627bc64290aa400 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Wed, 5 Jun 2019 17:00:01 -0400 Subject: [PATCH 06/40] added details and clarifications --- candidate-tuf-versions.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 092f7b8b..2b770377 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,7 +1,7 @@ * TAP: * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 19-December-2018 +* Last-Modified: 5-June-2018 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown @@ -9,31 +9,31 @@ # Abstract -This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. Breaking changes mean that a client and server must implement the changes at the same time in order to continue functioning as expected. This TAP will define a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the metadata they download. +This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. To facilitate finding breaking changes, this TAP will define a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the version of the metadata they download. # Motivation -Various TAPs, including TAPs 3 and 8 include changes that will make clients using the old version of the spec incompatible with servers using the new version. This TAP defines a procedure to ensure that clients are not missing important features to ensure the security of updates. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. +Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the spec will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. This TAP defines a procedure to ensure that clients are not missing important features that ensure the security of updates. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. # Rationale -This TAP clarifies that spec versions should be based on [semantic versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. +This TAP clarifies that spec versions should be based on [semantic versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In addition, the consistent format makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. -Breaking changes should only occur during a major release of the TUF spec (1.x.x to 2.x.x). The client should check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. +Breaking changes should only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The client should check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. + +The Backwards Compatibility section of a TAP can be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, for example in TAP 3 and TAP 8, then it will be part of a new major version. These changes add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it will be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. # Specification -The root metadata already contains the TUF spec-version. After downloading the root metadata, the client shall compare the spec-version in the root metadata (server spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described below. +The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in {Procedure}. -To allow for changes to the format of root metadata, an intermediate root metadata file will be used. This intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client updates to the new spec-version, they can download the root metadata file that follows the intermediate one, and continue with the update. This process will only be used for major spec-version updates. +To allow for changes to the format of root metadata during a major update, an intermediate root metadata file will be used. This intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client updates to the new spec-version, they can download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. ## Procedure -If the server spec-version is lower than the client spec-version, the client shall terminate the update and report an error. - -If the major version (the first digit) of the spec-version has been incremented, the client must update itself to a client supporting the same major version before proceeding. This could be an automatic process or an error could be reported, requesting a manual client update. +If the major versions on the client and repository do not match, the update cannot proceed until this discrepancy is resolved. If the repository spec-version is lower than the client spec-version, the client should terminate and report the mismatch to the user. The client's user may then choose to report this issue to the repository or take no action. The client will not be able to perform an update until the repository is updated or the client chooses to downgrade their TUF client. If the client spec-version is lower than the repository spec-version, the client may choose to automatically update to a new TUF client. If this option is not available, the client should report the mismatch to the user so that the user may manually update the client. Once the updated TUF client is installed, the client may proceed with the update. -If a minor version or patch number of the spec-version has been incremented, the client should report this and may update, but can chose to proceed without further action. +If a minor version or patch of the spec-version does not match, the client should report the mismatch to the user. However, the update can proceed without further action. A client may choose to update before proceeding with the update or simply log the error. Additionally, a client may act differently for minor version changes and patches. For example, a client may choose to require manual confirmation before proceeding with a mismatched minor version, but automatically continue with a mismatched patch. In general, it is best to update to the most recent TUF client as soon as possible. ## Version Number format @@ -45,7 +45,7 @@ There should be minimal security impact. Ensuring that the client is up to date # Backwards Compatibility -This TAP is backwards compatible, and should be implemented before any non-backwards compatible TAPs are released. +This TAP is backwards compatible, and should be implemented on all clients before any non-backwards compatible TAPs are released. # Augmented Reference Implementation From 8c4c1a39037bb81b96f64c0cc797d0d597cc2cc6 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 6 Jun 2019 16:51:53 -0400 Subject: [PATCH 07/40] minor edits --- candidate-tuf-versions.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 2b770377..f2d811d1 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -13,7 +13,7 @@ This TAP clarifies how to manage updates to the TUF spec that include non-backwa # Motivation -Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the spec will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. This TAP defines a procedure to ensure that clients are not missing important features that ensure the security of updates. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. +Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the specification will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. This TAP defines a procedure to ensure that clients are not missing important features that ensure the security of updates. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. # Rationale @@ -21,13 +21,13 @@ This TAP clarifies that spec versions should be based on [semantic versioning](h Breaking changes should only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The client should check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. -The Backwards Compatibility section of a TAP can be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, for example in TAP 3 and TAP 8, then it will be part of a new major version. These changes add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it will be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. +The Backwards Compatibility section of a TAP can be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it will be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. # Specification -The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in {Procedure}. +The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure]{#procedure}. -To allow for changes to the format of root metadata during a major update, an intermediate root metadata file will be used. This intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client updates to the new spec-version, they can download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. +To allow for changes to the format of root metadata during a major update, an intermediate root metadata file will be used. This intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client updates to the new spec-version, they should download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. ## Procedure From 6de5c5dbccb75165b0a468928befccc5cb5e490c Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 6 Jun 2019 16:53:23 -0400 Subject: [PATCH 08/40] fix link --- candidate-tuf-versions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index f2d811d1..5c139dc3 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -25,7 +25,7 @@ The Backwards Compatibility section of a TAP can be used to determine whether th # Specification -The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure]{#procedure}. +The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure](#procedure). To allow for changes to the format of root metadata during a major update, an intermediate root metadata file will be used. This intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client updates to the new spec-version, they should download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. From 0b71af60eec3c32760c48edaca019bb9ab148075 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 10 Jun 2019 14:52:29 -0400 Subject: [PATCH 09/40] Added detail and reorganized sections --- candidate-tuf-versions.md | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 5c139dc3..754b0750 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,7 +1,7 @@ * TAP: * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 5-June-2018 +* Last-Modified: 10-June-2018 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown @@ -9,36 +9,49 @@ # Abstract -This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. To facilitate finding breaking changes, this TAP will define a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the version of the metadata they download. +This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the version of the metadata they download. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. # Motivation -Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the specification will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. This TAP defines a procedure to ensure that clients are not missing important features that ensure the security of updates. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. +Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the specification will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. Because of this, clients need to check that the version of the specification that they are using is compatible with the version that a repository is using before validating metadata. # Rationale -This TAP clarifies that spec versions should be based on [semantic versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In addition, the consistent format makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. +In order to allow for breaking changes, this TAP clarifies how spec-versions should be formatted, defines a procedure for comparing versions, and describes how root metadata can be updated. -Breaking changes should only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The client should check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. +This TAP clarifies that the spec-versions field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. -The Backwards Compatibility section of a TAP can be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it will be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. +The consistent format of versions makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. To do this comparison, the client will check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. This ensures that the client and metadata are using the same version number before validation. + +As the client checks the metadata spec-version in the root metadata, the root metadata format will not be altered before this check takes place. In order to allow for updates to the root metadata format, an intermediate root metadata file will be created when the root metadata is updated. Root metadata updates will only occur as part of a major version. The intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client downloads the intermediate root metadata and updates to the new spec-version, they will download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. # Specification -The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure](#procedure). +The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec-version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec-version in root metadata. -To allow for changes to the format of root metadata during a major update, an intermediate root metadata file will be used. This intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client updates to the new spec-version, they should download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. +The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure](#procedure). ## Procedure -If the major versions on the client and repository do not match, the update cannot proceed until this discrepancy is resolved. If the repository spec-version is lower than the client spec-version, the client should terminate and report the mismatch to the user. The client's user may then choose to report this issue to the repository or take no action. The client will not be able to perform an update until the repository is updated or the client chooses to downgrade their TUF client. If the client spec-version is lower than the repository spec-version, the client may choose to automatically update to a new TUF client. If this option is not available, the client should report the mismatch to the user so that the user may manually update the client. Once the updated TUF client is installed, the client may proceed with the update. +When the client compares their spec-version to the spec-version found in the root metadata, there are a few possible cases. If the versions are the same, nothing needs to be done. If the major versions differ, the update halts and cannot be completed until the major versions match. If the major versions match but the minor or patch versions differ, the client may choose to continue the update or to update to a new client. These cases are discussed in detail below. + +If the major versions on the client and repository do not match, the update cannot proceed until this discrepancy is resolved. If the repository spec-version is lower than the client spec-version, the client should terminate and report the mismatch to the user. The client's user may then choose to report this issue to the repository or take no action. The client will not be able to perform an update until the repository is updated or the client chooses to downgrade their TUF client. If the client spec-version is lower than the repository spec-version, the client should try to automatically update to a new TUF client. If an updated client is not available, the client shall report the mismatch to the user so that the user may manually update the client. Once the updated TUF client is installed, the client should proceed with the update. -If a minor version or patch of the spec-version does not match, the client should report the mismatch to the user. However, the update can proceed without further action. A client may choose to update before proceeding with the update or simply log the error. Additionally, a client may act differently for minor version changes and patches. For example, a client may choose to require manual confirmation before proceeding with a mismatched minor version, but automatically continue with a mismatched patch. In general, it is best to update to the most recent TUF client as soon as possible. +If a minor version or patch of the spec-version does not match, the client should report the mismatch to the user. However, the update can proceed without further action. A client may choose to update before proceeding with the update or simply log the error. Additionally, a client may act differently for minor version changes and patches. For example, a client may choose to require manual confirmation before proceeding with a mismatched minor version, but automatically continue with a mismatched patch. + +## Changes to TAPs +TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. ## Version Number format TUF version numbers shall be determined based on [semantic versioning](https://semver.org/). This specification describes version numbers in the format MAJOR.MINOR.PATCH. In semantic versioning, only major changes are non-backwards compatible. +## Multiple Repositories +If a client downloads metadata from multiple repositories as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md), the client should check that the major version on all repositories and the client is the same. If any of the major versions do not match, the mismatch should be handled as discussed in [Procedure](#procedure). The client should not maintain multiple spec-versions for different repositories as this prevents the metadata from different repositories from being compared. + +## Updating Trusted Root Metadata +After a major version change, the client must update their trusted root metadata to the root metadata that complies with the new spec-version. To do so, the client first downloads and verifies the intermediate root metadata file, then downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. + # Security Analysis There should be minimal security impact. Ensuring that the client is up to date should improve security in the event that a security vulnerability is patched in a release of the spec. From 2c3fcd933b4ee4e6d5c06f0ab2e46eae5d0a6711 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 4 Jul 2019 13:08:30 -0400 Subject: [PATCH 10/40] clarifications and link to implementation --- candidate-tuf-versions.md | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 754b0750..fce98b26 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,7 +1,7 @@ * TAP: * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 10-June-2018 +* Last-Modified: 04-July-2018 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown @@ -9,29 +9,35 @@ # Abstract -This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the version of the metadata they download. This TAP will need to be implemented by clients before they update to a version of TUF that is not backwards compatible. +This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. This TAP allows clients to update from their current version of the TUF specification to a version that is not backwards compatible. # Motivation -Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the specification will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. Because of this, clients need to check that the version of the specification that they are using is compatible with the version that a repository is using before validating metadata. +The goal of this TAP is to prevent compatibility issues while allowing breaking changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the specification will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. Because of this, clients need to check that the version of the specification that they are using is compatible with the version that a repository is using before validating metadata. # Rationale -In order to allow for breaking changes, this TAP clarifies how spec-versions should be formatted, defines a procedure for comparing versions, and describes how root metadata can be updated. +In order to allow for breaking changes, a client must compare its spec-version to the spec-version of the repository then report and error or update the client if these versions are not compatible. To allow for this functionality, this TAP clarifies how spec-versions should be formatted, defines a procedure for comparing versions, and describes how root metadata can be updated. This TAP clarifies that the spec-versions field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. The consistent format of versions makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. To do this comparison, the client will check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. This ensures that the client and metadata are using the same version number before validation. -As the client checks the metadata spec-version in the root metadata, the root metadata format will not be altered before this check takes place. In order to allow for updates to the root metadata format, an intermediate root metadata file will be created when the root metadata is updated. Root metadata updates will only occur as part of a major version. The intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client downloads the intermediate root metadata and updates to the new spec-version, they will download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. +As the client checks the metadata spec-version in the root metadata, the root metadata format will not be altered before this check takes place. In order to allow for updates to the root metadata format, an intermediate root metadata file will be created when the root metadata is updated. Root metadata updates will only occur as part of a major version. The intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client downloads the intermediate root metadata and updates to the new spec-version, they will download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. The client already downloads root metadata until the most recent is found, so the intermediate root metadata will not be used to validate updates. # Specification -The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec-version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec-version in root metadata. +This TAP requires some setup on the client to ensure that the spec-versions are compatible before an update is performed. In addition, the repository needs to perform some configuration when the spec-version is updated. These processes are described in this section. -The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure](#procedure). +## Client Setup -## Procedure +The following steps must be implemented in the setup of a TUF client: + +* The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec-version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec-version in root metadata. + +* The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure](#procedure). + +### Procedure When the client compares their spec-version to the spec-version found in the root metadata, there are a few possible cases. If the versions are the same, nothing needs to be done. If the major versions differ, the update halts and cannot be completed until the major versions match. If the major versions match but the minor or patch versions differ, the client may choose to continue the update or to update to a new client. These cases are discussed in detail below. @@ -39,6 +45,10 @@ If the major versions on the client and repository do not match, the update cann If a minor version or patch of the spec-version does not match, the client should report the mismatch to the user. However, the update can proceed without further action. A client may choose to update before proceeding with the update or simply log the error. Additionally, a client may act differently for minor version changes and patches. For example, a client may choose to require manual confirmation before proceeding with a mismatched minor version, but automatically continue with a mismatched patch. +## How a Repository updates to a new spec-version + +If there are no changes to the root metadata format or metadata encoding, the repository simply creates and signs a new root metadata file that includes the new spec-version. If the root metadata format changes in any way, the repository must create two root metadata files. The first is formatted using the old specification, but includes the new spec-version. The second Is formatted using the new specification and the new spec-version. All other metadata files and images are handled as described in the specification. + ## Changes to TAPs TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. @@ -62,7 +72,7 @@ This TAP is backwards compatible, and should be implemented on all clients befor # Augmented Reference Implementation -TODO +This TAP is included in the reference implementation [here] (https://github.com/theupdateframework/tuf/pull/854). # Copyright From 1375ce5bb98a47ee0d5b66da81cc763e28a9e611 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Fri, 5 Jul 2019 15:27:45 -0400 Subject: [PATCH 11/40] added clarifications and example about how spec-version updates are handled by a repository --- candidate-tuf-versions.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index fce98b26..1f94bcd2 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -47,7 +47,21 @@ If a minor version or patch of the spec-version does not match, the client shoul ## How a Repository updates to a new spec-version -If there are no changes to the root metadata format or metadata encoding, the repository simply creates and signs a new root metadata file that includes the new spec-version. If the root metadata format changes in any way, the repository must create two root metadata files. The first is formatted using the old specification, but includes the new spec-version. The second Is formatted using the new specification and the new spec-version. All other metadata files and images are handled as described in the specification. +The repository handles updating to a new spec-version in one of two ways, depending on whether there are any changes to the format of root metadata. + +For minor or fix version updates, or for any major version updates that do not affect root metadata, the repository simply creates and signs a new root metadata file that includes the new spec-version. To do this, the repository manager would create a new root metadata file and fill out all of the information, including the new spec-version. This file would then be signed by the root role and uploaded to the repository. Clients performing and update will download this root file and update to a new version as described in [Procedure](#procedure) before performing an update. + +For major changes in which the root metadata format changes in any way, the repository must create two root metadata files. The first is formatted using the old specification, but includes the new spec-version. The second is formatted using the new specification and the new spec-version. More specifically if a repository is updating from version 2.5.1 to version 3.0.0 and the current root file is named 10.root.json (using consistent snapshots), the following steps must be performed: + +* Create a new root metadata file, 11.root.json, that includes all required fields and formatting for version 2.5.1, except that the spec-version field lists version 3.0.0. This file will not be used to perform updates. +* Sign 11.root.json with the root key. +* Create a new root metadata file, 12.root.json, that includes all required fields and formatting for version 3.0.0, including a spec-version of 3.0.0. +* Sign 12.root.json with the root key. +* Upload the signed 11.root.json and 12.root.json to the repository at the same time. + +After these steps are performed, a client performing an update will download 11.root.json and update to spec-version 3.0 as described in [Procedure](#procedure). The client will then download 12.root.json, and seeing that this is the last available root file, the client will proceed with the update. + +For both of these cases, all other metadata files and images are handled as described in the specification. ## Changes to TAPs TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. From 156897b0c95dcb3eb308bccdbd6f943a8bc910d3 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Fri, 5 Jul 2019 16:41:25 -0400 Subject: [PATCH 12/40] minor edits --- candidate-tuf-versions.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 1f94bcd2..127ad90e 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,7 +1,7 @@ * TAP: * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 04-July-2018 +* Last-Modified: 05-July-2018 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown @@ -17,13 +17,13 @@ The goal of this TAP is to prevent compatibility issues while allowing breaking # Rationale -In order to allow for breaking changes, a client must compare its spec-version to the spec-version of the repository then report and error or update the client if these versions are not compatible. To allow for this functionality, this TAP clarifies how spec-versions should be formatted, defines a procedure for comparing versions, and describes how root metadata can be updated. +In order to allow for breaking changes, a client must compare its spec-version to the spec-version of the repository then report an error or update the client if these versions are not compatible. To allow for this functionality, this TAP clarifies how spec-versions should be formatted, defines a procedure for comparing versions, and describes how root metadata can be updated. This TAP clarifies that the spec-versions field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. The consistent format of versions makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. To do this comparison, the client will check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. This ensures that the client and metadata are using the same version number before validation. -As the client checks the metadata spec-version in the root metadata, the root metadata format will not be altered before this check takes place. In order to allow for updates to the root metadata format, an intermediate root metadata file will be created when the root metadata is updated. Root metadata updates will only occur as part of a major version. The intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client downloads the intermediate root metadata and updates to the new spec-version, they will download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. The client already downloads root metadata until the most recent is found, so the intermediate root metadata will not be used to validate updates. +As the client checks the metadata spec-version in the root metadata, the root metadata format will not be altered before this check takes place. In order to allow for updates to the root metadata format, an intermediate root metadata file will be created when the root metadata is updated. Root metadata updates will only occur as part of a major version. The intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client downloads the intermediate root metadata and updates to the new spec-version, they will download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. The client already downloads root metadata until the most recent is found, so the intermediate root metadata will not be used to validate updates. This process is explained in more detail in [How a Repository updates to a new spec-version](#how-a-repository-updates-to-a-new-spec-version). # Specification @@ -49,9 +49,9 @@ If a minor version or patch of the spec-version does not match, the client shoul The repository handles updating to a new spec-version in one of two ways, depending on whether there are any changes to the format of root metadata. -For minor or fix version updates, or for any major version updates that do not affect root metadata, the repository simply creates and signs a new root metadata file that includes the new spec-version. To do this, the repository manager would create a new root metadata file and fill out all of the information, including the new spec-version. This file would then be signed by the root role and uploaded to the repository. Clients performing and update will download this root file and update to a new version as described in [Procedure](#procedure) before performing an update. +For minor or fix version updates, or for any major version updates that do not affect root metadata, the repository simply creates and signs a new root metadata file that includes the new spec-version. To do this, the repository manager would create a new root metadata file and fill out all of the information, including the new spec-version. This file would then be signed by the root role and uploaded to the repository. Clients performing an update will download this root file and update to a new spec-version as described in [Procedure](#procedure) before performing an update. -For major changes in which the root metadata format changes in any way, the repository must create two root metadata files. The first is formatted using the old specification, but includes the new spec-version. The second is formatted using the new specification and the new spec-version. More specifically if a repository is updating from version 2.5.1 to version 3.0.0 and the current root file is named 10.root.json (using consistent snapshots), the following steps must be performed: +For major changes in which the root metadata format changes in any way, the repository must create two root metadata files. The first is formatted using the old specification, but includes the new spec-version. The second is formatted using the new specification and includes the new spec-version. More specifically if a repository is updating from spec-version 2.5.1 to spec-version 3.0.0 and the current root file is named 10.root.json (using consistent snapshots), the following steps must be performed: * Create a new root metadata file, 11.root.json, that includes all required fields and formatting for version 2.5.1, except that the spec-version field lists version 3.0.0. This file will not be used to perform updates. * Sign 11.root.json with the root key. @@ -61,7 +61,7 @@ For major changes in which the root metadata format changes in any way, the repo After these steps are performed, a client performing an update will download 11.root.json and update to spec-version 3.0 as described in [Procedure](#procedure). The client will then download 12.root.json, and seeing that this is the last available root file, the client will proceed with the update. -For both of these cases, all other metadata files and images are handled as described in the specification. +For both of these cases, all other metadata files and images are handled as described in the TUF specification for the version of the specification listed in root metadata. ## Changes to TAPs TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. From 0c178cfea1994fda161d38b094661fa2b019804a Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 15 Jul 2019 17:09:17 -0400 Subject: [PATCH 13/40] Changes to address special cases. Specifically: allow for delegated targets using an old spec version add a next_spec_version field to root require multiple repositories used by a client (as in tap 4) to use the same spec version. some minor grammar and organizational edits --- candidate-tuf-versions.md | 101 ++++++++++++++++++++++++++++---------- 1 file changed, 75 insertions(+), 26 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 127ad90e..a6601226 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,7 +1,7 @@ * TAP: * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 05-July-2018 +* Last-Modified: 13-July-2018 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown @@ -17,51 +17,94 @@ The goal of this TAP is to prevent compatibility issues while allowing breaking # Rationale -In order to allow for breaking changes, a client must compare its spec-version to the spec-version of the repository then report an error or update the client if these versions are not compatible. To allow for this functionality, this TAP clarifies how spec-versions should be formatted, defines a procedure for comparing versions, and describes how root metadata can be updated. +In order to allow for breaking changes, a client must compare its spec version to the spec version of the repository then report an error or update the client if these versions are not compatible. To allow for this functionality, this TAP clarifies how spec versions should be formatted, defines a procedure for comparing versions, and describes some special cases. -This TAP clarifies that the spec-versions field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. +This TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. -The consistent format of versions makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. To do this comparison, the client will check the major version in the root metadata when the root metadata is downloaded. If a new major version is found the client must update to the new spec-version before performing any software updates. This ensures that the client and metadata are using the same version number before validation. +The consistent format of versions makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. To do this comparison, an additional field will be added to root metadata that lists the 'next_spec_version'. This field will list the spec version to be used for all future root metadata. The client will check this field, and if it is set the client will finish the current update cycle, then update to the new spec version when a next root file is available. So if 10.root.json specifies a spec_version of 5.6.1 and a next_spec_version of 6.0.0, the client will look for 11.root.json, and if it is available update to version 6.0.0 before downloading and verifying 11.root.json. This process ensures that the client and metadata are using the same version number before validation. If 11.root.json is invalid or deleted by an attacker, the client shall use the metadata in 10.root.json as the trusted root metadata until the repository corrects 11.root.json. + +In addition, there are are few special cases that must be considered. These include TUF setups with multiple repositories as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) and delegated targets that are unable to update simultaneously with the top level metadata. + +If there are multiple repositories in use the repositories should coordinate their major spec version updates to ensure clients are able to perform updates. The repository managers already must coordinate to ensure that the same targets metadata is signed by all relevant repositories, so the same communication channels can be used to ensure that the spec version is updated at the same time on all repositories. + +A client may choose to support delegated targets that comply with an old spec_version. This may be desirable if the targets are managed by many different individuals or organizations to allow the spec_version update to happen more quickly. To support this behavior, an additional field will be added to the delegation metadata. This field will include the spec_version of the delegation and will be used by the client to determine which client version to use when parsing the delegation metadata. To do this, the client should maintain any old client versions that will be supported for delegated targets. -As the client checks the metadata spec-version in the root metadata, the root metadata format will not be altered before this check takes place. In order to allow for updates to the root metadata format, an intermediate root metadata file will be created when the root metadata is updated. Root metadata updates will only occur as part of a major version. The intermediate root metadata file will contain the new spec-version, but will be formatted according to the old specification. After a client downloads the intermediate root metadata and updates to the new spec-version, they will download the root metadata file that follows the intermediate one, and continue with the update. If a client is multiple versions behind a repository, they will be able to access the root metadata files for each version during the root metadata download process. This will occur while the client is validating each of the root metadata files in the chain. The client already downloads root metadata until the most recent is found, so the intermediate root metadata will not be used to validate updates. This process is explained in more detail in [How a Repository updates to a new spec-version](#how-a-repository-updates-to-a-new-spec-version). # Specification -This TAP requires some setup on the client to ensure that the spec-versions are compatible before an update is performed. In addition, the repository needs to perform some configuration when the spec-version is updated. These processes are described in this section. +This TAP requires an additional field in root metadata and delegations, as well as some setup on the client to ensure that the spec versions are compatible before an update is performed. In addition, the repository needs to perform some configuration when the spec version is updated. These processes are described in this section. + +## The new file format of root metadata + +A 'next_spec_version' field is added to root metadata. With this field, the "signed" portion of root metadata is: + + { "_type" : "root", + "spec_version" : SPEC_VERSION, + "next_spec_version" : NEXT_SPEC_VERSION, + "consistent_snapshot": CONSISTENT_SNAPSHOT, + "version" : VERSION, + "expires" : EXPIRES, + "keys" : { + KEYID : KEY + , ... }, + "roles" : { + ROLE : { + "keyids" : [ KEYID, ... ] , + "threshold" : THRESHOLD } + , ... } + } + + NEXT_SPEC_VERSION is the version number of the specification for the next root metadata file. + +## The new format of delegations + +A 'spec_version' field is added to delegations. With this field, the format of the delegations object is: + + { "keys" : { + KEYID : KEY, + ... }, + "roles" : [{ + "name": ROLENAME, + "spec_version": DELEGATION_SPEC_VERSION + "keyids" : [ KEYID, ... ] , + "threshold" : THRESHOLD, + ("path_hash_prefixes" : [ HEX_DIGEST, ... ] | + "paths" : [ PATHPATTERN, ... ]), + "terminating": TERMINATING, + }, ... ] + } + DELEGATION_SPEC_VERSION is the spec version used by the delegated targets metadata file. ## Client Setup The following steps must be implemented in the setup of a TUF client: -* The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec-version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec-version in root metadata. +* The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. -* The root metadata already contains the TUF spec-version. After downloading and verifying the root metadata, the client shall compare the spec-version in the root metadata (repository spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described in [Procedure](#procedure). +* The client should check the next_spec_version field in root metadata. After downloading and verifying the root metadata, the client shall compare the next_spec_version in the root metadata (repository spec version) with the spec version of the local client (client spec version). If there is a next root file available on the repository, the client shall then proceed as described in [Procedure](#procedure). If the next root file is not available or cannot be downloaded, the client skips these steps and proceeds with the update using its current spec version. ### Procedure -When the client compares their spec-version to the spec-version found in the root metadata, there are a few possible cases. If the versions are the same, nothing needs to be done. If the major versions differ, the update halts and cannot be completed until the major versions match. If the major versions match but the minor or patch versions differ, the client may choose to continue the update or to update to a new client. These cases are discussed in detail below. - -If the major versions on the client and repository do not match, the update cannot proceed until this discrepancy is resolved. If the repository spec-version is lower than the client spec-version, the client should terminate and report the mismatch to the user. The client's user may then choose to report this issue to the repository or take no action. The client will not be able to perform an update until the repository is updated or the client chooses to downgrade their TUF client. If the client spec-version is lower than the repository spec-version, the client should try to automatically update to a new TUF client. If an updated client is not available, the client shall report the mismatch to the user so that the user may manually update the client. Once the updated TUF client is installed, the client should proceed with the update. +When the client compares their spec version to the 'next_spec_version' found in the root metadata, there are a few possible cases. If the versions are the same, nothing needs to be done. If the major versions differ, the update halts and cannot be completed until the major versions match. If the major versions match but the minor or patch versions differ, the client may choose to continue the update or to update to a new client. These cases are discussed in detail below. -If a minor version or patch of the spec-version does not match, the client should report the mismatch to the user. However, the update can proceed without further action. A client may choose to update before proceeding with the update or simply log the error. Additionally, a client may act differently for minor version changes and patches. For example, a client may choose to require manual confirmation before proceeding with a mismatched minor version, but automatically continue with a mismatched patch. +If the major versions on the client and repository do not match, the update cannot proceed until this discrepancy is resolved. If the repository spec version is lower than the client spec version, the client should terminate and report the mismatch to the user. The client's user may then choose to report this issue to the repository or take no action. The client will not be able to perform an update until the repository is updated or the client chooses to downgrade their TUF client. If the client spec version is lower than the repository spec version, the client should try to automatically update to a new TUF client. If an updated client is not available, the client shall report the mismatch to the user so that the user may manually update the client. Once the updated TUF client is installed, the client should proceed with the update. -## How a Repository updates to a new spec-version +If a minor version or patch of the spec version does not match, the client should report the mismatch to the user. However, the update can proceed without further action. A client may choose to update itself before proceeding with the update or simply log the error. Additionally, a client may act differently for minor version changes and patches. For example, a client may choose to require manual confirmation before proceeding with a mismatched minor version, but automatically continue with a mismatched patch. -The repository handles updating to a new spec-version in one of two ways, depending on whether there are any changes to the format of root metadata. +## How a Repository updates to a new specification -For minor or fix version updates, or for any major version updates that do not affect root metadata, the repository simply creates and signs a new root metadata file that includes the new spec-version. To do this, the repository manager would create a new root metadata file and fill out all of the information, including the new spec-version. This file would then be signed by the root role and uploaded to the repository. Clients performing an update will download this root file and update to a new spec-version as described in [Procedure](#procedure) before performing an update. +A repository handles updating to a new spec version by updating the 'next_spec_version' field in root metadata, then creating root metadata with the new spec version. To do this, the repository manager would create a new root metadata file before updating versions and fill out all of the information including a next_spec_version for the updated version. This file would then be signed by the root role and uploaded to the repository. Then, the repository would update to the new spec version and create, sign, and upload a root metadata file with the new version. Clients performing an update will download the first root file, see the next root metadata file, and update to a new spec version as described in [Procedure](#procedure) before performing an update. -For major changes in which the root metadata format changes in any way, the repository must create two root metadata files. The first is formatted using the old specification, but includes the new spec-version. The second is formatted using the new specification and includes the new spec-version. More specifically if a repository is updating from spec-version 2.5.1 to spec-version 3.0.0 and the current root file is named 10.root.json (using consistent snapshots), the following steps must be performed: +For example if a repository is updating from spec version 2.5.1 to spec version 3.0.0 and the current root file is named 10.root.json (using consistent snapshots), the following steps must be performed: -* Create a new root metadata file, 11.root.json, that includes all required fields and formatting for version 2.5.1, except that the spec-version field lists version 3.0.0. This file will not be used to perform updates. -* Sign 11.root.json with the root key. -* Create a new root metadata file, 12.root.json, that includes all required fields and formatting for version 3.0.0, including a spec-version of 3.0.0. -* Sign 12.root.json with the root key. -* Upload the signed 11.root.json and 12.root.json to the repository at the same time. +* Create a new root metadata file, 11.root.json, that includes all required fields and formatting for version 2.5.1, with a next_spec_version of 3.0.0. +* Sign 11.root.json with the root key and upload it to the repository. +* Create a new root metadata file, 12.root.json, that includes all required fields and formatting for version 3.0.0, including a spec_version of 3.0.0 and a next_spec_version of 3.0.0. +* Sign 12.root.json with the root key and upload it to the repository. -After these steps are performed, a client performing an update will download 11.root.json and update to spec-version 3.0 as described in [Procedure](#procedure). The client will then download 12.root.json, and seeing that this is the last available root file, the client will proceed with the update. +After these steps are performed, a client performing an update will download 11.root.json and update to spec version 3.0 as described in [Procedure](#procedure). The client will then download 12.root.json, and seeing that this is the last available root file, the client will proceed with the update. -For both of these cases, all other metadata files and images are handled as described in the TUF specification for the version of the specification listed in root metadata. +All other metadata files and images are handled as described in the TUF specification for the version of the specification listed in root metadata. ## Changes to TAPs TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. @@ -70,11 +113,17 @@ TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted TUF version numbers shall be determined based on [semantic versioning](https://semver.org/). This specification describes version numbers in the format MAJOR.MINOR.PATCH. In semantic versioning, only major changes are non-backwards compatible. +## Delegated Targets + +To support delegated targets with old spec versions, clients should maintain a version of the client from the previous major spec version. When delegation metadata indicates an old spec version in the spec_version field of the delegation, the client will determine if the spec version of the delegation is supported, and if so parse the metadata using the correct client version. For example if a client using spec version 3.0 sees a delegation that uses spec version 2.5, the client may parse this metadata using a 2.8 version of the client that is maintained on the system. + ## Multiple Repositories -If a client downloads metadata from multiple repositories as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md), the client should check that the major version on all repositories and the client is the same. If any of the major versions do not match, the mismatch should be handled as discussed in [Procedure](#procedure). The client should not maintain multiple spec-versions for different repositories as this prevents the metadata from different repositories from being compared. + +If a client downloads metadata from multiple repositories as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md), the client should check that the major version on all repositories and the client is the same. If any of the major versions do not match, the mismatch should be handled as discussed in [Procedure](#procedure). The client should not maintain multiple spec versions for different repositories as this prevents the metadata from different repositories from being compared. ## Updating Trusted Root Metadata -After a major version change, the client must update their trusted root metadata to the root metadata that complies with the new spec-version. To do so, the client first downloads and verifies the intermediate root metadata file, then downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. + +After a major version change, the client must update their trusted root metadata to the root metadata that complies with the new spec version. To do so, the client first downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. # Security Analysis @@ -82,7 +131,7 @@ There should be minimal security impact. Ensuring that the client is up to date # Backwards Compatibility -This TAP is backwards compatible, and should be implemented on all clients before any non-backwards compatible TAPs are released. +This TAP adds additional fields to TUF metadata and so is not backwards compatible. In addition, this TAP should be implemented on all clients before any other non-backwards compatible TAPs are released. # Augmented Reference Implementation From 4bb95ee483f0bc8e807be6dbac1d654bd889461c Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 22 Jul 2019 11:41:30 -0400 Subject: [PATCH 14/40] Added use cases and description of directory structure added a description of use cases to motivation changed the specification to use a directory structure for major version changes --- candidate-tuf-versions.md | 130 +++++++++++++++++--------------------- 1 file changed, 59 insertions(+), 71 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index a6601226..35fe99c7 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,7 +1,7 @@ * TAP: * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 13-July-2018 +* Last-Modified: 22-July-2019 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown @@ -9,117 +9,105 @@ # Abstract -This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. This TAP allows clients to update from their current version of the TUF specification to a version that is not backwards compatible. +This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. This includes any feature that affects the parsing or validation of metadata, or any other changes that mean a client can no longer safely and reliable perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. This TAP allows clients and repositories to update from their current version of the TUF specification to a version that is not backwards compatible. # Motivation The goal of this TAP is to prevent compatibility issues while allowing breaking changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the specification will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. Because of this, clients need to check that the version of the specification that they are using is compatible with the version that a repository is using before validating metadata. -# Rationale +This TAP aims to support a variety of use cases when it comes to managing updates to the TUF specification. + +## Use case 1: A repository updates to a new TUF spec version + +As new features become available in the TUF specification, repositories may wish to update to a new version of the specification to support these features. This includes adding support for TAPs with breaking changes. Clients should have a way to determine what specification version the repository is using to ensure that metadata can be safely and reliably verified. In addition, clients who are using an older TUF version should not use metadata from a new version that is not backwards compatible. + +## Use case 2: A client is unable to update to a new TUF spec version + +A client may wish to maintain an older version of the TUF specification after a new version is released. This may be due to development time needed to update to the new TUF specification version or compatibility issues. Repositories should be able to support these clients using older specification versions as well as any clients that are up to date. + +## Use case 3: A delegated targets role uses a different TUF spec version -In order to allow for breaking changes, a client must compare its spec version to the spec version of the repository then report an error or update the client if these versions are not compatible. To allow for this functionality, this TAP clarifies how spec versions should be formatted, defines a procedure for comparing versions, and describes some special cases. +A delegated role may make and sign metadata using an old version of the TUF specification even after the top level roles have been updated. Different roles may be managed by people in different organizations who are not able to coordinate updating to a new version of the TUF specification. To handle this, a client should be able to parse a delegation that uses a different TUF specification version than the repository. -This TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. +## Use case 4: A client downloads metadata from multiple repositories -The consistent format of versions makes it possible for clients to reliably compare the spec version that they are using to the spec version found in TUF metadata. To do this comparison, an additional field will be added to root metadata that lists the 'next_spec_version'. This field will list the spec version to be used for all future root metadata. The client will check this field, and if it is set the client will finish the current update cycle, then update to the new spec version when a next root file is available. So if 10.root.json specifies a spec_version of 5.6.1 and a next_spec_version of 6.0.0, the client will look for 11.root.json, and if it is available update to version 6.0.0 before downloading and verifying 11.root.json. This process ensures that the client and metadata are using the same version number before validation. If 11.root.json is invalid or deleted by an attacker, the client shall use the metadata in 10.root.json as the trusted root metadata until the repository corrects 11.root.json. +As described in TAP 4, TUF clients may download metadata from multiple repositories. These repositories do not need to coordinate, and so should not be required to update to a new TUF specification version at the same time. A client should be able to use one repository that uses TUF version 1.0.0 and another repository that uses TUF version 2.0.0. -In addition, there are are few special cases that must be considered. These include TUF setups with multiple repositories as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) and delegated targets that are unable to update simultaneously with the top level metadata. +## Use case 5: Existing TUF clients -If there are multiple repositories in use the repositories should coordinate their major spec version updates to ensure clients are able to perform updates. The repository managers already must coordinate to ensure that the same targets metadata is signed by all relevant repositories, so the same communication channels can be used to ensure that the spec version is updated at the same time on all repositories. +Existing TUF clients will still expect to download and parse metadata that uses the current TUF version even once this TAP is implemented on repositories. Before this TAP there was no method for determining compatibility between a client and repository, so existing clients should be supported while this change is being implemented on TUF repositories. -A client may choose to support delegated targets that comply with an old spec_version. This may be desirable if the targets are managed by many different individuals or organizations to allow the spec_version update to happen more quickly. To support this behavior, an additional field will be added to the delegation metadata. This field will include the spec_version of the delegation and will be used by the client to determine which client version to use when parsing the delegation metadata. To do this, the client should maintain any old client versions that will be supported for delegated targets. +# Rationale + +To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF specification. However, secure and reliable verification of metadata requires that the TUF client is able to compare metadata from multiple sources. + +Non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. +Repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. + +The directory structure for breaking changes can also be used to identify the spec version used by a delegation. When parsing a delegation, a client can determine what spec versions are supported by the client and use a supported version to validate the metadata. # Specification -This TAP requires an additional field in root metadata and delegations, as well as some setup on the client to ensure that the spec versions are compatible before an update is performed. In addition, the repository needs to perform some configuration when the spec version is updated. These processes are described in this section. +In order to allow clients and repositories to update TUF spec versions while maintaining support for old systems, this TAP defines a variety of procedures to be added to the update process. This section describes how version numbers will be defined for the TUF specification, what the repository and client must to do support specification updates, and some special cases. -## The new file format of root metadata +## Version Number Format -A 'next_spec_version' field is added to root metadata. With this field, the "signed" portion of root metadata is: +This TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. These backwards compatible TAPs add additional features to TUF, but clients that do not have these features will still be able to securely and reliably perform updates from repositories that support the TAPs. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. - { "_type" : "root", - "spec_version" : SPEC_VERSION, - "next_spec_version" : NEXT_SPEC_VERSION, - "consistent_snapshot": CONSISTENT_SNAPSHOT, - "version" : VERSION, - "expires" : EXPIRES, - "keys" : { - KEYID : KEY - , ... }, - "roles" : { - ROLE : { - "keyids" : [ KEYID, ... ] , - "threshold" : THRESHOLD } - , ... } - } +### Changes to TAPs +TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. - NEXT_SPEC_VERSION is the version number of the specification for the next root metadata file. +## How a repository updates -## The new format of delegations +When a repository manager chooses to update the repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. Clients will now be able to use these metadata files once their TUF spec versions are also updated. -A 'spec_version' field is added to delegations. With this field, the format of the delegations object is: +If the repository is updated to a new minor or patch spec version, this may be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the directory named 2.0.0. Minor version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. - { "keys" : { - KEYID : KEY, - ... }, - "roles" : [{ - "name": ROLENAME, - "spec_version": DELEGATION_SPEC_VERSION - "keyids" : [ KEYID, ... ] , - "threshold" : THRESHOLD, - ("path_hash_prefixes" : [ HEX_DIGEST, ... ] | - "paths" : [ PATHPATTERN, ... ]), - "terminating": TERMINATING, - }, ... ] - } - DELEGATION_SPEC_VERSION is the spec version used by the delegated targets metadata file. +A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. -## Client Setup +The version numbers used for consistent snapshots should be consistent across all supported spec versions. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. These files must use the same keys so that a client can find the next root file in whichever spec version they support. -The following steps must be implemented in the setup of a TUF client: +For existing TUF clients to continue operation while this TAP is implemented, repositories should store metadata from before TUF 1.0.0 in the top level repository (there will be no directory named 0.0.0). This allows existing clients to continue downloading metadata from the repository. -* The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. +## Changes to TUF clients -* The client should check the next_spec_version field in root metadata. After downloading and verifying the root metadata, the client shall compare the next_spec_version in the root metadata (repository spec version) with the spec version of the local client (client spec version). If there is a next root file available on the repository, the client shall then proceed as described in [Procedure](#procedure). If the next root file is not available or cannot be downloaded, the client skips these steps and proceeds with the update using its current spec version. +TUF clients must make a few changes to support changes to the TUF specification. -### Procedure +The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. -When the client compares their spec version to the 'next_spec_version' found in the root metadata, there are a few possible cases. If the versions are the same, nothing needs to be done. If the major versions differ, the update halts and cannot be completed until the major versions match. If the major versions match but the minor or patch versions differ, the client may choose to continue the update or to update to a new client. These cases are discussed in detail below. +In addition, TUF clients may maintain old versions of the client for compatibility reasons. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. -If the major versions on the client and repository do not match, the update cannot proceed until this discrepancy is resolved. If the repository spec version is lower than the client spec version, the client should terminate and report the mismatch to the user. The client's user may then choose to report this issue to the repository or take no action. The client will not be able to perform an update until the repository is updated or the client chooses to downgrade their TUF client. If the client spec version is lower than the repository spec version, the client should try to automatically update to a new TUF client. If an updated client is not available, the client shall report the mismatch to the user so that the user may manually update the client. Once the updated TUF client is installed, the client should proceed with the update. +## Changes to the update process -If a minor version or patch of the spec version does not match, the client should report the mismatch to the user. However, the update can proceed without further action. A client may choose to update itself before proceeding with the update or simply log the error. Additionally, a client may act differently for minor version changes and patches. For example, a client may choose to require manual confirmation before proceeding with a mismatched minor version, but automatically continue with a mismatched patch. +When a TUF client downloads metadata from a repository, the client must determine which spec version to use for the download. To do this, the client looks for the highest supported numbered directory on the repository. -## How a Repository updates to a new specification -A repository handles updating to a new spec version by updating the 'next_spec_version' field in root metadata, then creating root metadata with the new spec version. To do this, the repository manager would create a new root metadata file before updating versions and fill out all of the information including a next_spec_version for the updated version. This file would then be signed by the root role and uploaded to the repository. Then, the repository would update to the new spec version and create, sign, and upload a root metadata file with the new version. Clients performing an update will download the first root file, see the next root metadata file, and update to a new spec version as described in [Procedure](#procedure) before performing an update. +* If the highest number on the repository is equal to the client spec version, the client will use this directory to download metadata. +* If the highest number on the repository is less than the client spec version, the client may call functions from a previous spec version client to download the metadata. The client may support as many or as few versions as desired for the application. If a previous version is not available, the client shall report that an update is not possible due to an old spec version on the repository. +* If the highest number on the repository is higher than the client spec version, the client should report that it is not using the most up to date TUF spec version then proceed with the directory that corresponds with the client spec version if available. If no such directory exists, the client terminates the update. -For example if a repository is updating from spec version 2.5.1 to spec version 3.0.0 and the current root file is named 10.root.json (using consistent snapshots), the following steps must be performed: +Once the supported directory is determined, the client shall attempt the update using the metadata in this directory. -* Create a new root metadata file, 11.root.json, that includes all required fields and formatting for version 2.5.1, with a next_spec_version of 3.0.0. -* Sign 11.root.json with the root key and upload it to the repository. -* Create a new root metadata file, 12.root.json, that includes all required fields and formatting for version 3.0.0, including a spec_version of 3.0.0 and a next_spec_version of 3.0.0. -* Sign 12.root.json with the root key and upload it to the repository. +For example, if a client has a spec version of 3.5 and a repository has directories for 2.0.0, 3.0.0, and 4.0.0, the client will report that spec version 4.0 is available, then download metadata from 3.0.0. -After these steps are performed, a client performing an update will download 11.root.json and update to spec version 3.0 as described in [Procedure](#procedure). The client will then download 12.root.json, and seeing that this is the last available root file, the client will proceed with the update. +Alternatively, if the same client downloads metadata from a repository with directories 1.0.0 and 2.0.0, the client could download metadata from 2.0.0 using a 2.x version of the client. If a 2.x version is not available, the client will report that it is unable to perform an update. -All other metadata files and images are handled as described in the TUF specification for the version of the specification listed in root metadata. +Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file uses a version other than the supported version, the client will look for the next root file first in the supported version, then the previous versions until the next root file is found or the currently trusted root file's version is reached. All root files should be verified using the major version of the TUF client that corresponds with the major version of the root file. -## Changes to TAPs -TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. +So, if the currently trusted root file is named 4.root.json and uses version 1.0.0 and the highest supported version is 3.0.0, the client will look for 5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the client will look for 6.root.json using the same process. -## Version Number format +## Changes to map file procedure -TUF version numbers shall be determined based on [semantic versioning](https://semver.org/). This specification describes version numbers in the format MAJOR.MINOR.PATCH. In semantic versioning, only major changes are non-backwards compatible. +When a TUF client contains a map file for multiple repositories, an additional step must be added to ensure a consensus is possible across repositories that are using different spec versions. In order for this procedure to work, all repositories should maintain metadata according to old spec versions as explained in [How a repository updates](#how-a-repository-updates). -## Delegated Targets +When parsing a mapping, the client should first look for a threshold as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) using the client's TUF spec version. If a threshold is not met with this spec version, the client should try with the previous major spec version, and so on until a threshold is reached or the version becomes negative. -To support delegated targets with old spec versions, clients should maintain a version of the client from the previous major spec version. When delegation metadata indicates an old spec version in the spec_version field of the delegation, the client will determine if the spec version of the delegation is supported, and if so parse the metadata using the correct client version. For example if a client using spec version 3.0 sees a delegation that uses spec version 2.5, the client may parse this metadata using a 2.8 version of the client that is maintained on the system. +## Changes to delegations -## Multiple Repositories +Delegated targets should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). -If a client downloads metadata from multiple repositories as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md), the client should check that the major version on all repositories and the client is the same. If any of the major versions do not match, the mismatch should be handled as discussed in [Procedure](#procedure). The client should not maintain multiple spec versions for different repositories as this prevents the metadata from different repositories from being compared. +In order to support this behavior, TUF clients should maintain the ability to parse targets metadata files from old spec versions. This can be done through functions that call sections of the old client version and translate these to be used by the new client. The client may make an old version obsolete if they choose, but will risk being unable to download new targets from delegations that are not updated. ## Updating Trusted Root Metadata @@ -131,11 +119,11 @@ There should be minimal security impact. Ensuring that the client is up to date # Backwards Compatibility -This TAP adds additional fields to TUF metadata and so is not backwards compatible. In addition, this TAP should be implemented on all clients before any other non-backwards compatible TAPs are released. +This TAP is backwards compatible and should be implemented on all clients before any other non-backwards compatible TAPs are released. # Augmented Reference Implementation -This TAP is included in the reference implementation [here] (https://github.com/theupdateframework/tuf/pull/854). +TODO # Copyright From 87d57c2ef1517be43f61651f7298e07ca69abd4e Mon Sep 17 00:00:00 2001 From: marinamoore Date: Tue, 20 Aug 2019 15:52:17 -0400 Subject: [PATCH 15/40] clarifications and minor edits --- candidate-tuf-versions.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 35fe99c7..cead4ce1 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -39,7 +39,7 @@ Existing TUF clients will still expect to download and parse metadata that uses # Rationale -To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF specification. However, secure and reliable verification of metadata requires that the TUF client is able to compare metadata from multiple sources. +To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF specification. However, secure and reliable verification of metadata requires that the TUF client is able to compare metadata from multiple sources. This comparison is not possible if changes to the TUF specification add or change metadata fields. Non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. @@ -64,9 +64,9 @@ When a repository manager chooses to update the repository to a new major TUF sp If the repository is updated to a new minor or patch spec version, this may be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the directory named 2.0.0. Minor version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. -A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. +A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. The version can be phased out by removing the directory containing that version from the repository. -The version numbers used for consistent snapshots should be consistent across all supported spec versions. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. These files must use the same keys so that a client can find the next root file in whichever spec version they support. +The version numbers used for consistent snapshots should be consistent across all supported spec versions. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the same consistent snapshot number must additionally use the same keys so that a client can find the next root file in whichever spec version they support. For existing TUF clients to continue operation while this TAP is implemented, repositories should store metadata from before TUF 1.0.0 in the top level repository (there will be no directory named 0.0.0). This allows existing clients to continue downloading metadata from the repository. @@ -82,10 +82,9 @@ In addition, TUF clients may maintain old versions of the client for compatibili When a TUF client downloads metadata from a repository, the client must determine which spec version to use for the download. To do this, the client looks for the highest supported numbered directory on the repository. - * If the highest number on the repository is equal to the client spec version, the client will use this directory to download metadata. * If the highest number on the repository is less than the client spec version, the client may call functions from a previous spec version client to download the metadata. The client may support as many or as few versions as desired for the application. If a previous version is not available, the client shall report that an update is not possible due to an old spec version on the repository. -* If the highest number on the repository is higher than the client spec version, the client should report that it is not using the most up to date TUF spec version then proceed with the directory that corresponds with the client spec version if available. If no such directory exists, the client terminates the update. +* If the highest number on the repository is higher than the client spec version, the client should report to the user that it is not using the most up to date TUF spec version then proceed with the directory that corresponds with the client spec version if available. If no such directory exists, the client terminates the update. Once the supported directory is determined, the client shall attempt the update using the metadata in this directory. @@ -95,7 +94,7 @@ Alternatively, if the same client downloads metadata from a repository with dire Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file uses a version other than the supported version, the client will look for the next root file first in the supported version, then the previous versions until the next root file is found or the currently trusted root file's version is reached. All root files should be verified using the major version of the TUF client that corresponds with the major version of the root file. -So, if the currently trusted root file is named 4.root.json and uses version 1.0.0 and the highest supported version is 3.0.0, the client will look for 5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the client will look for 6.root.json using the same process. +So, if the currently trusted root file is named 4.root.json and uses version 1.0.0 and the highest supported version is 3.0.0, the client will look for 5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the client will look for 6.root.json using the same process. To facilitate this, the client should maintain functions to parse root files from previous spec versions. If the client does not support the spec version of a root file, the client shall terminate the update and report the spec version mismatch. ## Changes to map file procedure From 48cc83a6392f904ce48134611f1ece05b9539a7d Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 22 Aug 2019 16:54:47 -0400 Subject: [PATCH 16/40] added some description --- candidate-tuf-versions.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index cead4ce1..6b3f2a5e 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -39,11 +39,11 @@ Existing TUF clients will still expect to download and parse metadata that uses # Rationale -To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF specification. However, secure and reliable verification of metadata requires that the TUF client is able to compare metadata from multiple sources. This comparison is not possible if changes to the TUF specification add or change metadata fields. +To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF specification. However, secure and reliable verification of metadata requires that the TUF client is able to compare metadata from multiple sources. This comparison is not possible if changes to the TUF specification add or change metadata fields. This TAP proposes a system for maintaining multiple TUF versions while ensuring that a client is able to compare metadata from the same version. Non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. -Repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. +In this TAP, repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. The directory structure for breaking changes can also be used to identify the spec version used by a delegation. When parsing a delegation, a client can determine what spec versions are supported by the client and use a supported version to validate the metadata. @@ -53,20 +53,27 @@ In order to allow clients and repositories to update TUF spec versions while mai ## Version Number Format -This TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects, and makes the version numbers consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. These backwards compatible TAPs add additional features to TUF, but clients that do not have these features will still be able to securely and reliably perform updates from repositories that support the TAPs. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. +In order to make TUF version numbers consistent and easily understood, this TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. These backwards compatible TAPs add additional features to TUF, but clients that do not have these features will still be able to securely and reliably perform updates from repositories that support the TAPs. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. ### Changes to TAPs -TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. +In order to manage changes to TUF, TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. ## How a repository updates -When a repository manager chooses to update the repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. Clients will now be able to use these metadata files once their TUF spec versions are also updated. +When a repository manager chooses to update the repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. Clients will now be able to use these metadata files once their TUF spec versions are also updated. After an update to version 2.0.0, the repository structure may look like: -If the repository is updated to a new minor or patch spec version, this may be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the directory named 2.0.0. Minor version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. +``` +- 1.0.0 + |- metadata files +- 2.0.0 + |- metadata files +``` + +If the repository is updated to a new minor or patch spec version, this shall be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the directory named 2.0.0. Minor version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. The version can be phased out by removing the directory containing that version from the repository. -The version numbers used for consistent snapshots should be consistent across all supported spec versions. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the same consistent snapshot number must additionally use the same keys so that a client can find the next root file in whichever spec version they support. +In order to allow a client to find the current metadata files across spec versions, the version numbers used for consistent snapshots should be consistent across all supported spec versions. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the same consistent snapshot number must additionally use the same keys so that a client can find the next root file in whichever spec version they support. For existing TUF clients to continue operation while this TAP is implemented, repositories should store metadata from before TUF 1.0.0 in the top level repository (there will be no directory named 0.0.0). This allows existing clients to continue downloading metadata from the repository. @@ -76,7 +83,7 @@ TUF clients must make a few changes to support changes to the TUF specification. The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. -In addition, TUF clients may maintain old versions of the client for compatibility reasons. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. +For compatibility, TUF clients may maintain old versions of the client. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. ## Changes to the update process @@ -104,13 +111,13 @@ When parsing a mapping, the client should first look for a threshold as describe ## Changes to delegations -Delegated targets should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). +So that delegated targets can update to new spec versions, delegated targets should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). In order to support this behavior, TUF clients should maintain the ability to parse targets metadata files from old spec versions. This can be done through functions that call sections of the old client version and translate these to be used by the new client. The client may make an old version obsolete if they choose, but will risk being unable to download new targets from delegations that are not updated. ## Updating Trusted Root Metadata -After a major version change, the client must update their trusted root metadata to the root metadata that complies with the new spec version. To do so, the client first downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. +To allow for future updates after a major version change, the client must update their trusted root metadata to the root metadata that complies with the new spec version. To do so, the client first downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. # Security Analysis From 986b3e3b7d7ea52caec2cabfac2af091fbdaf0bb Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 26 Aug 2019 16:16:08 -0400 Subject: [PATCH 17/40] added clarifications and grammar fixes --- candidate-tuf-versions.md | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 6b3f2a5e..322168de 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -19,15 +19,15 @@ This TAP aims to support a variety of use cases when it comes to managing update ## Use case 1: A repository updates to a new TUF spec version -As new features become available in the TUF specification, repositories may wish to update to a new version of the specification to support these features. This includes adding support for TAPs with breaking changes. Clients should have a way to determine what specification version the repository is using to ensure that metadata can be safely and reliably verified. In addition, clients who are using an older TUF version should not use metadata from a new version that is not backwards compatible. +As new features become available in the TUF specification, repositories may wish to update to a new version of the specification to support these features. This includes adding support for TAPs with breaking changes. In addition to the ability to update to a new specification version, clients should have a way to determine what specification version the repository is using to ensure that metadata can be safely and reliably verified. In addition, clients who are using an older TUF version should not use metadata from a new version that is not backwards compatible. ## Use case 2: A client is unable to update to a new TUF spec version -A client may wish to maintain an older version of the TUF specification after a new version is released. This may be due to development time needed to update to the new TUF specification version or compatibility issues. Repositories should be able to support these clients using older specification versions as well as any clients that are up to date. +A client may wish to maintain an older version of the TUF specification after a new version is released on a repository. This may be due to development time needed to update to the new TUF specification version or compatibility issues. Repositories should be able to support these clients using older specification versions as well as any clients that are up to date. ## Use case 3: A delegated targets role uses a different TUF spec version -A delegated role may make and sign metadata using an old version of the TUF specification even after the top level roles have been updated. Different roles may be managed by people in different organizations who are not able to coordinate updating to a new version of the TUF specification. To handle this, a client should be able to parse a delegation that uses a different TUF specification version than the repository. +A delegated role may make and sign metadata using an old version of the TUF specification even after the repository hosting the top level roles has been updated. Different roles may be managed by people in different organizations who are not able to coordinate updating to a new version of the TUF specification. To handle this, a client should be able to parse a delegation that uses a different TUF specification version than the repository. ## Use case 4: A client downloads metadata from multiple repositories @@ -35,13 +35,13 @@ As described in TAP 4, TUF clients may download metadata from multiple repositor ## Use case 5: Existing TUF clients -Existing TUF clients will still expect to download and parse metadata that uses the current TUF version even once this TAP is implemented on repositories. Before this TAP there was no method for determining compatibility between a client and repository, so existing clients should be supported while this change is being implemented on TUF repositories. +Existing TUF clients will still expect to download and parse metadata that uses the current TUF version even once this TAP is implemented on repositories. Before this TAP there was no method for determining compatibility between a client and repository, so existing clients should be supported while this change is being implemented on TUF clients and repositories. # Rationale To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF specification. However, secure and reliable verification of metadata requires that the TUF client is able to compare metadata from multiple sources. This comparison is not possible if changes to the TUF specification add or change metadata fields. This TAP proposes a system for maintaining multiple TUF versions while ensuring that a client is able to compare metadata from the same version. -Non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. +As they do not affect compatibility, non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to at any time without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. In this TAP, repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. @@ -60,7 +60,7 @@ In order to manage changes to TUF, TAPs shall be tied to a version of the TUF sp ## How a repository updates -When a repository manager chooses to update the repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. Clients will now be able to use these metadata files once their TUF spec versions are also updated. After an update to version 2.0.0, the repository structure may look like: +When a repository manager chooses to update their repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. Clients will now be able to use these metadata files once their TUF spec versions are also updated. After an update to version 2.0.0, the repository structure may look like: ``` - 1.0.0 @@ -71,23 +71,29 @@ When a repository manager chooses to update the repository to a new major TUF sp If the repository is updated to a new minor or patch spec version, this shall be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the directory named 2.0.0. Minor version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. -A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. The version can be phased out by removing the directory containing that version from the repository. +A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. The version can be phased out by no longer creating new metadata files in that directory. In order to allow clients to parse the root metadata chain, root metadata files should not be deleted even once a version is deprecated. In order to allow a client to find the current metadata files across spec versions, the version numbers used for consistent snapshots should be consistent across all supported spec versions. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the same consistent snapshot number must additionally use the same keys so that a client can find the next root file in whichever spec version they support. -For existing TUF clients to continue operation while this TAP is implemented, repositories should store metadata from before TUF 1.0.0 in the top level repository (there will be no directory named 0.0.0). This allows existing clients to continue downloading metadata from the repository. +For existing TUF clients to continue operation while this TAP is implemented, repositories should store metadata from before TUF 1.0.0 in the top level repository (there will be no directory named 0.0.0). This allows existing clients to continue downloading metadata from the repository. So a TUF repository that updates to version 1.0.0 may look like: + +``` +- metadata files +- 1.0.0 + |- metadata files +``` ## Changes to TUF clients TUF clients must make a few changes to support changes to the TUF specification. -The client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. +A client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. -For compatibility, TUF clients may maintain old versions of the client. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. +For compatibility, TUF clients may maintain old versions of the client that support previous spec versions. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. ## Changes to the update process -When a TUF client downloads metadata from a repository, the client must determine which spec version to use for the download. To do this, the client looks for the highest supported numbered directory on the repository. +When a TUF client downloads metadata from a repository, the client must determine which spec version to use for the download. To do this, the client looks for the highest supported numbered directory on the repository then determines compatibility with the following procedure: * If the highest number on the repository is equal to the client spec version, the client will use this directory to download metadata. * If the highest number on the repository is less than the client spec version, the client may call functions from a previous spec version client to download the metadata. The client may support as many or as few versions as desired for the application. If a previous version is not available, the client shall report that an update is not possible due to an old spec version on the repository. @@ -107,7 +113,7 @@ So, if the currently trusted root file is named 4.root.json and uses version 1.0 When a TUF client contains a map file for multiple repositories, an additional step must be added to ensure a consensus is possible across repositories that are using different spec versions. In order for this procedure to work, all repositories should maintain metadata according to old spec versions as explained in [How a repository updates](#how-a-repository-updates). -When parsing a mapping, the client should first look for a threshold as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) using the client's TUF spec version. If a threshold is not met with this spec version, the client should try with the previous major spec version, and so on until a threshold is reached or the version becomes negative. +When parsing a mapping, the client should first look for a threshold as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) using the client's TUF spec version. If a threshold is not met with this spec version, the client should try with the previous major spec version, and so on until a threshold is reached, the client does not support the version, or the version becomes negative. ## Changes to delegations @@ -117,7 +123,7 @@ In order to support this behavior, TUF clients should maintain the ability to pa ## Updating Trusted Root Metadata -To allow for future updates after a major version change, the client must update their trusted root metadata to the root metadata that complies with the new spec version. To do so, the client first downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. +To allow for future updates after a major version change, the client must update their trusted root metadata to a root metadata that complies with the new spec version. To do so, the client first downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. # Security Analysis From f5999be3cedb544492bdd0acd4c61b020e801dbe Mon Sep 17 00:00:00 2001 From: marinamoore Date: Tue, 27 Aug 2019 12:52:42 -0400 Subject: [PATCH 18/40] fix backward compatibility description --- candidate-tuf-versions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 322168de..cb06144e 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -131,7 +131,7 @@ There should be minimal security impact. Ensuring that the client is up to date # Backwards Compatibility -This TAP is backwards compatible and should be implemented on all clients before any other non-backwards compatible TAPs are released. +This TAP is backwards compatible and should be implemented on all repositories before any non-backwards compatible TAPs are released. # Augmented Reference Implementation From 30551894f5a06735d6b65c19dff6f2cfd94f69c6 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 3 Oct 2019 14:47:24 -0400 Subject: [PATCH 19/40] add more rationale --- candidate-tuf-versions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index cb06144e..23781698 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -43,7 +43,7 @@ To allow for updates to the TUF specification while considering all of the above As they do not affect compatibility, non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to at any time without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. -In this TAP, repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. +In order to allow clients to update independently of repositories, the repositories need to continue support for old TUF versions for some period of time after updating. In this TAP, repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. The directory structure for breaking changes can also be used to identify the spec version used by a delegation. When parsing a delegation, a client can determine what spec versions are supported by the client and use a supported version to validate the metadata. @@ -87,7 +87,7 @@ For existing TUF clients to continue operation while this TAP is implemented, re TUF clients must make a few changes to support changes to the TUF specification. -A client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. +In order to find compatible updates on a repository, a client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. For compatibility, TUF clients may maintain old versions of the client that support previous spec versions. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. From 124e82fd54c5264b18525687140c18ff30e3ea3d Mon Sep 17 00:00:00 2001 From: marinamoore Date: Tue, 8 Oct 2019 15:36:42 -0400 Subject: [PATCH 20/40] add clarifications and security analysis --- candidate-tuf-versions.md | 55 ++++++++++++++++++++++++++------------- 1 file changed, 37 insertions(+), 18 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 23781698..8c8a1f86 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -9,11 +9,13 @@ # Abstract -This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. This includes any feature that affects the parsing or validation of metadata, or any other changes that mean a client can no longer safely and reliable perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. This TAP allows clients and repositories to update from their current version of the TUF specification to a version that is not backwards compatible. +This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change atomically in order to continue functioning as expected. This includes any feature that affects the parsing or validation of metadata, or any other changes that mean a client can no longer safely and reliable perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. + +This TAP adds support for breaking changes by making changes to both repositories and clients. Repositories will generate metadata for multiple TUF specification versions and place the metadata in a directory structure according to the specification version. Clients will then determine the most recent specification version support by both the client and repository and use this specification version to perform an update. In order for clients to update to a new TUF specification version, the process described in this TAP assumes that TUF clients are able to upgrade. # Motivation -The goal of this TAP is to prevent compatibility issues while allowing breaking changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository using a new version of the specification will be incompatible with clients using the old version of the specification. Similarly, clients using the new version will no longer be able to parse metadata from repositories using the old version of the specification. Because of this, clients need to check that the version of the specification that they are using is compatible with the version that a repository is using before validating metadata. +The goal of this TAP is to prevent compatibility issues while allowing breaking changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository in the format of a new TUF specification version will be incompatible with clients parsing metadata formatted using an old version of the specification. Similarly, clients using a new TUF specification version will no longer be able to parse metadata from repositories using an old version of the TUF specification. Because of this, clients need to check that the version of the specification that they are using is compatible with the version that a repository is using before validating metadata. For example, a client uses TUF version 2.0.0 will not be able to correctly parse metadata generated by a repository using TUF version 3.0.0 (a new version) or 1.0.0 (an old version). This TAP aims to support a variety of use cases when it comes to managing updates to the TUF specification. @@ -23,19 +25,19 @@ As new features become available in the TUF specification, repositories may wish ## Use case 2: A client is unable to update to a new TUF spec version -A client may wish to maintain an older version of the TUF specification after a new version is released on a repository. This may be due to development time needed to update to the new TUF specification version or compatibility issues. Repositories should be able to support these clients using older specification versions as well as any clients that are up to date. +A client may wish to continue parsing metadata generated using an older version of the TUF specification after a new specification version is released. This may be due to development time needed to update to the new TUF specification version or compatibility issues. Repositories should be able to support clients using older specification versions as well as any clients that are up to date by maintaining metadata in both the new and old specification formats. ## Use case 3: A delegated targets role uses a different TUF spec version -A delegated role may make and sign metadata using an old version of the TUF specification even after the repository hosting the top level roles has been updated. Different roles may be managed by people in different organizations who are not able to coordinate updating to a new version of the TUF specification. To handle this, a client should be able to parse a delegation that uses a different TUF specification version than the repository. +A delegated role may make and sign metadata using an old version of the TUF specification even after the repository hosting the top level roles has been updated. Different roles may be managed by people in different organizations who are not able to coordinate updating to a new version of the TUF specification. To handle this, a client should be able to parse a delegation that generates metatdata with a different TUF specification version than the repository. Therefore the client should be able to parse metadata from multiple TUF specification versions. ## Use case 4: A client downloads metadata from multiple repositories -As described in TAP 4, TUF clients may download metadata from multiple repositories. These repositories do not need to coordinate, and so should not be required to update to a new TUF specification version at the same time. A client should be able to use one repository that uses TUF version 1.0.0 and another repository that uses TUF version 2.0.0. +As described in TAP 4, TUF clients may download metadata from multiple repositories. These repositories do not need to coordinate, and so should not be required to update to a new TUF specification version at the same time. A client should be able to use one repository that uses TUF version 1.0.0 and another repository that uses TUF version 2.0.0. This functionality is only needed for TUF clients that will download metadata from multiple repositories. ## Use case 5: Existing TUF clients -Existing TUF clients will still expect to download and parse metadata that uses the current TUF version even once this TAP is implemented on repositories. Before this TAP there was no method for determining compatibility between a client and repository, so existing clients should be supported while this change is being implemented on TUF clients and repositories. +Existing TUF clients will still expect to download and parse metadata that uses the current TUF specification version even once this TAP is implemented. Before this TAP there was no method for determining compatibility between a client and repository, so existing TUF repositories should be able to continue to generate backwards compatible metadata to support existing clients. This means that non-backwards compatible TUF metadata must be stored in a different location of the repository from existing metadata. # Rationale @@ -43,9 +45,9 @@ To allow for updates to the TUF specification while considering all of the above As they do not affect compatibility, non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to at any time without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. -In order to allow clients to update independently of repositories, the repositories need to continue support for old TUF versions for some period of time after updating. In this TAP, repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. +In order to allow clients to update independently of repositories, the repositories need to continue support for old TUF versions for some period of time after updating. In this TAP, repositories support multiple versions by separating the metadata from versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. -The directory structure for breaking changes can also be used to identify the spec version used by a delegation. When parsing a delegation, a client can determine what spec versions are supported by the client and use a supported version to validate the metadata. +To save space, target files should remain in the parent directory on the repository. The metadata files (in directories according to their spec version) will point to target files relative to the parent directory. # Specification @@ -53,16 +55,23 @@ In order to allow clients and repositories to update TUF spec versions while mai ## Version Number Format -In order to make TUF version numbers consistent and easily understood, this TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. These backwards compatible TAPs add additional features to TUF, but clients that do not have these features will still be able to securely and reliably perform updates from repositories that support the TAPs. Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. +In order to make the meaning of a TUF version numbers change consistent and easily understood, this TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects. +In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. +Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. +If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. These backwards compatible TAPs add additional features to TUF, but clients that do not have these features will still be able to securely and reliably perform updates from repositories that support the TAPs. There may be minor differences in which updates a client installs across minor versions. If a repository maintainer is worried about the impact of a minor spec change on clients, they may choose to wait to implement the change with a major specification version (for example, they could remain on version 2.4.6 after the release of 2.5). +Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. ### Changes to TAPs In order to manage changes to TUF, TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. ## How a repository updates -When a repository manager chooses to update their repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. Clients will now be able to use these metadata files once their TUF spec versions are also updated. After an update to version 2.0.0, the repository structure may look like: +In order to allow for the above use cases, repositories must support multiple TUF specification versions. In order to do so, this TAP proposes a new directory structure for repositories. + +When a repository manager chooses to update their repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. Target files will not be included in this directory. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. The root file should be signed by both the new root key and the current root key (the root key from the most recent metadata in the previous major spec version). Clients will now be able to use the new metadata files once their TUF spec versions are also updated. After an update to version 2.0.0, the repository structure may look like: ``` +- Target files - 1.0.0 |- metadata files - 2.0.0 @@ -87,9 +96,9 @@ For existing TUF clients to continue operation while this TAP is implemented, re TUF clients must make a few changes to support changes to the TUF specification. -In order to find compatible updates on a repository, a client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. +In order to find compatible updates on a repository, a client must keep track of the TUF specification versions it supports. To do so, a global variable or other local storage option should contain the client spec version, or spec version range. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. -For compatibility, TUF clients may maintain old versions of the client that support previous spec versions. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. +For compatibility, TUF clients may maintain support for old versions of the TUF specification. This support can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. ## Changes to the update process @@ -101,25 +110,25 @@ When a TUF client downloads metadata from a repository, the client must determin Once the supported directory is determined, the client shall attempt the update using the metadata in this directory. -For example, if a client has a spec version of 3.5 and a repository has directories for 2.0.0, 3.0.0, and 4.0.0, the client will report that spec version 4.0 is available, then download metadata from 3.0.0. +For example, if a client has a spec version of 3.5 and a repository has directories for 2.0.0, 3.0.0, and 4.0.0, the client will report that spec version 4.0 is available, then download metadata from 3.0.0. This reporting is up to the discretion of an implementer, but it should be used to encourage updating the client to the most recent specification version. Alternatively, if the same client downloads metadata from a repository with directories 1.0.0 and 2.0.0, the client could download metadata from 2.0.0 using a 2.x version of the client. If a 2.x version is not available, the client will report that it is unable to perform an update. -Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file uses a version other than the supported version, the client will look for the next root file first in the supported version, then the previous versions until the next root file is found or the currently trusted root file's version is reached. All root files should be verified using the major version of the TUF client that corresponds with the major version of the root file. +Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file saved on the client uses a version other than the supported version, the client will look for the next root file first in the supported version, then the previous versions until the next root file is found or the currently trusted root file's version is reached. All root files should be verified using the major version of the TUF client that corresponds with the major version of the root file. So, if the currently trusted root file is named 4.root.json and uses version 1.0.0 and the highest supported version is 3.0.0, the client will look for 5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the client will look for 6.root.json using the same process. To facilitate this, the client should maintain functions to parse root files from previous spec versions. If the client does not support the spec version of a root file, the client shall terminate the update and report the spec version mismatch. ## Changes to map file procedure -When a TUF client contains a map file for multiple repositories, an additional step must be added to ensure a consensus is possible across repositories that are using different spec versions. In order for this procedure to work, all repositories should maintain metadata according to old spec versions as explained in [How a repository updates](#how-a-repository-updates). +Metadata that is generated using different major versions of the TUF specification cannot be safely and reliable compared. To deal with this a TUF client that contains a map file for multiple repositories must perform an additional step to ensure a consensus is possible across repositories that are using different spec versions. In order for this procedure to work, all repositories should maintain metadata according to old spec versions as explained in [How a repository updates](#how-a-repository-updates). -When parsing a mapping, the client should first look for a threshold as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) using the client's TUF spec version. If a threshold is not met with this spec version, the client should try with the previous major spec version, and so on until a threshold is reached, the client does not support the version, or the version becomes negative. +When parsing a mapping, the client should first look for a threshold as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) using the client's TUF spec version. If a threshold is not met with this spec version, the client should try with the previous major spec version, and so on until a threshold is reached or the client does not support the version. ## Changes to delegations -So that delegated targets can update to new spec versions, delegated targets should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). +So that delegated targets can update to new spec versions, delegated targets metadata should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). -In order to support this behavior, TUF clients should maintain the ability to parse targets metadata files from old spec versions. This can be done through functions that call sections of the old client version and translate these to be used by the new client. The client may make an old version obsolete if they choose, but will risk being unable to download new targets from delegations that are not updated. +In order to support this behavior, TUF clients should maintain the ability to parse targets metadata files from old spec versions. This can be done through functions that call sections of the old client version and translate these to be used by the new client. The client may make an old version obsolete if they choose, but will risk being unable to download new targets from delegations that are not updated. TUF implementers should decide how many old specification versions to support based on the expected usage of their implementation. ## Updating Trusted Root Metadata @@ -129,6 +138,16 @@ To allow for future updates after a major version change, the client must update There should be minimal security impact. Ensuring that the client is up to date should improve security in the event that a security vulnerability is patched in a release of the spec. +A downgrade attack on the TUF specification version may be possible if an attacker is able to block access to a directory on the repository. This would mean that a client would use metadata from a previous specification version when performing an update. However, the metadata would still have to be current and properly signed. To mitigate the damage from a downgrade attack in the case that a security flaw is found in a version of the TUF specification, the vulnerable version of the specification should no longer be supported on the repository (the metadata files should be revoked or allowed to expire). In addition, clients should be updated to no longer support a vulnerable specification version. + +The TUF version update system described in this TAP uses the root key rotation mechanism to ensure that the client only uses valid TUF metadata. The first time a new TUF version is used by the client, the client checks the repository's root metadata in the new version's format. If the root metadata is not signed by the previous trusted root key, the update does not proceed, and the client halts the update. In this way, updates to the repository's TUF version does not impact the security of an update. + +When the client updates to a new TUF version, it should use TUF to ensure that it downloads a valid new client. To do so, it should use its current TUF version to update to the new version of TUF. For example, if a client running TUF version 2.5.0 wants to update to version 3.0.0, the client should use metadata from TUF version 2.x.x to self update. Using this technique, the TUF client can ensure that it downloads the intended update when upgrading to a new TUF version. + +As always, authors of TUF clients should ensure that the client is a valid implementation of TUF and that no security flaws are introduced in code. In addition, supply chain security techniques (for example [in-toto](https://in-toto.io/in-toto.html)) should be used to ensure that vulnerabilities are not introduced into TUF clients between when they are written and installed. + +Note that an attacker on the same network could inject false metadata to block the client from updating. However this denial of service attack is always possible for an attacker on the network with the ability to alter network traffic. + # Backwards Compatibility This TAP is backwards compatible and should be implemented on all repositories before any non-backwards compatible TAPs are released. From 468c6b8db59437737395416732e5ffc1205d82b8 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 10 Oct 2019 16:29:50 -0400 Subject: [PATCH 21/40] Formatting changes wrapped all text at 80 chars to enable easier diff viewing for future versions. --- candidate-tuf-versions.md | 383 +++++++++++++++++++++++++++++++------- 1 file changed, 313 insertions(+), 70 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 8c8a1f86..02405573 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -9,66 +9,188 @@ # Abstract -This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change atomically in order to continue functioning as expected. This includes any feature that affects the parsing or validation of metadata, or any other changes that mean a client can no longer safely and reliable perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. - -This TAP adds support for breaking changes by making changes to both repositories and clients. Repositories will generate metadata for multiple TUF specification versions and place the metadata in a directory structure according to the specification version. Clients will then determine the most recent specification version support by both the client and repository and use this specification version to perform an update. In order for clients to update to a new TUF specification version, the process described in this TAP assumes that TUF clients are able to upgrade. +This TAP clarifies how to manage updates to the TUF specification that include +non-backwards compatible (breaking) changes. A breaking change means that a +client and repository must implement the change atomically in order to continue +functioning as expected. This includes any feature that affects the parsing or +validation of metadata, or any other changes that means a client can no longer +safely and reliable perform an update if the repository is updated out of sync +with the client. To facilitate finding breaking changes, this TAP defines a +procedure for TUF clients to ensure that their version of the TUF specification +is compatible with the specification version of the metadata they download. + +This TAP adds support for breaking changes by making changes to both +repositories and clients. Repositories will generate metadata for multiple TUF +specification versions and place the metadata in a directory structure according +to the specification version. Clients will then determine the most recent +specification version support by both the client and repository and use this +specification version to perform an update. In order for clients to update to a +new TUF specification version, the process described in this TAP assumes that +TUF clients are able to upgrade. # Motivation -The goal of this TAP is to prevent compatibility issues while allowing breaking changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include changes that are not backwards compatible. That means that metadata generated by a repository in the format of a new TUF specification version will be incompatible with clients parsing metadata formatted using an old version of the specification. Similarly, clients using a new TUF specification version will no longer be able to parse metadata from repositories using an old version of the TUF specification. Because of this, clients need to check that the version of the specification that they are using is compatible with the version that a repository is using before validating metadata. For example, a client uses TUF version 2.0.0 will not be able to correctly parse metadata generated by a repository using TUF version 3.0.0 (a new version) or 1.0.0 (an old version). - -This TAP aims to support a variety of use cases when it comes to managing updates to the TUF specification. +The goal of this TAP is to prevent compatibility issues while allowing breaking +changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include +changes that are not backwards compatible. That means that metadata generated +by a repository in the format of a new TUF specification version will be +incompatible with clients parsing metadata formatted using an old version of +the specification. Similarly, clients using a new TUF specification version +will no longer be able to parse metadata from repositories using an old version +of the TUF specification. Because of this, clients need to check that the +version of the specification that they are using is compatible with the version +that a repository is using before validating metadata. For example, a client +uses TUF version 2.0.0 will not be able to correctly parse metadata generated +by a repository using TUF version 3.0.0 (a new version) or 1.0.0 (an old +version). + +This TAP aims to support a variety of use cases when it comes to managing +updates to the TUF specification. ## Use case 1: A repository updates to a new TUF spec version -As new features become available in the TUF specification, repositories may wish to update to a new version of the specification to support these features. This includes adding support for TAPs with breaking changes. In addition to the ability to update to a new specification version, clients should have a way to determine what specification version the repository is using to ensure that metadata can be safely and reliably verified. In addition, clients who are using an older TUF version should not use metadata from a new version that is not backwards compatible. +As new features become available in the TUF specification, repositories may +wish to update to a new version of the specification to support these features. +This includes adding support for TAPs with breaking changes. In addition to the +ability to update to a new specification version, clients should have a way to +determine what specification version the repository is using to ensure that +metadata can be safely and reliably verified. In addition, clients who are +using an older TUF version should not use metadata from a new version that is +not backwards compatible. ## Use case 2: A client is unable to update to a new TUF spec version -A client may wish to continue parsing metadata generated using an older version of the TUF specification after a new specification version is released. This may be due to development time needed to update to the new TUF specification version or compatibility issues. Repositories should be able to support clients using older specification versions as well as any clients that are up to date by maintaining metadata in both the new and old specification formats. +A client may wish to continue parsing metadata generated using an older version +of the TUF specification after a new specification version is released. This +may be due to development time needed to update to the new TUF specification +version or compatibility issues. Repositories should be able to support clients +using older specification versions as well as any clients that are up to date +by maintaining metadata in both the new and old specification formats. ## Use case 3: A delegated targets role uses a different TUF spec version -A delegated role may make and sign metadata using an old version of the TUF specification even after the repository hosting the top level roles has been updated. Different roles may be managed by people in different organizations who are not able to coordinate updating to a new version of the TUF specification. To handle this, a client should be able to parse a delegation that generates metatdata with a different TUF specification version than the repository. Therefore the client should be able to parse metadata from multiple TUF specification versions. +A delegated role may make and sign metadata using an old version of the TUF +specification even after the repository hosting the top level roles has been +updated. Different roles may be managed by people in different organizations +who are not able to coordinate updating to a new version of the TUF +specification. To handle this, a client should be able to parse a delegation +that generates metatdata with a different TUF specification version than the +repository. Therefore the client should be able to parse metadata from multiple +TUF specification versions. ## Use case 4: A client downloads metadata from multiple repositories -As described in TAP 4, TUF clients may download metadata from multiple repositories. These repositories do not need to coordinate, and so should not be required to update to a new TUF specification version at the same time. A client should be able to use one repository that uses TUF version 1.0.0 and another repository that uses TUF version 2.0.0. This functionality is only needed for TUF clients that will download metadata from multiple repositories. +As described in TAP 4, TUF clients may download metadata from multiple +repositories. These repositories do not need to coordinate, and so should not +be required to update to a new TUF specification version at the same time. A +client should be able to use one repository that uses TUF version 1.0.0 and +another repository that uses TUF version 2.0.0. This functionality is only +needed for TUF clients that will download metadata from multiple repositories. ## Use case 5: Existing TUF clients -Existing TUF clients will still expect to download and parse metadata that uses the current TUF specification version even once this TAP is implemented. Before this TAP there was no method for determining compatibility between a client and repository, so existing TUF repositories should be able to continue to generate backwards compatible metadata to support existing clients. This means that non-backwards compatible TUF metadata must be stored in a different location of the repository from existing metadata. +Existing TUF clients will still expect to download and parse metadata that uses +the current TUF specification version even once this TAP is implemented. Before +this TAP there was no method for determining compatibility between a client and +repository, so existing TUF repositories should be able to continue to generate +backwards compatible metadata to support existing clients. This means that +non-backwards compatible TUF metadata must be stored in a different location of +the repository from existing metadata. # Rationale -To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF specification. However, secure and reliable verification of metadata requires that the TUF client is able to compare metadata from multiple sources. This comparison is not possible if changes to the TUF specification add or change metadata fields. This TAP proposes a system for maintaining multiple TUF versions while ensuring that a client is able to compare metadata from the same version. - -As they do not affect compatibility, non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to at any time without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. - -In order to allow clients to update independently of repositories, the repositories need to continue support for old TUF versions for some period of time after updating. In this TAP, repositories support multiple versions by separating the metadata from versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. - -To save space, target files should remain in the parent directory on the repository. The metadata files (in directories according to their spec version) will point to target files relative to the parent directory. +To allow for updates to the TUF specification while considering all of the +above use cases, a client must be able to parse metadata from multiple +repositories or roles that are using different versions of the TUF +specification. However, secure and reliable verification of metadata requires +that the TUF client is able to compare metadata from multiple sources. This +comparison is not possible if changes to the TUF specification add or change +metadata fields. This TAP proposes a system for maintaining multiple TUF +versions while ensuring that a client is able to compare metadata from the same +version. + +As they do not affect compatibility, non-breaking changes may happen +independently on clients and repositories. Any specification versions that do +not contain breaking changes may be updated to at any time without worrying +about compatibility. This TAP clarifies how TUF version numbers are formatted +so that breaking changes are distinct from non-breaking changes. + +In order to allow clients to update independently of repositories, the +repositories need to continue support for old TUF versions for some period of +time after updating. In this TAP, repositories support multiple versions by +separating the metadata from versions with breaking changes into different +directories. These directories allow a client to choose the most recent +metadata they support while allowing for flexibility in how long a client will +take to update to a new spec version. Similarly, allowing the client to +maintain multiple versions allows the client to use metadata from multiple +directories that are not in communication and so do not coordinate an update to +the most recent TUF specification. + +To save space, target files should remain in the parent directory on the +repository. The metadata files (in directories according to their spec version) +will point to target files relative to the parent directory. # Specification -In order to allow clients and repositories to update TUF spec versions while maintaining support for old systems, this TAP defines a variety of procedures to be added to the update process. This section describes how version numbers will be defined for the TUF specification, what the repository and client must to do support specification updates, and some special cases. +In order to allow clients and repositories to update TUF spec versions while +maintaining support for old systems, this TAP defines a variety of procedures +to be added to the update process. This section describes how version numbers +will be defined for the TUF specification, what the repository and client must +to do support specification updates, and some special cases. ## Version Number Format -In order to make the meaning of a TUF version numbers change consistent and easily understood, this TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects. -In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then it will be part of a new major version. -Non backwards compatible versions add features that change the way that TUF processes updates, and need to be implemented on both the client and repository to maintain security and functionality. Examples of breaking changes are TAP 3 and TAP 8. -If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor version. These backwards compatible TAPs add additional features to TUF, but clients that do not have these features will still be able to securely and reliably perform updates from repositories that support the TAPs. There may be minor differences in which updates a client installs across minor versions. If a repository maintainer is worried about the impact of a minor spec change on clients, they may choose to wait to implement the change with a major specification version (for example, they could remain on version 2.4.6 after the release of 2.5). -Patches are used to fix typos and make small changes to existing features. More details about what constitutes a major, minor, or patch change can be found at https://semver.org/. +In order to make the meaning of a TUF version numbers change consistent and +easily understood, this TAP clarifies that the 'spec_version' field of root +metadata shall be based on [Semantic Versioning](https://semver.org/), with +version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used +in other open source projects. +In accordance with Semantic Versioning, breaking changes will only occur during +a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards +Compatibility section of a TAP should be used to determine whether the TAP +creates a breaking change. If the change is not backwards compatible, then +it will be part of a new major version. +Non backwards compatible versions add features that change the way that TUF +processes updates, and need to be implemented on both the client and repository +to maintain security and functionality. Examples of breaking changes are TAP 3 +and TAP 8. +If the change adds a new feature that is backwards compatible, for example in +TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor +version. These backwards compatible TAPs add additional features to TUF, but +clients that do not have these features will still be able to securely and +reliably perform updates from repositories that support the TAPs. There may be +minor differences in which updates a client installs across minor versions. If +a repository maintainer is worried about the impact of a minor spec change on +clients, they may choose to wait to implement the change with a major +specification version (for example, they could remain on version 2.4.6 after + the release of 2.5). +Patches are used to fix typos and make small changes to existing features. More +details about what constitutes a major, minor, or patch change can be found at +https://semver.org/. ### Changes to TAPs -In order to manage changes to TUF, TAPs shall be tied to a version of the TUF specification. Once a TAP is accepted the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include this field. +In order to manage changes to TUF, TAPs shall be tied to a version of the TUF +specification. Once a TAP is accepted the header should be updated to include +the first TUF version that will include the TAP. The Preamble Header +description in TAP 1 shall be updated to include this field. ## How a repository updates -In order to allow for the above use cases, repositories must support multiple TUF specification versions. In order to do so, this TAP proposes a new directory structure for repositories. - -When a repository manager chooses to update their repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. Target files will not be included in this directory. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. The root file should be signed by both the new root key and the current root key (the root key from the most recent metadata in the previous major spec version). Clients will now be able to use the new metadata files once their TUF spec versions are also updated. After an update to version 2.0.0, the repository structure may look like: +In order to allow for the above use cases, repositories must support multiple +TUF specification versions. In order to do so, this TAP proposes a new +directory structure for repositories. + +When a repository manager chooses to update their repository to a new major TUF +spec version, they create a new directory on the repository named for the major +version (for example 2.0.0). This directory will contain all metadata files +with the new spec version. Target files will not be included in this directory. +After creating the directory, the repository creates and signs root, snapshot, +timestamp, and top level targets metadata using the new TUF spec version and +places these metadata files in the directory. The root file should be signed by +both the new root key and the current root key (the root key from the most + recent metadata in the previous major spec version). Clients will now be able + to use the new metadata files once their TUF spec versions are also updated. + After an update to version 2.0.0, the repository structure may look like: ``` - Target files @@ -78,13 +200,34 @@ When a repository manager chooses to update their repository to a new major TUF |- metadata files ``` -If the repository is updated to a new minor or patch spec version, this shall be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the directory named 2.0.0. Minor version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. - -A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. The version can be phased out by no longer creating new metadata files in that directory. In order to allow clients to parse the root metadata chain, root metadata files should not be deleted even once a version is deprecated. - -In order to allow a client to find the current metadata files across spec versions, the version numbers used for consistent snapshots should be consistent across all supported spec versions. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the same consistent snapshot number must additionally use the same keys so that a client can find the next root file in whichever spec version they support. - -For existing TUF clients to continue operation while this TAP is implemented, repositories should store metadata from before TUF 1.0.0 in the top level repository (there will be no directory named 0.0.0). This allows existing clients to continue downloading metadata from the repository. So a TUF repository that updates to version 1.0.0 may look like: +If the repository is updated to a new minor or patch spec version, this shall +be done by uploading new metadata files in the new format to the proper +directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata +would go in the directory named 2.0.0. Minor version changes are backwards +compatible, so clients using version 2.0.0 will still be able to parse metadata +written using version 2.1.0. + +A repository may continue to support old major TUF spec versions by creating +metadata both in the old location and in the new directory. The repository may +maintain as many versions as the repository manager wishes. If there are +security concerns with an old spec version, that version should be phased out +as soon as possible. The version can be phased out by no longer creating new +metadata files in that directory. In order to allow clients to parse the root +metadata chain, root metadata files should not be deleted even once a version +is deprecated. + +In order to allow a client to find the current metadata files across spec +versions, the version numbers used for consistent snapshots should be +consistent across all supported spec versions. This means that there may be a +file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the +same consistent snapshot number must additionally use the same keys so that a +client can find the next root file in whichever spec version they support. + +For existing TUF clients to continue operation while this TAP is implemented, +repositories should store metadata from before TUF 1.0.0 in the top level +repository (there will be no directory named 0.0.0). This allows existing +clients to continue downloading metadata from the repository. So a TUF +repository that updates to version 1.0.0 may look like: ``` - metadata files @@ -96,61 +239,161 @@ For existing TUF clients to continue operation while this TAP is implemented, re TUF clients must make a few changes to support changes to the TUF specification. -In order to find compatible updates on a repository, a client must keep track of the TUF specification versions it supports. To do so, a global variable or other local storage option should contain the client spec version, or spec version range. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. +In order to find compatible updates on a repository, a client must keep track +of the TUF specification versions it supports. To do so, a global variable or +other local storage option should contain the client spec version, or spec +version range. For simplicity, this field should be formatted according to +Semantic Versioning so that it can be directly compared to the spec version in +root metadata. -For compatibility, TUF clients may maintain support for old versions of the TUF specification. This support can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. +For compatibility, TUF clients may maintain support for old versions of the TUF +specification. This support can be used if the client downloads metadata from a +repository or delegated role that does not support the current TUF +specification. To allow for this behavior, when a new version of the TUF client +is implemented it may contain the ability to call certain functions from the +old TUF client for parsing and validating metadata. ## Changes to the update process -When a TUF client downloads metadata from a repository, the client must determine which spec version to use for the download. To do this, the client looks for the highest supported numbered directory on the repository then determines compatibility with the following procedure: - -* If the highest number on the repository is equal to the client spec version, the client will use this directory to download metadata. -* If the highest number on the repository is less than the client spec version, the client may call functions from a previous spec version client to download the metadata. The client may support as many or as few versions as desired for the application. If a previous version is not available, the client shall report that an update is not possible due to an old spec version on the repository. -* If the highest number on the repository is higher than the client spec version, the client should report to the user that it is not using the most up to date TUF spec version then proceed with the directory that corresponds with the client spec version if available. If no such directory exists, the client terminates the update. - -Once the supported directory is determined, the client shall attempt the update using the metadata in this directory. - -For example, if a client has a spec version of 3.5 and a repository has directories for 2.0.0, 3.0.0, and 4.0.0, the client will report that spec version 4.0 is available, then download metadata from 3.0.0. This reporting is up to the discretion of an implementer, but it should be used to encourage updating the client to the most recent specification version. - -Alternatively, if the same client downloads metadata from a repository with directories 1.0.0 and 2.0.0, the client could download metadata from 2.0.0 using a 2.x version of the client. If a 2.x version is not available, the client will report that it is unable to perform an update. - -Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file saved on the client uses a version other than the supported version, the client will look for the next root file first in the supported version, then the previous versions until the next root file is found or the currently trusted root file's version is reached. All root files should be verified using the major version of the TUF client that corresponds with the major version of the root file. - -So, if the currently trusted root file is named 4.root.json and uses version 1.0.0 and the highest supported version is 3.0.0, the client will look for 5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the client will look for 6.root.json using the same process. To facilitate this, the client should maintain functions to parse root files from previous spec versions. If the client does not support the spec version of a root file, the client shall terminate the update and report the spec version mismatch. +When a TUF client downloads metadata from a repository, the client must +determine which spec version to use for the download. To do this, the client +looks for the highest supported numbered directory on the repository then +determines compatibility with the following procedure: + +* If the highest number on the repository is equal to the client spec version, +the client will use this directory to download metadata. +* If the highest number on the repository is less than the client spec version, +the client may call functions from a previous spec version client to download +the metadata. The client may support as many or as few versions as desired for +the application. If a previous version is not available, the client shall +report that an update is not possible due to an old spec version on the +repository. +* If the highest number on the repository is higher than the client spec +version, the client should report to the user that it is not using the most up +to date TUF spec version then proceed with the directory that corresponds with +the client spec version if available. If no such directory exists, the client +terminates the update. + +Once the supported directory is determined, the client shall attempt the update +using the metadata in this directory. + +For example, if a client has a spec version of 3.5 and a repository has +directories for 2.0.0, 3.0.0, and 4.0.0, the client will report that spec +version 4.0 is available, then download metadata from 3.0.0. This reporting is +up to the discretion of an implementer, but it should be used to encourage +updating the client to the most recent specification version. + +Alternatively, if the same client downloads metadata from a repository with +directories 1.0.0 and 2.0.0, the client could download metadata from 2.0.0 +using a 2.x version of the client. If a 2.x version is not available, the +client will report that it is unable to perform an update. + +Once the supported directory is determined, the client must validate root +metadata from this directory. If the currently trusted root file saved on the +client uses a version other than the supported version, the client will look +for the next root file first in the supported version, then the previous +versions until the next root file is found or the currently trusted root file's +version is reached. All root files should be verified using the major version +of the TUF client that corresponds with the major version of the root file. + +So, if the currently trusted root file is named 4.root.json and uses version +1.0.0 and the highest supported version is 3.0.0, the client will look for +5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the +client will look for 6.root.json using the same process. To facilitate this, +the client should maintain functions to parse root files from previous spec +versions. If the client does not support the spec version of a root file, the +client shall terminate the update and report the spec version mismatch. ## Changes to map file procedure -Metadata that is generated using different major versions of the TUF specification cannot be safely and reliable compared. To deal with this a TUF client that contains a map file for multiple repositories must perform an additional step to ensure a consensus is possible across repositories that are using different spec versions. In order for this procedure to work, all repositories should maintain metadata according to old spec versions as explained in [How a repository updates](#how-a-repository-updates). +Metadata that is generated using different major versions of the TUF +specification cannot be safely and reliable compared. To deal with this a TUF +client that contains a map file for multiple repositories must perform an +additional step to ensure a consensus is possible across repositories that are +using different spec versions. In order for this procedure to work, all +repositories should maintain metadata according to old spec versions as +explained in [How a repository updates](#how-a-repository-updates). -When parsing a mapping, the client should first look for a threshold as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) using the client's TUF spec version. If a threshold is not met with this spec version, the client should try with the previous major spec version, and so on until a threshold is reached or the client does not support the version. +When parsing a mapping, the client should first look for a threshold as +described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) +using the client's TUF spec version. If a threshold is not met with this spec +version, the client should try with the previous major spec version, and so on +until a threshold is reached or the client does not support the version. ## Changes to delegations -So that delegated targets can update to new spec versions, delegated targets metadata should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). - -In order to support this behavior, TUF clients should maintain the ability to parse targets metadata files from old spec versions. This can be done through functions that call sections of the old client version and translate these to be used by the new client. The client may make an old version obsolete if they choose, but will risk being unable to download new targets from delegations that are not updated. TUF implementers should decide how many old specification versions to support based on the expected usage of their implementation. +So that delegated targets can update to new spec versions, delegated targets +metadata should be stored in directories with their major spec version just +like top level metadata on the repository. For each targets metadata file, a +TUF client should download the highest supported version. This highest +supported version will be found using the same procedure as described in +[Changes to the update process](#changes-to-the-update-process). + +In order to support this behavior, TUF clients should maintain the ability to +parse targets metadata files from old spec versions. This can be done through +functions that call sections of the old client version and translate these to +be used by the new client. The client may make an old version obsolete if they +choose, but will risk being unable to download new targets from delegations +that are not updated. TUF implementers should decide how many old specification +versions to support based on the expected usage of their implementation. ## Updating Trusted Root Metadata -To allow for future updates after a major version change, the client must update their trusted root metadata to a root metadata that complies with the new spec version. To do so, the client first downloads and verifies the current version root metadata file. Once verified, this current version root metadata file must be stored as the trusted root metadata. In future updates, the client will start from the trusted root metadata when finding the next available update. +To allow for future updates after a major version change, the client must +update their trusted root metadata to a root metadata that complies with the +new spec version. To do so, the client first downloads and verifies the current +version root metadata file. Once verified, this current version root metadata +file must be stored as the trusted root metadata. In future updates, the client +will start from the trusted root metadata when finding the next available update. # Security Analysis -There should be minimal security impact. Ensuring that the client is up to date should improve security in the event that a security vulnerability is patched in a release of the spec. - -A downgrade attack on the TUF specification version may be possible if an attacker is able to block access to a directory on the repository. This would mean that a client would use metadata from a previous specification version when performing an update. However, the metadata would still have to be current and properly signed. To mitigate the damage from a downgrade attack in the case that a security flaw is found in a version of the TUF specification, the vulnerable version of the specification should no longer be supported on the repository (the metadata files should be revoked or allowed to expire). In addition, clients should be updated to no longer support a vulnerable specification version. - -The TUF version update system described in this TAP uses the root key rotation mechanism to ensure that the client only uses valid TUF metadata. The first time a new TUF version is used by the client, the client checks the repository's root metadata in the new version's format. If the root metadata is not signed by the previous trusted root key, the update does not proceed, and the client halts the update. In this way, updates to the repository's TUF version does not impact the security of an update. - -When the client updates to a new TUF version, it should use TUF to ensure that it downloads a valid new client. To do so, it should use its current TUF version to update to the new version of TUF. For example, if a client running TUF version 2.5.0 wants to update to version 3.0.0, the client should use metadata from TUF version 2.x.x to self update. Using this technique, the TUF client can ensure that it downloads the intended update when upgrading to a new TUF version. - -As always, authors of TUF clients should ensure that the client is a valid implementation of TUF and that no security flaws are introduced in code. In addition, supply chain security techniques (for example [in-toto](https://in-toto.io/in-toto.html)) should be used to ensure that vulnerabilities are not introduced into TUF clients between when they are written and installed. - -Note that an attacker on the same network could inject false metadata to block the client from updating. However this denial of service attack is always possible for an attacker on the network with the ability to alter network traffic. +There should be minimal security impact. Ensuring that the client is up to date +should improve security in the event that a security vulnerability is patched +in a release of the spec. + +A downgrade attack on the TUF specification version may be possible if an +attacker is able to block access to a directory on the repository. This would +mean that a client would use metadata from a previous specification version +when performing an update. However, the metadata would still have to be current +and properly signed. To mitigate the damage from a downgrade attack in the case +that a security flaw is found in a version of the TUF specification, the +vulnerable version of the specification should no longer be supported on the +repository (the metadata files should be revoked or allowed to expire). In +addition, clients should be updated to no longer support a vulnerable +specification version. + +The TUF version update system described in this TAP uses the root key rotation +mechanism to ensure that the client only uses valid TUF metadata. The first +time a new TUF version is used by the client, the client checks the +repository's root metadata in the new version's format. If the root metadata is +not signed by the previous trusted root key, the update does not proceed, and +the client halts the update. In this way, updates to the repository's TUF +version does not impact the security of an update. + +When the client updates to a new TUF version, it should use TUF to ensure that +it downloads a valid new client. To do so, it should use its current TUF +version to update to the new version of TUF. For example, if a client running +TUF version 2.5.0 wants to update to version 3.0.0, the client should use +metadata from TUF version 2.x.x to self update. Using this technique, the TUF +client can ensure that it downloads the intended update when upgrading to a new +TUF version. + +As always, authors of TUF clients should ensure that the client is a valid +implementation of TUF and that no security flaws are introduced in code. In +addition, supply chain security techniques (for example [in-toto](https://in-toto.io/in-toto.html)) +should be used to ensure that vulnerabilities are not introduced into TUF +clients between when they are written and installed. + +Note that an attacker on the same network could inject false metadata to block +the client from updating. However this denial of service attack is always +possible for an attacker on the network with the ability to alter network +traffic. # Backwards Compatibility -This TAP is backwards compatible and should be implemented on all repositories before any non-backwards compatible TAPs are released. +This TAP is backwards compatible and should be implemented on all repositories +before any non-backwards compatible TAPs are released. # Augmented Reference Implementation From 2b7e8d70da9da6cdb97b825c31f0a1edbe81440d Mon Sep 17 00:00:00 2001 From: marinamoore Date: Fri, 18 Oct 2019 15:57:11 -0400 Subject: [PATCH 22/40] clarifications to the multiple repositories case --- candidate-tuf-versions.md | 248 +++++++++++++++++++++----------------- 1 file changed, 137 insertions(+), 111 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 02405573..2d49add8 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -12,8 +12,9 @@ This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change atomically in order to continue -functioning as expected. This includes any feature that affects the parsing or -validation of metadata, or any other changes that means a client can no longer +functioning as expected. Breaking changes to the TUF specification +include any feature that affects the parsing or +validation of metadata and any other change that means a client can no longer safely and reliable perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification @@ -22,18 +23,24 @@ is compatible with the specification version of the metadata they download. This TAP adds support for breaking changes by making changes to both repositories and clients. Repositories will generate metadata for multiple TUF specification versions and place the metadata in a directory structure according -to the specification version. Clients will then determine the most recent -specification version support by both the client and repository and use this -specification version to perform an update. In order for clients to update to a +to the specification version. Additionally, clients may maintain support for +previous TUF specification versions (spec versions) by having functions that parse metadata +from previous TUF versions. Clients will then determine the most recent +specification version supported by both the client and repository and use this +specification version to perform the update. In order for clients to upgrade to a new TUF specification version, the process described in this TAP assumes that -TUF clients are able to upgrade. +TUF clients are able to self upgrade. + +For clarity, throughout this TAP upgrading refers to the process of moving from +one TUF specification version to another while updating refers to the process +of downloading and validating packages as specified by TUF. # Motivation The goal of this TAP is to prevent compatibility issues while allowing breaking changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include -changes that are not backwards compatible. That means that metadata generated -by a repository in the format of a new TUF specification version will be +changes that are not backwards compatible. When breaking changes are added to a +repository in a new specification version, metadata generated by the repository will be incompatible with clients parsing metadata formatted using an old version of the specification. Similarly, clients using a new TUF specification version will no longer be able to parse metadata from repositories using an old version @@ -50,20 +57,21 @@ updates to the TUF specification. ## Use case 1: A repository updates to a new TUF spec version As new features become available in the TUF specification, repositories may -wish to update to a new version of the specification to support these features. +wish to upgrade to a new version of the specification to support these features. This includes adding support for TAPs with breaking changes. In addition to the -ability to update to a new specification version, clients should have a way to +ability to upgrade to a new specification version, clients should have a way to determine what specification version the repository is using to ensure that -metadata can be safely and reliably verified. In addition, clients who are +metadata can be safely and reliably verified. Clients who are using an older TUF version should not use metadata from a new version that is not backwards compatible. -## Use case 2: A client is unable to update to a new TUF spec version +## Use case 2: A client is unable to upgrade to a new TUF spec version A client may wish to continue parsing metadata generated using an older version of the TUF specification after a new specification version is released. This -may be due to development time needed to update to the new TUF specification -version or compatibility issues. Repositories should be able to support clients +may be due to development time needed to upgrade to the new TUF specification +version or compatibility issues with other parts of the client system (for +example a signature algorithm that is not supported). Repositories should be able to support clients using older specification versions as well as any clients that are up to date by maintaining metadata in both the new and old specification formats. @@ -71,10 +79,10 @@ by maintaining metadata in both the new and old specification formats. A delegated role may make and sign metadata using an old version of the TUF specification even after the repository hosting the top level roles has been -updated. Different roles may be managed by people in different organizations -who are not able to coordinate updating to a new version of the TUF +upgraded. Different roles may be managed by people in different organizations +who are not able to coordinate upgrading to a new version of the TUF specification. To handle this, a client should be able to parse a delegation -that generates metatdata with a different TUF specification version than the +that generates metadata with a different TUF specification version than the repository. Therefore the client should be able to parse metadata from multiple TUF specification versions. @@ -82,19 +90,18 @@ TUF specification versions. As described in TAP 4, TUF clients may download metadata from multiple repositories. These repositories do not need to coordinate, and so should not -be required to update to a new TUF specification version at the same time. A +be required to upgrade to a new TUF specification version at the same time. A client should be able to use one repository that uses TUF version 1.0.0 and -another repository that uses TUF version 2.0.0. This functionality is only -needed for TUF clients that will download metadata from multiple repositories. +another repository that uses TUF version 2.0.0. ## Use case 5: Existing TUF clients Existing TUF clients will still expect to download and parse metadata that uses the current TUF specification version even once this TAP is implemented. Before this TAP there was no method for determining compatibility between a client and -repository, so existing TUF repositories should be able to continue to generate +repository, so existing TUF repositories should continue to generate backwards compatible metadata to support existing clients. This means that -non-backwards compatible TUF metadata must be stored in a different location of +non-backwards compatible TUF metadata must be stored in a different location on the repository from existing metadata. # Rationale @@ -102,37 +109,47 @@ the repository from existing metadata. To allow for updates to the TUF specification while considering all of the above use cases, a client must be able to parse metadata from multiple repositories or roles that are using different versions of the TUF -specification. However, secure and reliable verification of metadata requires -that the TUF client is able to compare metadata from multiple sources. This -comparison is not possible if changes to the TUF specification add or change -metadata fields. This TAP proposes a system for maintaining multiple TUF -versions while ensuring that a client is able to compare metadata from the same -version. - -As they do not affect compatibility, non-breaking changes may happen -independently on clients and repositories. Any specification versions that do -not contain breaking changes may be updated to at any time without worrying -about compatibility. This TAP clarifies how TUF version numbers are formatted -so that breaking changes are distinct from non-breaking changes. - -In order to allow clients to update independently of repositories, the -repositories need to continue support for old TUF versions for some period of -time after updating. In this TAP, repositories support multiple versions by +specification. In addition, a repository must support clients that parse +metadata with different TUF versions. This TAP proposes a system for allowing +TUF specification version changes on both clients and repositories. + +There are three possible approaches to managing versions between clients and +repositories. First, each repository could maintain multiple TUF versions while the +clients only maintain one version. In this case, TUF clients would not be able +to use metadata from multiple repositories if the repositories do not support +the same TUF version (Use Case 4). Second, each client could maintain multiple versions while +the repositories each maintain a single version. In this case, if a repository +upgrades to a new TUF version, clients will be unable to perform updates until +support for the new version is added (Use Case 2). The third option is to support multiple +versions on both clients and repositories. This option allows the clients and +repositories to be upgraded independently to support all use cases. + +Repositories need to continue support for old TUF versions for some period of +time after upgrading. In this TAP, repositories support multiple versions by separating the metadata from versions with breaking changes into different directories. These directories allow a client to choose the most recent -metadata they support while allowing for flexibility in how long a client will -take to update to a new spec version. Similarly, allowing the client to -maintain multiple versions allows the client to use metadata from multiple -directories that are not in communication and so do not coordinate an update to -the most recent TUF specification. - -To save space, target files should remain in the parent directory on the +metadata they support, allowing for flexibility in how long a client will +take to upgrade to a new spec version. To save space, target files should remain +in the parent directory on the repository. The metadata files (in directories according to their spec version) -will point to target files relative to the parent directory. +can point to target files relative to the parent directory. + +Similarly, clients need to maintain multiple specification versions to allow +for communication with various repositories. To do so, this TAP recommends that +clients maintain functions that can be used to validate metadata from various +TUF specification versions. These functions allow a client to maintain old +versions of the specification while still supporting the most recent version. + +As they do not affect compatibility, non-breaking changes can happen at any +time without affecting TUF's operation. To support this ability, TUF needs a +way to determine if new specification versions include any breaking changes. +This TAP clarifies how TUF version numbers are formatted +so that breaking changes are distinct from non-breaking changes. It does so +using a scheme popular across open source projects, Semantic Versioning. # Specification -In order to allow clients and repositories to update TUF spec versions while +In order to allow clients and repositories to upgrade to new TUF spec versions while maintaining support for old systems, this TAP defines a variety of procedures to be added to the update process. This section describes how version numbers will be defined for the TUF specification, what the repository and client must @@ -140,7 +157,7 @@ to do support specification updates, and some special cases. ## Version Number Format -In order to make the meaning of a TUF version numbers change consistent and +In order to make the meaning of a TUF version number change consistent and easily understood, this TAP clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used @@ -149,13 +166,13 @@ In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, then -it will be part of a new major version. +it will be part of a new major version of the TUF specification. Non backwards compatible versions add features that change the way that TUF -processes updates, and need to be implemented on both the client and repository -to maintain security and functionality. Examples of breaking changes are TAP 3 -and TAP 8. +processes updates, and need to be implemented on both clients and repositories +atomically to maintain security and functionality. Examples of breaking changes +are TAP 3 and TAP 8. If the change adds a new feature that is backwards compatible, for example in -TAP 4 and TAP 10, it is not a breaking change and should be part of a new minor +TAP 4 and TAP 10, it should be part of a new minor version. These backwards compatible TAPs add additional features to TUF, but clients that do not have these features will still be able to securely and reliably perform updates from repositories that support the TAPs. There may be @@ -170,17 +187,17 @@ https://semver.org/. ### Changes to TAPs In order to manage changes to TUF, TAPs shall be tied to a version of the TUF -specification. Once a TAP is accepted the header should be updated to include +specification. Once a TAP is accepted a TUF Version field in the header should be updated to include the first TUF version that will include the TAP. The Preamble Header -description in TAP 1 shall be updated to include this field. +description in TAP 1 shall be updated to include the TUF Version field. ## How a repository updates -In order to allow for the above use cases, repositories must support multiple +As described in the [Rationale](#rationale), repositories must support multiple TUF specification versions. In order to do so, this TAP proposes a new directory structure for repositories. -When a repository manager chooses to update their repository to a new major TUF +When a repository manager chooses to upgrade their repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. Target files will not be included in this directory. @@ -188,9 +205,9 @@ After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. The root file should be signed by both the new root key and the current root key (the root key from the most - recent metadata in the previous major spec version). Clients will now be able - to use the new metadata files once their TUF spec versions are also updated. - After an update to version 2.0.0, the repository structure may look like: +recent metadata in the previous major spec version). Clients will now be able +to use the new metadata files once their TUF spec versions are also updated. +After an update to version 2.0.0, the repository structure may look like: ``` - Target files @@ -203,7 +220,7 @@ both the new root key and the current root key (the root key from the most If the repository is updated to a new minor or patch spec version, this shall be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata -would go in the directory named 2.0.0. Minor version changes are backwards +would go in the directory named 2.0.0. Minor and patch version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. @@ -213,7 +230,7 @@ maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. The version can be phased out by no longer creating new metadata files in that directory. In order to allow clients to parse the root -metadata chain, root metadata files should not be deleted even once a version +metadata chain, root metadata files shall not be deleted even once a version is deprecated. In order to allow a client to find the current metadata files across spec @@ -223,16 +240,17 @@ file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the same consistent snapshot number must additionally use the same keys so that a client can find the next root file in whichever spec version they support. -For existing TUF clients to continue operation while this TAP is implemented, -repositories should store metadata from before TUF 1.0.0 in the top level -repository (there will be no directory named 0.0.0). This allows existing +For existing TUF clients to continue operation after this TAP is implemented, +repositories may store metadata from before TUF 1.0.0 in the top level +repository (with no directory named 0.0.0). This allows existing clients to continue downloading metadata from the repository. So a TUF -repository that updates to version 1.0.0 may look like: +repository that upgrades from version 0.12.0 to version 1.0.0 may look like: ``` -- metadata files +- Targets files +- 0.12.0 metadata files - 1.0.0 - |- metadata files + |- 1.0.0 metadata files ``` ## Changes to TUF clients @@ -246,32 +264,38 @@ version range. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. -For compatibility, TUF clients may maintain support for old versions of the TUF +TUF clients may maintain support for pervious versions of the TUF specification. This support can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the -old TUF client for parsing and validating metadata. +old TUF client for parsing and validating metadata. The client may make an old +specification version obsolete if they choose by removing the functions for that specification +version, but will risk being unable to download new targets from delegations +that use the old specification version. TUF implementers should decide how many old specification +versions to support based on the expected usage of their implementation. ## Changes to the update process When a TUF client downloads metadata from a repository, the client must determine which spec version to use for the download. To do this, the client -looks for the highest supported numbered directory on the repository then -determines compatibility with the following procedure: +looks for the highest supported version on the repository using the following procedure: -* If the highest number on the repository is equal to the client spec version, +* The client determines the latest version available on the repository by +looking for the directory with the largest version number. +* If the latest version on the repository is equal to the client spec version, the client will use this directory to download metadata. -* If the highest number on the repository is less than the client spec version, +* If the latest version on the repository is a version before the client spec version, the client may call functions from a previous spec version client to download the metadata. The client may support as many or as few versions as desired for -the application. If a previous version is not available, the client shall +the application. If the previous version is not available, the client shall report that an update is not possible due to an old spec version on the repository. -* If the highest number on the repository is higher than the client spec +* If the latest version on the repository is higher than the client spec version, the client should report to the user that it is not using the most up -to date TUF spec version then proceed with the directory that corresponds with -the client spec version if available. If no such directory exists, the client +to date TUF spec version (the method of reporting is left to the discretion of +the client) then proceed with the directory that corresponds with +the latest client spec version if available. If no such directory exists, the client terminates the update. Once the supported directory is determined, the client shall attempt the update @@ -285,12 +309,12 @@ updating the client to the most recent specification version. Alternatively, if the same client downloads metadata from a repository with directories 1.0.0 and 2.0.0, the client could download metadata from 2.0.0 -using a 2.x version of the client. If a 2.x version is not available, the +using a 2.x version of the client. If 2.x is not supported by the client, the client will report that it is unable to perform an update. Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file saved on the -client uses a version other than the supported version, the client will look +client uses a spec version other than the supported version, the client will look for the next root file first in the supported version, then the previous versions until the next root file is found or the currently trusted root file's version is reached. All root files should be verified using the major version @@ -304,39 +328,41 @@ the client should maintain functions to parse root files from previous spec versions. If the client does not support the spec version of a root file, the client shall terminate the update and report the spec version mismatch. -## Changes to map file procedure - -Metadata that is generated using different major versions of the TUF -specification cannot be safely and reliable compared. To deal with this a TUF -client that contains a map file for multiple repositories must perform an -additional step to ensure a consensus is possible across repositories that are -using different spec versions. In order for this procedure to work, all -repositories should maintain metadata according to old spec versions as -explained in [How a repository updates](#how-a-repository-updates). - -When parsing a mapping, the client should first look for a threshold as -described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) -using the client's TUF spec version. If a threshold is not met with this spec -version, the client should try with the previous major spec version, and so on -until a threshold is reached or the client does not support the version. +## Multiple Repositories + +A TUF client that performs updates using multiple repositories may need to +access repositories that use different TUF specification versions. When +comparing the metadata from multiple repositories, the goal is to ensure that +the target file that is downloaded is verified by all repositories. Therefore +the client can perform validation for each repository independently, then +compare the results of the validation. + +To do so, the client needs to ensure that the metadata from each +repository is valid for the given targets file. +If the repositories use different versions of the TUF specification, +the client should use the TUF version that corresponds to each repository to +validate the metadata from that repository, then compare the results. The +update is only valid if valid metadata from both repositories points to the same +target file, and this target file matches the hashes provided by each repository. +Note that different TUF versions may use different hashing algorithms. If this +is the case, both hashes should be verified independently. ## Changes to delegations -So that delegated targets can update to new spec versions, delegated targets -metadata should be stored in directories with their major spec version just +The TUF version used by a delegation does not need to match the TUF version +used by the top level metadata. The TUF client is responsible for parsing the +metadata from the top-level roles and delegations using the appropriate +specification version, similar to the process used for multiple repositories. +If there is no compatible spec version between the client +and delegation, the client should report this and terminate the update. + +So that delegated targets can upgrade to new spec versions, delegated targets +metadata should be stored in directories corresponding to their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). -In order to support this behavior, TUF clients should maintain the ability to -parse targets metadata files from old spec versions. This can be done through -functions that call sections of the old client version and translate these to -be used by the new client. The client may make an old version obsolete if they -choose, but will risk being unable to download new targets from delegations -that are not updated. TUF implementers should decide how many old specification -versions to support based on the expected usage of their implementation. - ## Updating Trusted Root Metadata To allow for future updates after a major version change, the client must @@ -360,10 +386,10 @@ and properly signed. To mitigate the damage from a downgrade attack in the case that a security flaw is found in a version of the TUF specification, the vulnerable version of the specification should no longer be supported on the repository (the metadata files should be revoked or allowed to expire). In -addition, clients should be updated to no longer support a vulnerable +addition, clients should be upgraded to no longer support a vulnerable specification version. -The TUF version update system described in this TAP uses the root key rotation +The TUF version upgrade system described in this TAP uses the root key rotation mechanism to ensure that the client only uses valid TUF metadata. The first time a new TUF version is used by the client, the client checks the repository's root metadata in the new version's format. If the root metadata is @@ -371,10 +397,10 @@ not signed by the previous trusted root key, the update does not proceed, and the client halts the update. In this way, updates to the repository's TUF version does not impact the security of an update. -When the client updates to a new TUF version, it should use TUF to ensure that +When a client upgrades to a new TUF version, it should use TUF to ensure that it downloads a valid new client. To do so, it should use its current TUF -version to update to the new version of TUF. For example, if a client running -TUF version 2.5.0 wants to update to version 3.0.0, the client should use +version to upgrade to the new version of TUF. For example, if a client running +TUF version 2.5.0 wants to upgrade to version 3.0.0, the client should use metadata from TUF version 2.x.x to self update. Using this technique, the TUF client can ensure that it downloads the intended update when upgrading to a new TUF version. @@ -386,7 +412,7 @@ should be used to ensure that vulnerabilities are not introduced into TUF clients between when they are written and installed. Note that an attacker on the same network could inject false metadata to block -the client from updating. However this denial of service attack is always +a client from updating. However this denial of service attack is always possible for an attacker on the network with the ability to alter network traffic. From c9912d52fdcb18be6fade95c429200db029530a2 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 31 Oct 2019 15:19:58 -0400 Subject: [PATCH 23/40] rework intro sections --- candidate-tuf-versions.md | 265 +++++++++++++++++++++----------------- 1 file changed, 147 insertions(+), 118 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 2d49add8..cba1aa6d 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -9,122 +9,140 @@ # Abstract -This TAP clarifies how to manage updates to the TUF specification that include -non-backwards compatible (breaking) changes. A breaking change means that a -client and repository must implement the change atomically in order to continue -functioning as expected. Breaking changes to the TUF specification -include any feature that affects the parsing or -validation of metadata and any other change that means a client can no longer -safely and reliable perform an update if the repository is updated out of sync -with the client. To facilitate finding breaking changes, this TAP defines a -procedure for TUF clients to ensure that their version of the TUF specification -is compatible with the specification version of the metadata they download. - -This TAP adds support for breaking changes by making changes to both -repositories and clients. Repositories will generate metadata for multiple TUF -specification versions and place the metadata in a directory structure according -to the specification version. Additionally, clients may maintain support for -previous TUF specification versions (spec versions) by having functions that parse metadata -from previous TUF versions. Clients will then determine the most recent -specification version supported by both the client and repository and use this -specification version to perform the update. In order for clients to upgrade to a -new TUF specification version, the process described in this TAP assumes that -TUF clients are able to self upgrade. - -For clarity, throughout this TAP upgrading refers to the process of moving from -one TUF specification version to another while updating refers to the process -of downloading and validating packages as specified by TUF. +TUF currently does not support breaking changes to the specification. TUF +clients and repositories have to use the same version of the specification in +order for metadata generated on a repository to be safely and reliably +verified on a client. This TAP allows TUF implementations to manage updates +independently on clients and repositories so that TUF can continue to function +after a breaking changes is implemented on either the client or repository. In +order for clients to upgrade to a new TUF specification version, the process +described in this TAP assumes that TUF clients are able to self upgrade. + +To manage breaking changes, this TAP requires a change to both the way the TUF +specification manages versions and how updates are performed. Together these +changes ensure that TUF clients are able to download metadata and updates from a +compatible repository. First, this TAP requires that the TUF specification use +Semantic Versioning to separate breaking changes from non-breaking changes. +Second, this TAP requires that both clients and repositories maintain support +for old versions of the TUF specification to allow reliable updates to continue +throughout the process of updating the TUF specification. To do so, repositories +will generate metadata for multiple TUF specification versions and place the +metadata in a directory structure according to the specification version. In +addition, clients will include support for metadata from previous TUF versions. +To determine the version to use for an update, clients will determine the most +recent specification version supported by both the client and repository. # Motivation -The goal of this TAP is to prevent compatibility issues while allowing breaking -changes to the TUF specification. Various TAPs, including TAPs 3 and 8 include -changes that are not backwards compatible. When breaking changes are added to a -repository in a new specification version, metadata generated by the repository will be -incompatible with clients parsing metadata formatted using an old version of -the specification. Similarly, clients using a new TUF specification version -will no longer be able to parse metadata from repositories using an old version -of the TUF specification. Because of this, clients need to check that the -version of the specification that they are using is compatible with the version -that a repository is using before validating metadata. For example, a client -uses TUF version 2.0.0 will not be able to correctly parse metadata generated -by a repository using TUF version 3.0.0 (a new version) or 1.0.0 (an old -version). - -This TAP aims to support a variety of use cases when it comes to managing -updates to the TUF specification. +Various TAPs, including TAPs 3 and 8, propose changes that are not backwards +compatible. Because these changes are not compatible with previous TUF versions, +current TUF implementations that use the existing specification are not able to +use these new features. This TAP will support adding TAPs with non-backwards +compatible (breaking) changes to existing TUF implementations. To do so, +breaking changes need to be handled for a variety of use cases. ## Use case 1: A repository updates to a new TUF spec version As new features become available in the TUF specification, repositories may -wish to upgrade to a new version of the specification to support these features. -This includes adding support for TAPs with breaking changes. In addition to the -ability to upgrade to a new specification version, clients should have a way to -determine what specification version the repository is using to ensure that -metadata can be safely and reliably verified. Clients who are -using an older TUF version should not use metadata from a new version that is -not backwards compatible. - -## Use case 2: A client is unable to upgrade to a new TUF spec version - -A client may wish to continue parsing metadata generated using an older version -of the TUF specification after a new specification version is released. This -may be due to development time needed to upgrade to the new TUF specification -version or compatibility issues with other parts of the client system (for -example a signature algorithm that is not supported). Repositories should be able to support clients -using older specification versions as well as any clients that are up to date -by maintaining metadata in both the new and old specification formats. +wish to upgrade to a new version to support these features. +This could include adding TAPs with breaking changes. When the repository +upgrades, clients that use an older version of the TUF spec will no longer be +able to safely parse metadata from the repository. Because of this, a couple of +issues need to be addressed. First, clients need a way to determine whether +their version of the TUF spec is compatible with the version used by the +repository. This ensures that clients always parse metadata according to the +spec version that generated that metadata. Additionally, the client may not be +not able to upgrade immediately to the spec version used on the repository due +to development time or other constraints. The client needs some way to process +updates after this upgrade occurs on the repository to ensure reliable access +to updates. + +## Use case 2: A client updates to a new TUF spec version + +Just as a repository may be upgraded, a TUF client may wish to upgrade to a new +TUF spec version to use new features. When the client implements an upgrade that +includes a breaking change, it cannot be sure that all repositories have also +upgraded to the new spec version. Therefore after the client upgrades they must +ensure that any metadata downloaded from a repository was generated using the +new spec version. If the repository is using an older spec version, the client +should have some way to allow the update to proceed. ## Use case 3: A delegated targets role uses a different TUF spec version -A delegated role may make and sign metadata using an old version of the TUF -specification even after the repository hosting the top level roles has been -upgraded. Different roles may be managed by people in different organizations +A delegated role may make and sign metadata using different version of the TUF +specification than the repository hosting the top level roles. Delegated roles +and the top level repository may be managed by people in different organizations who are not able to coordinate upgrading to a new version of the TUF -specification. To handle this, a client should be able to parse a delegation -that generates metadata with a different TUF specification version than the -repository. Therefore the client should be able to parse metadata from multiple -TUF specification versions. +specification. To handle this, a client should be able to parse +delegations that generate metadata with a different TUF specification version +than the repository. ## Use case 4: A client downloads metadata from multiple repositories As described in TAP 4, TUF clients may download metadata from multiple -repositories. These repositories do not need to coordinate, and so should not -be required to upgrade to a new TUF specification version at the same time. A -client should be able to use one repository that uses TUF version 1.0.0 and -another repository that uses TUF version 2.0.0. +repositories. These repositories do not coordinate with each other, and so may +not upgrade to a new TUF specification version at the same time. A +client should be able to use multiple repositories that do not use the same TUF +spec version. For example, a client may download metadata from one repository +that uses TUF version 1.0.0 and another repository that uses TUF version 2.0.0. -## Use case 5: Existing TUF clients +## Use case 5: Allowing Backwards Compatibility Existing TUF clients will still expect to download and parse metadata that uses -the current TUF specification version even once this TAP is implemented. Before -this TAP there was no method for determining compatibility between a client and -repository, so existing TUF repositories should continue to generate -backwards compatible metadata to support existing clients. This means that -non-backwards compatible TUF metadata must be stored in a different location on -the repository from existing metadata. +the current TUF specification version even after this TAP is implemented on +repositories. Before this TAP there was no method for determining compatibility +between a client and a repository, so existing TUF repositories should continue +to generate metadata using the current method to support existing clients. This +means that the mechanism for allowing backwards compatible updates must itself +be backwards compatible. # Rationale -To allow for updates to the TUF specification while considering all of the -above use cases, a client must be able to parse metadata from multiple -repositories or roles that are using different versions of the TUF -specification. In addition, a repository must support clients that parse -metadata with different TUF versions. This TAP proposes a system for allowing -TUF specification version changes on both clients and repositories. - -There are three possible approaches to managing versions between clients and -repositories. First, each repository could maintain multiple TUF versions while the -clients only maintain one version. In this case, TUF clients would not be able -to use metadata from multiple repositories if the repositories do not support -the same TUF version (Use Case 4). Second, each client could maintain multiple versions while +In order to allow for breaking changes to the TUF specification as discussed +above, there are two main issues that need to be addressed. First, clients need +to be able to determine whether the TUF spec version they are using is +compatible with the spec version from which they are receiving metadata. +This ensures that clients parse metadata according to the correct spec +version to prevent errors from any changes to the +metadata in the new version. Second, clients need to use this information to +determine where to access metadata that is compatible with their version of the +TUF specification. + +To address the first issue, this TAP standardizes the spec version field to use specification +separate breaking changes from non-breaking changes and ensure that the field +can be compared across clients and repositories. Separating out non-breaking +changes allows these backwards compatible changes to happen at any time without +affecting TUF's operation. Therefore clients and repositories can still +coordinate after a non-breaking change occurs. One common framework used to +separate versions by type is Semantic Versioning. Semantic Versioning is a +versioning scheme popular across open source projects that breaks versions into +segments by how critical the changes are. This format allows breaking changes to +increment the MAJOR version number, while non-breaking changes increment the +MINOR version number. This allows a client to +easily separate breaking changes from non-breaking changes. In addition, +Semantic Versioning has a standard way to format version numbers so that they +can be parsed and compared across implementations. + +Once the spec version is determined, clients and repositories need a process for +ensuring compatibility. There are three possible approaches to managing versions +between clients and repositories that we considered. First, each repository +could maintain multiple TUF versions while the clients only maintain one +version. In this case, TUF clients would not be able to use metadata from +multiple repositories if the repositories do not support the same TUF version +(Use Case 4). Second, each client could maintain multiple versions while the repositories each maintain a single version. In this case, if a repository upgrades to a new TUF version, clients will be unable to perform updates until -support for the new version is added (Use Case 2). The third option is to support multiple +support for the new version is added (Use Case 1). The third option is to support multiple versions on both clients and repositories. This option allows the clients and repositories to be upgraded independently to support all use cases. -Repositories need to continue support for old TUF versions for some period of +In picking this third option, supporting version changes on both clients and repositories, +both clients and repositories will need to maintain multiple versions of the +TUF spec. This will require changes to the way repositories store metadata and +clients parse this metadata. + +For the repositories this means that repositories need to continue support for +old TUF versions for some period of time after upgrading. In this TAP, repositories support multiple versions by separating the metadata from versions with breaking changes into different directories. These directories allow a client to choose the most recent @@ -134,34 +152,34 @@ in the parent directory on the repository. The metadata files (in directories according to their spec version) can point to target files relative to the parent directory. -Similarly, clients need to maintain multiple specification versions to allow +On the client side, the client is +responsible for maintaining multiple specification versions to allow for communication with various repositories. To do so, this TAP recommends that -clients maintain functions that can be used to validate metadata from various +clients maintain functions that can be used to validate metadata from previous TUF specification versions. These functions allow a client to maintain old versions of the specification while still supporting the most recent version. -As they do not affect compatibility, non-breaking changes can happen at any -time without affecting TUF's operation. To support this ability, TUF needs a -way to determine if new specification versions include any breaking changes. -This TAP clarifies how TUF version numbers are formatted -so that breaking changes are distinct from non-breaking changes. It does so -using a scheme popular across open source projects, Semantic Versioning. # Specification -In order to allow clients and repositories to upgrade to new TUF spec versions while -maintaining support for old systems, this TAP defines a variety of procedures -to be added to the update process. This section describes how version numbers -will be defined for the TUF specification, what the repository and client must -to do support specification updates, and some special cases. +This TAP defines a variety of procedures to be added to the update process that +will allow for breaking changes to the TUF specification. These processes occur +on both clients and repos and include a change to how version numbers are +determined, new directories on repositories, new metadata parsing functions on +clients, and a new process for finding compatible metadata. + +For clarity, throughout this TAP upgrading refers to the process of moving from +one TUF specification version to another while updating refers to the process +of downloading and validating packages as specified by TUF. ## Version Number Format -In order to make the meaning of a TUF version number change consistent and -easily understood, this TAP clarifies that the 'spec_version' field of root +Clarifies that the 'spec_version' field of root metadata shall be based on [Semantic Versioning](https://semver.org/), with -version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used -in other open source projects. +version numbers in the format MAJOR.MINOR.PATCH. + +This is a standard format used in other open source projects and makes the +meaning of a TUF version number change consistent and easily understood. In accordance with Semantic Versioning, breaking changes will only occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards Compatibility section of a TAP should be used to determine whether the TAP @@ -187,17 +205,21 @@ https://semver.org/. ### Changes to TAPs In order to manage changes to TUF, TAPs shall be tied to a version of the TUF -specification. Once a TAP is accepted a TUF Version field in the header should be updated to include +specification. + +Once a TAP is accepted a TUF Version field in the header should be updated to include the first TUF version that will include the TAP. The Preamble Header description in TAP 1 shall be updated to include the TUF Version field. ## How a repository updates +Repositories will add metadata for new TUF specification versions in new +directories. + As described in the [Rationale](#rationale), repositories must support multiple TUF specification versions. In order to do so, this TAP proposes a new -directory structure for repositories. - -When a repository manager chooses to upgrade their repository to a new major TUF +directory structure for repositories. When a repository manager chooses to +upgrade their repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. Target files will not be included in this directory. @@ -255,7 +277,8 @@ repository that upgrades from version 0.12.0 to version 1.0.0 may look like: ## Changes to TUF clients -TUF clients must make a few changes to support changes to the TUF specification. +TUF clients must store the TUF specification versions they support and may add +functions to maintain old versions of the TUF specification. In order to find compatible updates on a repository, a client must keep track of the TUF specification versions it supports. To do so, a global variable or @@ -328,7 +351,9 @@ the client should maintain functions to parse root files from previous spec versions. If the client does not support the spec version of a root file, the client shall terminate the update and report the spec version mismatch. -## Multiple Repositories +## Special Cases + +### Multiple Repositories A TUF client that performs updates using multiple repositories may need to access repositories that use different TUF specification versions. When @@ -347,7 +372,7 @@ target file, and this target file matches the hashes provided by each repository Note that different TUF versions may use different hashing algorithms. If this is the case, both hashes should be verified independently. -## Changes to delegations +### Changes to delegations The TUF version used by a delegation does not need to match the TUF version used by the top level metadata. The TUF client is responsible for parsing the @@ -363,7 +388,7 @@ TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). -## Updating Trusted Root Metadata +### Updating Trusted Root Metadata To allow for future updates after a major version change, the client must update their trusted root metadata to a root metadata that complies with the @@ -374,10 +399,11 @@ will start from the trusted root metadata when finding the next available update # Security Analysis -There should be minimal security impact. Ensuring that the client is up to date -should improve security in the event that a security vulnerability is patched -in a release of the spec. +Overall, this TAP will increase the security of TUF systems by allowing for +upgrades to TUF clients and repositories. However, there are a few attacks and +changes to the security model that should be discussed. +## Downgrade Attack A downgrade attack on the TUF specification version may be possible if an attacker is able to block access to a directory on the repository. This would mean that a client would use metadata from a previous specification version @@ -389,6 +415,7 @@ repository (the metadata files should be revoked or allowed to expire). In addition, clients should be upgraded to no longer support a vulnerable specification version. +## Root Key Rotation The TUF version upgrade system described in this TAP uses the root key rotation mechanism to ensure that the client only uses valid TUF metadata. The first time a new TUF version is used by the client, the client checks the @@ -397,6 +424,7 @@ not signed by the previous trusted root key, the update does not proceed, and the client halts the update. In this way, updates to the repository's TUF version does not impact the security of an update. +## How a client upgrades When a client upgrades to a new TUF version, it should use TUF to ensure that it downloads a valid new client. To do so, it should use its current TUF version to upgrade to the new version of TUF. For example, if a client running @@ -405,6 +433,7 @@ metadata from TUF version 2.x.x to self update. Using this technique, the TUF client can ensure that it downloads the intended update when upgrading to a new TUF version. +## Other considerations As always, authors of TUF clients should ensure that the client is a valid implementation of TUF and that no security flaws are introduced in code. In addition, supply chain security techniques (for example [in-toto](https://in-toto.io/in-toto.html)) From c1921b7b004f7462f95f3d10da2b7606583d237f Mon Sep 17 00:00:00 2001 From: marinamoore Date: Wed, 6 Nov 2019 16:03:16 -0500 Subject: [PATCH 24/40] minor edits and fixes to abstract and rationale --- candidate-tuf-versions.md | 579 +++++++++++++++++++------------------- 1 file changed, 289 insertions(+), 290 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index cba1aa6d..7b932ec8 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -9,207 +9,208 @@ # Abstract -TUF currently does not support breaking changes to the specification. TUF -clients and repositories have to use the same version of the specification in -order for metadata generated on a repository to be safely and reliably -verified on a client. This TAP allows TUF implementations to manage updates -independently on clients and repositories so that TUF can continue to function -after a breaking changes is implemented on either the client or repository. In -order for clients to upgrade to a new TUF specification version, the process -described in this TAP assumes that TUF clients are able to self upgrade. - -To manage breaking changes, this TAP requires a change to both the way the TUF -specification manages versions and how updates are performed. Together these -changes ensure that TUF clients are able to download metadata and updates from a -compatible repository. First, this TAP requires that the TUF specification use -Semantic Versioning to separate breaking changes from non-breaking changes. -Second, this TAP requires that both clients and repositories maintain support -for old versions of the TUF specification to allow reliable updates to continue -throughout the process of updating the TUF specification. To do so, repositories -will generate metadata for multiple TUF specification versions and place the -metadata in a directory structure according to the specification version. In +The TUF specification does not currently support breaking changes or changes +that are not backwards compatible. If a repository and a client are not using +the same version of the TUF specification, metadata can not be safely and +reliably verified. This TAP addresses this clash of versions by allowing TUF +implementations to independently manage updates on clients and repositories. +This in turn ensures that TUF will continue functioning after breaking changes +are implemented. + +To manage breaking changes without loss of functionality, this TAP requires two +changes: one to the way the TUF specification manages versions, and the other to +how it performs updates. The former is accomplished by having this TAP require +that the specification use Semantic Versioning to separate breaking from +non-breaking changes. And the latter is achieved by requiring both clients and +repositories to maintain support for older versions of the TUF specification so +reliable updates can continue throughout the process of the specification +upgrade. To do so, repositories will generate metadata for multiple TUF +specification versions and maintain them in an accessible directory. In addition, clients will include support for metadata from previous TUF versions. -To determine the version to use for an update, clients will determine the most -recent specification version supported by both the client and repository. +To determine the version to use for an update, clients will select the most +recent specification version supported by both the client and the repository. + # Motivation Various TAPs, including TAPs 3 and 8, propose changes that are not backwards compatible. Because these changes are not compatible with previous TUF versions, -current TUF implementations that use the existing specification are not able to -use these new features. This TAP will support adding TAPs with non-backwards -compatible (breaking) changes to existing TUF implementations. To do so, -breaking changes need to be handled for a variety of use cases. +current implementations that use the existing specification can not access the +new features in these TAPs. By creating a way to deal with non-backwards +compatible (breaking) changes, TUF will be able to handle a variety of use +cases, including those that appear below. ## Use case 1: A repository updates to a new TUF spec version -As new features become available in the TUF specification, repositories may -wish to upgrade to a new version to support these features. -This could include adding TAPs with breaking changes. When the repository -upgrades, clients that use an older version of the TUF spec will no longer be -able to safely parse metadata from the repository. Because of this, a couple of -issues need to be addressed. First, clients need a way to determine whether -their version of the TUF spec is compatible with the version used by the -repository. This ensures that clients always parse metadata according to the -spec version that generated that metadata. Additionally, the client may not be -not able to upgrade immediately to the spec version used on the repository due -to development time or other constraints. The client needs some way to process -updates after this upgrade occurs on the repository to ensure reliable access -to updates. +As new features become available in the TUF specification, repositories may wish +to upgrade to a new version. This could include adding TAPs with breaking +changes. When the repository upgrades, clients that use an older version of the +TUF spec will no longer be able to safely parse metadata from the repository. +To resolve this problem, clients will first need a way to determine whether +their version of TUF spec is compatible with the version used by the repository. +This is to ensure that clients always parse metadata according to the version +that generated that metadata. Then, they will need some way to process updates +after this upgrade occurs on the repository to ensure reliable access to +updates, even if they are not able to upgrade immediately due to development +time or other constraints. ## Use case 2: A client updates to a new TUF spec version Just as a repository may be upgraded, a TUF client may wish to upgrade to a new TUF spec version to use new features. When the client implements an upgrade that includes a breaking change, it cannot be sure that all repositories have also -upgraded to the new spec version. Therefore after the client upgrades they must -ensure that any metadata downloaded from a repository was generated using the -new spec version. If the repository is using an older spec version, the client +upgraded to the new specification version. Therefore, after the client upgrades +they must ensure that any metadata downloaded from a repository was generated +using the new version. If the repository is using an older version, the client should have some way to allow the update to proceed. -## Use case 3: A delegated targets role uses a different TUF spec version +## Use case 3: A delegated targets role uses a different TUF spec version than the repository -A delegated role may make and sign metadata using different version of the TUF +A delegated role may make and sign metadata using a different version of the TUF specification than the repository hosting the top level roles. Delegated roles and the top level repository may be managed by people in different organizations who are not able to coordinate upgrading to a new version of the TUF -specification. To handle this, a client should be able to parse -delegations that generate metadata with a different TUF specification version -than the repository. +specification. In this case, a client should be able to parse delegations that +generate metadata with a different TUF specification version than the repository. ## Use case 4: A client downloads metadata from multiple repositories As described in TAP 4, TUF clients may download metadata from multiple repositories. These repositories do not coordinate with each other, and so may -not upgrade to a new TUF specification version at the same time. A -client should be able to use multiple repositories that do not use the same TUF -spec version. For example, a client may download metadata from one repository -that uses TUF version 1.0.0 and another repository that uses TUF version 2.0.0. +not upgrade to a new TUF specification version at the same time. A client should +be able to use multiple repositories that do not use the same version of TUF. +For example, a client may download metadata from one repository that uses TUF +version 1.0.0 and another repository that uses version 2.0.0. ## Use case 5: Allowing Backwards Compatibility Existing TUF clients will still expect to download and parse metadata that uses -the current TUF specification version even after this TAP is implemented on +the current TUF specification version, even after this TAP is implemented on repositories. Before this TAP there was no method for determining compatibility between a client and a repository, so existing TUF repositories should continue to generate metadata using the current method to support existing clients. This -means that the mechanism for allowing backwards compatible updates must itself +means that this mechanism for allowing backwards compatible updates must itself be backwards compatible. # Rationale -In order to allow for breaking changes to the TUF specification as discussed -above, there are two main issues that need to be addressed. First, clients need -to be able to determine whether the TUF spec version they are using is -compatible with the spec version from which they are receiving metadata. -This ensures that clients parse metadata according to the correct spec -version to prevent errors from any changes to the -metadata in the new version. Second, clients need to use this information to -determine where to access metadata that is compatible with their version of the -TUF specification. - -To address the first issue, this TAP standardizes the spec version field to use specification -separate breaking changes from non-breaking changes and ensure that the field -can be compared across clients and repositories. Separating out non-breaking -changes allows these backwards compatible changes to happen at any time without -affecting TUF's operation. Therefore clients and repositories can still -coordinate after a non-breaking change occurs. One common framework used to -separate versions by type is Semantic Versioning. Semantic Versioning is a -versioning scheme popular across open source projects that breaks versions into -segments by how critical the changes are. This format allows breaking changes to -increment the MAJOR version number, while non-breaking changes increment the -MINOR version number. This allows a client to -easily separate breaking changes from non-breaking changes. In addition, -Semantic Versioning has a standard way to format version numbers so that they -can be parsed and compared across implementations. - -Once the spec version is determined, clients and repositories need a process for -ensuring compatibility. There are three possible approaches to managing versions -between clients and repositories that we considered. First, each repository -could maintain multiple TUF versions while the clients only maintain one -version. In this case, TUF clients would not be able to use metadata from -multiple repositories if the repositories do not support the same TUF version -(Use Case 4). Second, each client could maintain multiple versions while -the repositories each maintain a single version. In this case, if a repository -upgrades to a new TUF version, clients will be unable to perform updates until -support for the new version is added (Use Case 1). The third option is to support multiple -versions on both clients and repositories. This option allows the clients and -repositories to be upgraded independently to support all use cases. - -In picking this third option, supporting version changes on both clients and repositories, -both clients and repositories will need to maintain multiple versions of the -TUF spec. This will require changes to the way repositories store metadata and -clients parse this metadata. - -For the repositories this means that repositories need to continue support for -old TUF versions for some period of -time after upgrading. In this TAP, repositories support multiple versions by -separating the metadata from versions with breaking changes into different -directories. These directories allow a client to choose the most recent -metadata they support, allowing for flexibility in how long a client will -take to upgrade to a new spec version. To save space, target files should remain -in the parent directory on the -repository. The metadata files (in directories according to their spec version) -can point to target files relative to the parent directory. - -On the client side, the client is -responsible for maintaining multiple specification versions to allow -for communication with various repositories. To do so, this TAP recommends that -clients maintain functions that can be used to validate metadata from previous -TUF specification versions. These functions allow a client to maintain old -versions of the specification while still supporting the most recent version. +We propose this TAP because as TUF continues to evolve, the need for TUF clients +and repositories to upgrade will grow. This sets up the potential for clients +that are no longer able to perform updates because they cannot parse the +metadata generated by repositories, or for clients to install the wrong images +due to changes in TUF. Both of these issues prevent clients from installing the +images intended by repository managers, and could lead to critical security or +functionality problems. TUF clients and repositories need to be able to upgrade +to new versions without preventing secure and reliable access to software +updates. + +In trying to create more flexible functionality between servers and clients when +it comes to dealing with different versions of TUF, we identified two main +issues that need to be addressed. First, clients need a way to be able to +determine whether their current TUF specification is compatible with the version +from which they are receiving new metadata. This ensures that clients parse +metadata according to the correct version to prevent errors from any changes to +the metadata in the new version. Second, clients need a way to use this +information to determine how to access metadata that is compatible with their +version of the TUF specification. + +To address the first issue, this TAP proposes standardizing the specification +version field to separate breaking changes from non-breaking changes and +ensure that the field can be compared across clients and repositories. +Separating out non-breaking changes allows these backwards compatible changes to +happen at any time without affecting TUF's operation. Therefore, clients and +repositories can still coordinate after a non-breaking change occurs. One common +framework used to separate versions by type is Semantic Versioning. Semantic +Versioning is a versioning scheme popular across open source projects that +categorizes specification versions by the scope and criticality of their +changes. Breaking changes in a specification would warrant a MAJOR version +number increase, while non-breaking changes would warrant only a MINOR version +number increase. In addition, Semantic Versioning has a standard way to format +version numbers so that they can be parsed and compared across implementations. + +To address the second issue, of accessing a compatible version, there are three +possible approaches. First, each repository could maintain multiple TUF versions +while the clients only maintain one version. In this case, TUF clients could not +use metadata from multiple repositories unless they all support the same TUF +version (Use Case 4). In the second approach, each client could maintain +multiple versions while the repositories each maintain a single version. In this +case, if a repository upgrades to a new TUF version, clients will be unable to +perform updates until support for the new version is added (Use Case 1). The +third option is to support multiple versions on both clients and repositories. +This option allows the clients and repositories to be upgraded independently to +support all use cases. + +The specification that follows adopts the third option, and supports version +changes on both clients and repositories. This requires both clients and +repositories to maintain multiple versions of the TUF spec, as well as changes +to the way repositories store metadata and clients parse this metadata. + +For the repositories this means continuing support for old TUF versions for some +period of time after upgrading. This grace period gives the client time to +upgrade to a new specification version. Versions with breaking changes are kept +in separate directories that allow a client to choose the most recent metadata +they support. To save space, target files should remain in the parent directory +on the repository. The metadata files (in directories according to their spec + version) can point to target files relative to the parent directory. + +On the client side, this TAP also requires maintenance of multiple specification +versions to allow for communication with various repositories. To do so, it is +recommended that clients maintain functions that can be used to validate +metadata from previous TUF specification versions. These functions allow a +client to maintain old versions of the specification while still supporting the +most recent version. # Specification -This TAP defines a variety of procedures to be added to the update process that -will allow for breaking changes to the TUF specification. These processes occur -on both clients and repos and include a change to how version numbers are -determined, new directories on repositories, new metadata parsing functions on -clients, and a new process for finding compatible metadata. +As stated above, this TAP defines a variety of procedures to be added to the +update process on both clients and repos. These changes include the way in which +version numbers are determined, the addition of new directories on repositories, +new metadata parsing functions on clients, and a new process for finding +compatible metadata. + +For clarity, throughout this TAP *upgrading* refers to the process of moving +from one TUF specification version to another while *updating* refers to the +process of downloading and validating packages as specified by TUF. The rest of +this section details the changes to the TUF specification recommended by this +TAP. -For clarity, throughout this TAP upgrading refers to the process of moving from -one TUF specification version to another while updating refers to the process -of downloading and validating packages as specified by TUF. ## Version Number Format -Clarifies that the 'spec_version' field of root -metadata shall be based on [Semantic Versioning](https://semver.org/), with -version numbers in the format MAJOR.MINOR.PATCH. - -This is a standard format used in other open source projects and makes the -meaning of a TUF version number change consistent and easily understood. -In accordance with Semantic Versioning, breaking changes will only occur during -a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The Backwards -Compatibility section of a TAP should be used to determine whether the TAP -creates a breaking change. If the change is not backwards compatible, then -it will be part of a new major version of the TUF specification. -Non backwards compatible versions add features that change the way that TUF -processes updates, and need to be implemented on both clients and repositories -atomically to maintain security and functionality. Examples of breaking changes -are TAP 3 and TAP 8. +Using [Semantic Versioning](https://semver.org/), breaking changes will only +occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The +'spec_version' field of root metadata will follow this convention, with version +numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other +open source projects that makes the meaning of a TUF version number change +consistent and easily understood. The Backwards Compatibility section of a TAP +should be used to determine whether the TAP creates a breaking change. If the +change is not backwards compatible, such as those in TAP 3 and 8, then it will +be part of a new major version of the TUF specification. Non backwards +compatible versions add features that change the way that TUF processes updates +and need to be implemented on both clients and repositories atomically to +maintain security and functionality. + If the change adds a new feature that is backwards compatible, for example in -TAP 4 and TAP 10, it should be part of a new minor -version. These backwards compatible TAPs add additional features to TUF, but -clients that do not have these features will still be able to securely and -reliably perform updates from repositories that support the TAPs. There may be -minor differences in which updates a client installs across minor versions. If -a repository maintainer is worried about the impact of a minor spec change on -clients, they may choose to wait to implement the change with a major -specification version (for example, they could remain on version 2.4.6 after - the release of 2.5). -Patches are used to fix typos and make small changes to existing features. More -details about what constitutes a major, minor, or patch change can be found at -https://semver.org/. +TAP 4 and TAP 10, it should be part of a new minor version. These TAPs add +additional features to TUF, but clients that do not have these features will +still be able to securely and reliably perform updates from repositories that +support the TAPs. There may be minor differences as to which updates a client +installs across minor versions. If a repository maintainer is worried about the +impact of a minor change on clients, it may choose to wait to implement the +change with a major specification version (for example, they could remain on +version 2.4.6 after the release of 2.5). Patches are used to fix typos and make +small changes to existing features. More details about what constitutes a major, +minor, or patch change can be found at https://semver.org/. ### Changes to TAPs In order to manage changes to TUF, TAPs shall be tied to a version of the TUF specification. -Once a TAP is accepted a TUF Version field in the header should be updated to include -the first TUF version that will include the TAP. The Preamble Header -description in TAP 1 shall be updated to include the TUF Version field. +Once a TAP is accepted, a TUF Version field in the header should list the first +TUF version that will include the TAP. The Preamble Header description in TAP 1 +shall be updated to include the TUF Version field. + ## How a repository updates @@ -217,19 +218,19 @@ Repositories will add metadata for new TUF specification versions in new directories. As described in the [Rationale](#rationale), repositories must support multiple -TUF specification versions. In order to do so, this TAP proposes a new -directory structure for repositories. When a repository manager chooses to -upgrade their repository to a new major TUF -spec version, they create a new directory on the repository named for the major -version (for example 2.0.0). This directory will contain all metadata files -with the new spec version. Target files will not be included in this directory. -After creating the directory, the repository creates and signs root, snapshot, -timestamp, and top level targets metadata using the new TUF spec version and -places these metadata files in the directory. The root file should be signed by -both the new root key and the current root key (the root key from the most -recent metadata in the previous major spec version). Clients will now be able -to use the new metadata files once their TUF spec versions are also updated. -After an update to version 2.0.0, the repository structure may look like: +TUF specification versions. In order to do so, this TAP proposes a new directory +structure for repositories. When a repository manager chooses to upgrade to a +new major TUF spec version, they create a new directory on the repository named +for the major version (for example 2.0.0). This directory will contain all +metadata files for the new spec version, but not target files. After creating +the directory, the repository creates and signs root, snapshot, timestamp, and +top level targets metadata using the new TUF spec version and places these +metadata files in the directory. The root file should be signed by both the new +root key and the current root key (the root key from the most recent metadata in +the previous major spec version). Clients will now be able to use the new +metadata files once their TUF spec versions are also updated. After an update to +version 2.0.0, the repository structure may look like: + ``` - Target files @@ -239,34 +240,34 @@ After an update to version 2.0.0, the repository structure may look like: |- metadata files ``` -If the repository is updated to a new minor or patch spec version, this shall -be done by uploading new metadata files in the new format to the proper -directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata -would go in the directory named 2.0.0. Minor and patch version changes are backwards -compatible, so clients using version 2.0.0 will still be able to parse metadata -written using version 2.1.0. +Repository updates to a new minor or patch specification version shall be done +by uploading new metadata files in the new format to the proper directory. So if +a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the +directory named 2.0.0. Minor and patch version changes are backwards compatible, +so clients using version 2.0.0 will still be able to parse metadata written +using version 2.1.0. A repository may continue to support old major TUF spec versions by creating -metadata both in the old location and in the new directory. The repository may -maintain as many versions as the repository manager wishes. If there are -security concerns with an old spec version, that version should be phased out -as soon as possible. The version can be phased out by no longer creating new -metadata files in that directory. In order to allow clients to parse the root -metadata chain, root metadata files shall not be deleted even once a version -is deprecated. - -In order to allow a client to find the current metadata files across spec -versions, the version numbers used for consistent snapshots should be -consistent across all supported spec versions. This means that there may be a -file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the -same consistent snapshot number must additionally use the same keys so that a -client can find the next root file in whichever spec version they support. +metadata in both the old location and the new directory. A repository manager +may maintain as many versions as desired, though any version with security +concerns should be phased out as soon as possible. This can be done by stopping +the creation of new metadata files in the directory for the phased out version. +In order to allow clients to parse the root metadata chain, root metadata files +shall not be deleted even once a version is deprecated. + +Version numbers used for consistent snapshots should be consistent across all +supported spec versions or a client might have difficulty finding the current +metadata files. This means that there may be a file at 1.0.0/3.root.json as well +as 2.0.0/3.root.json. Root files with the same consistent snapshot number must +also use the same keys so that a client can find the next root file in whichever +spec version they support. For existing TUF clients to continue operation after this TAP is implemented, repositories may store metadata from before TUF 1.0.0 in the top level -repository (with no directory named 0.0.0). This allows existing -clients to continue downloading metadata from the repository. So a TUF -repository that upgrades from version 0.12.0 to version 1.0.0 may look like: +repository (with no directory named 0.0.0). This allows existing clients to +continue downloading metadata from the repository. So a TUF repository that +upgrades from version 0.12.0 to version 1.0.0 may look like: + ``` - Targets files @@ -278,47 +279,45 @@ repository that upgrades from version 0.12.0 to version 1.0.0 may look like: ## Changes to TUF clients TUF clients must store the TUF specification versions they support and may add -functions to maintain old versions of the TUF specification. - -In order to find compatible updates on a repository, a client must keep track -of the TUF specification versions it supports. To do so, a global variable or -other local storage option should contain the client spec version, or spec -version range. For simplicity, this field should be formatted according to -Semantic Versioning so that it can be directly compared to the spec version in -root metadata. - -TUF clients may maintain support for pervious versions of the TUF -specification. This support can be used if the client downloads metadata from a -repository or delegated role that does not support the current TUF -specification. To allow for this behavior, when a new version of the TUF client -is implemented it may contain the ability to call certain functions from the -old TUF client for parsing and validating metadata. The client may make an old -specification version obsolete if they choose by removing the functions for that specification -version, but will risk being unable to download new targets from delegations -that use the old specification version. TUF implementers should decide how many old specification -versions to support based on the expected usage of their implementation. +functions to maintain old versions. In order to find compatible updates on a +repository, a client must keep track of the TUF specification versions it +supports. To do so, a global variable or other local storage option should +contain the client spec version, or spec version range. For simplicity, this +field should be formatted according to Semantic Versioning so that it can be +directly compared to the spec version in root metadata. + +There may be multiple versions of the specification supported by each TUF +client. This allows metadata downloads from a repository or delegated role that +does not support the current specification. To allow for this behavior, when a +new version of the TUF client is implemented, it may contain the ability to call +certain functions from the old client for parsing and validating metadata. +Should the client choose to make an old specification version obsolete, it can +remove the functions for that version. However, it does risk being unable to +download new targets from delegations that use the old specification version. +TUF implementers should decide how many old specification versions to support +based on the expected usage of their implementation. + ## Changes to the update process When a TUF client downloads metadata from a repository, the client must -determine which spec version to use for the download. To do this, the client -looks for the highest supported version on the repository using the following procedure: +determine which specification version to use for the download. To do this, +the client looks for the highest supported version on the repository using the +following procedure: * The client determines the latest version available on the repository by looking for the directory with the largest version number. -* If the latest version on the repository is equal to the client spec version, -the client will use this directory to download metadata. -* If the latest version on the repository is a version before the client spec version, -the client may call functions from a previous spec version client to download -the metadata. The client may support as many or as few versions as desired for -the application. If the previous version is not available, the client shall -report that an update is not possible due to an old spec version on the -repository. +* If the latest version on the repository is equal to that of the client, it +will use this directory to download metadata. +* If the latest version pre-dates the client spec version, it may call functions +from a previous client version to download the metadata. The client may support +as many or as few versions as desired for the application. If the previous +version is not available, the client shall report that an update can not be +performed due to an old spec version on the repository. * If the latest version on the repository is higher than the client spec version, the client should report to the user that it is not using the most up -to date TUF spec version (the method of reporting is left to the discretion of -the client) then proceed with the directory that corresponds with -the latest client spec version if available. If no such directory exists, the client +to date version then proceed with the directory that corresponds with the latest +client spec version, if available. If no such directory exists, the client terminates the update. Once the supported directory is determined, the client shall attempt the update @@ -331,119 +330,119 @@ up to the discretion of an implementer, but it should be used to encourage updating the client to the most recent specification version. Alternatively, if the same client downloads metadata from a repository with -directories 1.0.0 and 2.0.0, the client could download metadata from 2.0.0 -using a 2.x version of the client. If 2.x is not supported by the client, the -client will report that it is unable to perform an update. +directories 1.0.0 and 2.0.0, the client could download metadata from 2.0.0 using +a 2.x version of the client. If 2.x is not supported by the client, the client +will report that it is unable to perform an update. Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file saved on the -client uses a spec version other than the supported version, the client will look -for the next root file first in the supported version, then the previous -versions until the next root file is found or the currently trusted root file's -version is reached. All root files should be verified using the major version -of the TUF client that corresponds with the major version of the root file. +client uses a spec version other than the supported version, the client will +look for the next root file. This process will start in the supported version, +then move to the previous versions until the next root file is found, or until +the currently trusted root file's version is reached. All root files should be +verified using the major version of the TUF client that corresponds with the +major version of the root file. So, if the currently trusted root file is named 4.root.json and uses version 1.0.0 and the highest supported version is 3.0.0, the client will look for 5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the -client will look for 6.root.json using the same process. To facilitate this, -the client should maintain functions to parse root files from previous spec +client will look for 6.root.json using the same process. To facilitate this, the +client should maintain functions to parse root files from previous spec versions. If the client does not support the spec version of a root file, the client shall terminate the update and report the spec version mismatch. + ## Special Cases ### Multiple Repositories A TUF client that performs updates using multiple repositories may need to -access repositories that use different TUF specification versions. When -comparing the metadata from multiple repositories, the goal is to ensure that -the target file that is downloaded is verified by all repositories. Therefore -the client can perform validation for each repository independently, then -compare the results of the validation. - -To do so, the client needs to ensure that the metadata from each -repository is valid for the given targets file. -If the repositories use different versions of the TUF specification, -the client should use the TUF version that corresponds to each repository to -validate the metadata from that repository, then compare the results. The -update is only valid if valid metadata from both repositories points to the same -target file, and this target file matches the hashes provided by each repository. -Note that different TUF versions may use different hashing algorithms. If this -is the case, both hashes should be verified independently. +access repositories that use different specification versions. When comparing +metadata from multiple repositories, the goal is to ensure that the downloaded +target file is verified by all repositories. Therefore the client can perform +validation for each repository independently, and then compare the results. + +To do so, the client needs to ensure that the metadata from each repository is +valid for the given targets file. If the repositories use different versions of +the TUF specification, the client should use the version that corresponds to +each repository to validate its metadata. Results from all repositories, could +then be compared. The update will only be considered valid if valid metadata +from both repositories point to the same target file, and it matches the hashes +provided by each repository. Note that different TUF versions may use different +hashing algorithms. If this is the case, both hashes should be verified +independently. + ### Changes to delegations -The TUF version used by a delegation does not need to match the TUF version -used by the top level metadata. The TUF client is responsible for parsing the -metadata from the top-level roles and delegations using the appropriate -specification version, similar to the process used for multiple repositories. -If there is no compatible spec version between the client -and delegation, the client should report this and terminate the update. - -So that delegated targets can upgrade to new spec versions, delegated targets -metadata should be stored in directories corresponding to their major spec version just -like top level metadata on the repository. For each targets metadata file, a -TUF client should download the highest supported version. This highest -supported version will be found using the same procedure as described in +The TUF version used by a delegation does not need to match the version used by +the top level metadata. The TUF client is responsible for parsing the metadata +from the top-level roles and delegations using the appropriate specification +version, similar to the process used for multiple repositories. If there is no +compatible specification version between the client and delegation, the client +should report this and terminate the update. + +So that delegated targets can upgrade to new specification versions, their +targets metadata should be stored in directories corresponding to their major +version, just like the top level metadata on the repository. For each targets +metadata file, a TUF client should download the highest supported version, which +will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). ### Updating Trusted Root Metadata -To allow for future updates after a major version change, the client must -update their trusted root metadata to a root metadata that complies with the -new spec version. To do so, the client first downloads and verifies the current -version root metadata file. Once verified, this current version root metadata -file must be stored as the trusted root metadata. In future updates, the client -will start from the trusted root metadata when finding the next available update. +To allow for future updates after a major version change, the client must update +its trusted root metadata to one that complies with the new specification +version. To do so, the client first downloads and verifies the current version +root metadata file, which then must be stored as the trusted root metadata. In +future updates, the client will start from the trusted root metadata to find the +next available update. # Security Analysis Overall, this TAP will increase the security of TUF systems by allowing for -upgrades to TUF clients and repositories. However, there are a few attacks and -changes to the security model that should be discussed. +faster upgrades to TUF clients and repositories. However, there are a few +attacks and changes to the security model that should be discussed. ## Downgrade Attack A downgrade attack on the TUF specification version may be possible if an attacker is able to block access to a directory on the repository. This would -mean that a client would use metadata from a previous specification version -when performing an update. However, the metadata would still have to be current -and properly signed. To mitigate the damage from a downgrade attack in the case -that a security flaw is found in a version of the TUF specification, the -vulnerable version of the specification should no longer be supported on the -repository (the metadata files should be revoked or allowed to expire). In -addition, clients should be upgraded to no longer support a vulnerable -specification version. +mean that a client would use metadata from a previous specification version when +performing an update. However, the metadata would still have to be current and +properly signed. To mitigate the damage from a downgrade attack in case a +security flaw is found in a version of the TUF specification, the vulnerable +version should no longer be supported on the repository (the metadata files +should be revoked or allowed to expire). In addition, clients should be upgraded +to no longer support a vulnerable specification version. ## Root Key Rotation The TUF version upgrade system described in this TAP uses the root key rotation -mechanism to ensure that the client only uses valid TUF metadata. The first -time a new TUF version is used by the client, the client checks the -repository's root metadata in the new version's format. If the root metadata is -not signed by the previous trusted root key, the update does not proceed, and -the client halts the update. In this way, updates to the repository's TUF -version does not impact the security of an update. +mechanism to ensure that the client only uses valid TUF metadata. The first time +a new TUF version is used by the client, it checks the repository's root +metadata in the new version's format. If the root metadata is not signed by the +previous trusted root key, the update does not proceed, and the client halts the +update. In this way, updates to the repository's TUF version do not impact the +security of an update. ## How a client upgrades -When a client upgrades to a new TUF version, it should use TUF to ensure that -it downloads a valid new client. To do so, it should use its current TUF -version to upgrade to the new version of TUF. For example, if a client running -TUF version 2.5.0 wants to upgrade to version 3.0.0, the client should use -metadata from TUF version 2.x.x to self update. Using this technique, the TUF -client can ensure that it downloads the intended update when upgrading to a new -TUF version. +When a client upgrades itself to a new TUF version, it should use TUF for the +self update to ensure that it downloads a valid new client. To do so, it should +use its current TUF version to upgrade to the new version of TUF. For example, +if a client running TUF version 2.5.0 wants to upgrade to version 3.0.0, the +client should use metadata from TUF version 2.x.x to self update. Using this +technique, the TUF client can ensure that it downloads the intended update when +upgrading to a new version. ## Other considerations As always, authors of TUF clients should ensure that the client is a valid -implementation of TUF and that no security flaws are introduced in code. In -addition, supply chain security techniques (for example [in-toto](https://in-toto.io/in-toto.html)) +implementation of TUF and that no security flaws are introduced in code. +In addition, supply chain security techniques (for example [in-toto](https://in-toto.io/in-toto.html)) should be used to ensure that vulnerabilities are not introduced into TUF clients between when they are written and installed. -Note that an attacker on the same network could inject false metadata to block -a client from updating. However this denial of service attack is always -possible for an attacker on the network with the ability to alter network -traffic. +Note that an attacker on the same network could inject false metadata to block a +client from updating. However, this denial of service attack is always possible +for an attacker on the network with the ability to alter network traffic. # Backwards Compatibility From 5ba968f2628dc3ef390f2b428630c40ba4716409 Mon Sep 17 00:00:00 2001 From: mnm678 Date: Fri, 3 Jan 2020 14:28:00 -0500 Subject: [PATCH 25/40] Apply suggestions from code review Co-Authored-By: Trishank Karthik Kuppusamy <33133073+trishankatdatadog@users.noreply.github.com> --- candidate-tuf-versions.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 7b932ec8..b96ecd4b 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -21,7 +21,7 @@ To manage breaking changes without loss of functionality, this TAP requires two changes: one to the way the TUF specification manages versions, and the other to how it performs updates. The former is accomplished by having this TAP require that the specification use Semantic Versioning to separate breaking from -non-breaking changes. And the latter is achieved by requiring both clients and +non-breaking changes. The latter is achieved by requiring both clients and repositories to maintain support for older versions of the TUF specification so reliable updates can continue throughout the process of the specification upgrade. To do so, repositories will generate metadata for multiple TUF @@ -128,7 +128,7 @@ number increase, while non-breaking changes would warrant only a MINOR version number increase. In addition, Semantic Versioning has a standard way to format version numbers so that they can be parsed and compared across implementations. -To address the second issue, of accessing a compatible version, there are three +To address the second issue of accessing a compatible version, there are three possible approaches. First, each repository could maintain multiple TUF versions while the clients only maintain one version. In this case, TUF clients could not use metadata from multiple repositories unless they all support the same TUF @@ -145,7 +145,7 @@ changes on both clients and repositories. This requires both clients and repositories to maintain multiple versions of the TUF spec, as well as changes to the way repositories store metadata and clients parse this metadata. -For the repositories this means continuing support for old TUF versions for some +For repositories, this means continuing support for old TUF versions for some period of time after upgrading. This grace period gives the client time to upgrade to a new specification version. Versions with breaking changes are kept in separate directories that allow a client to choose the most recent metadata @@ -316,7 +316,7 @@ version is not available, the client shall report that an update can not be performed due to an old spec version on the repository. * If the latest version on the repository is higher than the client spec version, the client should report to the user that it is not using the most up -to date version then proceed with the directory that corresponds with the latest +to date version, and then proceed with the directory that corresponds with the latest client spec version, if available. If no such directory exists, the client terminates the update. From 30202ded1f2389b88d8174b294ef37cc556747c2 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Fri, 3 Jan 2020 14:22:58 -0800 Subject: [PATCH 26/40] Added clarification based on review. Specifically: * Made changes to clarify that clients and repositories write code that implements a TUF specification version * Clarified that clients are repositories are managed by different people * Various wording fixes to add detail and clarity --- candidate-tuf-versions.md | 110 +++++++++++++++++++++----------------- 1 file changed, 60 insertions(+), 50 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index b96ecd4b..b45b79f3 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -19,11 +19,11 @@ are implemented. To manage breaking changes without loss of functionality, this TAP requires two changes: one to the way the TUF specification manages versions, and the other to -how it performs updates. The former is accomplished by having this TAP require +how TUF implementations perform updates. The former is accomplished by having this TAP require that the specification use Semantic Versioning to separate breaking from non-breaking changes. The latter is achieved by requiring both clients and -repositories to maintain support for older versions of the TUF specification so -reliable updates can continue throughout the process of the specification +repositories to maintain sets of TUF metadata that follow different versions of the TUF specification +so that reliable updates can continue throughout the process of the specification upgrade. To do so, repositories will generate metadata for multiple TUF specification versions and maintain them in an accessible directory. In addition, clients will include support for metadata from previous TUF versions. @@ -46,6 +46,8 @@ As new features become available in the TUF specification, repositories may wish to upgrade to a new version. This could include adding TAPs with breaking changes. When the repository upgrades, clients that use an older version of the TUF spec will no longer be able to safely parse metadata from the repository. +However, repositories and clients may be managed by different people with +different interests who may not be able to coordinate the upgrade to a new TUF version. To resolve this problem, clients will first need a way to determine whether their version of TUF spec is compatible with the version used by the repository. This is to ensure that clients always parse metadata according to the version @@ -84,11 +86,11 @@ version 1.0.0 and another repository that uses version 2.0.0. ## Use case 5: Allowing Backwards Compatibility -Existing TUF clients will still expect to download and parse metadata that uses -the current TUF specification version, even after this TAP is implemented on +Existing TUF clients will still expect to download and parse metadata without +knowledge of the supported TUF version, even after this TAP is implemented on repositories. Before this TAP there was no method for determining compatibility between a client and a repository, so existing TUF repositories should continue -to generate metadata using the current method to support existing clients. This +to distribute metadata using their existing method to support existing clients. This means that this mechanism for allowing backwards compatible updates must itself be backwards compatible. @@ -106,13 +108,13 @@ updates. In trying to create more flexible functionality between servers and clients when it comes to dealing with different versions of TUF, we identified two main -issues that need to be addressed. First, clients need a way to be able to -determine whether their current TUF specification is compatible with the version -from which they are receiving new metadata. This ensures that clients parse +issues that need to be addressed. First, clients need to be able to determine +whether the TUF specification version they implement is compatible with the TUF +version of the metadata they receive. This ensures that clients parse metadata according to the correct version to prevent errors from any changes to the metadata in the new version. Second, clients need a way to use this -information to determine how to access metadata that is compatible with their -version of the TUF specification. +information to determine how to access metadata that is compatible with the +version of the TUF specification they implement. To address the first issue, this TAP proposes standardizing the specification version field to separate breaking changes from non-breaking changes and @@ -129,31 +131,37 @@ number increase. In addition, Semantic Versioning has a standard way to format version numbers so that they can be parsed and compared across implementations. To address the second issue of accessing a compatible version, there are three -possible approaches. First, each repository could maintain multiple TUF versions -while the clients only maintain one version. In this case, TUF clients could not -use metadata from multiple repositories unless they all support the same TUF -version (Use Case 4). In the second approach, each client could maintain -multiple versions while the repositories each maintain a single version. In this -case, if a repository upgrades to a new TUF version, clients will be unable to -perform updates until support for the new version is added (Use Case 1). The -third option is to support multiple versions on both clients and repositories. -This option allows the clients and repositories to be upgraded independently to -support all use cases. - -The specification that follows adopts the third option, and supports version -changes on both clients and repositories. This requires both clients and +possible approaches. First, each repository could maintain sets of metadata that +implement multiple TUF versions while the clients only implement one TUF +specification version. In this case, TUF clients could not use metadata from +multiple repositories unless they all generate metadata that supports the same +TUF version (Use Case 4). In the second approach, each client could maintain +multiple versions that implement different versions of the TUF specification +while the repositories each generate metadata following a single TUF +specification version. In this case, if a repository upgrades to support a new +TUF version, clients will be unable to perform updates until support for the new +version is added to the client (Use Case 1). The third option is to implement +support for multiple TUF versions on both clients and repositories. This option +allows clients and repositories to be upgraded to support new versions of the +TUF specification independently and supports all use cases mentioned in this TAP. + +This TAP adopts the third option, and describes procedures for both clients and +repositories to implement multiple TUF specification versions at the same time. +This requires both clients and repositories to maintain multiple versions of the TUF spec, as well as changes to the way repositories store metadata and clients parse this metadata. For repositories, this means continuing support for old TUF versions for some -period of time after upgrading. This grace period gives the client time to -upgrade to a new specification version. Versions with breaking changes are kept -in separate directories that allow a client to choose the most recent metadata -they support. To save space, target files should remain in the parent directory -on the repository. The metadata files (in directories according to their spec - version) can point to target files relative to the parent directory. - -On the client side, this TAP also requires maintenance of multiple specification +period of time after upgrading. This grace period gives existing clients that +implement old versions of the TUF specification time to implement support for a +new specification version. Repositories achieve this using a directory structure +with a directory for each supported TUF specification version. These directories +contain metadata that supports the given TUF specification version. Using these +directories, a client is able to choose the most recent metadata they support. +More details about this directory structure are contained in the [specification](#how-a-repository-updates). + +On the client side, this TAP also requires maintenance of multiple versions that +support different TUF specification versions to allow for communication with various repositories. To do so, it is recommended that clients maintain functions that can be used to validate metadata from previous TUF specification versions. These functions allow a @@ -188,8 +196,8 @@ should be used to determine whether the TAP creates a breaking change. If the change is not backwards compatible, such as those in TAP 3 and 8, then it will be part of a new major version of the TUF specification. Non backwards compatible versions add features that change the way that TUF processes updates -and need to be implemented on both clients and repositories atomically to -maintain security and functionality. +and so clients and repositories need to use code that supports the same major +version when performing an update in order to maintain security and functionality. If the change adds a new feature that is backwards compatible, for example in TAP 4 and TAP 10, it should be part of a new minor version. These TAPs add @@ -222,7 +230,9 @@ TUF specification versions. In order to do so, this TAP proposes a new directory structure for repositories. When a repository manager chooses to upgrade to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all -metadata files for the new spec version, but not target files. After creating +metadata files for the new spec version, but target files will remain in the +parent directory to save space. The metadata files (in directories according to their spec +version) can point to target files relative to the parent directory. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. The root file should be signed by both the new @@ -255,7 +265,10 @@ the creation of new metadata files in the directory for the phased out version. In order to allow clients to parse the root metadata chain, root metadata files shall not be deleted even once a version is deprecated. -Version numbers used for consistent snapshots should be consistent across all +A repository should generate all TUF metadata, including root metadata, for all +TUF versions that the repository supports. Any update should be reflected across +all of these versions. For clarity, version numbers used for consistent +snapshots should be consistent across all supported spec versions or a client might have difficulty finding the current metadata files. This means that there may be a file at 1.0.0/3.root.json as well as 2.0.0/3.root.json. Root files with the same consistent snapshot number must @@ -337,7 +350,9 @@ will report that it is unable to perform an update. Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file saved on the client uses a spec version other than the supported version, the client will -look for the next root file. This process will start in the supported version, +look for the next root file. The next root file may have been generated using +the supported version, or it may have been generated using a previous TUF +version. The client will look for the root file first in the supported version, then move to the previous versions until the next root file is found, or until the currently trusted root file's version is reached. All root files should be verified using the major version of the TUF client that corresponds with the @@ -372,23 +387,18 @@ provided by each repository. Note that different TUF versions may use different hashing algorithms. If this is the case, both hashes should be verified independently. +### Delegations -### Changes to delegations - -The TUF version used by a delegation does not need to match the version used by -the top level metadata. The TUF client is responsible for parsing the metadata -from the top-level roles and delegations using the appropriate specification -version, similar to the process used for multiple repositories. If there is no +The TUF specification version(s) supported by a delegation does not need to +match the version(s) supported by the top level metadata. A delegation may use +the directory structure described above to indicate which TUF specification +versions it supports. The TUF client is responsible for parsing the metadata +from delegations using code that implements the appropriate specification +version (found using the same procedure as described in +[Changes to the update process](#changes-to-the-update-process)). If there is no compatible specification version between the client and delegation, the client should report this and terminate the update. -So that delegated targets can upgrade to new specification versions, their -targets metadata should be stored in directories corresponding to their major -version, just like the top level metadata on the repository. For each targets -metadata file, a TUF client should download the highest supported version, which -will be found using the same procedure as described in -[Changes to the update process](#changes-to-the-update-process). - ### Updating Trusted Root Metadata To allow for future updates after a major version change, the client must update From 9790a0c88ac46e2a8091d245d1ed3dc1cb73fb28 Mon Sep 17 00:00:00 2001 From: mnm678 Date: Tue, 14 Jan 2020 09:18:39 -0800 Subject: [PATCH 27/40] Apply suggestions from code review Co-Authored-By: Justin Cappos --- candidate-tuf-versions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index b45b79f3..6250ffbf 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -11,7 +11,7 @@ The TUF specification does not currently support breaking changes or changes that are not backwards compatible. If a repository and a client are not using -the same version of the TUF specification, metadata can not be safely and +the same version of the TUF specification, differences in metadata format may mean that metadata cannot be safely and reliably verified. This TAP addresses this clash of versions by allowing TUF implementations to independently manage updates on clients and repositories. This in turn ensures that TUF will continue functioning after breaking changes @@ -78,7 +78,7 @@ generate metadata with a different TUF specification version than the repository ## Use case 4: A client downloads metadata from multiple repositories As described in TAP 4, TUF clients may download metadata from multiple -repositories. These repositories do not coordinate with each other, and so may +repositories. These repositories may not coordinate with each other, and so may not upgrade to a new TUF specification version at the same time. A client should be able to use multiple repositories that do not use the same version of TUF. For example, a client may download metadata from one repository that uses TUF @@ -97,7 +97,7 @@ be backwards compatible. # Rationale We propose this TAP because as TUF continues to evolve, the need for TUF clients -and repositories to upgrade will grow. This sets up the potential for clients +and repositories to upgrade their TUF metadata format will grow. This sets up the potential for clients that are no longer able to perform updates because they cannot parse the metadata generated by repositories, or for clients to install the wrong images due to changes in TUF. Both of these issues prevent clients from installing the From 4cee0bdbb6d7d79086cdd40751e10c52ba8c3914 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 23 Jan 2020 13:00:15 -0500 Subject: [PATCH 28/40] Added clarifications based on review. Especially: * defined breaking changes * clarified the goals of the tap * described which metadata files must have the same version --- candidate-tuf-versions.md | 50 ++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 14 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 6250ffbf..8a61d3a9 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -11,13 +11,16 @@ The TUF specification does not currently support breaking changes or changes that are not backwards compatible. If a repository and a client are not using -the same version of the TUF specification, differences in metadata format may mean that metadata cannot be safely and -reliably verified. This TAP addresses this clash of versions by allowing TUF +the same version of the TUF specification, differences in metadata format may +mean that metadata cannot be safely and reliably verified. Any changes that +affect the metadata in this way are considered breaking changes. This TAP +addresses this clash of versions by allowing TUF implementations to independently manage updates on clients and repositories. This in turn ensures that TUF will continue functioning after breaking changes are implemented. -To manage breaking changes without loss of functionality, this TAP requires two +To allow TUF implementers to adopt breaking changes without an interruption to +providing and verifying updates, this TAP requires two changes: one to the way the TUF specification manages versions, and the other to how TUF implementations perform updates. The former is accomplished by having this TAP require that the specification use Semantic Versioning to separate breaking from @@ -34,23 +37,27 @@ recent specification version supported by both the client and the repository. # Motivation Various TAPs, including TAPs 3 and 8, propose changes that are not backwards -compatible. Because these changes are not compatible with previous TUF versions, +compatible. These non backwards compatible, or breaking, changes add or change +functionality such that the new version is not compatible with previous versions. +Because these changes are not compatible with previous TUF versions, current implementations that use the existing specification can not access the new features in these TAPs. By creating a way to deal with non-backwards -compatible (breaking) changes, TUF will be able to handle a variety of use -cases, including those that appear below. +compatible (breaking) changes, TUF will be able to continuously provide secure +updates when non backwards compatible changes are made to the repository or +client. The use cases addressed in this TAP are listed below. ## Use case 1: A repository updates to a new TUF spec version As new features become available in the TUF specification, repositories may wish to upgrade to a new version. This could include adding TAPs with breaking changes. When the repository upgrades, clients that use an older version of the -TUF spec will no longer be able to safely parse metadata from the repository. +TUF spec will no longer be able to provide the security properties of TUF when +parsing metadata from the repository as the fields in the metadata may have changed. However, repositories and clients may be managed by different people with different interests who may not be able to coordinate the upgrade to a new TUF version. To resolve this problem, clients will first need a way to determine whether their version of TUF spec is compatible with the version used by the repository. -This is to ensure that clients always parse metadata according to the version +This is to ensure that clients always parse metadata according to the TUF version that generated that metadata. Then, they will need some way to process updates after this upgrade occurs on the repository to ensure reliable access to updates, even if they are not able to upgrade immediately due to development @@ -62,8 +69,8 @@ Just as a repository may be upgraded, a TUF client may wish to upgrade to a new TUF spec version to use new features. When the client implements an upgrade that includes a breaking change, it cannot be sure that all repositories have also upgraded to the new specification version. Therefore, after the client upgrades -they must ensure that any metadata downloaded from a repository was generated -using the new version. If the repository is using an older version, the client +they must ensure that any metadata downloaded from a repository is parsed using +a compatible TUF version. If the repository is using an older version, the client should have some way to allow the update to proceed. ## Use case 3: A delegated targets role uses a different TUF spec version than the repository @@ -84,11 +91,10 @@ be able to use multiple repositories that do not use the same version of TUF. For example, a client may download metadata from one repository that uses TUF version 1.0.0 and another repository that uses version 2.0.0. -## Use case 5: Allowing Backwards Compatibility +## Use case 5: Backwards compatibility for existing clients -Existing TUF clients will still expect to download and parse metadata without -knowledge of the supported TUF version, even after this TAP is implemented on -repositories. Before this TAP there was no method for determining compatibility +Existing TUF clients (written before this TAP is implemented) will still expect +to download and parse metadata. Before this TAP there was no method for determining compatibility between a client and a repository, so existing TUF repositories should continue to distribute metadata using their existing method to support existing clients. This means that this mechanism for allowing backwards compatible updates must itself @@ -168,6 +174,22 @@ metadata from previous TUF specification versions. These functions allow a client to maintain old versions of the specification while still supporting the most recent version. +The top level TUF metadata (root, snapshot, timestamp, and top-level targets) +used to install an update should all implement the same TUF specification +version. A specification change may rely on more than one metadata file (for +example a change in the signing process would affect all metadata types), so +using the same specification version for top level metadata allows for these +large changes to the specification. However, delegated targets may not be +managed by the same parties as the top level metadata. For this reason, this TAP +allows clients to use a different TUF specification version for delegated +targets. This compromise allows TUF specification changes to affect multiple +metadata files while allowing flexibility in how repositories and delegations +are managed. + +So, during the course of an update a client could use 4.0.0_parse_root, +4.0.0_parse_snapshot, 4.0.0_parse_timestamp, 4.0.0_parse_targets, +4.0.0_parse_delegation, and 3.0.0_parse_delegation. + # Specification From e927183b37e6074a1cac1379f240c4078e2ff985 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 10 Feb 2020 13:30:18 -0500 Subject: [PATCH 29/40] Add clarifications, especially: * Define breaking changes once at the beginning, then use the term consistently * Describe how this TAP relates to TAP 5 * Various minor edits --- candidate-tuf-versions.md | 46 ++++++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 13 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 8a61d3a9..165e0bf6 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -9,7 +9,7 @@ # Abstract -The TUF specification does not currently support breaking changes or changes +The TUF specification does not currently support changes that are not backwards compatible. If a repository and a client are not using the same version of the TUF specification, differences in metadata format may mean that metadata cannot be safely and reliably verified. Any changes that @@ -36,14 +36,12 @@ recent specification version supported by both the client and the repository. # Motivation -Various TAPs, including TAPs 3 and 8, propose changes that are not backwards -compatible. These non backwards compatible, or breaking, changes add or change -functionality such that the new version is not compatible with previous versions. +Various TAPs, including TAPs 3 and 8, propose breaking changes. Because these changes are not compatible with previous TUF versions, -current implementations that use the existing specification can not access the -new features in these TAPs. By creating a way to deal with non-backwards -compatible (breaking) changes, TUF will be able to continuously provide secure -updates when non backwards compatible changes are made to the repository or +current implementations that use the existing specification cannot access the +new features in these TAPs. By creating a way to deal with +breaking changes, TUF will be able to continuously provide secure +updates when breaking changes are made to the repository or client. The use cases addressed in this TAP are listed below. ## Use case 1: A repository updates to a new TUF spec version @@ -179,10 +177,12 @@ used to install an update should all implement the same TUF specification version. A specification change may rely on more than one metadata file (for example a change in the signing process would affect all metadata types), so using the same specification version for top level metadata allows for these -large changes to the specification. However, delegated targets may not be +large changes to the specification. For information about how this relates to +url pinning and TAP 5, see [Special Cases](#tap-5). However, delegated targets may not be managed by the same parties as the top level metadata. For this reason, this TAP allows clients to use a different TUF specification version for delegated -targets. This compromise allows TUF specification changes to affect multiple +targets by maintaining functions to parse metadata that conforms to different +specification versions. This compromise allows TUF specification changes to affect multiple metadata files while allowing flexibility in how repositories and delegations are managed. @@ -247,13 +247,13 @@ shall be updated to include the TUF Version field. Repositories will add metadata for new TUF specification versions in new directories. -As described in the [Rationale](#rationale), repositories must support multiple +As described in the [Rationale](#rationale), repositories should support multiple TUF specification versions. In order to do so, this TAP proposes a new directory structure for repositories. When a repository manager chooses to upgrade to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files for the new spec version, but target files will remain in the -parent directory to save space. The metadata files (in directories according to their spec +parent directory to save space. In this case the metadata files (in directories according to their spec version) can point to target files relative to the parent directory. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these @@ -351,7 +351,7 @@ version is not available, the client shall report that an update can not be performed due to an old spec version on the repository. * If the latest version on the repository is higher than the client spec version, the client should report to the user that it is not using the most up -to date version, and then proceed with the directory that corresponds with the latest +to date version, and then perform the update with the directory that corresponds with the latest client spec version, if available. If no such directory exists, the client terminates the update. @@ -409,6 +409,26 @@ provided by each repository. Note that different TUF versions may use different hashing algorithms. If this is the case, both hashes should be verified independently. +### TAP 5 + +A TUF client that supports TAP 5 may download top-level metadata from multiple +sources. However, breaking changes may require that multiple top-level metadata +files use the same TUF specification version. To manage this, a root metadata +file that includes a list of urls for metadata as described in TAP 5 should +only include links to metadata files using the same TUF specification version as +the root metadata file. + +The owner of root metadata is responsible for ensuring that all metadata files +linked in the root metadata use the same TUF specification version as the root +metadata. If the repository that contains the root metadata supports multiple +TUF specification versions, each root metadata file should list top-level +metadata corresponding with its TUF specification version. + +Clients should verify that all top-level metadata use the same TUF specification +version as the root metadata. If a metadata file linked in root metadata does +not meet this criteria, it will be considered invalid and the client should +use the next listed metadata file or terminate the update. + ### Delegations The TUF specification version(s) supported by a delegation does not need to From 3f78127bef354c8b574bfa7e0ebf5fedd8abd523 Mon Sep 17 00:00:00 2001 From: Marina Moore Date: Thu, 9 Jul 2020 19:05:02 -0400 Subject: [PATCH 30/40] Apply suggestions from code review Co-authored-by: Joshua Lock --- candidate-tuf-versions.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 165e0bf6..23315fab 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -103,9 +103,9 @@ be backwards compatible. We propose this TAP because as TUF continues to evolve, the need for TUF clients and repositories to upgrade their TUF metadata format will grow. This sets up the potential for clients that are no longer able to perform updates because they cannot parse the -metadata generated by repositories, or for clients to install the wrong images +metadata generated by repositories, or for clients to install the wrong targets due to changes in TUF. Both of these issues prevent clients from installing the -images intended by repository managers, and could lead to critical security or +targets intended by repository managers, and could lead to critical security or functionality problems. TUF clients and repositories need to be able to upgrade to new versions without preventing secure and reliable access to software updates. @@ -129,7 +129,7 @@ repositories can still coordinate after a non-breaking change occurs. One common framework used to separate versions by type is Semantic Versioning. Semantic Versioning is a versioning scheme popular across open source projects that categorizes specification versions by the scope and criticality of their -changes. Breaking changes in a specification would warrant a MAJOR version +changes. Breaking changes in the specification would warrant a MAJOR version number increase, while non-breaking changes would warrant only a MINOR version number increase. In addition, Semantic Versioning has a standard way to format version numbers so that they can be parsed and compared across implementations. @@ -239,7 +239,7 @@ specification. Once a TAP is accepted, a TUF Version field in the header should list the first TUF version that will include the TAP. The Preamble Header description in TAP 1 -shall be updated to include the TUF Version field. +has been updated to include the TUF Version field. ## How a repository updates From cb810167bb5acffdc307dda55d18c6f805f32f1e Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 16 Jul 2020 11:08:00 -0700 Subject: [PATCH 31/40] Update example as TUF is on version 1.0.x --- candidate-tuf-versions.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 23315fab..95f66db9 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -128,7 +128,7 @@ happen at any time without affecting TUF's operation. Therefore, clients and repositories can still coordinate after a non-breaking change occurs. One common framework used to separate versions by type is Semantic Versioning. Semantic Versioning is a versioning scheme popular across open source projects that -categorizes specification versions by the scope and criticality of their +categorizes project versions by the scope and criticality of their changes. Breaking changes in the specification would warrant a MAJOR version number increase, while non-breaking changes would warrant only a MINOR version number increase. In addition, Semantic Versioning has a standard way to format @@ -298,17 +298,17 @@ also use the same keys so that a client can find the next root file in whichever spec version they support. For existing TUF clients to continue operation after this TAP is implemented, -repositories may store metadata from before TUF 1.0.0 in the top level -repository (with no directory named 0.0.0). This allows existing clients to +repositories may store metadata from before TUF 2.0.0 in the top level +repository (with no directory named 1.0.0). This allows existing clients to continue downloading metadata from the repository. So a TUF repository that -upgrades from version 0.12.0 to version 1.0.0 may look like: +upgrades from version 1.12.0 to version 2.0.0 may look like: ``` - Targets files -- 0.12.0 metadata files -- 1.0.0 - |- 1.0.0 metadata files +- 1.12.0 metadata files +- 2.0.0 + |- 2.0.0 metadata files ``` ## Changes to TUF clients From 817e993b75d9a5bc5bf7e1d8c955b468e5784c02 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 20 Jul 2020 14:47:32 -0700 Subject: [PATCH 32/40] spec to specification and other minor edits for consistency and accuracy --- candidate-tuf-versions.md | 102 +++++++++++++++++++------------------- 1 file changed, 51 insertions(+), 51 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 95f66db9..49bf0c49 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -12,7 +12,7 @@ The TUF specification does not currently support changes that are not backwards compatible. If a repository and a client are not using the same version of the TUF specification, differences in metadata format may -mean that metadata cannot be safely and reliably verified. Any changes that +mean that metadata cannot be safely and reliably verified. Any changes to the specification that affect the metadata in this way are considered breaking changes. This TAP addresses this clash of versions by allowing TUF implementations to independently manage updates on clients and repositories. @@ -24,12 +24,12 @@ providing and verifying updates, this TAP requires two changes: one to the way the TUF specification manages versions, and the other to how TUF implementations perform updates. The former is accomplished by having this TAP require that the specification use Semantic Versioning to separate breaking from -non-breaking changes. The latter is achieved by requiring both clients and +non-breaking changes. The latter is achieved by allowing both clients and repositories to maintain sets of TUF metadata that follow different versions of the TUF specification so that reliable updates can continue throughout the process of the specification -upgrade. To do so, repositories will generate metadata for multiple TUF +upgrade. To do so, repositories can generate metadata for multiple TUF specification versions and maintain them in an accessible directory. In -addition, clients will include support for metadata from previous TUF versions. +addition, clients can include support for metadata from previous TUF versions. To determine the version to use for an update, clients will select the most recent specification version supported by both the client and the repository. @@ -44,38 +44,38 @@ breaking changes, TUF will be able to continuously provide secure updates when breaking changes are made to the repository or client. The use cases addressed in this TAP are listed below. -## Use case 1: A repository updates to a new TUF spec version +## Use case 1: A repository updates to a new TUF specification version As new features become available in the TUF specification, repositories may wish to upgrade to a new version. This could include adding TAPs with breaking changes. When the repository upgrades, clients that use an older version of the -TUF spec will no longer be able to provide the security properties of TUF when +TUF specification will no longer be able to provide the security properties of TUF when parsing metadata from the repository as the fields in the metadata may have changed. However, repositories and clients may be managed by different people with different interests who may not be able to coordinate the upgrade to a new TUF version. To resolve this problem, clients will first need a way to determine whether -their version of TUF spec is compatible with the version used by the repository. +their version of TUF specification is compatible with the version used by the repository. This is to ensure that clients always parse metadata according to the TUF version that generated that metadata. Then, they will need some way to process updates after this upgrade occurs on the repository to ensure reliable access to updates, even if they are not able to upgrade immediately due to development time or other constraints. -## Use case 2: A client updates to a new TUF spec version +## Use case 2: A client updates to a new TUF specification version Just as a repository may be upgraded, a TUF client may wish to upgrade to a new -TUF spec version to use new features. When the client implements an upgrade that +TUF specification version to use new features. When the client implements an upgrade that includes a breaking change, it cannot be sure that all repositories have also upgraded to the new specification version. Therefore, after the client upgrades they must ensure that any metadata downloaded from a repository is parsed using a compatible TUF version. If the repository is using an older version, the client should have some way to allow the update to proceed. -## Use case 3: A delegated targets role uses a different TUF spec version than the repository +## Use case 3: A delegated targets role uses a different TUF specification version than the repository A delegated role may make and sign metadata using a different version of the TUF -specification than the repository hosting the top level roles. Delegated roles -and the top level repository may be managed by people in different organizations +specification than the repository hosting the top-level roles. Delegated roles +and the top-level repository may be managed by people in different organizations who are not able to coordinate upgrading to a new version of the TUF specification. In this case, a client should be able to parse delegations that generate metadata with a different TUF specification version than the repository. @@ -103,18 +103,18 @@ be backwards compatible. We propose this TAP because as TUF continues to evolve, the need for TUF clients and repositories to upgrade their TUF metadata format will grow. This sets up the potential for clients that are no longer able to perform updates because they cannot parse the -metadata generated by repositories, or for clients to install the wrong targets +metadata generated by repositories, or for clients to install the wrong target files due to changes in TUF. Both of these issues prevent clients from installing the -targets intended by repository managers, and could lead to critical security or +targets intended by repository managers, and in extreme cases could lead to critical security or functionality problems. TUF clients and repositories need to be able to upgrade to new versions without preventing secure and reliable access to software updates. In trying to create more flexible functionality between servers and clients when -it comes to dealing with different versions of TUF, we identified two main +it comes to dealing with different versions of the TUF specification, we identified two main issues that need to be addressed. First, clients need to be able to determine whether the TUF specification version they implement is compatible with the TUF -version of the metadata they receive. This ensures that clients parse +specification version of the metadata they receive. This ensures that clients parse metadata according to the correct version to prevent errors from any changes to the metadata in the new version. Second, clients need a way to use this information to determine how to access metadata that is compatible with the @@ -122,7 +122,7 @@ version of the TUF specification they implement. To address the first issue, this TAP proposes standardizing the specification version field to separate breaking changes from non-breaking changes and -ensure that the field can be compared across clients and repositories. +ensure that the field can be compared across TUF clients and repositories. Separating out non-breaking changes allows these backwards compatible changes to happen at any time without affecting TUF's operation. Therefore, clients and repositories can still coordinate after a non-breaking change occurs. One common @@ -172,21 +172,21 @@ metadata from previous TUF specification versions. These functions allow a client to maintain old versions of the specification while still supporting the most recent version. -The top level TUF metadata (root, snapshot, timestamp, and top-level targets) +The top-level TUF metadata (root, snapshot, timestamp, and top-level targets) used to install an update should all implement the same TUF specification version. A specification change may rely on more than one metadata file (for example a change in the signing process would affect all metadata types), so -using the same specification version for top level metadata allows for these +using the same specification version for top-level metadata allows for these large changes to the specification. For information about how this relates to -url pinning and TAP 5, see [Special Cases](#tap-5). However, delegated targets may not be -managed by the same parties as the top level metadata. For this reason, this TAP +url pinning and TAP 5, see [Special Cases](#tap-5). However, delegated targets metadata may not be +managed by the same parties as the top-level metadata. For this reason, this TAP allows clients to use a different TUF specification version for delegated targets by maintaining functions to parse metadata that conforms to different specification versions. This compromise allows TUF specification changes to affect multiple metadata files while allowing flexibility in how repositories and delegations are managed. -So, during the course of an update a client could use 4.0.0_parse_root, +For example, during the course of an update a client could use 4.0.0_parse_root, 4.0.0_parse_snapshot, 4.0.0_parse_timestamp, 4.0.0_parse_targets, 4.0.0_parse_delegation, and 3.0.0_parse_delegation. @@ -194,9 +194,9 @@ So, during the course of an update a client could use 4.0.0_parse_root, # Specification As stated above, this TAP defines a variety of procedures to be added to the -update process on both clients and repos. These changes include the way in which -version numbers are determined, the addition of new directories on repositories, -new metadata parsing functions on clients, and a new process for finding +update process on both clients and repositories. These changes include the way in which +version numbers are determined, the addition of new directories on repositories +and new metadata parsing functions on clients, and a new process for finding compatible metadata. For clarity, throughout this TAP *upgrading* refers to the process of moving @@ -209,13 +209,13 @@ TAP. ## Version Number Format Using [Semantic Versioning](https://semver.org/), breaking changes will only -occur during a major release of the TUF spec (e.g. 1.x.x to 2.x.x). The +occur during a major release of the TUF specification (e.g. 1.x.x to 2.x.x). The 'spec_version' field of root metadata will follow this convention, with version numbers in the format MAJOR.MINOR.PATCH. This is a standard format used in other open source projects that makes the meaning of a TUF version number change consistent and easily understood. The Backwards Compatibility section of a TAP should be used to determine whether the TAP creates a breaking change. If the -change is not backwards compatible, such as those in TAP 3 and 8, then it will +change is not backwards compatible, such as those in TAPs 3 and 8, then it will be part of a new major version of the TUF specification. Non backwards compatible versions add features that change the way that TUF processes updates and so clients and repositories need to use code that supports the same major @@ -250,17 +250,17 @@ directories. As described in the [Rationale](#rationale), repositories should support multiple TUF specification versions. In order to do so, this TAP proposes a new directory structure for repositories. When a repository manager chooses to upgrade to a -new major TUF spec version, they create a new directory on the repository named +new major TUF specification version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all -metadata files for the new spec version, but target files will remain in the +metadata files for the new specification version, but target files will remain in the parent directory to save space. In this case the metadata files (in directories according to their spec version) can point to target files relative to the parent directory. After creating the directory, the repository creates and signs root, snapshot, timestamp, and -top level targets metadata using the new TUF spec version and places these +top-level targets metadata using the new TUF specification version and places these metadata files in the directory. The root file should be signed by both the new root key and the current root key (the root key from the most recent metadata in -the previous major spec version). Clients will now be able to use the new -metadata files once their TUF spec versions are also updated. After an update to +the previous major specification version). Clients will now be able to use the new +metadata files once their TUF specification versions are also updated. After an update to version 2.0.0, the repository structure may look like: @@ -279,7 +279,7 @@ directory named 2.0.0. Minor and patch version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. -A repository may continue to support old major TUF spec versions by creating +A repository may continue to support old major TUF specification versions by creating metadata in both the old location and the new directory. A repository manager may maintain as many versions as desired, though any version with security concerns should be phased out as soon as possible. This can be done by stopping @@ -291,14 +291,14 @@ A repository should generate all TUF metadata, including root metadata, for all TUF versions that the repository supports. Any update should be reflected across all of these versions. For clarity, version numbers used for consistent snapshots should be consistent across all -supported spec versions or a client might have difficulty finding the current +supported specification versions so that a client can find the current metadata files. This means that there may be a file at 1.0.0/3.root.json as well -as 2.0.0/3.root.json. Root files with the same consistent snapshot number must -also use the same keys so that a client can find the next root file in whichever -spec version they support. +as 2.0.0/3.root.json. Root metadata files with the same consistent snapshot number must +also use the same keys so that a client can find the next root metadata file in whichever +specification version they support. For existing TUF clients to continue operation after this TAP is implemented, -repositories may store metadata from before TUF 2.0.0 in the top level +repositories may store metadata from before TUF 2.0.0 in the top-level repository (with no directory named 1.0.0). This allows existing clients to continue downloading metadata from the repository. So a TUF repository that upgrades from version 1.12.0 to version 2.0.0 may look like: @@ -317,9 +317,9 @@ TUF clients must store the TUF specification versions they support and may add functions to maintain old versions. In order to find compatible updates on a repository, a client must keep track of the TUF specification versions it supports. To do so, a global variable or other local storage option should -contain the client spec version, or spec version range. For simplicity, this +contain the client specification version, or specification version range. For simplicity, this field should be formatted according to Semantic Versioning so that it can be -directly compared to the spec version in root metadata. +directly compared to the specification version in root metadata. There may be multiple versions of the specification supported by each TUF client. This allows metadata downloads from a repository or delegated role that @@ -344,23 +344,23 @@ following procedure: looking for the directory with the largest version number. * If the latest version on the repository is equal to that of the client, it will use this directory to download metadata. -* If the latest version pre-dates the client spec version, it may call functions +* If the latest version pre-dates the client specification version, it may call functions from a previous client version to download the metadata. The client may support as many or as few versions as desired for the application. If the previous version is not available, the client shall report that an update can not be -performed due to an old spec version on the repository. +performed due to an old specification version on the repository. * If the latest version on the repository is higher than the client spec version, the client should report to the user that it is not using the most up to date version, and then perform the update with the directory that corresponds with the latest -client spec version, if available. If no such directory exists, the client +client specification version, if available. If no such directory exists, the client terminates the update. Once the supported directory is determined, the client shall attempt the update using the metadata in this directory. -For example, if a client has a spec version of 3.5 and a repository has +For example, if a client has a specification version of 3.5 and a repository has directories for 2.0.0, 3.0.0, and 4.0.0, the client will report that spec -version 4.0 is available, then download metadata from 3.0.0. This reporting is +version 4.x is available, then download metadata from the 3.0.0 directory. This reporting is up to the discretion of an implementer, but it should be used to encourage updating the client to the most recent specification version. @@ -371,12 +371,12 @@ will report that it is unable to perform an update. Once the supported directory is determined, the client must validate root metadata from this directory. If the currently trusted root file saved on the -client uses a spec version other than the supported version, the client will +client uses a specification version other than the supported version, the client will look for the next root file. The next root file may have been generated using the supported version, or it may have been generated using a previous TUF version. The client will look for the root file first in the supported version, then move to the previous versions until the next root file is found, or until -the currently trusted root file's version is reached. All root files should be +the currently trusted root file's version is reached. All root files must be verified using the major version of the TUF client that corresponds with the major version of the root file. @@ -385,8 +385,8 @@ So, if the currently trusted root file is named 4.root.json and uses version 5.root.json first in 3.0.0, then 2.0.0, then 1.0.0. If this file is found, the client will look for 6.root.json using the same process. To facilitate this, the client should maintain functions to parse root files from previous spec -versions. If the client does not support the spec version of a root file, the -client shall terminate the update and report the spec version mismatch. +versions. If the client does not support the specification version of a root file, the +client shall terminate the update and report the specification version mismatch. ## Special Cases @@ -402,7 +402,7 @@ validation for each repository independently, and then compare the results. To do so, the client needs to ensure that the metadata from each repository is valid for the given targets file. If the repositories use different versions of the TUF specification, the client should use the version that corresponds to -each repository to validate its metadata. Results from all repositories, could +each repository to validate its metadata. Results from all repositories could then be compared. The update will only be considered valid if valid metadata from both repositories point to the same target file, and it matches the hashes provided by each repository. Note that different TUF versions may use different @@ -432,7 +432,7 @@ use the next listed metadata file or terminate the update. ### Delegations The TUF specification version(s) supported by a delegation does not need to -match the version(s) supported by the top level metadata. A delegation may use +match the version(s) supported by the top-level metadata. A delegation may use the directory structure described above to indicate which TUF specification versions it supports. The TUF client is responsible for parsing the metadata from delegations using code that implements the appropriate specification From 161db514d546c70763d61e7aee40b182a13c20a2 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 20 Jul 2020 14:49:51 -0700 Subject: [PATCH 33/40] update with semantic versioning implementation link --- candidate-tuf-versions.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 49bf0c49..03021afc 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -1,7 +1,7 @@ * TAP: * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 22-July-2019 +* Last-Modified: 20-July-2020 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown @@ -503,7 +503,8 @@ before any non-backwards compatible TAPs are released. # Augmented Reference Implementation -TODO +Semantic Versioning was added to the TUF Reference Implementation in [#914](https://github.com/theupdateframework/tuf/pull/914). +The rest of this proposal has not yet been implemented. # Copyright From f218b829c1f7a235b53fd5efdc381d86166e67d1 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Mon, 20 Jul 2020 15:19:29 -0700 Subject: [PATCH 34/40] add deprecation_timestamp field for targets metadata --- candidate-tuf-versions.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 03021afc..033eb5dc 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -287,6 +287,30 @@ the creation of new metadata files in the directory for the phased out version. In order to allow clients to parse the root metadata chain, root metadata files shall not be deleted even once a version is deprecated. +A repository may indicate the planned phase out of a major version using an +optional `deprecation_timestamp` field in targets metadata. This field can be +added to targets metadata once a date is set for deprecation of a TUF +specification version by the repository and will include the timestamp after +which the metadata for the current specification version will no longer be +maintained. Inclusion of this field will signal to clients both that they will +need to upgrade to the next specification version before the timestamp, and +that there will not be metadata available in the current directory after this +point. Both top-level and delegated targets metadata may use this field to +indicate the point after which the given TUF specification version will no +longer be supported. With the addition of this field, the "signed" portion of +targets metadata will include the following: + +``` +{ "_type" : "targets", + "spec_version" : SPEC_VERSION, + "version" : VERSION, + "expires" : EXPIRES, + "targets" : TARGETS, + ("deprecation_timestamp": DEPRECATION_TIMESTAMP), + ("delegations" : DELEGATIONS) +} +``` + A repository should generate all TUF metadata, including root metadata, for all TUF versions that the repository supports. Any update should be reflected across all of these versions. For clarity, version numbers used for consistent @@ -388,6 +412,17 @@ client should maintain functions to parse root files from previous spec versions. If the client does not support the specification version of a root file, the client shall terminate the update and report the specification version mismatch. +When a client validates targets metadata, they may check for the +`deprecation_timestamp` field. If this field is present, the client should do +the following: +* If the current time is after the time listed in `deprecation_timestamp`, the +client should report to the user that the update is not available due to a +version mismatch and consider the current targets metadata file invalid. If +this is the top-level targets file, the update should be terminated. +* Otherwise, the client should warn the user that the TUF specification version +they are using will be deprecated at the time indicated by `deprecation_timestamp` +and proceed with the update. + ## Special Cases From 94183f944f32a3af6de6baae2249aed28b7998c6 Mon Sep 17 00:00:00 2001 From: Marina Moore Date: Tue, 21 Jul 2020 07:59:09 -0700 Subject: [PATCH 35/40] Apply suggestions from code review Co-authored-by: Joshua Lock --- candidate-tuf-versions.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 033eb5dc..353dd006 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -287,12 +287,12 @@ the creation of new metadata files in the directory for the phased out version. In order to allow clients to parse the root metadata chain, root metadata files shall not be deleted even once a version is deprecated. -A repository may indicate the planned phase out of a major version using an -optional `deprecation_timestamp` field in targets metadata. This field can be -added to targets metadata once a date is set for deprecation of a TUF -specification version by the repository and will include the timestamp after -which the metadata for the current specification version will no longer be -maintained. Inclusion of this field will signal to clients both that they will +A repository may indicate the planned phase out of metadata conforming to a major version +of the TUF specification using an optional `deprecation_timestamp` field in targets metadata. +This field can be added to targets metadata once a date is set for deprecation of metadata +adhering to a TUF specification version by the repository and will include the timestamp after +which the metadata for the current specification version will no longer be maintained. +Inclusion of this field will signal to clients both that they will need to upgrade to the next specification version before the timestamp, and that there will not be metadata available in the current directory after this point. Both top-level and delegated targets metadata may use this field to @@ -325,12 +325,12 @@ For existing TUF clients to continue operation after this TAP is implemented, repositories may store metadata from before TUF 2.0.0 in the top-level repository (with no directory named 1.0.0). This allows existing clients to continue downloading metadata from the repository. So a TUF repository that -upgrades from version 1.12.0 to version 2.0.0 may look like: +upgrades from version 1.0.0 to version 2.0.0 may look like: ``` - Targets files -- 1.12.0 metadata files +- 1.0.0 metadata files - 2.0.0 |- 2.0.0 metadata files ``` From d75e5c91da21ca19d38cec3553d0f269dc709906 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Tue, 21 Jul 2020 08:15:07 -0700 Subject: [PATCH 36/40] replace deprecation_timestamp with becomes_obsolete --- candidate-tuf-versions.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 353dd006..554fd951 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -288,10 +288,10 @@ In order to allow clients to parse the root metadata chain, root metadata files shall not be deleted even once a version is deprecated. A repository may indicate the planned phase out of metadata conforming to a major version -of the TUF specification using an optional `deprecation_timestamp` field in targets metadata. +of the TUF specification using an optional `becomes_obsolete` field in targets metadata. This field can be added to targets metadata once a date is set for deprecation of metadata adhering to a TUF specification version by the repository and will include the timestamp after -which the metadata for the current specification version will no longer be maintained. +which the metadata for the current specification version will no longer be maintained. Inclusion of this field will signal to clients both that they will need to upgrade to the next specification version before the timestamp, and that there will not be metadata available in the current directory after this @@ -306,7 +306,7 @@ targets metadata will include the following: "version" : VERSION, "expires" : EXPIRES, "targets" : TARGETS, - ("deprecation_timestamp": DEPRECATION_TIMESTAMP), + ("becomes_obsolete": BECOMES_OBSOLETE), ("delegations" : DELEGATIONS) } ``` @@ -413,14 +413,14 @@ versions. If the client does not support the specification version of a root fil client shall terminate the update and report the specification version mismatch. When a client validates targets metadata, they may check for the -`deprecation_timestamp` field. If this field is present, the client should do +`becomes_obsolete` field. If this field is present, the client should do the following: -* If the current time is after the time listed in `deprecation_timestamp`, the +* If the current time is after the time listed in `becomes_obsolete`, the client should report to the user that the update is not available due to a version mismatch and consider the current targets metadata file invalid. If this is the top-level targets file, the update should be terminated. * Otherwise, the client should warn the user that the TUF specification version -they are using will be deprecated at the time indicated by `deprecation_timestamp` +they are using will be deprecated at the time indicated by `becomes_obsolete` and proceed with the update. From 3fe4e522ec4cf6aaadef58edb823ba82c73cfcf4 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Tue, 21 Jul 2020 09:50:48 -0700 Subject: [PATCH 37/40] additionally add becomes_obsolete to the timestamp metadata --- candidate-tuf-versions.md | 41 +++++++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 554fd951..ba30478b 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -287,18 +287,23 @@ the creation of new metadata files in the directory for the phased out version. In order to allow clients to parse the root metadata chain, root metadata files shall not be deleted even once a version is deprecated. -A repository may indicate the planned phase out of metadata conforming to a major version -of the TUF specification using an optional `becomes_obsolete` field in targets metadata. -This field can be added to targets metadata once a date is set for deprecation of metadata -adhering to a TUF specification version by the repository and will include the timestamp after -which the metadata for the current specification version will no longer be maintained. -Inclusion of this field will signal to clients both that they will +A repository may indicate the planned phase out of metadata conforming to a +major version of the TUF specification using an optional `becomes_obsolete` +field in targets and timestamp metadata. This field can be added to targets and +timestamp metadata once a date is set for deprecation of metadata adhering to a +TUF specification version by the repository and will include the timestamp after +which the metadata for the current specification version will no longer be +maintained. Inclusion of this field will signal to clients both that they will need to upgrade to the next specification version before the timestamp, and that there will not be metadata available in the current directory after this -point. Both top-level and delegated targets metadata may use this field to -indicate the point after which the given TUF specification version will no -longer be supported. With the addition of this field, the "signed" portion of -targets metadata will include the following: +point. The client can check the `becomes_obsolete` field in timestamp metadata +before checking the metadata expiration. This allows them to differentiate +between metadata that is expired due to deprecation and metadata that is expired +due to an attack. Both timestamp and delegated targets metadata may use this +field to indicate the point after which the given TUF specification version will +no longer be supported. This means that delegated targets may include this field +independent of top-level metadata. With the addition of this field, the "signed" +portion of targets metadata will include the following: ``` { "_type" : "targets", @@ -311,6 +316,18 @@ targets metadata will include the following: } ``` +And the "signed" portion of timestamp will include: + +``` +{ "_type" : "timestamp", + "spec_version" : SPEC_VERSION, + "version" : VERSION, + "expires" : EXPIRES, + ("becomes_obsolete": BECOMES_OBSOLETE), + "meta" : METAFILES +} +``` + A repository should generate all TUF metadata, including root metadata, for all TUF versions that the repository supports. Any update should be reflected across all of these versions. For clarity, version numbers used for consistent @@ -412,13 +429,13 @@ client should maintain functions to parse root files from previous spec versions. If the client does not support the specification version of a root file, the client shall terminate the update and report the specification version mismatch. -When a client validates targets metadata, they may check for the +When a client validates timestamp and targets metadata, they may check for the `becomes_obsolete` field. If this field is present, the client should do the following: * If the current time is after the time listed in `becomes_obsolete`, the client should report to the user that the update is not available due to a version mismatch and consider the current targets metadata file invalid. If -this is the top-level targets file, the update should be terminated. +this is the timestamp or top-level targets file, the update should be terminated. * Otherwise, the client should warn the user that the TUF specification version they are using will be deprecated at the time indicated by `becomes_obsolete` and proceed with the update. From a6b31052fe55143a3ae37572ea1f16749c8ee387 Mon Sep 17 00:00:00 2001 From: marinamoore Date: Fri, 24 Jul 2020 10:26:36 -0700 Subject: [PATCH 38/40] move becomes_obsolete to root Signed-off-by: marinamoore --- candidate-tuf-versions.md | 37 ++++++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 15 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index ba30478b..80201a92 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -289,42 +289,49 @@ shall not be deleted even once a version is deprecated. A repository may indicate the planned phase out of metadata conforming to a major version of the TUF specification using an optional `becomes_obsolete` -field in targets and timestamp metadata. This field can be added to targets and -timestamp metadata once a date is set for deprecation of metadata adhering to a +field in root and targets metadata. This field can be added to root and targets +metadata once a date is set for deprecation of metadata adhering to a TUF specification version by the repository and will include the timestamp after which the metadata for the current specification version will no longer be maintained. Inclusion of this field will signal to clients both that they will need to upgrade to the next specification version before the timestamp, and that there will not be metadata available in the current directory after this -point. The client can check the `becomes_obsolete` field in timestamp metadata +point. The client can check the `becomes_obsolete` field in root metadata before checking the metadata expiration. This allows them to differentiate between metadata that is expired due to deprecation and metadata that is expired -due to an attack. Both timestamp and delegated targets metadata may use this -field to indicate the point after which the given TUF specification version will -no longer be supported. This means that delegated targets may include this field -independent of top-level metadata. With the addition of this field, the "signed" -portion of targets metadata will include the following: +due to an attack. Delegated targets may include this field independent of +top-level metadata to indicate when the metadata they are responsible for will +no longer support a given TUF specification version. With the addition of this +field, the "signed" portion of targets metadata will include the following: ``` { "_type" : "targets", "spec_version" : SPEC_VERSION, "version" : VERSION, "expires" : EXPIRES, - "targets" : TARGETS, ("becomes_obsolete": BECOMES_OBSOLETE), + "targets" : TARGETS, ("delegations" : DELEGATIONS) } ``` -And the "signed" portion of timestamp will include: +And the "signed" portion of root will include: ``` -{ "_type" : "timestamp", +{ "_type" : "root", "spec_version" : SPEC_VERSION, + "consistent_snapshot": CONSISTENT_SNAPSHOT, "version" : VERSION, "expires" : EXPIRES, ("becomes_obsolete": BECOMES_OBSOLETE), - "meta" : METAFILES + "keys" : { + KEYID : KEY + , ... }, + "roles" : { + ROLE : { + "keyids" : [ KEYID, ... ] , + "threshold" : THRESHOLD } + , ... } } ``` @@ -429,13 +436,13 @@ client should maintain functions to parse root files from previous spec versions. If the client does not support the specification version of a root file, the client shall terminate the update and report the specification version mismatch. -When a client validates timestamp and targets metadata, they may check for the +When a client validates root and targets metadata, they may check for the `becomes_obsolete` field. If this field is present, the client should do the following: * If the current time is after the time listed in `becomes_obsolete`, the client should report to the user that the update is not available due to a -version mismatch and consider the current targets metadata file invalid. If -this is the timestamp or top-level targets file, the update should be terminated. +version mismatch and consider the current metadata file invalid. If +this is the root or top-level targets file, the update should be terminated. * Otherwise, the client should warn the user that the TUF specification version they are using will be deprecated at the time indicated by `becomes_obsolete` and proceed with the update. From b5d3bd92fbe2dc645c2db762a95e2a10a3159beb Mon Sep 17 00:00:00 2001 From: marinamoore Date: Thu, 6 Aug 2020 14:56:26 -0700 Subject: [PATCH 39/40] Add clarifications from review --- candidate-tuf-versions.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/candidate-tuf-versions.md b/candidate-tuf-versions.md index 80201a92..19910776 100644 --- a/candidate-tuf-versions.md +++ b/candidate-tuf-versions.md @@ -165,10 +165,10 @@ directories, a client is able to choose the most recent metadata they support. More details about this directory structure are contained in the [specification](#how-a-repository-updates). On the client side, this TAP also requires maintenance of multiple versions that -support different TUF specification +support different major TUF specification versions to allow for communication with various repositories. To do so, it is recommended that clients maintain functions that can be used to validate -metadata from previous TUF specification versions. These functions allow a +metadata from previous major TUF specification versions. These functions allow a client to maintain old versions of the specification while still supporting the most recent version. @@ -307,9 +307,9 @@ field, the "signed" portion of targets metadata will include the following: ``` { "_type" : "targets", "spec_version" : SPEC_VERSION, + ("becomes_obsolete": BECOMES_OBSOLETE), "version" : VERSION, "expires" : EXPIRES, - ("becomes_obsolete": BECOMES_OBSOLETE), "targets" : TARGETS, ("delegations" : DELEGATIONS) } @@ -320,10 +320,10 @@ And the "signed" portion of root will include: ``` { "_type" : "root", "spec_version" : SPEC_VERSION, + ("becomes_obsolete": BECOMES_OBSOLETE), "consistent_snapshot": CONSISTENT_SNAPSHOT, "version" : VERSION, "expires" : EXPIRES, - ("becomes_obsolete": BECOMES_OBSOLETE), "keys" : { KEYID : KEY , ... }, @@ -335,6 +335,8 @@ And the "signed" portion of root will include: } ``` +where `BECOMES_OBSOLETE` is a timestamp. + A repository should generate all TUF metadata, including root metadata, for all TUF versions that the repository supports. Any update should be reflected across all of these versions. For clarity, version numbers used for consistent @@ -364,8 +366,8 @@ upgrades from version 1.0.0 to version 2.0.0 may look like: TUF clients must store the TUF specification versions they support and may add functions to maintain old versions. In order to find compatible updates on a repository, a client must keep track of the TUF specification versions it -supports. To do so, a global variable or other local storage option should -contain the client specification version, or specification version range. For simplicity, this +supports. To do so, local storage on the client should contain the client +specification version, or specification version range. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the specification version in root metadata. @@ -447,7 +449,6 @@ this is the root or top-level targets file, the update should be terminated. they are using will be deprecated at the time indicated by `becomes_obsolete` and proceed with the update. - ## Special Cases ### Multiple Repositories @@ -492,9 +493,9 @@ use the next listed metadata file or terminate the update. The TUF specification version(s) supported by a delegation does not need to match the version(s) supported by the top-level metadata. A delegation may use -the directory structure described above to indicate which TUF specification +the directory structure described above to indicate which major TUF specification versions it supports. The TUF client is responsible for parsing the metadata -from delegations using code that implements the appropriate specification +from delegations using code that implements the appropriate major specification version (found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process)). If there is no compatible specification version between the client and delegation, the client @@ -560,6 +561,10 @@ for an attacker on the network with the ability to alter network traffic. This TAP is backwards compatible and should be implemented on all repositories before any non-backwards compatible TAPs are released. +Existing TUF implementations should continue to store TUF 1.0.0 metadata +using their existing method to maintain backwards compatibility as described +in the specification section. + # Augmented Reference Implementation Semantic Versioning was added to the TUF Reference Implementation in [#914](https://github.com/theupdateframework/tuf/pull/914). From b6b04e9e778cd92fa5c2e7238b64541c0944878d Mon Sep 17 00:00:00 2001 From: marinamoore Date: Fri, 7 Aug 2020 08:57:33 -0700 Subject: [PATCH 40/40] Add TAP number 14 to the version management TAP Signed-off-by: marinamoore --- README.md | 1 + candidate-tuf-versions.md => tap14.md | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) rename candidate-tuf-versions.md => tap14.md (99%) diff --git a/README.md b/README.md index 0eaf39db..ac3d662d 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ * [TAP 5: Setting URLs for roles in the root metadata file](tap5.md) * [TAP 8: Key rotation and explicit self-revocation](tap8.md) +* [TAP 14: Managing TUF Versions](tap14.md) ## Rejected diff --git a/candidate-tuf-versions.md b/tap14.md similarity index 99% rename from candidate-tuf-versions.md rename to tap14.md index 19910776..387af470 100644 --- a/candidate-tuf-versions.md +++ b/tap14.md @@ -1,7 +1,7 @@ -* TAP: +* TAP: 14 * Title: Managing TUF Versions * Version: 1 -* Last-Modified: 20-July-2020 +* Last-Modified: 07-August-2020 * Author: Marina Moore, Justin Cappos * Status: Draft * Content-Type: text/markdown