repository design: survey existing implementation

[jku]

Let's starts #1136 with a "competitor analysis": survey existing TUF repository implementations from a adopter/integrator perspective:

• Take a look at existing implementations, document the APIs they provide and use cases they seem to cater for:
  • python-tuf repository_tool / repository_lib
  • current Warehouse integration branch
  • go-tuf
  • tough
  • repository-editor-for-tuf
  • others?

As bonus tasks:

• Try to define the spectrum of potential downstreams (integration into a webservice vs stand-alone tool, files vs postgres, etc)
• Maybe start defining the potential supported use cases (snapshot update, repository initialization, etc)

[joshuagl]

The current warehouse integration branch is, I believe, pypi/warehouse#7488

Some potential use cases / areas:

• initial repository setup
• key management (e.g. safe key rotations)
• delegation management
• developer actions: adding targets at least (anything else?)
• snapshot update
• timestamp update
• ensuring repository validity (this comes up in several other use cases)

[jku]

Documenting a few items from first discussion:

• Jussi: a lot of the code we will look at may be designed with command line interfaces in mind: this probably won't prevent the components from being useful in e.g. Warehouse-like integrations but let's keep the possible bias in mind
• Lukas: a lot of the design in repository_lib/tool is sound, it's just the implementation that is the issue
• Let's try to look at a few code bases, document
  • roughly what API (or tools) are provided for repository side
  • what kind of components are used to do that
• Lukas documents python-tuf, Jussi looks at tough and also writes something about repository-editor-for-tuf
• Meet again in a week, compare notes: see if it makes sense to write a document that collects the info in same style and form in one place

[joshuagl]

This diagram may be useful for the repository_lib/tool survey repository_tool-diagram.png

[lukpueh]

Updated on 2021-10-28 to add link to warehouse review

---

Just discussed existing implementations with @jku and @rdimitrov (thanks!).

Here are links to our reviews:

• python-tuf
• repository-editor-for-tuf
• go-tuf and php-tuf
• AWS Tough
• Warehouse

And here are some randomly ordered reoccurring issues from our discussion:

• state keeping in memory and on disk
• (fail late) validation of repository consistency
• library vs. REPL vs. CLI use
• metdata update interdependencies (see e.g. #958)
• being able to work on parts of a repository without (read/write/sign) access to all metadata or target files or keys
• TUF spec has little instructions for repository side activities

The repository tool re-design will be tracked in #1136 and further brain storming may take place in this TUF (Re-)Design Document.

[jku]

WRT review: the big missing part is Warehouse review, no-one's yet had a good look at it.

[lukpueh]

Just updated my comment above to add a link to the Warehouse review.

[lukpueh]

Closing with #1136 (comment)