Transporting CV graphs with CVs shared with separately transported graphs causes problems

[mandelsoft]

What happened:

If there are two aggregated component versions sharing a common CV, transportation is hardly possible for signed components.

• even for unsigned CVs the transport fails for the transport of the second CV, because the shared one is already there.
  The check (basically a tweaked deep equal) for valid existing components fails, because the access methods might be touched.
  So far, it is not possible to detect whether a component is really the same without reading and comparing the blobs of both variants.
  The only solution so far is the option -f, which overwrites the existing CVs in the target
• just overwriting is no solution because:
  • it should be able to fail if the shared CV contains different technical resources
  • digests and signature would be lost if the second transport contains no or different digests
  • what should happen is both aggregated CVs are digested with different algorithms.

What you expected to happen:

It must be possible to transport partly shared CV graphs, which are independently signed in a safe way, guaranteeing that all possible
transportation sequences finally provide the same result.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know:

Environment:

[mandelsoft]

Basically, we urgently have to rework the creation of component versions.

First: The only way to attack this problem is to provide the necessary resource digests as early as possible: during the CV creation.

This means:

• the digest algorithm is determined already when creating a CV
  -> no different algorithms possible for different signing sequences for different aggregation CVs
  -> during the transport it is possible to check the identity of resources, just by comparing the digest.
  -> creating with access instead of input will be more expensive, because all blobs must be read
  -> creating resources by access will only be possible if the appropriate access type is known to the actual version
  of the toolset
• add check to equality to transport if CV is already present at target based on the now available digests.
  • fail if content differs and accept is content is identical
  • still support the -f option to be able to resolve conflicts.
  • problem: transport of signatures

Second: During transport the signatures must be merged. Even if the CV describes the same content, the signatures in the
target must potentially be adapted to reflect both, the target and the source signatures.

Problem: how to handle resigning with the same signature identity?
Transporting should update the signature -> we need a signing timestamp to identify the latest one. Is this really secure?

[mandelsoft]

The next problem arises with labels, which are not signature relevant.
Those labels are intended to be attached on-demand. to annotate a CV with additional information.

One useful scenario here could be the implementation of a routing slip which gathers information about executed processing steps on the CV.

Here, we also have a merging problem during the transport of shard components versions (the existence of a label as well as different values). But in contrast to the ones described in the initial problem description, the labels are formal fields, but fields maintained by external tools. They are not formally known by the core model. Nevertheless, we cannot just always throw an error for non-matching label content or just ignore it.

One solution would be (similar to identity matchers) introduce formal label mergers. A label may additionally declare the matcher to use. There will be a set of standard mergers (default is just add a non-existing label). and we have to introduce a registration mechanism similar to the one for access methods.

Later on, the existing plugin concept could be used to extend the API to also support such mergers, Then they could automatically be downloaded on-the-fly, when required by a transport step. (The download and update mechanisms for executables and plugins are already supported by the OCM library).

[mandelsoft]

The following scenario must be handled correctly, also:

• an already transported CV is locally signed in the target environment
• an update with (non-signature relevant changes, for example routing slip is imported)...
• ...with potentially additional signatures.

Here, also the early digesting and the proposed merge operation could help to support this scenario.