Change notice related business terms - modelling guidance required

[odscjen]

A number of business terms relate to change notices. Some guidance explaining how eForms' modelling of change is represented in OCDS is required. Related business terms include:

• BT-140
• BT-141
• BT-1501(n)-Contract
• BT-1501(s)-Contract

[duncandewhurst]

See https://docs.ted.europa.eu/eforms/1.1.1/schema/all-in-one.html#changesElementsSection for a full list of related business terms.

A change notice in eForms can have multiple changes but only one change reason, like the example which has two changes, each relating to a different lot, and one change reason.

The draft guidance for BT-140 (change reason) is to create an amendment object and the draft guidance for BT-141 (change description) is to map to the object created for BT-140. I'm not sure that this makes sense given the above context. The alternative would be to create an amendment object for each change and repeat the information from BT-140 in each amendment object. @jpmckinney what do you think?

Edit: Updated links since version 1.1.0 seems to have disappeared from the eForms documentation.

[jpmckinney]

I'm not clear on the issue with having one amendment object per set of changes?

[duncandewhurst]

I was thinking about:

• The loss of structure that would result from concatenating many change descriptions into one Amendment.description.
• How to map many 'Change Previous Section Identifier' (BT-13716), 'Procurement Documents Change Date' (BT-719) and 'Procurement Documents Change Indicator' (BT-718) to one amendment.

[jpmckinney]

Aha, yes, eForms has a facility to group multiple Change into Changes (i.e. a changeset), and the Changes section can appear across multiple notices. OCDS, on the other hand, has only one relevant amendments array, which can grow across releases, but it has no second-level of organization to match eForms' changeset.

So, in OCDS, we would need to duplicate the rationale.

To count the number of changesets, I guess a user would need to count the number of releases with a 'tenderAmendment' tag rather than count the number of amendments (which is instead the number of Change elements).

[duncandewhurst]

Sounds good!

[duncandewhurst]

The guidance also needs to explain which array to add the amendment to: tender.amendments, awards.amendments or contracts.amendments.

A change in eForms has two fields that reference what changed:

id	name	description	repeatable	mandatory	notes
BT-758-notice	Change Notice Version Identifier	The reference to the version of the previous notice that should be changed. This helps, for example, to avoid errors caused by multiple change notices sent at a similar time.	false	true
BT-13716-notice	Change Previous Section Identifier	An identifier of one or more sections within the changed notice. The information in the change section refers to this section or these sections.	true	false	Possible values are PROCEDURE, BUYER or RESULT, or a section identifier of the form PAR-XXXX, LOT-XXXX, GLO-XXXX, RES-XXXX or ORG-XXXX.

I think that BT-758 can be discarded since the release for a change notice will be related to previous notices by its ocid.

If BT-13716 references a LotResult identifier (RES-XXXX), then an amendment can be added to the .amendments array of the associated Award. Otherwise, I think the only option is to add the amendment to tender.amendments since there's no way of determining which award or contract a change relates to. I think that BT-13716 can itself be discarded as it relates to the structure of eForms rather than OCDS.

Each change notice must include at least one efac:Change. However, none of the properties under efac:Change are mandatory. Apart from BT-758, which is discarded, the only mandatory field in <efac:Changes> is BT-140 (Change Reason Code).

Therefore I think it's best to document the guidance on adding amendment objects as part of BT-140 and to reference that in the guidance for the other business terms.

Draft guidance

BT-140 (Change Reason Code)

For each ancestor::efac:Changes/efac:Change:

• If the change's /efbc:ChangedSectionIdentifier (BT-13716) references a LotResult (RES-XXXX), get the award for the LotResult and add an Amendment object to the award's .amendments array. Otherwise, add an Amendment object to the tender.amendments array.
• If the change's /efbc:ChangedSectionIdentifier (BT-13716) references a Lot (LOT-XXXX), add the Lot's identifier to the amendment's .relatedLots array.
• Set the amendment's .id sequentially across all notices for this procedure. For example, if the first change notice for a procedure has three changes, it uses id's '1' through '3'. A second change notice for the same procedure then uses id's '4' and up, etc.
• Add a Classification object to the amendment's rationaleClassifications array and map to its .id.
• Look up the code's label in the authority table and map it to the classification's .description.
• Set the classification's .scheme to 'change-corrig-justification'.

BT-141 (Change Description)

These values map to the same Amendment objects as created for BT-140. Update the corresponding Amendment object and map to its .description.

BT-762 (Change Reason Description)

These values map to the same Amendment objects as created for BT-140. Update the corresponding Amendment objects and map to its .rationale.

BT-719 (Procurement Documents Change Date)

These values map to the same Amendment objects as created for BT-140. Update the corresponding Amendment object and map to its .documentsChangeDate.

BT-718 (Procurement Documents Change Indicator)

These values map to the same Amendment objects as created for BT-140. Update the corresponding Amendment object and map to its .documentsChanged.

@jpmckinney does that sound good?

[duncandewhurst]

We could add a relatedLots field to Amendment so that, when BT-13716-notice references a lot (LOT-XXXX), we can link the amendment to the specific lots the change relates to. I need to think more about how to handle references to parts and groups of lots.

[jpmckinney]

Yes, guidance looks good.

For groups of lots, we can maybe return to it after #71.

[duncandewhurst]

Sounds good. I've updated guidance.yaml with the proposed mappings.

[duncandewhurst]

We could add a relatedLots field to Amendment so that, when BT-13716-notice references a lot (LOT-XXXX), we can link the amendment to the specific lots the change relates to. I need to think more about how to handle references to parts and groups of lots.

Based on #71 (comment), I think that we should add a relatedLotGroups field to Amendment for use when BT-13716-notice references a lots group (GLO-XXXX).

[odscjen]

BT-1501(n) and BT-1501(s) appear to have been forgotten about in this thread. The current (UNREVIEWED) guidance is:

No guidance. Relationships are either implied by the XML structure or explicit via references from a section to the organization.

Which I don't think is particularly clear or consistent in style with the rest of the guidance. Suggest changing it to:

Discard. The notice/section(s) changed can be implied through comparison of previous releases with the current amended release.

Where notice would be used for BT-1501(n) and section(s) for BT-1501(s)

@jpmckinney @duncandewhurst

[jpmckinney]

A bit of copy-editing:

Discard. The notice and sections that were modified can be determined by comparing previous releases with the current release.

[odscjen]

That's a much better sentence :) I've updated guidance.yaml

[odscjen]

All the BT's referenced in this issue have been updated in guidance.yaml so I'm closing this issue

[jpmckinney]

Copying discussion from a deleted extension related to BT-719:

odscjen

The document object contains the field dateModified that is similar to the new field documentsChangeDate in this extension. Should the README contain some general guidance on when to use this extension instead of or in addition to documents.dateModified? I can see this being confusing if anyone tries to use this extension for non-eforms data.

Otherwise this all looks fine.

duncandewhurst

Ah, good spot. Given that the semantics of the two fields are the same I wonder if we should instead map BT-719 to tender.documents.dateModified. The procurement documents fields in eForms are mandatory so there should also be a document to set .dateModified on.

The only downsides that I can see are:

• Only the latest modification date would be reflected in a compiled release. Whereas, all the dates would be available if mapped to tender.amendments.documentsChangeDate.
• The mapping would be more complicated because, for changes associated with a lot, it would involve looking up the document in tender.documents with 'biddingDocuments' in .role and the lot identified in <efbc:ChangedSectionIdentifier> in .relatedLots.
  @jpmckinney what do you think?

jpmckinney

It makes sense to map to documents. I think it's similar complexity, because looking up the award (per BT-140) is a bit complicated already.

I think it's fine to preserve only the most recent date (in the case of a document being amended multiple times).

If we change the mapping for BT-719 in this way, what about BT-718? Right now it adds a documentsChanged boolean, but as a user, I'm probably comparing e.g. my last access date to the modifiedDate, so maybe it can be discarded?

jpmckinney

By the way, GMail is merging multiple "Author extension" PRs into one thread, which is confusing, so for any additional extensions you can add the title of the extension in the PR name to avoid that.

duncandewhurst

By the way, GMail is merging multiple "Author extension" PRs into one thread, which is confusing, so for any additional extensions you can add the title of the extension in the PR name to avoid that.

Will do. I'll rename the existing PRs as I come to them, too.

duncandewhurst

If we change the mapping for BT-719 in this way, what about BT-718? Right now it adds a documentsChanged boolean, but as a user, I'm probably comparing e.g. my last access date to the modifiedDate, so maybe it can be discarded?

From the perspective of a bidder, I think checking dateModified is probably sufficient, so it depends on whether there are any use cases that depend on knowing whether a specific amendment included a change to documents, or counting the number of amendments that include a change to documents. I don't know if there are use cases relating to integrity or efficiency to do with identifying processes in which the documents changed many times. That said, that could still be done by looking at dateModified in a versioned release. Unless you know of any such use cases, I'm happy to discard it.

jpmckinney

Let's discard it. I think it's probably a bit too narrow for there to be integrity red flags developed around it.

duncandewhurst

I've updated guidance.yaml (9c2cba5).

This extension is no longer needed so, in order to preserve this discussion, I'll archive it rather than delete it.