BT-330, BT-1375, BT-556: Write guidance for groups of lots

[ColinMaudry]

We should clarify in guidance how groups of lots translate from eForms to OCDS.

Groups of lots exist in eForms to:

• let buyers group lots that share the same award criteria (BT-330, BT-1375)
• express the maximum or estimated value of a group of lots within a framework agreement (BT-556)

[ColinMaudry]

I will add links the lot extension page in the guidance, I think the example explains well how OCDS lots and groups of lots work.

[jpmckinney]

Possibly relevant: open-contracting/ocds-extensions#183

[duncandewhurst]

From @jpmckinney via Slack:

Since there are a lot of cases where eForms allows groups of lots, we should maybe just have a general guidance page saying that, in the case of a group of lots, perform the mapping for each of the group’s lots

[duncandewhurst]

Some fields map to properties of LotGroup (BT-22-LotsGroup, BT-21-LotGroups, BT-24-LotGroups) and for others the mapping should be performed for each lot in the group, so I think we need two sets of guidance for lot groups:

For fields that map to properties of LotGroup:

Get the lot group for a ProcurementProjectLot
Get the LotGroup in tender.lotGroups whose .id is equal to the value of the XPath ancestor::cac:ProcurementProjectLot/cac:ID. If none exists yet, add a LotGroup to tender.lotGroups and set its id to the value of the XPath ancestor::cac:ProcurementProjectLot/cac:ID.

Then we can update the guidance for BT-21-LotGroups, for example, to:

Get the lot group for the ProcurementProjectLot and map to its .title

For fields that map to each lot in a lot group:

Get the lots in a lot group
Get the cac:TenderingTerms/cac:LotDistribution/cac:LotsGroup whose /cbc:LotsGroupID is equal to the value of the XPath ancestor::cac:ProcurementProjectLot/cac:ID. For each /cac:ProcurementProjectLotReference, get the Lot in tender.lots whose .id is equal to the value of /cbc:ID. If none exists yet, add a Lot to tender.lots and set its id to the value of /cbc:ID.

Then we can update the guidance for the related fields to

Get the lots in the lot group and map to each lot's...

---

On closer inspection, I'm not sure the latter case actually exists. I need to look into why, for example, the eForms guidance for BT-122 and BT-123 mentions both lots and groups of lots, but guidance.yaml only has BT-122-Lot and BT-123-Lot and not the -LotsGroup equivalents. The xpathAbsolute for these fields seems to restrict their use to lots only by setting @schemeName to 'Lot', e.g. /*/cac:ProcurementProjectLot[cbc:ID/@schemeName='Lot']/cac:TenderingProcess/cac:AuctionTerms/cbc:Description.

[jpmckinney]

The eForms SDK GitHub discussions are monitored - you can ask for clarification there.

[duncandewhurst]

Great. I've done that: OP-TED/eForms-SDK#134

[duncandewhurst]

On closer inspection, I'm not sure the latter case actually exists. I need to look into why, for example, the eForms guidance for BT-122 and BT-123 mentions both lots and groups of lots, but guidance.yaml only has BT-122-Lot and BT-123-Lot and not the -LotsGroup equivalents.

The eForms team confirmed that these business terms only apply to lots. They will update the documentation.

I checked through all the fields with a -LotsGroup suffix to determine whether the approach proposed in #71 (comment) of performing mappings for all lots in a group is actually needed. The details are below, but my conclusion is that the approach only applies to one field (BT-726-LotsGroup) so we can just document the guidance in the mapping for that field.

Fields mapped to tender.lotGroups

The following fields map to tender.lotGroups so there's no need to perform the mapping for each lot in the group:

• BT-157-LotsGroup (Group Framework Estimated Maximum Value)
• BT-21-LotsGroup (Title)
• BT-22-LotsGroup (Internal Identifier)
• BT-24-LotsGroup (Description)
• BT-27-LotsGroup (Estimated Value)
• BT-300-LotsGroup (Additional Information)

LotsGroup award criteria

The following fields are currently mapped to tender.lots.awardCriteria:

• BT-539-LotsGroup (Award Critereon Type)
• BT-540-LotsGroup (Award Criterion Description)
• BT-541-LotsGroup (Award Criterion Number)
• BT-5421-LotsGroup (Award Criterion Number Weight)
• BT-5422-LotsGroup (Award Criterion Number Fixed)
• BT-5423-LotsGroup (Award Criterion Number Threshold)
• BT-543-LotsGroup (Award Criteria Complicated)
• BT-733-LotsGroup (Award Criteria Order Justification)
• BT-734-LotsGroup (Award Criterion Name)

However, according to https://docs.ted.europa.eu/eforms/1.1.1/schema/all-in-one.html#selectionCriteriaSection:

When Lots of a Group may be awarded individually, it may be necessary to distinguish Group and Lots, and have criteria expressed for both [...] The same logic applies to awarding criteria

Therefore, I think that these fields should be mapped to tender.lotGroups instead.

Fields to be discarded

I've asked for clarification, but I think that OPT-090-LotsGroup is an error and, actually, OPT-090 can only appear in the context of individual lots. In any case, I think that this field can be discarded since it only serves to contextualise BT-111-Lot within the eForms XML structure. In OCDS, the mapping of BT-111-Lot provides the necessary context.

Fields related to unpublished information

The following fields will be addressed by #112:

• BT-195(BT-XXX)-LotsGroup
• BT-196(BT-XXX)-LotsGroup
• BT-197(BT-XXX)-LotsGroup
• BT-198(BT-XXX)-LotsGroup

Identifiers

BT-137-LotsGroup and BT-330-Procedure are both the identifier of a lots group. I think that these fields should both map to tender.lotGroups.id with the following guidance:

If there is a LotGroup in tender.lotGroups whose .id is equal to the value of this field, discard. Otherwise, add a LotGroup to tender.lotGroups and map to its .id.

Fields for which a mapping needs to be performed for each lot in a lots group

For BT-726-LotsGroup (Suitable For SMEs), I can't see how there could be different values for a lots group and the individual lots in the group, so I think the proposal in #71 (comment) to perform the mapping for each of the group's lots makes sense.

[duncandewhurst]

BT-1375-Procedure is also related to groups of lots. Proposed guidance:

Get the LotGroup in tender.lotGroups whose id is equal to the value of the XPath ancestor::cac:LotsGroup/cbc:LotsGroupID. If none exists yet, add a LotGroup to tender.lotGroups and set it's id to the value of the XPath ancestor::cac:LotsGroup/cbc:LotsGroupID.

If not already present, add to the lot group's .relatedLots.

[duncandewhurst]

Fields to be discarded

I've asked for clarification, but I think that OPT-090-LotsGroup is an error and, actually, OPT-090 can only appear in the context of individual lots. In any case, I think that this field can be discarded since it only serves to contextualise BT-111-Lot within the eForms XML structure. In OCDS, the mapping of BT-111-Lot provides the necessary context.

The eForms team confirmed that OPT-090-LotsGroup is an error. It will be removed.

[duncandewhurst]

Fields for which a mapping needs to be performed for each lot in a lots group

For BT-726-LotsGroup (Suitable For SMEs), I can't see how there could be different values for a lots group and the individual lots in the group, so I think the proposal in #71 (comment) to perform the mapping for each of the group's lots makes sense.

On reflection, I think that it's better to map this field to tender.lotGroups. Otherwise, we'll need to author logic to handle situations where publishers provide incoherent values for BT-726-Lot and BT-726-LotsGroup. Mapping to tender.lotGroups is also consistent with other -LotsGroup business terms.

[duncandewhurst]

I've updated guidance.yaml according to the proposals in this issue.

[odscjen]

Also need to refer to LotsGroup for BT-156 (Group Framework Value) via it's associated BT-556 (Group Framework Value Lot Identifier). The guidance proposed in #71 (comment) doesn't work for this as the LotsGroup ID is in XPATH sibling::cac:TenderLot/cac:ID.

Proposed guidance for these incorporating the adjusted general guidance are:

BT-156

Get the LotGroup in tender.lotGroups whose .id is equal to the value of the XPath sibling::cac:TenderLot/cac:ID. If none exists yet, add a LotGroup to tender.lotGroups and set its .id to the value of the XPath sibling::cac:TenderLot/cac:ID., map to its .techniques.frameworkAgreement.value.amount and map currencyID to its techniques.frameworkAgreement.value.currency.

BT-556

If there is a LotGroup in tender.lotGroups whose .id is equal to the value of this field, discard. Otherwise, add a LotGroup to tender.lotGroups and map to its .id.

[duncandewhurst]

I think that this issue can be closed since we decided to discard BT-156 and BT-556 in #121

[jpmckinney]

I don't see 156 mentioned in #121 and I can't find other discussion about it. (I'm retracing if we explained why we discarded for #196).

[duncandewhurst]

BT-156 is mentioned several times in #121. However, BT-556 isn't mentioned - is that what you meant?

If I understood correctly, BT-156 is the value and BT-556 is the identifier of the LotGroup to which the value relates so if we're discarding BT-156 then we should discard BT-556 too because, without the value, the identifier alone is useless.

[jpmckinney]

I'm not sure what I meant. Maybe I couldn't find where it was decided to discard, but, looking again, I think that's in #121 (comment)

It looks like these (plus BT-161-NoticeResult and some currency and ID fields) are the only NoticeResult fields (along with their withheld publication fields). I'm not entirely sure why they exist in eForms. Let's assume for now that they can be derived from BT-660-LotResult and BT-709-LotResult.

Edit: Anyhow, we do indeed currently discard both BT-156 and BT-556.

[duncandewhurst]

Yep, that's the comment I took to mean that we should discard them!