Add explicit fields instead of relying on conditional semantics

[jpmckinney]

For example, if tender.status is 'planning', then tender.value means 'cost estimate', the pre-tender estimated value of the contracting process. If tender.status is 'active', it means the estimated value at the announcement of the contracting process. In the compiled release, the cost estimate is lost; the only way to access it is to use a versioned release (which only one publisher provides), or to look through all the releases.

Instead, OC4IDS adds a costEstimate field. Similarly, it has a pair of contractValue and finalValue fields for the initial and final values of the contract. (see #575 (comment))

This issue is to identify which fields have conditional semantics, and which should be preserved such that the information is easy to access and interpret.

[duncandewhurst]

I reviewed the field and code descriptions on the 1.2-dev branch to identify conditional semantics.

Tender section
The following codes modify the semantics of the tender section:

Codelist	Code	Title	Description
releaseTag.csv	planning	Planning	A contracting process is proposed or planned. Information in the tender section describes the proposed process. The tender.status field should be used to identify whether the planning is at an early pipeline stage, or whether there are detailed plans for a tender developed.
tenderStatus.csv	Planning	Planning	A future contracting process is being considered. Early information about the process can be provided in the tender section. A process with this status might provide information on early engagement or consultation opportunities, during which the details of a subsequent tender can be shaped.
tenderStatus.csv	Planned	Planned	A contracting process is scheduled, but is not yet taking place. Details of the anticipated dates can be provided in the tender block.

These codes effectively modify the semantics of all of the fields in the tender section to be pre-tender estimates or opportunities for early engagement or consultation.

To decide which values should be preserved, I checked for occurrences of these fields in the list of OCDS indicators:

• UC069 maps 'tender value estimate' to tender.value and compares it to contracts.value to measure savings, but the indicator doesn't specify whether the pre-tender estimate or the value of the time of the tender announcement is required.
• No indicators require values from early engagement or consultation exercises.

#896 (comment) proposes to narrow the semantics of tender.value to:

The total estimated value of the procurement, as estimated when publishing the tender information".

If that happens, we should definitely add another field to disclose the pre-tender estimate as it could no longer be disclosed in tender.value.

Procuring entity and contract items
The descriptions of the following fields include conditions. However, the conditions are about avoiding repetition of the same information in multiple fields, rather than the semantics of the fields themselves, so no information is lost when releases are compiled:

section	path	title	description
tender	tender/procuringEntity	Procuring entity	The organization managing the contracting process. If an organization is both a buyer and a procuring entity (as can be the case in simple contracting processes), it should be disclosed using the buyer field, but not this field.
contracts	contracts/items	Items contracted	The goods, services, and any intangible outcomes in this contract. If the items contracted are identical to the items awarded, this field should be omitted.

Contract value and period
No codes or field descriptions explicitly modify the semantics of the contract section. However, contracts.value and contracts.period may be updated if a contract is amended, in which case the original values from the signed contract would be lost from the compiled release, unless provided in the awards section. From OC4IDS, we know there are use cases for identifying overruns by comparing at least the initial and final contract value and period. For active contracts, similar use cases are likely to apply for comparing the initial and latest contract value and period.

In the case of the contract value, assuming values can't change between the award and the signed contract, comparing awards.value and contracts.value is OK. However, I think this is largely implicit/discussed in Github issues, so we should improve the descriptions of awards.value and contracts.value to clarify that the former is the initial value and the latter is updated with any post-signature changes.

In the case of the contract period, we may need an additional field to preserve values. Presumably, a challenge from an unsuccessful bidder during the standstill period, even if unsuccessful, could delay the start of the contract beyond the start of the contract period specified in the award. In which case, using awards.period and contracts.period to identify overruns could lead to false positives.

[jpmckinney]

Thanks for the analysis!

For tender.value, if the new guidance is to disclose planning information under a different OCID than the procurement procedure per #866 (though I presume it's also okay to retroactively disclose planning information once a procurement procedure has started under one same OCID), then the semantics would be consistent in the context of a given OCID, and the value would not be updated and given new meaning by a publisher, so the situation is improved.

That said, in the case of retroactive disclosure of planning information, there is no place for the pre-tender estimate, unless the publisher creates a "synthetic" release, which is a lot of overhead, so I think it is best to add a new field, and deprecate the "pre-tender estimate" semantics of tender.value (since we can't change the semantics outright).

For contract values, I tend to prefer your suggestion of comparing awards and contracts. That said, we also have this extension, which adds new fields instead: https://extensions.open-contracting.org/en/extensions/contract_completion/master/

[JachymHercher]

(though I presume it's also okay to retroactively disclose planning information once a procurement procedure has started under one same OCID),

The current conclusion of #866 (comment) (also discussed in a few comments above that) is that it is not, i.e. when publishing planning information, you always must do it under a new planning process, even if it is retrospective. If we stick to that (I believe we should), then I think we wouldn't need a new field.

---

For example, if tender.status is 'planning', then tender.value means 'cost estimate', the pre-tender estimated value of the contracting process. If tender.status is 'active', it means the estimated value at the announcement of the contracting process.

Second reason to not add the field: is the semantic difference between "cost estimate" and "estimated value at the announcement of the contracting process" important enough in practice to merit two fields? I would have the tendency to treat both as the same thing - a non-binding, somewhat unimportant estimate - which gradually improves over time. Also, #914 (comment) says "No indicators require values from early engagement or consultation exercises.".

[duncandewhurst]

If planning information must always be disclosed under a separate planning process, then I agree that there's no need to add a field to preserve the pre-tender estimate for tender.value.

is the semantic difference between "cost estimate" and "estimated value at the announcement of the contracting process" important enough in practice to merit two fields?

Based on OCP's procurement indicators, no. However, presumably, this issue was prompted by a use case for separating the concepts, so this might be a topic for consultation/further research if retroactive disclosure of planning data can happen under the same OCID.

@JachymHercher I would be interested in your feedback on whether the scenario of the contract period differing between the award and the signed contract is actually possible:

In the case of the contract period, we may need an additional field to preserve values. Presumably, a challenge from an unsuccessful bidder during the standstill period, even if unsuccessful, could delay the start of the contract beyond the start of the contract period specified in the award. In which case, using awards.contractPeriod and contracts.period to identify overruns could lead to false positives.

(I edited the final paragraph of my original comment to use the correct field names: awards.contractPeriod and contracts.period)

[jpmckinney]

The current conclusion of #866 (comment) (also discussed in a few comments above that) is that it is not, i.e. when publishing planning information, you always must do it under a new planning process, even if it is retrospective. If we stick to that (I believe we should), then I think we wouldn't need a new field.

Aha, yes, happy to stick to that. That way, there is only one way to disclose the pre-tender estimate, rather than two.

[JachymHercher]

@JachymHercher I would be interested in your feedback on whether the scenario of the contract period differing between the award and the signed contract is actually possible: [...] Presumably, a challenge from an unsuccessful bidder during the standstill period, even if unsuccessful, could delay the start of the contract beyond the start of the contract period specified in the award

The scenario "challenge leads to a delay" seems totally realistic. I'm not sure what the solution is - besides telling people "if you're analyzing modifications, you need to go through individual releases".

[duncandewhurst]

The scenario "challenge leads to a delay" seems totally realistic. I'm not sure what the solution is - besides telling people "if you're analyzing modifications, you need to go through individual releases".

An alternative would be to add an explicit field to preserve the initial period in the signed contract. Otherwise, no option is perfect with respect to identifying changes to the contract period:

• Relying on individual releases - most publishers don't provide individual releases, most tools and methodologies are based on compiled releases.
• Comparing awards and contracts - if there is a difference between awards.contractPeriod and contracts.period in a compiled release, it's not possible to determine whether the difference is due to a delay to signing the contract, or a post-signature change to the period.
• Using the completion extension - if contracts.period.endDate and contracts.implementation.endDate are equal in a compiled release, it's not possible to determine whether the contract was completed according to the original period in the signed contract, or whether the contract was amended and contracts.period.endDate updated.

@jpmckinney - what do you think?

@JachymHercher - I'm not sure if the same issue applies to contract values. In #914 (comment) I assumed that the value in the award decision and the value in the signed contract would never differ. Is that correct? Or could a challenge lead to a change to the contract value?

[JachymHercher]

@JachymHercher - I'm not sure if the same issue applies to contract values. In #914 (comment) I assumed that the value in the award decision and the value in the signed contract would never differ. Is that correct? Or could a challenge lead to a change to the contract value?

I would have the same reflex as you do, i.e. an award value will always be the same as the contract signed in the initial contract (and then the contract value gets modified, while the award value does not.) I would argue this stems from the fact that a value is a crucial part of the bid and awards are based on bids, so unless you modify the bids, you can't change the content of an award.

(Now, trying to be dreamy and difficult: perhaps some contracts have "parametric" values? For example, construction projects depend on the cost of inputs and if the input price changes dramatically they want to pass it on to the buyer, so the contract could have a clause of "if signed before 2022-01-01, then we charge Y; if signed later, we will charge Y+10". In such a scenario, an unexpected delay in contract signature could also lead to a value change. Just thinking out loud though - feel free to ignore or ask a practitioner :-). )

[jpmckinney]

I guess it's a question of balance and priority. We don't want three fields for every concept (initial, current, final). The dates and values of the main objects (tender, award, contract) are more important, so we might allow more time-based fields for them. However, this isn't really about "conditional semantics", since a contract can have the same status, etc. and its period can be amended. This issue started with tender.value meaning different things based on tender.status.

[duncandewhurst]

Sounds good. In terms of conditional semantics, we agreed that we don't need to add any fields. Shall we close this issue and open a new issue about adding time-based fields to preserve values of important fields?

[jpmckinney]

Yes, please create the new issue and then close this one.