Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid examples that are incoherent, or that are too abstract #1449

Open
jpmckinney opened this issue Nov 22, 2021 · 26 comments · May be fixed by #1680
Open

Avoid examples that are incoherent, or that are too abstract #1449

jpmckinney opened this issue Nov 22, 2021 · 26 comments · May be fixed by #1680
Assignees
Labels
Focus - UX Making information easier to find and interpret
Milestone

Comments

@jpmckinney
Copy link
Member

Users prefer specific examples. We now have many specific examples on deeper pages (e.g. for the Map step).

However, much of the older content still has generic/abstract examples – or no examples.

We should review the documentation to check for examples that need to be added or improved. We can then list the pages as a to-do list in this issue, and close the issue once a sufficient number of pages have been updated.

@jpmckinney jpmckinney added the Focus - UX Making information easier to find and interpret label Nov 22, 2021
@jpmckinney jpmckinney added this to the 1.2.0 milestone Nov 22, 2021
@jpmckinney
Copy link
Member Author

cc @sabahfromlondon to identify/prioritize the pages on which to add/improve examples.

@odscjen
Copy link
Contributor

odscjen commented Sep 27, 2022

@jpmckinney is it ok if I start working on this or shall I leave it for Sabah?

@sabahfromlondon
Copy link

@jpmckinney I think I missed this! Sorry.

I'll compile the list this week.

Thanks for flagging @odscjen

@sabahfromlondon
Copy link

@jpmckinney I've completed a review of the documentation pages. I also thought it was worth checking the depper pages, even though that was not quite within the scope of this ticker. It didn't take much longer and I have some thoughts / suggestions it might be worth discussing..

Here's the link to the spreadsheet.

Shall we discuss on our call on Wednesday?

cc: @odscjen

@jpmckinney
Copy link
Member Author

Thank you! We discussed and answered some of the questions in the sheet. I'll leave this with analysts to find appropriate examples.

@neelima-j
Copy link

neelima-j commented Dec 6, 2022

The sheet has identified 2 required tasks and 9 potential tasks

Example Required

  • Include a real world example of what OCDS looks like in JSON and CSV, below the green box at How does the OCDS work?
  • Have real world examples of a release and record from the same dataset. Use the dropdown for the user to toggle between the two as per other pages in the documentation, at How is OCDS data published?

@jpmckinney for the 'Maybe' examples, is the sheet the best place to ask clarifying questions/ have discussions, or would you rather we brought it here as tasks?

@jpmckinney
Copy link
Member Author

@neelima-j I think your first link is meant to be to https://standard.open-contracting.org/latest/en/primer/how/?

For the maybe examples, let's move it here as notifications are easier to track.

@odscjen
Copy link
Contributor

odscjen commented Sep 11, 2023

Maybe examples copied in from the spreadsheet. Initial comments also from the spreadsheet, italic comments added by me as I transferred this list to here:

  • Awards and contracts - The JSON example is an external link probably becuase it is long - suggestion is to leave as it. There are also tabular examples, which should also be in JSON?
  • Electronic catalogs - Could be helpful, but possibly no examples for this?
  • Updates and amendments - The examples look fictional, but still very easy to understand. I assume the task is to see if there are any real examples which could be used instead?
  • Milestones - The examples look fictional, but still very easy to understand. Perhaps a tender milestones example could be added? Again assume the maybe is around looking for real world examples to use instead here?
  • Buyers and suppliers - Example could be in JSON in addition to/instead of tabular. The example is for the Department of Health and Social Care, but UK could also be added to the description.
  • Organization references - Perhaps we should change the name of the organisation in the example to a name users will recognise as a possible tenderer and then supplier e.g. Microsoft? Maybe we could add an example of how the JSON looks for multiple organisations and with different roles?
  • Record reference - Real world examples are welcome here
  • Merging - Real world examples are welcome here
  • Identifiers - It's possible that users might want to copy what another publisher has done, so may want to look up the whole dataset i.e. via the registry so it would be good to have real world examples throughout this page

@jpmckinney
Copy link
Member Author

@yolile Which of these would you prioritize for added/improved examples? Are any good enough as-is?

@yolile
Copy link
Member

yolile commented Sep 21, 2023

Include a real world example of what OCDS looks like in JSON and CSV, below the green box at How does the OCDS work?

We often hear and need to show partners what OCDS data looks like, so this sounds good. The only thing to consider when preparing the example is to make sure that we state that this is just an example and not all the example fields are required nor does the example contain all the existing fields etc.

Have real world examples of a release and record from the same dataset. Use the dropdown for the user to toggle between the two as per other pages in the documentation, at How is OCDS data published?

For this one, I think we have worked examples at the Merging section and the Change History guidance. Maybe we can just add a sentence saying that an example is available on the Change History page?

Awards and contracts - The JSON example is an external link probably becuase it is long - suggestion is to leave as it. There are also tabular examples, which should also be in JSON?

The tabular examples are tabular on purpose to easily demonstrate the issue and solution. I think we can leave it as it is.

Electronic catalogs - Could be helpful, but possibly no examples for this?

Leave it as it is as there is no special mapping to show

Buyers and suppliers - Example could be in JSON in addition to/instead of tabular.

Agree that the example should be in JSON and not in tabular format.

Identifiers - It's possible that users might want to copy what another publisher has done, so may want to look up the whole dataset i.e. via the registry so it would be good to have real world examples throughout this page

The identifiers section has a lot of things to be improved and it has its own issues already documented.

For all the others, do we really want/need real-world examples? I can see some value in having a real example to follow, but I'm afraid of 1) broken links in the future and 2) the reader following any bad practice that the referenced publisher might have. We could avoid both things by only adding a copy of the real JSON file but I don't see much difference with having fictional examples (other than knowing who else is publishing that in OCDS)

@jpmckinney
Copy link
Member Author

jpmckinney commented Sep 21, 2023

@yolile When this issue was created, we had identified that users find it harder to relate to an example if it is totally fictional, e.g. "Department of Bubble Gum buys 10 blister packs of chewing gum from Happy Smiles Inc."

If our fictional examples are close enough to "real", then they are fine. If there are deficiencies in a publisher's data, we can just correct them in the documentation - we aren't making a promise that the data is identical.

Edit: Another reason for using real data was to avoid creating incoherent examples. We sometimes had such examples in the past, where we wanted to demonstrate a lot of fields at once, but it started to make no sense in reality.

@odscjen
Copy link
Contributor

odscjen commented Oct 16, 2023

For the release and record of the same dataset, Ghana publish records which include versioned and compiled releases so they're a good example for this e.g use ocds-uhveoc-680235 from https://www.ghaneps.gov.gh/ocds/services/recordpackage/getrecordpackage/2023/10 which has 2 releases. However the data will need cleaned up, e.g. they're not used parties.ids correctly and we probably don't need to include all of the fields they have. This could replace the fictional example in merging and then be linked to in How is OCDS data published along with Change history as suggested by @yolile in #1449 (comment).

Ebonyi State in Nigeria could be used for the json and csv example, they provide both json and excel downloads, e.g. https://ebonyieprocure.eb.gov.ng/do_award_contract_details.php?id=ocds-zinqhl-014665-EB/WTR/EP002-NG. Again this will need corrected, e.g. id's are not being used correctly, and we'll remove some of the fields so that it's not an overly long example.

I've picked African examples for both of these as:
a) most of the current examples seem to be either from the UK, Moldova, or Latin America
b) they're both publishers that are still publishing OCDS

@jpmckinney @yolile if you're happy with those examples I'll get started on drafting them up taking into consideration the need to make it clear that the actual data may not be identical to the data the publisher has published.

Re. the other points are you happy with:

  • leaving the Buyers and suppliers example as fictional as long as it's converted into json? (it seems like a coherent example to me using real companies)
  • the fictional example in Milestones left as is as it looks coherent and close to real
  • the fictional example in amendements also gets left as is as it looks coherent and close to real
  • leaving as is the example used in Organization references as it is a real company (although a small one), I'm not sure I see the need to change it to a large multinational (although I may be biased in this particular case as ODS is the example company!)

I've updated the above comment that lists the maybes to strike out those that @yolile has already confirmed can be left as is

@jpmckinney
Copy link
Member Author

@yolile I defer to you.

@yolile
Copy link
Member

yolile commented Oct 18, 2023

All sounds good to me, thanks @odscjen

@duncandewhurst
Copy link
Contributor

duncandewhurst commented Dec 11, 2023

I'm following up on @jpmckinney's request in #1661 (comment) for me to see if any of the existing examples mentioned in this issue are deficient, and if so, propose some improvement.

Regarding avoiding totally fictional or incoherent examples (the purpose of this issue), I propose the following improvements:

  • Amendments guidance (example 1) and Records reference (record structure) (Done in Make examples minimal #1680)
    • Deficiencies:
      • The buyer (Open Data Services) is not a government agency and appears elsewhere in the documentation as a supplier, which might be confusing to readers.
      • The object of the procurement (a data merging tool) closely relates to the subject of the example (updates and amendments), which might be confusing for readers.
    • Proposed improvements:
      • Replace the buyer with a government agency
      • Replace the object of the procurement with something unrelated to the OCDS releases and records model, e.g. office supplies.
  • Records reference (embedded releases) (Done in Make examples minimal #1680)
    • Deficiencies:
      • The supplier is totally fictional (AnyCorp)
      • The example doesn't follow a common thread with the earlier examples on the page
    • Proposed improvements:
      • Replace with an embedded releases version of the example featured earlier in the page (noting the improvements proposed below)

Having reviewed the examples mentioned in this issue, I think there are many other opportunities for improvement. In most cases, with respect to the concepts they are supposed to illustrate, the examples are either unnecessarily complex or include fields which are irrelevant, or both.

My other observation from reviewing the changes in #1661 is that it's more trouble than it's worth to find real OCDS data and to try to correct errors and trim it down for brevity and clarity so that it can serve as example data.

I think that we should:

  1. Get 1449 improve examples #1661 merged without adding to its scope
  2. Update the contributor guidelines for authoring examples as follows:
  • Explain what is meant by 'real' (i.e. featuring real organisations and plausible and coherent field values, not being real data from an OCDS publisher)
  • Examples ought to feature the minimum level of complexity and minimum set of fields with respect to the concepts they are supposed to illustrate.
  • Examples on the same page ought to follow a common thread/context/scenario.
  1. Review and update all examples to conform to the guidelines. (incorporating Make JSON examples minimal #1666 so that we don't need to go through the examples twice).

I'm sharing my notes from reviewing the examples mentioned in this issue so that they aren't lost:

  • Awards and contracts guidance (changes example):
  • Awards and contracts guidance (multiple decisions example):
    • Deficiencies:
      • The example data are tabular rather than JSON, which is inconsistent with most of the OCDS documentation.
      • The link to the real data in the Paraguay API is at risk of link rot.
      • The real data returned by the Paraguay API is likely to be overwhelming for readers because it contains many fields that are irrelevant to the subject of the example.
    • Proposed improvements:
      • Replace tabular examples with minimal JSON examples.
      • Remove the link to the Paraguay API.
  • Amendments guidance (example 1) and Records reference (record structure) (Done in Make examples minimal #1680)
    • Deficiencies (further to those mentioned earlier in this comment):
      • The tender amendment release is unnecessarily complex: it amends two fields (tender.value and tender.period), when only one is needed to illustrate how amendments are modelled.
      • The example data contain many fields that are irrelevant to the subject of the example, e.g. tender.status, tender.procurementMethod, tender.awardPeriod etc.
    • Proposed improvements:
      • Update the tender amendment release so that only tender.period is amended
      • Remove irrelevant fields
  • Amendments guidance (example 2)
    • Deficiencies:
      • The value of contracts.title ("4501062723") does not conform to the schema
      • The example data contains fields that are irrelevant to the subject of the example, e.g. contracts.items.classification.
    • Proposed improvements:
      • Update contracts.title to conform to the schema
      • Remove irrelevant fields
  • Milestones guidance (planning example)
    • Deficiencies: The example data contains fields that are irrelevant to the subject of the example, e.g. planning.budget
    • Proposed improvements: Remove irrelevant fields
  • Milestones guidance (project data example):
    • Deficiencies: The example is unnecessarily complex: it contains two milestones and two implementation update releases, when only one of each is required to illustrate how delivery milestones are modelled.
    • Proposed improvements: Update the example to cover only one implementation milestone and one implementation update release, reframe as 'delivery milestones'.
  • Milestones guidance (delivery and payment data example)
    • Deficiencies: The example is unnecessarily complex: it covers two delivery and two payment milestones, when it need cover only single payment milestone because delivery milestones are already covered by the previous example.
    • Proposed improvements: Update the example to cover only a single payment milestone, reframe as 'payment milestones'.
  • Organization reference guidance:
    • Deficiencies:
      • The tenderer/supplier (Open Data Services) appears elsewhere in the OCDS documentation as a buyer, which might be confusing to readers.
      • The example data contains fields that are irrelevant to the example, e.g. tender.status, awards.value, awards.contractPeriod etc.
      • The worked example is unnecessarily long and verbose given the simplicity of the concepts it concerns.
    • Proposed improvements:
      • Remove the worked example and instead expand the example in the OrganizationReference section of the schema reference to show both an organization reference and the referenced item in the parties array.

Also noting that the amendments guidance twice invites readers to use the Data Review Tool to explore changes in the contracting process. IIRC, the Data Review Tool no longer provides that functionality (OCDS Show) so I think those mentions should be removed.

@jpmckinney
Copy link
Member Author

All sounds good to me – thank you, @duncandewhurst ! You can create an issue about "Update the contributor guidelines" in the handbook.

@jpmckinney
Copy link
Member Author

IIRC, the Data Review Tool no longer provides that functionality (OCDS Show) so I think those mentions should be removed.

It does – we had planned to remove it as part of a full redesign, which hasn't occurred.

@duncandewhurst
Copy link
Contributor

duncandewhurst commented Feb 28, 2024

Moved to #1666 (comment)

@jpmckinney
Copy link
Member Author

jpmckinney commented Feb 28, 2024

Moved to #1666 (comment)

@duncandewhurst duncandewhurst changed the title Review shallow pages to add or improve examples Avoid examples that are incoherent, or that are too abstract Feb 28, 2024
@duncandewhurst
Copy link
Contributor

duncandewhurst commented Feb 28, 2024

I've updated the issue title to reflect the purpose expressed in #1661 (comment) and I've moved the previous two comments to #1666 (comment) (I'd forgotten I'd opened a separate issue about making examples minimal).

@duncandewhurst duncandewhurst linked a pull request Feb 29, 2024 that will close this issue
1 task
@odscjen odscjen assigned duncandewhurst and unassigned odscjen Feb 29, 2024
@duncandewhurst
Copy link
Contributor

duncandewhurst commented Mar 11, 2024

Whilst reviewing examples for #1666, I noted a couple more overly generic examples, which I'll update as part of #1680:

@jpmckinney
Copy link
Member Author

This issue has some resolutions in #1680 so I won't mark #1661 as closing it.

@jpmckinney
Copy link
Member Author

jpmckinney commented Mar 28, 2024

@duncandewhurst I think the second checklist item is not part of #1680, correct?

Is there anything else left from this issue, otherwise?

@duncandewhurst
Copy link
Contributor

@duncandewhurst I think the second checklist item is not part of #1680, correct?

Whoops, yes. I should've listed it in #1666. Let me know if I should add it to #1680

Is there anything else left from this issue, otherwise?

Nope, I think we're done.

@jpmckinney
Copy link
Member Author

Let me know if I should add it to #1680

Sure, should be quick.

@duncandewhurst
Copy link
Contributor

Done in 6f058dc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Focus - UX Making information easier to find and interpret
Projects
Status: Review in progress
Development

Successfully merging a pull request may close this issue.

6 participants