Return more fields from extra section

[jmertic]

The previous landscape API would return all fields listed in the extra section, while the landscape2 API only returns a set number of fields. It would be great to have the previous behavior.

Example - landscape v1

{
  "name": "OpenEXR",
  "second_path": [
    "Imaging and Color / Image Formats, I/O, and Processing Libraries",
    "Dev Days 2024 / Participating Project"
  ],
  "description": "The goal of the OpenEXR project is to keep the format reliable and modern and to maintain its place as the preferred image format for entertainment content creation.",
  "homepage_url": "https://www.openexr.com/",
  "project": "adopted",
  "repo_url": "https://github.com/AcademySoftwareFoundation/openexr",
  "logo": "openexr.svg",
  "crunchbase": "https://www.crunchbase.com/organization/academy-software-foundation",
  "extra": {
    "artwork_url": "https://artwork.aswf.io/projects/openexr/",
    "mailing_list_url": "https://lists.aswf.io/g/openexr-dev",
    "slack_url": "https://slack.aswf.io",
    "chat_channel": "#openexr",
    "calendar_url": "https://zoom-lfx.platform.linuxfoundation.org/meetings/openexr",
    "ical_url": "https://webcal.prod.itx.linuxfoundation.org/lfx/a092M00001If9ujQAB",
    "documentation_url": "https://docs.openexr.com",
    "contributed_by": "Lucasfilm",
    "slug": "openexr",
    "accepted": "2020-04-24",
    "annual_review_date": "2024-04-17",
    "annual_review_url": "https://github.com/AcademySoftwareFoundation/tac/issues/483",
    "next_annual_review_date": "2025-04-16",
    "dev_stats_url": "https://insights.lfx.linuxfoundation.org/foundation/aswf/overview?project=openexr",
    "chair": "Cary Phillips"
  },
  "github_data": {
    "languages": [
      {
        "name": "C",
        "value": 9334263,
        "color": "#555555"
      },
      {
        "name": "C++",
        "value": 5737890,
        "color": "#f34b7d"
      },
      {
        "name": "Python",
        "value": 174811,
        "color": "#3572A5"
      },
      {
        "name": "CMake",
        "value": 154271
      },
      {
        "name": "M4",
        "value": 26297
      },
      {
        "name": "Shell",
        "value": 24834,
        "color": "#89e051"
      },
      {
        "name": "Starlark",
        "value": 22785
      },
      {
        "name": "Batchfile",
        "value": 6522,
        "color": "#C1F12E"
      },
      {
        "name": "PowerShell",
        "value": 3917,
        "color": "#012456"
      },
      {
        "name": "HTML",
        "value": 1144,
        "color": "#e34c26"
      }
    ],
    "contributions": "6;4;0;0;1;0;0;0;0;1;7;0;1;1;0;11;4;5;12;4;7;15;9;15;8;7;12;10;23;14;18;13;0;0;3;0;0;0;0;0;0;8;10;2;10;0;0;11;9;23;15;12",
    "firstWeek": "2023-10-08Z",
    "stars": 2107,
    "license": "BSD 3-Clause \"New\" or \"Revised\" License",
    "description": "The OpenEXR project provides the specification and reference implementation of the EXR file format, the professional-grade image storage format of the motion picture industry.",
    "latest_commit_date": "2024-10-04T21:58:42Z",
    "latest_commit_link": "/AcademySoftwareFoundation/openexr/commit/c41984b9563b63397f4671bc0c0d86223736c6e7",
    "release_date": "2024-09-30T16:06:56Z",
    "release_link": "https://github.com/AcademySoftwareFoundation/openexr/releases",
    "contributors_count": 138,
    "contributors_link": "https://github.com/AcademySoftwareFoundation/openexr/graphs/contributors"
  },
  "repos": [
    {
      "url": "https://github.com/AcademySoftwareFoundation/openexr",
      "stars": 1625
    },
    {
      "url": "https://github.com/AcademySoftwareFoundation/Imath",
      "stars": 376
    },
    {
      "url": "https://github.com/AcademySoftwareFoundation/openexr-images",
      "stars": 98
    },
    {
      "url": "https://github.com/AcademySoftwareFoundation/openexr-examples",
      "stars": 7
    },
    {
      "url": "https://github.com/AcademySoftwareFoundation/openexr-website",
      "stars": 1
    }
  ],
  "github_start_commit_data": {
    "start_commit_link": "/AcademySoftwareFoundation/openexr/commit/6e37ef8aed4e20ea834f570845533b65a97a3bc4",
    "start_date": "2003-01-21T06:34:28Z"
  },
  "image_data": {
    "fileName": "open-exr.svg",
    "hash": "k4f/wYPCQyNQytS590ldXMhL97lQkfHWzrRT42eijlA="
  },
  "firstCommitDate": "2003-01-21T06:34:28Z",
  "firstCommitLink": "https://github.com/AcademySoftwareFoundation/openexr/commit/6e37ef8aed4e20ea834f570845533b65a97a3bc4",
  "latestCommitDate": {
    "text": "this week",
    "value": "990",
    "original": "2024-10-04T21:58:42Z"
  },
  "latestCommitLink": "https://github.com/AcademySoftwareFoundation/openexr/commit/c41984b9563b63397f4671bc0c0d86223736c6e7",
  "releaseDate": {
    "text": "this week",
    "value": "990",
    "original": "2024-09-30T16:06:56Z"
  },
  "releaseLink": "https://github.com/AcademySoftwareFoundation/openexr/releases",
  "commitsThisYear": 311,
  "contributorsCount": 138,
  "contributorsLink": "https://github.com/AcademySoftwareFoundation/openexr/graphs/contributors",
  "language": "C",
  "stars": 2107,
  "license": "BSD 3-Clause \"New\" or \"Revised\" License",
  "headquarters": "San Francisco, California",
  "twitter": "https://twitter.com/AcademySwf",
  "latestTweetDate": "2023-04-12T17:25:09.000Z",
  "organization": "Academy Software Foundation",
  "crunchbaseData": {
    "name": "Academy Software Foundation",
    "description": "Academy Software Foundation is an open source foundation intended for developers in the motion picture and media space.",
    "homepage": "http://www.aswf.io",
    "city": "San Francisco",
    "region": "California",
    "country": "United States",
    "twitter": "https://twitter.com/AcademySwf",
    "linkedin": "https://www.linkedin.com/company/academysoftwarefoundation/",
    "acquisitions": [
      {
        "date": "2023-04-01",
        "acquiree": "OpenImageIO"
      }
    ],
    "parents": [
      "https://www.crunchbase.com/organization/linux-foundation"
    ],
    "stockExchange": null,
    "company_type": "Non Profit",
    "industries": [
      "Film",
      "Media and Entertainment",
      "Open Source",
      "Software"
    ],
    "numEmployeesMin": null,
    "numEmployeesMax": null
  },
  "path": "ASWF Projects / All",
  "landscape": "ASWF Projects / All",
  "category": "ASWF Projects",
  "amount": "N/A",
  "oss": true,
  "href": "logos/open-exr.svg",
  "bestPracticeBadgeId": 2799,
  "bestPracticePercentage": 100,
  "industries": [
    "Film",
    "Media and Entertainment",
    "Open Source",
    "Software"
  ],
  "starsPresent": true,
  "starsAsText": "2,107",
  "marketCapPresent": false,
  "marketCapAsText": "N/A",
  "id": "open-exr",
  "flatName": "OpenEXR",
  "member": false,
  "relation": "adopted",
  "isSubsidiaryProject": false
}

landscape v2

{
    "category": "ASWF Projects",
    "homepage_url": "https://www.openexr.com/",
    "id": "aswf-projects--all--openexr",
    "logo_url": "https://aswf.landscape2.io/logos/91f168e585f3f5f16c79eb60b1e658d6cfdea17dfa1529eba6aabdff689d9f93.svg",
    "name": "OpenEXR",
    "subcategory": "All",
    "accepted_at": "2020-04-24",
    "additional_categories": [
      {
        "category": "Dev Days 2024",
        "subcategory": "Participating Project"
      }
    ],
    "artwork_url": "https://artwork.aswf.io/projects/openexr/",
    "chat_channel": "#openexr",
    "country": "United States",
    "crunchbase_url": "https://www.crunchbase.com/organization/academy-software-foundation",
    "description": "The goal of the OpenEXR project is to keep the format reliable and modern and to maintain its place as the preferred image format for entertainment content creation.",
    "devstats_url": "https://insights.lfx.linuxfoundation.org/foundation/aswf/overview?project=openexr",
    "mailing_list_url": "https://lists.aswf.io/g/openexr-dev",
    "maturity": "adopted",
    "oss": true,
    "repositories": [
      {
        "url": "https://github.com/AcademySoftwareFoundation/openexr",
        "languages": {
          "Batchfile": 6522,
          "C": 8300833,
          "C++": 3738166,
          "CMake": 85070,
          "HTML": 572,
          "M4": 26297,
          "Python": 146173,
          "Shell": 17470,
          "Starlark": 22785
        },
        "primary": true
      },
      {
        "url": "https://github.com/AcademySoftwareFoundation/Imath",
        "languages": {
          "C": 1022478,
          "C++": 1931864,
          "CMake": 68673,
          "HTML": 572,
          "PowerShell": 3917,
          "Python": 28638,
          "Shell": 7364
        },
        "primary": false
      },
      {
        "url": "https://github.com/AcademySoftwareFoundation/openexr-images",
        "languages": { },
        "primary": false
      },
      {
        "url": "https://github.com/AcademySoftwareFoundation/openexr-examples",
        "languages": {
          "C": 10952,
          "C++": 63723,
          "CMake": 479
        },
        "primary": false
      },
      {
        "url": "https://github.com/AcademySoftwareFoundation/openexr-website",
        "languages": { },
        "primary": false
      }
    ],
    "slack_url": "https://slack.aswf.io"
  },

[tegioz]

Hi @jmertic 👋

Which ones are you missing? We'd be happy to consider adding any of them back.

Please note that landscape v2 only reads the extra fields defined in the corresponding type (also documented here). So fields like ical_url or contributed_by in your initial example do not actually 'exist' in v2, and are ignored when the landscape data file is processed.

[jmertic]

Hi @tegioz!

Ideally - I'd like to have any non-documented ones listed; this would align with the v1 behavior. I'd be fine with it if you wanted to put it under another level in the JSON response, but we should have them returned.

[tegioz]

Allowing mixing landscape and user-defined fields at the same object level (in the extra section) could be confusing and potentially lead to problems. It could be confusing for any user interested in the landscape seeing fields that are not actually related to any landscape functionality. But more importantly, this could create problems if at some point any of the user-defined fields matches a landscape one. If the matching field was using a different format or an invalid value, it would break the landscape build. We don't want to encourage this behavior, so we'd rather not to go this way.

One option would be to add a new field so that landscape users can attach information that's relevant to their applications, something similar to the Kubernetes annotations. You could move the fields in extra that are not related to the landscape to this new annotations object. And we could return them from the API so that your application can use them. This would provide the same functionality in a clean and error-free way.

Alternatively, you can always consume the landscape.yml data file directly from your application, instead of relying on the landscape v2 API. The information we return from the API comes entirely from that file (we are not allowed to use Crunchbase data on it, and likely GitHub's neither), so the main difference is how the data is organized. Please note that this approach still exposes you to a potential conflict in the case a landscape field name ends up matching one of the user-defined ones.

Happy to go with the annotations way if that would help you 🙂

[jmertic]

annotations is a great way to address this - thanks!

[tegioz]

Awesome, we'll take care of it 👍

[tegioz]

Annotations added!

Closed by #721