Add custom schema checks

.github/styles/config/vocabularies/Docs/accept.txt

@@ -43,6 +43,7 @@ Doyensec
 Dynatrace
 Entra
 [Ee]nablement
+ESLint
 (?i)enums?
 GCUs?
 Gradle

src/content/graphos/delivery/check-configurations.mdx

@@ -11,10 +11,13 @@ You can configure schema checks behavior to suit your use case using either Grap
 - Check schema changes against multiple graph variants.
 - Ignore changes to default values.
 - Ignore potentially breaking changes when an operations check runs against zero operations.
+- Change the [severity level](./schema-linter/#setting-severity-levels) of linter rules.
+- Enable [custom checks](./custom-checks).
 
 ## Using GraphOS Studio (recommended)
 
-In [GraphOS Studio](https://studio.apollographql.com?referrer=docs-content), you can configure default rules that are applied to every executed schema check. You define these rules from the **Configuration** tab of your variant's **Checks** page:
+In [GraphOS Studio](https://studio.apollographql.com?referrer=docs-content), you can configure default rules that are applied to every executed schema check.
+You define these rules from the **Configuration** tab of your variant's **Checks** page:
 
 <img
   className="screenshot"
@@ -25,16 +28,36 @@ In [GraphOS Studio](https://studio.apollographql.com?referrer=docs-content), you
 
 ### Configuration options
 
+You configure operations, linter, and proposals checks.
+
+#### Operations check configurations
+
+- Whether operations checks are included in check runs. (On by default.)
 - **Time range**: Include all distinct operations executed within this range. The default value is 7 days ("Within the last week").
   - **Operation count threshold**: Exclude all operations executed fewer than this number of times within the specified **time range**.
 - **Included Variants**: Include all distinct operations executed against each selected variant of your graph. The default value is **Base variant**—whichever variant schema checks are being run against.
 - **Excluded Clients**: Exclude all operations executed by particular clients, such as clients used exclusively for development and testing.
 - **Excluded Operations**: Exclude specific named operations.
 - **Ignored Conditions**: Ignore certain kinds of schema changes. [Learn more.](./run-checks/#ignored-conditions-settings)
+- **Custom Checks**: Enable custom checks. [Learn more.](./custom-checks)
+
+#### Linter check configurations
+
+Linter check configurations include whether to include linting checks (on by default) and linter rules' [severity levels](./schema-linter/#setting-severity-levels).
+See [Linter configuration](./schema-linter/#linter-configuration) for details.
+
+#### Proposals check configurations
+
+You can configure schema checks to include a Proposals task that verifies whether schema changes have matching and approved schema proposals.
+(Off by default.) [Learn more.](./schema-proposals/configuration#configure-schema-checks)
+
+#### Custom check configurations
+
+Enable custom checks and register your validation webhook. [Learn more.](./custom-checks)
 
 ## Using the Rover CLI
 
-You can customize a single run of schema checks by providing options to the Rover CLI. If you've also [configured default rules](#using-graphos-studio-recommended) for schema checks, any command-line options you provide take precedence over those rules.
+You can customize a single schema check run by providing options to the Rover CLI. If you've also [configured default rules](#using-graphos-studio-recommended) for schema checks, any command-line options you provide take precedence over those rules.
 
 ### Validation period
 

src/content/graphos/delivery/checks-reference.mdx

@@ -4,11 +4,11 @@ subtitle: Reference for types of schema changes that checks can detect
 description: GraphOS Schema checks reference for schema changes in your GraphQL schema.
 ---
 
-This reference describes the operations checks statuses you may see in GraphOS Studio and the schema change types reported by [the Rover CLI `subgraph check` response](./run-checks#the-check-response).
+This reference describes the [operations checks](./schema-checks#operations-checks) statuses you see in GraphOS Studio and the schema change types reported by [the Rover CLI `subgraph check` response](./run-checks#the-check-response).
 
 ## Operations check statuses
 
-GraphOS classifies [operations checks](./schema-checks#types-of-checks) in Studio with the following statuses:
+GraphOS classifies operations checks with the following statuses:
 
 <table class="field-table">
   <thead>

src/content/graphos/delivery/config.json

@@ -20,6 +20,10 @@
     "Schema Checks": {
       "Overview": "/schema-checks",
       "Run Schema Checks": "/run-checks",
+      "Custom Checks": ["/custom-checks",
+      [
+        "enterprise"
+      ]],
       "Check Configurations": "/check-configurations",
       "Connect to GitHub": "/github-integration",
       "Checks Reference": "/checks-reference"

src/content/graphos/delivery/custom-checks.mdx

@@ -0,0 +1,261 @@
+---
+title: Run Custom Schema Checks
+subtitle: Extend GraphOS schema checks with custom validations
+description: Learn how to extend the GraphOS schema checks workflow by configuring custom schema checks that allow your organization to enforce unique validation rules using your own services.
+---
+
+<EnterpriseFeature linkWithAnchor="https://www.apollographql.com/pricing#delivery-pipeline" />
+
+Your organization's validation needs might extend beyond default schema checks.
+For example, you might want to enforce schema standards in addition to default [GraphOS linter rules](./linter-rules).
+Or you might require validations that depend on external data, such as ensuring schema changes are only submitted from a specific branch in version control.
+
+_Custom schema checks_ let you supplement default checks by executing custom validations on your own services.
+
+## How custom checks work
+
+Once you [configure custom checks](./custom-checks), GraphOS sends schema change information to an endpoint you specify.
+Your endpoint can then perform validations and report potential issues back to GraphOS via the [Platform API](/graphos/platform-api/).
+GraphOS Studio displays the issues you report along with other check result details.
+
+<img
+  className="screenshot"
+  src="../img/schema-checks/custom-check-result.jpg"
+  alt="To-do: Schema check run showing custom check details in GraphOS Studio"
+  width="700"
+/>
+
+<Note>
+
+You can only register one endpoint for custom checks, but that endpoint can perform as many different schema validations as you want.
+
+</Note>
+
+Custom checks run alongside other check types—like [operations and linter checks](./schema-checks#types-of-checks)—as part of every schema check run.
+Like those other check types, custom checks only run [after a build check has successfully completed](schema-checks#build-checks-1).
+
+## Configuration
+
+Custom checks require you to set up an HTTPS endpoint that's accessible via the public internet.
+You can set secret tokens to ensure that your endpoint only responds to requests sent by GraphOS.
+Once you register your endpoint in GraphOS Studio, GraphOS sends it webhook notifications as POST requests.
+
+### Set up a validation endpoint
+
+Your validation endpoint should:
+
+1. Be able to receive and process a POST request with a payload with [the specified JSON format](#webhook-format).
+1. Send a `200` response to the request.
+    - When GraphOS dispatches a webhook with check information, it has a 30-second timeout to receive a `200` response.
+    - If the request times out or doesn't receive a `200`, it will retry up to five times, with an exponential backoff delay starting at 5 seconds and going up to 60 seconds.
+1. [Submit validation check results](#submitting-a-check-result) to the GraphOS Platform API.
+
+Outside of these requirements, you have the flexibility to implement any validations you need.
+
+#### Webhook format
+
+On every schema run, GraphOS sends your registered endpoint a POST request.
+The request has a JSON payload with the following shape:
+
+```json
+{
+  // Type of event, indicating a custom schema check in Apollo GraphOS
+  "eventType": "APOLLO_CUSTOM_CHECK",
+  // Version of the payload shape
+  "TODO:version": "...",
+  "checkStep": {
+    // The unique identifier of the graph being validated
+    "graphId": "string",
+    // The variant of the graph (e.g., "current", "staging")
+    "graphVariant": "string",
+    // A unique identifier for the specific custom check task
+    "taskId": "UUID",
+    // A unique identifier for the workflow that initiated this check
+    "workflowId": "UUID",
+    "gitContext": {
+      // The branch from which the schema change is being submitted
+      "branch": "string",
+      // The commit hash associated with the schema change
+      "commit": "string",
+      // The name or email of the person who committed the change
+      "committer": "string",
+      // The commit message describing the schema change
+      "message": "string",
+      // The URL of the remote Git repository (e.g., GitHub or GitLab)
+      "remoteUrl": "string"
+    }
+  },
+  "baseSchema": {
+    // The hash of the current base schema
+    "hash": "string",
+    // A list of subgraphs that make up the base schema (if using Apollo Federation)
+    "subgraphs": [
+      {
+        // The hash of the subgraph schema
+        "hash": "string",
+        // The name of the subgraph
+        "name": "string"
+      }
+    ]
+  },
+  "proposedSchema": {
+    // The hash of the proposed schema
+    "hash": "string",
+    // A list of subgraphs in the proposed schema (if using Apollo Federation)
+    "subgraphs": [
+      {
+        // The hash of the subgraph schema
+        "hash": "string",
+        // The name of the subgraph
+        "name": "string"
+      }
+    ]
+  }
+}
+```
+
+<Note>
+
+The `version` attribute is included because payloads may include additional properties in the future.
+
+</Note>
+
+Because schemas can be large, the payload provides schema hashes rather than full-text schemas.
+Your endpoint can request the full schema contents using the [GraphOS Platform API](/graphos/platform-api/).
+
+Using the schema contents and payload information, your endpoint can perform whatever validations you need.
+
+Refer to [Platform API docs](https://studio.apollographql.com/public/apollo-platform/home?variant=main) for instructions on making and authenticating requests.
+See the examples below for fetching schema contents by hash.
+
+#### Fetch a single schema
+
+To fetch a single supergraph or subgraph schema contents by hash, use the following GraphQL query on the Platform API:
+
+```graphql
+query schema($graphId: ID!, $hash: SHA256) {
+  graph(id: $graphId) {
+    doc(hash: $hash) {
+      source
+    }
+  }
+}
+```
+
+#### Fetch all schemas
+
+To request all subgraphs and supergraph schema in one request, use the following GraphQL query on the Platform API:
+
+```graphql
+query schema(
+  $graphId: ID!
+  $supergraphHash: SHA256
+  $subgraphAHash: SHA256
+  $subgraphBHash: SHA256
+) {
+  graph(id: $graphId) {
+    supergraph: doc(hash: $supergraphHash) {
+      source
+    }
+    subgraphA: doc(hash: $subgraphAHash) {
+      source
+    }
+    subgraphB: doc(hash: $subgraphBHash) {
+      source
+    }
+  }
+}
+```
+
+#### Submitting a check result
+
+Once your endpoint has completed its validation, it should submit the results back to the triggering schema check run using the Platform API.
+The triggering schema check run waits for a response for ten minutes and doesn't complete until it receives it.
+If ten minutes pass without a response, the schema check run is marked as failed.
+
+
+The results should include a list of violations, each with the following:
+
+- `level`: Violation [severity level](./schema-linter/#setting-severity-levels), either `ERROR`, `WARN`, or `IGNORE`
+- `rule`: Name of the violation type
+- `message`: Error message
+
+
+
+#### Example check results mutation
+
+You can execute use the following GraphQL mutation on the Platform API to submit custom check results:
+
+```graphql
+mutation SchemaCheckCallback(
+  $input: CustomCheckCallbackInput!
+  $name: String!
+  $graphId: ID!
+) {
+  graph(id: $graphId) {
+    id
+    variant(name: $name) {
+      id
+      customCheckCallback(input: $input) {
+        __typename
+        ... on CustomCheckResult {
+          violations {
+            level
+            message
+            rule
+          }
+        }
+        ... on PermissionError {
+          message
+        }
+        ... on TaskError {
+          message
+        }
+        ... on ValidationError {
+          message
+        }
+      }
+    }
+  }
+}
+```
+
+### Enable custom checks in Studio
+
+Once your endpoint is ready, you need to register it in GraphOS Studio.
+
+1. In [GraphOS Studio](https://studio.apollographql.com?referrer=docs-content), go to your graph's **Checks** page.
+2. Select **Configuration** in the upper right to open the checks configuration page.
+3. From the checks configuration page, open the **Custom Checks** section.
+4. Toggle on the switch next to **Enable Custom Checks**.
+5. Click the **Edit** button next to **Registered webhook** and enter the **Endpoint URL**.
+6. Optionally, enter a **Secret Token**.
+
+    If you enter a token, each notification HTTP request includes an `x-apollo-signature` header whose value is a [Hash Message Authentication Code (HMAC)](https://en.wikipedia.org/wiki/HMAC) generated using the token, the request body as the message, and the SHA256 hash function. The `x-apollo-signature` header has the format `sha256=<hmac-value>`.
+
+    <ExpansionPanel title="See an example">
+
+    Given the following inputs:
+
+    **Secret token** (key): `your_secret_token`
+
+    **Request body** (message):
+
+    ```json
+    {
+        "to":"do"
+    }
+    ```
+
+    **Hash function**: SHA256
+
+    The `x-apollo-header` value would be `sha256=to-do`.
+    </ExpansionPanel>
+
+    Refer to this [guide from Okta](https://www.okta.com/identity-101/hmac/) to learn more about implementation and see additional resources.
+
+You should now see your configured webhook on the checks configuration page and begin to receive custom check events at the specified endpoint whenever schema checks run.
+
+## Example implementation
+
+Refer to the [example implementation](https://github.com/apollosolutions/custom-check-examples) for a custom check validation service that enforces GraphQL ESLint rules.

src/content/graphos/delivery/run-checks.mdx

@@ -8,10 +8,10 @@ This guide covers running your first schema check and integrating checks into yo
 
 ## Prerequisites
 
-To enable schema checks for your supergraph:
+Before running your first schema check do the following:
 
 1. Ensure you've published all your subgraph schemas to GraphOS and that those schemas are up to date. [Learn how to publish schemas.](./)
-2. For operations checks, ensure your supergraph is sending operation metrics to GraphOS. GraphOS uses historical metrics to determine whether a potentially dangerous schema change is safe.
+2. Ensure your supergraph is sending operation metrics to GraphOS. [Operations checks](./schema-checks#operations-checks-1) uses historical metrics to determine whether a potentially dangerous schema change is safe.
 
    - If you have a [cloud supergraph](../graphs/#cloud-supergraphs), your router reports metrics automatically.
    - [See setup for other graph types.](../metrics/sending-operation-metrics/)
@@ -125,9 +125,8 @@ You can rerun checks from GraphOS Studio. Select the check and click **Rerun che
   width="300"
 />
 
-The new checks run incorporates any changes made to excluded or included clients, checked variants, and any operations marked as safe or ignored.
-
-The new run uses the current [check configuration options](./check-configurations), regardless of the configuration at the time of the original run. Similarly, the new run's time window is based on the current time, not when the original check ran.
+The new check run uses the current [check configuration options](./check-configurations), regardless of the configuration at the time of the original run.
+Similarly, the new run's time window is based on the current time, not when the original check ran.
 
 <Note>
 
@@ -238,7 +237,8 @@ jobs:
 
 ## Next steps
 
-Once you've enabled schema checks, you can [configure them](./check-configurations) to suit your use case. Configurations include excluding certain historical operations from operations checks, ignoring certain types of schema changes, and more.
+Once you've enabled schema checks, you can [configure them](./check-configurations) to suit your use case.
+Configurations include excluding certain historical operations from operations checks, ignoring certain types of schema changes, enabling [custom checks](./custom-checks) and more.
 
 ### Integrating with GitHub