From 727ef232e5d9db86aa077488141e4485730ad790 Mon Sep 17 00:00:00 2001 From: Mike Date: Thu, 11 Apr 2024 13:46:36 -0400 Subject: [PATCH] Api 33629 update bi docs (#16103) * Updated BI docs with better words * more changes in BI md docs --------- Co-authored-by: Kristen Brown <11942904+kristen-brown@users.noreply.github.com> --- .../vba_documents/v1/controller_swagger.rb | 9 + .../swagger/vba_documents/v1/description.md | 173 ++++++++-------- .../swagger/vba_documents/v2/description.md | 192 ++++++++---------- 3 files changed, 187 insertions(+), 187 deletions(-) diff --git a/modules/vba_documents/app/swagger/vba_documents/v1/controller_swagger.rb b/modules/vba_documents/app/swagger/vba_documents/v1/controller_swagger.rb index b83922f4e5f..045f4cab2bf 100644 --- a/modules/vba_documents/app/swagger/vba_documents/v1/controller_swagger.rb +++ b/modules/vba_documents/app/swagger/vba_documents/v1/controller_swagger.rb @@ -85,6 +85,9 @@ class ControllerSwagger extend VBADocuments::Responses::ForbiddenError key :summary, 'Get status for a previous benefits document upload' key :operationId, 'getBenefitsDocumentUploadStatus' + key :description, 'A request to the `/uploads/{id}` endpoint will return a real-time status for that GUID, ' \ + 'and update its status in `/uploads/report`.\n\n' \ + 'The `updated_at` field indicates the last time the status for a given GUID was updated.' key :tags, [ VBA_TAG @@ -173,6 +176,12 @@ class ControllerSwagger key :summary, 'Get a bulk status report for a list of previous uploads' key :operationId, 'getBenefitsDocumentUploadStatusReport' + key :description, 'Due to current system limitations, data for the `/uploads/report` endpoint is cached for one hour.' \ + '\n\n' \ + 'A request to the `/uploads/{id}` endpoint will return a real-time status for that GUID, ' \ + 'and update its status in `/uploads/report`.' \ + '\n\n' \ + 'The `updated_at` field indicates the last time the status for a given GUID was updated.' security do key :apikey, [] diff --git a/modules/vba_documents/app/swagger/vba_documents/v1/description.md b/modules/vba_documents/app/swagger/vba_documents/v1/description.md index 3ec795ed7a3..f0efc266428 100644 --- a/modules/vba_documents/app/swagger/vba_documents/v1/description.md +++ b/modules/vba_documents/app/swagger/vba_documents/v1/description.md @@ -1,4 +1,5 @@ -The Benefits Intake API allows authorized third-party systems used by Veteran Service Organizations (VSOs), agencies, and Veterans to digitally submit VA benefits claim documents directly to the Veterans Benefits Administration's (VBA) claims intake process. This API handles documents related to the following benefit claim types: +The Benefits Intake API enables authorized third-party systems, such as those used by Veteran Service Organizations (VSOs), agencies, and Veterans, to digitally submit documents for VA benefits claims directly to the claims intake process of the Veterans Benefits Administration (VBA). This API handles documents related to the following benefit claim types: + * Compensation * Pension/Survivors Benefits @@ -8,105 +9,109 @@ The Benefits Intake API allows authorized third-party systems used by Veteran Se * Veteran Readiness & Employment (VRE) * Board of Veteran Appeals (BVA) -This API also provides submission status updates until documents are successfully established for VBA claim processing, eliminating the need for users to switch between systems to manually check whether documents have been successfully uploaded. - -**Note**: This API may not be used for Veterans Health Administration (VHA) benefits, including 10-10EZ and 10-10d. +Additionally, the API offers real-time status updates on document submissions until they've been successfully accepted by VBA or another VA system for claims processing. -## Background -This API provides a secure, efficient, and tracked alternative to mail or fax for VA benefit claim document submissions. Documents are uploaded directly to the VBA so they can be processed as quickly as possible. +**Important**: This API **may not** be used for Veterans Health Administration (VHA) benefits forms, including the Instructions and Enrollment Application for Health Benefits form (10-10EZ) and the application for CHAMPVA benefits form (10-10d). ## Technical overview -The Benefits Intake API first provides an upload location and unique submission identifier, and then accepts a payload consisting of a document in PDF format, zero or more optional attachments in PDF format, and some JSON metadata. - -The metadata describes the document and attachments, and identifies the person for whom it is being submitted. This payload is encoded as binary multipart/form-data (not base64). The unique identifier supplied with the payload can subsequently be used to request the processing status of the uploaded document package. - -To avoid errors and processing delays, API consumers are encouraged to validate the `zipcode`,`fileNumber`, `veteranFirstName`, `veteranLastName` and `businessLine` fields before submission according to their description in the DocumentUploadMetadata model and use the 'businessLine' attribute for the most efficient processing. Additionally, please ensure no PDF user passwords are used in submitted PDFs. - -### Attachment & file size limits -There is no limit on the number of files a payload can contain, but size limits do apply. - -* Uploaded documents cannot be larger than 78" x 101" -* The entire payload cannot exceed 5 GB -* No single file in a payload can exceed 100 MB - -### Date of receipt -The date that documents are successfully submitted through the Benefits Intake API is used as the official VA date of receipt. However, note that until a document status of `received`, `processing`, `success`, or `vbms` is returned, a client cannot consider the document received by VA. - -A status of `received` means that the document package has been transmitted, but may not be validated. Any errors with the document package, such as unreadable PDFs or a Veteran not found, will cause the status to change to `error`. - -If the document status is `error`, VA has not received the submission and cannot honor the submission date as the date of receipt. +The Benefits Intake API first provides an upload location and unique submission identifier, and then accepts a payload consisting of a PDF document, other optional PDF attachments, and JSON metadata. The metadata describes the PDF attachments and identifies the Veteran who the benefits are related to. ### Authentication and Authorization -API requests are authorized through a symmetric API token, provided in an HTTP header with name 'apikey'. [Request an API key](https://developer.va.gov/explore/api/benefits-intake/sandbox-access). - -### Testing in the sandbox environment -In the sandbox environment, the final status of a submission is `received` and submissions do not actually progress to the central mail repository or VBMS. - -Progress beyond the `received` status can be simulated for testing. We allow passing in a `Status-Override` header on the `/uploads/{id}` endpoint so that you can change the status of your submission to simulate the various scenarios. - -The available statuses are `pending`, `uploaded`, `received`, `processing`, `success`, `vbms`, and `error`. The meaning of the various statuses is listed below in Models under DocumentUploadStatusAttributes. +API requests are authorized through a symmetric API token, provided in an HTTP header with name `apikey`. [Get access to sandbox](https://developer.va.gov/explore/api/benefits-intake/sandbox-access). ### Test data -We use mock test data in the sandbox environment. Data is not sent upstream and it is not necessary to align submitted test data with any other systems' data. +Since the primary purpose of this API is to accept user submissions, it doesn't have mock test data available. In the sandbox environment, data submitted to the API is not sent downstream. This API can return data that has been previously submitted by a consumer. -### Validating documents -Use the POST `/uploads/validate_document` endpoint to make sure your documents will pass system file requirements and -validations before you send them through the submissions process. This step is optional but decreases the likelihood -of individual document errors during the submission process. +### Document Submission Statuses -Validations performed: -* Document is a valid PDF (Note: `Content-Type` header value must be "application/pdf") -* Document does not have a user password (an owner password is acceptable) -* File size does not exceed 100 MB -* Page size does not exceed 78" x 101" +After a successful upload, a submission status can be tracked using the GET /uploads/{id} endpoint. If multiple uploads have been made, the POST /uploads/report endpoint can be used to track the status of multiple submissions. For more information, please see steps 5 and 6 in the **How to Upload** section of this documentation or check out the example curl and response sections for these endpoints. -### Upload operation -Allows a client to upload a multi-part document package (form + attachments + metadata). +Detailed descriptions of what each status means are found in this table. +| Status | What it means | +| --- | --- | +| **Pending** | - This is the initial status. Indicates no document submission has been uploaded yet.

- Date of Receipt is not yet established with this status. | +| **Uploaded** | - Indicates document submission has been successfully uploaded (PUT) to the API server.

- Submission has not yet been validated.

- Date of Receipt is not yet established with this status.

- Any errors with the document submission, such as having an unreadable PDF, may cause an Error status. | +| **Received** | - Indicates document submission has been received downstream of the API and is awaiting processing.

- The VA Date of Receipt is set when this status is achieved.

- Any errors with the document submission, such as having an unreadable PDF, may cause an Error status.

- This is the final status in the sandbox environment unless further progress is simulated by the user. | +| **Processing**| - Indicates the document package is being validated, processed, and made ready to route and work.

- Any errors with the document submission, such as having an unreadable PDF, may cause an Error status.| +| **Success** | - Indicates the document submission has been successfully received within VA's mail handling system.

- Success is the final status for a small percentage of submissions with claim types, Veteran types, or exception processes that are not worked in VBMS.

- Most submissions reach a Success status within 1 business day.

- A small portion will take longer. However, some submissions may take up to 2 weeks to reach a Success status.| +| **VBMS** | - Indicates the document submission was successfully uploaded into a Veteran's eFolder within VBMS.

- On average, submissions reach VBMS status within 3 business days. However, processing times vary and some submissions may remain in a Success status for several weeks before reaching a VBMS status.

- Some document packages are worked in VA systems other than VBMS. For these submissions, Success is the final status. | +| **Error** | - Indicates that there was an error. Refer to the error code and detail for further information. | +| **Expired** | - After a POST request, there is a 15-minute window during which documents must be uploaded via a PUT request.

- An Expired status means the documents were not successfully uploaded within this 15-minute window.

- We recommend coding to retry unsuccessful uploads within 15 minutes using the same submission in case of connection issues. | + +In the sandbox environment: +* The final status of a submission is `received` since submissions do not actually progress to the Central Mail repository or VBMS. +* Progress beyond the `received` status can be simulated for testing. A `Status-Override` header can be passed on the `/uploads/{id}` endpoint to change the status of a submission to simulate the various status scenarios. + +In the production environment: +* The date that the Benefits Intake API submits the documents downstream (indicated by the `received` status) is used as the official VA date of receipt. This is usually the same day as when the documents are submitted by users, but not always. +* If the document's final destination is VBMS, it can fall into an `error` state until the status of VBMS is returned. +* If the document's final destination is another VA system besides VBMS, it can fall into an `error` state until the status of `success` is returned. + + +### How to avoid submission errors +The Benefits Intake API returns three different categories of errors: +* HTTP errors relating to the client request. These errors will be immediate. +* Metadata and PDF validation errors run by the system before passing the submission downstream to subsequent services. These errors will be asynchronous but generally come within minutes of submission. +* Downstream errors returned by subsequent services, usually relating to the content of the PDFs or metadata. These errors will be asynchronous and can take minutes to hours or longer to receive. + +To prevent submission delays and errors: +* Ensure that submitted PDFs are not locked by user passwords and that they are within the file and page size limits before submitting your payload. +* Be sure to validate all metadata fields against their data requirements as defined in the DocumentUploadMetadata schema before submitting your payload. +* The businessLine field is optional, but when included, it will ensure the fastest possible processing. If not specified, businessLine will default to CMP, the business line for Compensation requests. +* The POST /uploads/validate_document endpoint can be used to ensure individual PDF documents meet system requirements prior to submission. + +For information on how to monitor the status of submissions, please see Steps 5 and 6 of the **How to Upload** section of this documentation. -1. Client Request: POST https://sandbox-api.va.gov/services/vba_documents/v1/ - * No request body or parameters required +### Attachment & file size limits +There is no limit on the number of files a payload can contain, but file size and page dimension limits do apply. +* Uploaded PDFs can't be larger than 78" x 101". +* Uploaded PDFs can't exceed 100 MB. +* The entire payload can't exceed 5 GB. +### How to Upload +The Upload operation lets a client upload a multi-part document submission, specifically PDFs and metadata. Uploading is detailed below. +1. Client Request: POST https://sandbox-api.va.gov/services/vba_documents/v1/ + * No request body or parameters required. 2. Service Response: A JSON API object with the following attributes: - * `guid`: An identifier used for subsequent status requests - * `location`: A URL to which the actual document package payload can be submitted in the next step. The URL is specific to this upload request, and should not be re-used for subsequent uploads. The URL is valid for 900 seconds (15 minutes) from the time of this response. If the location is not used within 15 minutes, the GUID will expire. Once expired, status checks on the GUID will return a status of `expired`. - * Note: If, after you've submitted a document, the status hasn't changed to `uploaded` before 15 minutes has elapsed, we recommend retrying the upload in order to make sure the document properly reaches our servers. If the upload continues to fail, try encoding the payload as Base64 (See below). - - 3. Client Request: PUT to the location URL returned in Step 2. - * Request body should be encoded as binary multipart/form-data (base64 also available - see details below), equivalent to that generated by an HTML form submission or using "curl -F…". The format is described in more detail below. - * No `apikey` authorization header is required for this request, as authorization is embedded in the signed location URL. - -4. Service Response: The HTTP status indicates whether the upload was successful. - * Additionally, the response includes an ETag header containing an MD5 hash of the submitted payload. This can be compared to the submitted payload to ensure data integrity of the upload. - -### Status caching -Due to current system limitations, data for the `/uploads/report` endpoint is cached for one hour. + * `guid`: An identifier used for subsequent status requests. + * `location`: A URL to which the actual document submission payload can be submitted in the next step. The URL is specific to this upload request, and should not be re-used for subsequent uploads. The URL is valid for 15 minutes (900 seconds) from the time of this response. If the location is not used within 15 minutes, the GUID will expire. Once expired, status checks on the GUID will return a status of `expired`. + * **Note**: If the status hasn't changed to uploaded within 15 minutes of submission, retry the POST request in step 1 to generate a new location URL. +3. Client Request: PUT to the location URL returned in step 2. + * The request body should use the binary multipart/form-data format . The body can optionally be encoded as Base64. + * No `apikey` authorization header is required for this request, as authorization is embedded in the signed location URL. +4. Service Response: The HTTP status indicates whether the upload was submitted. + * Additionally, the response includes an ETag header containing an MD5 hash of the submitted payload. This can be compared to the submitted payload to ensure data integrity of the upload. + * **Note**: A successful document submission **_does not_** mean it was successfully received by VA. Successful document submissions can fail later, as discussed above in the "How to avoid submission errors" section. +5. Client Request: GET https://sandbox-api.va.gov/services/vba_documents/v2/uploads/{id} + * `id`: An identifier (`guid`) returned in step 2 by the previous submission. +6. Service Response: A JSON API object with the following attributes: + * `guid`: The identifier of the submission for which a status was requested. + * `status`: The current status of the submission. See the previous "Document Submission Statuses" section for more information about the specific statuses. + * `code`: Only present if `status` is `error`. An error code specifying why the submission is in an error state. For a list of the error codes, see the schema for the GET /uploads/{id} 200 response. + * `detail`: Only present if `status` is `error`. Plain language detailing of the error corresponding to the error code and the specific submission. + * `updated_at`: The last time the submission status was updated. + + +If submissions to the API are made frequently, the bulk status endpoint should be used in place of the above Step 5 to request updates for multiple submissions: + +5. Client Request: POST https://sandbox-api.va.gov/services/vba_documents/v1/uploads/report + * `ids`: A list of identifiers ( guid) returned in step 2 by the previous submissions. +6. Service Response: A JSON API object with the following attributes for each `id`: + * `guid`: The identifier of the submission for which a status was requested. + * `status`: The current status of the submission. See the previous "Document Submission Statuses" section for more information about the specific statuses. + * `code`: Only present if `status` is `error`. An error code specifying why the submission is in an error state. For a list of the error codes, see the schema for the GET /uploads/{id} 200 response. + * `detail`: Only present if `status` is `error`. Plain language detailing of the error corresponding to the error code and the specific submission. + * `updated_at`: The last time the submission status was updated. + * Due to current system limitations, data for the /uploads/report endpoint is cached for one hour. The /uploads/{id} endpoint isn't cached. -A request to the `/uploads/{id}` endpoint will return a real-time status for that GUID, and update its status in `/uploads/report`. - -The `updated_at` field indicates the last time the status for a given GUID was updated. - -### Document Submission Statuses -**Important note:** a submission has not been received by VA until it has a status of Received, Processing, Success, -or VBMS. Detailed descriptions of what each status means are found in this table. - -| Status | What it means | -| --- | --- | -| **Pending** | Initial status.

Indicates no document package has been uploaded yet.

Date of Receipt is not yet established with this status | -| **Uploaded** | Indicates document package has been successfully uploaded (PUT) from your system to the API server but has not yet been validated.

Date of Receipt is not yet established with this status. Any errors with the document package, such as having an unreadable PDF, may cause an Error status. | -| **Received** | Indicates document package has been received upstream of the API and is awaiting Processing.

The VA Date of Receipt is set when this status is achieved.

This is the final status in the sandbox environment unless further progress is simulated. | -| **Processing**| Indicates the document package is being validated, processed, and made ready to route and work. | -| **Success** | Indicates the document package has been successfully received within VA's mail handling system.

Success is the final status for a small percentage of submitted packages with claim types, Veteran types, or exception processes that are not worked in VBMS. Most submissions reach a Success status within 1 business day. A small portion will take longer; however, some submissions may take up to 2 weeks to reach a Success status. | -| **VBMS** | Indicates this document package was successfully uploaded into a Veteran's eFolder within VBMS.

On average, submissions reach VBMS status within 3 business days; however, processing times vary and some submissions may remain in a Success status for several weeks before reaching a VBMS status.

Some document packages are worked in VA systems other than VBMS. For these submissions, Success is the final status. | -| **Error** | Indicates that there was an error. Refer to the error code and message for further information. | -| **Expired** | After a POST request, there is a 15-minute window during which documents must be uploaded via a PUT request.

An Expired status means the documents were not successfully uploaded within this 15-minute window. We recommend coding to retry unsuccessful uploads within 15 minutes using the same submission in case of connection issues. | ### Optional Base64 encoding +Base64 is an encoding scheme that converts binary data into text format, so that encoded textual data can be easily transported over networks uncorrupted and without data loss. -Base64 is an encoding scheme that converts binary data into text format, so that encoded textual data can be easily transported over networks uncorrupted and without data loss. - -Base64 can be used to encode binary multipart/form-data it in its entirety. Note that the whole payload must be encoded, not individual parts/attachments. +Base64 can be used to encode binary multipart/form-data in its entirety. Note that the whole payload must be encoded, not individual parts/attachments. -After encoding your payload, you'll be required to preface your base64 string with `data:multipart/form-data;base64,` in order to allow our system to distinguish the file type. Your final string payload would look something like `data:multipart/form-data;base64,(encryption string)==` and close with the standard == marker. Note that the multipart boundaries i.e. -----WebKitFormBoundaryVfOwzCyvug0JmWYo and ending ------WebKitFormBoundaryVfOwzCyvug0JmWYo- must also be included. +After encoding the payload, the base64 string must be prefaced with `data:multipart/form-data;base64` in order to allow the system to distinguish the file type. -### Consumer onboarding process -When you're ready to move to production, [request a production API key.](https://developer.va.gov/go-live) +The final string payload will look something like this: `data:multipart/form-data;base64,(encryption string)==`, and close with the standard == marker. Note that the multipart boundaries, that is,\ +-----WebKitFormBoundaryVfOwzCyvug0JmWYo and ending in\ +------WebKitFormBoundaryVfOwzCyvug0JmWYo- must also be included. \ No newline at end of file diff --git a/modules/vba_documents/app/swagger/vba_documents/v2/description.md b/modules/vba_documents/app/swagger/vba_documents/v2/description.md index 1ad9abafda2..8b5a5b290bc 100644 --- a/modules/vba_documents/app/swagger/vba_documents/v2/description.md +++ b/modules/vba_documents/app/swagger/vba_documents/v2/description.md @@ -1,4 +1,5 @@ -The Benefits Intake API allows authorized third-party systems used by Veteran Service Organizations (VSOs), agencies, and Veterans to digitally submit VA benefits claim documents directly to the Veterans Benefits Administration's (VBA) claims intake process. This API handles documents related to the following benefit claim types: +The Benefits Intake API enables authorized third-party systems, such as those used by Veteran Service Organizations (VSOs), agencies, and Veterans, to digitally submit documents for VA benefits claims directly to the claims intake process of the Veterans Benefits Administration (VBA). This API handles documents related to the following benefit claim types: + * Compensation * Pension/Survivors Benefits @@ -8,126 +9,111 @@ The Benefits Intake API allows authorized third-party systems used by Veteran Se * Veteran Readiness & Employment (VRE) * Board of Veteran Appeals (BVA) -This API also provides submission status updates until documents are successfully established for VBA claim processing, eliminating the need for users to switch between systems to manually check whether documents have been successfully uploaded. - -**Note**: This API may not be used for Veterans Health Administration (VHA) benefits, including 10-10EZ and 10-10d. +Additionally, the API offers real-time status updates on document submissions until they've been successfully accepted by VBA or another VA system for claims processing. -## Background -This API provides a secure, efficient, and tracked alternative to mail or fax for VA benefit claim document submissions. Documents are uploaded directly to the VBA so they can be processed as quickly as possible. +**Important**: This API **may not** be used for Veterans Health Administration (VHA) benefits forms, including the Instructions and Enrollment Application for Health Benefits form (10-10EZ) and the application for CHAMPVA benefits form (10-10d). ## Technical overview -The Benefits Intake API first provides an upload location and unique submission identifier, and then accepts a payload consisting of a document in PDF format, zero or more optional attachments in PDF format, and some JSON metadata. - -The metadata describes the document and attachments, and identifies the person for whom it is being submitted. This payload is encoded as binary multipart/form-data (not base64). The unique identifier supplied with the payload can subsequently be used to request the processing status of the uploaded document package. - -To avoid errors and processing delays, API consumers are encouraged to validate the `zipcode`,`fileNumber`, `veteranFirstName`, `veteranLastName` and `businessLine` fields before submission according to their description in the DocumentUploadMetadata model and use the 'businessLine' attribute for the most efficient processing. Additionally, please ensure no PDF user passwords are used in submitted PDFs. - -### Attachment & file size limits -There is no limit on the number of files a payload can contain, but size limits do apply. - -* Uploaded documents cannot be larger than 78" x 101" -* The entire payload cannot exceed 5 GB -* No single file in a payload can exceed 100 MB - -### Date of receipt -The date that documents are successfully submitted through the Benefits Intake API is used as the official VA date of receipt. However, note that until a document status of `received`, `processing`, `success`, or `vbms` is returned, a client cannot consider the document received by VA. - -A status of `received` means that the document package has been transmitted, but may not be validated. Any errors with the document package, such as unreadable PDFs or a Veteran not found, will cause the status to change to `error`. - -If the document status is `error`, VA has not received the submission and cannot honor the submission date as the date of receipt. +The Benefits Intake API first provides an upload location and unique submission identifier, and then accepts a payload consisting of a PDF document, other optional PDF attachments, and JSON metadata. The metadata describes the PDF attachments and identifies the Veteran who the benefits are related to. ### Authentication and Authorization -API requests are authorized through a symmetric API token, provided in an HTTP header with name 'apikey'. [Request an API key](https://developer.va.gov/explore/api/benefits-intake/sandbox-access). - -### Testing in the sandbox environment -In the sandbox environment, the final status of a submission is `received` and submissions do not actually progress to the central mail repository or VBMS. +API requests are authorized through a symmetric API token, provided in an HTTP header with name `apikey`. [Get access to sandbox](https://developer.va.gov/explore/api/benefits-intake/sandbox-access). -Progress beyond the `received` status can be simulated for testing. We allow passing in a `Status-Override` header on the `/uploads/{id}` endpoint so that you can change the status of your submission to simulate the various scenarios. +### Test data +Since the primary purpose of this API is to accept user submissions, it doesn't have mock test data available. In the sandbox environment, data submitted to the API is not sent downstream. This API can return data that has been previously submitted by a consumer. -The available statuses are `pending`, `uploaded`, `received`, `processing`, `success`, `vbms`, and `error`. The meaning of the various statuses is listed below in Models under DocumentUploadStatusAttributes. +### Document Submission Statuses -There are additional tools that can help developers explore how the API works. There is a "download" endpoint that can help developers see how the server consumes the data. This endpoint is only available in sandbox and more information can be seen [here](https://github.com/department-of-veterans-affairs/vets-api/blob/master/modules/vba_documents/app/swagger/vba_documents/v2/downloads.md). +After a successful upload, a submission status can be tracked using the GET /uploads/{id} endpoint. If multiple uploads have been made, the POST /uploads/report endpoint can be used to track the status of multiple submissions. For more information, please see steps 5 and 6 in the **How to Upload** section of this documentation or check out the example curl and response sections for these endpoints. -In addition, there are Postman collections and a ping method that are only available in sandbox. More information can be found [here](https://github.com/department-of-veterans-affairs/vets-api/blob/master/lib/webhooks/postman_webhooks/ping-pong). -### Test data -We use mock test data in the sandbox environment. Data is not sent upstream and it is not necessary to align submitted test data with any other systems' data. +Detailed descriptions of what each status means are found in this table. +| Status | What it means | +| --- | --- | +| **Pending** | - This is the initial status. Indicates no document submission has been uploaded yet.

- Date of Receipt is not yet established with this status. | +| **Uploaded** | - Indicates document submission has been successfully uploaded (PUT) to the API server.

- Submission has not yet been validated.

- Date of Receipt is not yet established with this status.

- Any errors with the document submission, such as having an unreadable PDF, may cause an Error status. | +| **Received** | - Indicates document submission has been received downstream of the API and is awaiting processing.

- The VA Date of Receipt is set when this status is achieved.

- Any errors with the document submission, such as having an unreadable PDF, may cause an Error status.

- This is the final status in the sandbox environment unless further progress is simulated by the user. | +| **Processing**| - Indicates the document package is being validated, processed, and made ready to route and work.

- Any errors with the document submission, such as having an unreadable PDF, may cause an Error status.| +| **Success** | - Indicates the document submission has been successfully received within VA's mail handling system.

- Success is the final status for a small percentage of submissions with claim types, Veteran types, or exception processes that are not worked in VBMS.

- Most submissions reach a Success status within 1 business day.

- A small portion will take longer. However, some submissions may take up to 2 weeks to reach a Success status.| +| **VBMS** | - Indicates the document submission was successfully uploaded into a Veteran's eFolder within VBMS.

- On average, submissions reach VBMS status within 3 business days. However, processing times vary and some submissions may remain in a Success status for several weeks before reaching a VBMS status.

- Some document packages are worked in VA systems other than VBMS. For these submissions, Success is the final status. | +| **Error** | - Indicates that there was an error. Refer to the error code and detail for further information. | +| **Expired** | - After a POST request, there is a 15-minute window during which documents must be uploaded via a PUT request.

- An Expired status means the documents were not successfully uploaded within this 15-minute window.

- We recommend coding to retry unsuccessful uploads within 15 minutes using the same submission in case of connection issues. | + +In the sandbox environment: +* The final status of a submission is `received` since submissions do not actually progress to the Central Mail repository or VBMS. +* Progress beyond the `received` status can be simulated for testing. A `Status-Override` header can be passed on the `/uploads/{id}` endpoint to change the status of a submission to simulate the various status scenarios. + +In the production environment: +* The date that the Benefits Intake API submits the documents downstream (indicated by the `received` status) is used as the official VA date of receipt. This is usually the same day as when the documents are submitted by users, but not always. +* If the document's final destination is VBMS, it can fall into an `error` state until the status of VBMS is returned. +* If the document's final destination is another VA system besides VBMS, it can fall into an `error` state until the status of `success` is returned. + + +### How to avoid submission errors +The Benefits Intake API returns three different categories of errors: +* HTTP errors relating to the client request. These errors will be immediate. +* Metadata and PDF validation errors run by the system before passing the submission downstream to subsequent services. These errors will be asynchronous but generally come within minutes of submission. +* Downstream errors returned by subsequent services, usually relating to the content of the PDFs or metadata. These errors will be asynchronous and can take minutes to hours or longer to receive. + +To prevent submission delays and errors: +* Ensure that submitted PDFs are not locked by user passwords and that they are within the file and page size limits before submitting your payload. +* Be sure to validate all metadata fields against their data requirements as defined in the DocumentUploadMetadata schema before submitting your payload. +* The businessLine field is optional, but when included, it will ensure the fastest possible processing. If not specified, businessLine will default to CMP, the business line for Compensation requests. +* The POST /uploads/validate_document endpoint can be used to ensure individual PDF documents meet system requirements prior to submission. + +For information on how to monitor the status of submissions, please see Steps 5 and 6 of the **How to Upload** section of this documentation. -### Upload operation -Allows a client to upload a multi-part document package (form + attachments + metadata). Subscribing to the webhook callback in step 1 is optional. If polling is desired, begin with step 2. +### Attachment & file size limits +There is no limit on the number of files a payload can contain, but file size and page dimension limits do apply. +* Uploaded PDFs can't be larger than 78" x 101". +* Uploaded PDFs can't exceed 100 MB. +* The entire payload can't exceed 5 GB. +### How to Upload +The Upload operation lets a client upload a multi-part document submission, specifically PDFs and metadata. Uploading is detailed below. 1. Client Request (Optional): POST https://dev-api.va.gov/webhooks/v1/register * Webhooks: Pass the `webhook` object to subscribe URL(s) to the status change event `gov.va.developer.benefits-intake.status_change`. This can be sent as a JSON file or as JSON text data. Please refer to the endpoint Webhook schema below for additional details. - -2. Client Request: POST https://dev-api.va.gov/services/vba_documents/v2/uploads - * No request body or parameters required - +2. Client Request: POST https://sandbox-api.va.gov/services/vba_documents/v2/ + * No request body or parameters required. 3. Service Response: A JSON API object with the following attributes: - * `guid`: An identifier used for subsequent status requests - * `location`: A URL to which the actual document package payload can be submitted in the next step. The URL is specific to this upload request, and should not be re-used for subsequent uploads. The URL is valid for 900 seconds (15 minutes) from the time of this response. If the location is not used within 15 minutes, the GUID will expire. Once expired, status checks on the GUID will return a status of `expired`. - * Note: If, after you've submitted a document, the status hasn't changed to `uploaded` before 15 minutes has elapsed, we recommend retrying the upload in order to make sure the document properly reaches our servers. If the upload continues to fail, try encoding the payload as Base64 (See below). - -4. Client Request: PUT to the location URL returned in Step 3. - * Request body should be encoded as binary multipart/form-data (base64 also available - see details below), equivalent to that generated by an HTML form submission or using "curl -F…". The format is described in more detail below. - * No `apikey` authorization header is required for this request, as authorization is embedded in the signed location URL. - -5. Service Response: The HTTP status indicates whether the upload was successful. - * Additionally, the response includes an ETag header containing an MD5 hash of the submitted payload. This can be compared to the submitted payload to ensure data integrity of the upload. - -### Status updates -Once you submit a file upload, you may check its status using multiple methods. - - -* Polling: to check once or at regular intervals: - * For a single GUID, make GET requests to the /uploads/{guid} endpoint. - * For multiple GUIDs, make POST requests to the /uploads/report endpoint. -* Webhooks: we return the status changes to your subscribed URL (from step 1) as shown below. No polling or additional action is needed. - -``` -{ - api_name: 'vba_documents-v2', - timestamp: 1631048257, - notifications: [ - { - guid: 'a5a404d6-4547-4747-a9e1-31eca18d2e1f', - event: 'gov.va.developer.benefits-intake.status_change', - status_to: 'uploaded', - epoch_time: 1631047688, - status_from: 'pending' - }, - { - guid: 'a5a404d6-4547-4747-a9e1-31eca18d2e1f', - event: 'gov.va.developer.benefits-intake.status_change', - status_to: 'received', - epoch_time: 1631047697, - status_from: 'uploaded' - } - ] -} -``` + * `guid`: An identifier used for subsequent status requests. + * `location`: A URL to which the actual document submission payload can be submitted in the next step. The URL is specific to this upload request, and should not be re-used for subsequent uploads. The URL is valid for 15 minutes (900 seconds) from the time of this response. If the location is not used within 15 minutes, the GUID will expire. Once expired, status checks on the GUID will return a status of `expired`. + * **Note**: If the status hasn't changed to uploaded within 15 minutes of submission, retry the POST request in step 1 to generate a new location URL. +4. Client Request: PUT to the location URL returned in step 2. + * The request body should use the binary multipart/form-data format . The body can optionally be encoded as Base64. + * No `apikey` authorization header is required for this request, as authorization is embedded in the signed location URL. +5. Service Response: The HTTP status indicates whether the upload was submitted. + * Additionally, the response includes an ETag header containing an MD5 hash of the submitted payload. This can be compared to the submitted payload to ensure data integrity of the upload. + * **Note**: A successful document submission **_does not_** mean it was successfully received by VA. Successful document submissions can fail later, as discussed above in the "How to avoid submission errors" section. +6. Client Request: GET https://sandbox-api.va.gov/services/vba_documents/v2/uploads/{id} + * `id`: An identifier (`guid`) returned in step 2 by the previous submission. +7. Service Response: A JSON API object with the following attributes: + * `guid`: The identifier of the submission for which a status was requested. + * `status`: The current status of the submission. See the previous "Document Submission Statuses" section for more information about the specific statuses. + * `code`: Only present if `status` is `error`. An error code specifying why the submission is in an error state. For a list of the error codes, see the schema for the GET /uploads/{id} 200 response. + * `detail`: Only present if `status` is `error`. Plain language detailing of the error corresponding to the error code and the specific submission. + * `updated_at`: The last time the submission status was updated. + + +If submissions to the API are made frequently, the bulk status endpoint should be used in place of the above Step 5 to request updates for multiple submissions: + +4. Client Request: POST https://sandbox-api.va.gov/services/vba_documents/v2/uploads/report + * `ids`: A list of identifiers ( guid) returned in step 2 by the previous submissions. +5. Service Response: A JSON API object with the following attributes for each `id`: + * `guid`: The identifier of the submission for which a status was requested. + * `status`: The current status of the submission. See the previous "Document Submission Statuses" section for more information about the specific statuses. + * `code`: Only present if `status` is `error`. An error code specifying why the submission is in an error state. For a list of the error codes, see the schema for the GET /uploads/{id} 200 response. + * `detail`: Only present if `status` is `error`. Plain language detailing of the error corresponding to the error code and the specific submission. + * `updated_at`: The last time the submission status was updated. + * Due to current system limitations, data for the /uploads/report endpoint is cached for one hour. The /uploads/{id} endpoint isn't cached. -### Document Submission Statuses - -**Important note:** a submission has not been received by VA until it has a status of Received, Processing, Success, -or VBMS. Detailed descriptions of what each status means are found in this table. - -| Status | What it means | -| --- | --- | -| **Pending** | Initial status.

Indicates no document package has been uploaded yet.

Date of Receipt is not yet established with this status | -| **Uploaded** | Indicates document package has been successfully uploaded (PUT) from your system to the API server but has not yet been validated.

Date of Receipt is not yet established with this status. Any errors with the document package, such as having an unreadable PDF, may cause an Error status. | -| **Received** | Indicates document package has been received upstream of the API and is awaiting Processing.

The VA Date of Receipt is set when this status is achieved.

This is the final status in the sandbox environment unless further progress is simulated. | -| **Processing**| Indicates the document package is being validated, processed, and made ready to route and work. | -| **Success** | Indicates the document package has been successfully received within VA's mail handling system.

Success is the final status for a small percentage of submitted packages with claim types, Veteran types, or exception processes that are not worked in VBMS. Most submissions reach a Success status within 1 business day. A small portion will take longer; however, some submissions may take up to 2 weeks to reach a Success status. | -| **VBMS** | Indicates this document package was successfully uploaded into a Veteran's eFolder within VBMS.

On average, submissions reach VBMS status within 3 business days; however, processing times vary and some submissions may remain in a Success status for several weeks before reaching a VBMS status.

Some document packages are worked in VA systems other than VBMS. For these submissions, Success is the final status. | -| **Error** | Indicates that there was an error. Refer to the error code and message for further information. | -| **Expired** | After a POST request, there is a 15-minute window during which documents must be uploaded via a PUT request.

An Expired status means the documents were not successfully uploaded within this 15-minute window. We recommend coding to retry unsuccessful uploads within 15 minutes using the same submission in case of connection issues. | ### Optional Base64 encoding - Base64 is an encoding scheme that converts binary data into text format, so that encoded textual data can be easily transported over networks uncorrupted and without data loss. -Base64 can be used to encode binary multipart/form-data it in its entirety. Note that the whole payload must be encoded, not individual parts/attachments. +Base64 can be used to encode binary multipart/form-data in its entirety. Note that the whole payload must be encoded, not individual parts/attachments. -After encoding your payload, you'll be required to preface your base64 string with `data:multipart/form-data;base64,` in order to allow our system to distinguish the file type. Your final string payload would look something like `data:multipart/form-data;base64,(encryption string)==` and close with the standard == marker. Note that the multipart boundaries i.e. -----WebKitFormBoundaryVfOwzCyvug0JmWYo and ending ------WebKitFormBoundaryVfOwzCyvug0JmWYo- must also be included. +After encoding the payload, the base64 string must be prefaced with `data:multipart/form-data;base64` in order to allow the system to distinguish the file type. -### Consumer onboarding process -When you're ready to move to production, [request a production API key.](https://developer.va.gov/go-live) +The final string payload will look something like this: `data:multipart/form-data;base64,(encryption string)==`, and close with the standard == marker. Note that the multipart boundaries, that is,\ +-----WebKitFormBoundaryVfOwzCyvug0JmWYo and ending in\ +------WebKitFormBoundaryVfOwzCyvug0JmWYo- must also be included. \ No newline at end of file