Skip to content

Commit

Permalink
[Internal] Update Jobs GetRun API to support paginated responses for …
Browse files Browse the repository at this point in the history
…jobs and ForEach tasks (#1089)

## What changes are proposed in this pull request?

Introduces extension for jobs GetRun call that paginates tasks and
iterations arrays in the response. This change is necessary to prepare
for jobs API 2.2 release that serves paginated response. Pagination is
over once the `next_page_token` is absent from the response. The
pagination logic is not exposed to the customer.

## How is this tested?

Unit test and manual test. I wrote go code that called the new logic and
verified that the returned result contained full object when the API was
paginated.
  • Loading branch information
gkiko10 authored Nov 8, 2024
1 parent 776b63c commit 1981951
Show file tree
Hide file tree
Showing 2 changed files with 405 additions and 0 deletions.
34 changes: 34 additions & 0 deletions service/jobs/ext_api.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package jobs

import "context"

// GetRun retrieves a run based on the provided request.
// It handles pagination if the run contains multiple iterations or tasks.
func (a *JobsAPI) GetRun(ctx context.Context, request GetRunRequest) (*Run, error) {
run, err := a.jobsImpl.GetRun(ctx, request)
if err != nil {
return nil, err
}

// When querying a Job run, a page token is returned when there are more than 100 tasks. No iterations are defined for a Job run. Therefore, the next page in the response only includes the next page of tasks.
// When querying a ForEach task run, a page token is returned when there are more than 100 iterations. Only a single task is returned, corresponding to the ForEach task itself. Therefore, the client only reads the iterations from the next page and not the tasks.
isPaginatingIterations := run.Iterations != nil && len(run.Iterations) > 0

pageToken := run.NextPageToken
for pageToken != "" {
request.PageToken = pageToken
nextRun, err := a.jobsImpl.GetRun(ctx, request)
if err != nil {
return nil, err
}

if isPaginatingIterations {
run.Iterations = append(run.Iterations, nextRun.Iterations...)
} else {
run.Tasks = append(run.Tasks, nextRun.Tasks...)
}
pageToken = nextRun.NextPageToken
}

return run, nil
}
Loading

0 comments on commit 1981951

Please sign in to comment.