Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Reporting] API endpoint for exporting CSV from Saved Search ID #148775

Closed
tsullivan opened this issue Jan 12, 2023 · 2 comments · Fixed by #149172
Closed

[Reporting] API endpoint for exporting CSV from Saved Search ID #148775

tsullivan opened this issue Jan 12, 2023 · 2 comments · Fixed by #149172
Assignees
@tsullivan
Labels
enhancement New value added to drive a business result impact:critical This issue should be addressed immediately due to a critical level of impact on the product. Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience)

Comments

@tsullivan
Copy link
Member

tsullivan commented Jan 12, 2023 •
edited
Loading

This project will bring the /api/reporting/v1/generate/csv/saved-object/search:$SAVED_SEARCH_ID API to the 8.x series. The purpose of this API is a precursor for schedule CSV reports in Kibana, based on an arbitrary saved search. See #148030 for the API implementation in 7.17.

The feature will combine functionality from different domains:

  • Reporting: the framework that accepts requests to generate reports and executes them as asynchronous jobs.
  • Discover: the saved search object contains the configuration of a search the user wishes to export
  • Data: the saved search encodes the configuration for a SearchSource object, which can be abstracts how the ES query is constructed

These domains have to expose functionality that constitutes the "execute" task of a new Reporting export type.
Primarily, Discover needs to provide a method for extracting a usable SearchSource object from a saved search object.

Project document (internal link) https://docs.google.com/document/d/1KNkdm4XDbvvfb3db5zrHIPRDoFvIxNarG_lrp0p9Y_E/edit?usp=sharing
@tsullivan tsullivan added Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) enhancement New value added to drive a business result impact:critical This issue should be addressed immediately due to a critical level of impact on the product. labels Jan 12, 2023
@tsullivan tsullivan self-assigned this Jan 12, 2023
@tsullivan tsullivan mentioned this issue Jan 18, 2023
Merged
1 task
@tsullivan
Copy link
Member Author

Primarily, Discover needs to provide a method for extracting a usable SearchSource object from a saved search object.

It turns out there is an issue in the Discover team's backlog about implementing this, exactly as I had in mind: #114515

cc @davismcphee

@davismcphee davismcphee mentioned this issue Feb 6, 2023
Closed
This was referenced Feb 9, 2023
Merged
Closed
tsullivan added a commit that referenced this issue Feb 16, 2023
@tsullivan
Create Extensions of DiscoverAppLocator for DiscoverServerPlugin (#15…
807625c 
…0631)

## Summary

Adds `DiscoverServerPluginLocatorService`, a set of utilities that
allows consumers to extract search info from `DiscoverAppLocatorParams`.

Needed for #148775

## Refactoring changes
* Moved some code from `src/plugins/discover/public/utils/sorting` to
`common`, which was needed in the server-side context.
* Moved the definition of the `SavedSearch` interface from
`src/plugins/saved_search/public` to `common`

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
@tsullivan
Copy link
Member Author

PR: #149172

@tsullivan tsullivan mentioned this issue Feb 21, 2023
Closed
tsullivan added a commit that referenced this issue Feb 22, 2023
@tsullivan @kibanamachine
[Reporting] New API alternative endpoint for exporting CSV (#149172)
e09ecd0 
## Summary

This new CSV endpoint will be usable for scripted reporting. As of this
PR, the endpoint is not going to be used in the UI: that will be handled
by [this issue](#151190). The
new endpoint is not documented yet, but should be once the endpoint is
used in the UI, according to [this
issue](#151745).

Depends on #150631
Closes #148775

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

## Using the endpoint from a script
A request to generate a report has a string of rison-encoded job
parameters. The parameters can now be made into a template for customers
to inject their own savedSearchID. The time range of the search,
filters, and query can also be dynamically filled in at the time of
requesting a report - much more easily than before.

```sh
SAVED_SEARCH_ID="d779c5d0-a1c3-11ed-bd60-6957c24fc275"
 
# Inject the search ID into the "jobParams" template
JOB_PARAMS="(browserTimezone:UTC,locatorParams:!((id:DISCOVER_APP_LOCATOR,params:(savedSearchId:'${SAVED_SEARCH_ID}',timeRange:(from:now-1M,to:now)))),objectType:search)"
 
# Send a request to generate a report
curl -XPOST $ENDPOINT \
  -H "${AUTH_HEADER}" \
  -H 'Content-Type: application/json' \
  -H 'kbn-xsrf: reporting' 
  -d '{ "jobParams": "'$JOB_PARAMS'"}'
```

## Limitations
A locator without a `savedSearchId` is currently not supported. Getting
that support will make it possible to create exports based on content in
Discover tied only to a Data View (unsaved search support).

## (Demo) Developer Example app
_The demo app is descoped from this PR_
~~This PR updates the Reporting example app to feature a "CSV Explorer."
In this app, you can select a saved search, build up a
DiscoverAppLocatorParams object, and send a request to generate a
report.~~


https://user-images.githubusercontent.com/908371/217356050-f556f897-33c6-4623-aa06-9af191378e48.mp4

## Release Note
We are skipping the release note for now, while waiting for
documentation.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tsullivan tsullivan

Labels
enhancement New value added to drive a business result impact:critical This issue should be addressed immediately due to a critical level of impact on the product. Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Reporting] New API alternative endpoint for exporting CSV tsullivan/kibana
1 participant
@tsullivan