Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enrich jobs endpoint with filtering functionality #26

Merged
merged 1 commit into from
Sep 12, 2016

Conversation

venisa
Copy link

@venisa venisa commented Aug 31, 2016

This PR adds filtering capability to Job endpoint.
The default job payload also contains userId information

Please note: This PR is based on #23
Please ignore the first commit

@venisa venisa added the WIP label Aug 31, 2016
jobViews = apiRequest.getJobViews();
} else {
jobViews = apiRequest.getFilteredJobViews(apiRequest.buildJobStoreFilter(filters));
}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I look at this...I am not sure if it's better to do this here or to call a generic getJobViews(filters) and have JobsApiRequest perform this check and call the correct function accordingly

Copy link
Contributor

@archolewa archolewa Sep 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better if we did what the DataServlet does: pass in the filters to the JobsApiRequest and let it parse the filters internally. Then, when getJobViews() is invoked, the JobsApiRequest applies the filters if present. So, in the DataServlet, the code would look like:

JobsApiRequest apiRequest = new JobsApiRequest (
                    format,
                    null,  //asyncAfter is null so it behaves like a synchronous request
                    perPage,
                    page,
                    filters, //Note the addition of the filters to the constructor
                    uriInfo,
                    jobPayloadBuilder,
                    apiJobStore,
                    preResponseStore,
                    broadcastChannel
);
... //Other stuff
jobViews = apiRequest.getJobViews(); //Inside JobsApiRequest::getJobViews, JobsApiRequest::getFilteredJobViews is invoked if there are filters

@venisa
Copy link
Author

venisa commented Aug 31, 2016

@dayamr @archolewa @cdeszaq Please review

*/
public Observable<Map<String, String>> getFilteredJobViews(Set<ApiJobStoreFilter> apiJobStoreFilters) {
return apiJobStore.getFilteredRows(apiJobStoreFilters)
.map(this::mapJobRowsToJobViews);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be unwrapped.

@archolewa
Copy link
Contributor

Finished my first round. Overall it looks pretty good, though the testing feels a bit sparse. It would be nice if we could have some more tests involving different operations than just eq, and combinations of operations. The SearchProviderSpec has some good examples of very thorough filter tests. I don't feel a strong need to be quite that thorough here, but we may want to lift a few of the more complex filter tests from that specification.

@venisa
Copy link
Author

venisa commented Sep 6, 2016

@archolewa I will add the tests for the filters in #23 as that is where the code was added

@venisa
Copy link
Author

venisa commented Sep 6, 2016

@archolewa
Added tests for each operator: 89ca91d

Added test for multiple filter query with different operators: a93cb6c

Added test for ensuring we return empty list when no filters are satisfied: d01a771

We already have tests that the different filter operators select the correct JobRows:
https://github.com/yahoo/fili/blob/a93cb6cc2169661fe8a701b6aa638e98334e83a1/fili-core/src/test/groovy/com/yahoo/bard/webservice/async/jobs/stores/HashJobStoreSpec.groovy

String result = makeRequest("/jobs", [filters : ["userId-eq[pikachu]"]])

then: "We only get the job payload that satisfies the filter"
GroovyTestUtils.compareJson(result, "{\"jobs\":[]}", JsonSortStrategy.SORT_BOTH)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use single quotes to delimitate the string, you don't have to escape the double quotes around jobs.

@archolewa
Copy link
Contributor

One small comment. 👍 once it's been addressed.

Observable<Map<String, String>> jobViews;

if (filters == null || "".equals(filters)) {
jobViews = apiRequest.getJobViews();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you give filters a default value in the parameter list for this method, then you wouldn't have to test the null case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to be late bringing this up, but in the other ApiRequest objects we've avoided using the 'get' verb when the method is essentially a factory. Since the ApiRequest continues to be both a data object and a factory, it would be completely obscure whether 'getFoo()' was retrieving a built foo or an accessor for an already built foo.

This applies to getJobViews, but not by any means to just getJobViews in this code now.

@venisa venisa force-pushed the Enrich-Jobs-endpoint-with-filtering-functionality branch from d01a771 to baf6b86 Compare September 8, 2016 20:15
@venisa venisa removed the NEED REBASE label Sep 8, 2016
@venisa
Copy link
Author

venisa commented Sep 8, 2016

@michael-mclawhorn This has been rebased on top of master after merging #23

* Returns an Observable containing a stream of job payloads for all the jobs in the ApiJobStore. If, for any
* JobRow, the mapping from JobRow to job view fails, an Observable over JobRequestFailedException is returned. If
* the ApiJobStore is empty, we return an empty Observable.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Space not needed

String result = makeRequest("/jobs", [filters : ["userId-eq[momo]"]])

then: "We only get the job payload that satisfies the filter"
GroovyTestUtils.compareJson(result, expectedResponse, JsonSortStrategy.SORT_BOTH)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorting on Map (the default I think) should be sufficient, right?

@cdeszaq
Copy link
Collaborator

cdeszaq commented Sep 9, 2016

👍 after those last couple of small tweaks

@cdeszaq cdeszaq force-pushed the Enrich-Jobs-endpoint-with-filtering-functionality branch from 2f3e8f2 to de1de63 Compare September 12, 2016 15:06
@cdeszaq cdeszaq merged commit de1de63 into master Sep 12, 2016
@cdeszaq cdeszaq deleted the Enrich-Jobs-endpoint-with-filtering-functionality branch September 13, 2016 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants