Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(search): support sorting on multiple fields #10775

Merged

Conversation

RyanHolstien
Copy link
Collaborator

@RyanHolstien RyanHolstien commented Jun 25, 2024

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Summary by CodeRabbit

  • New Features

    • Enhanced search functionality by allowing multiple sorting criteria instead of a single criterion, improving flexibility for users.
    • Introduced a new optional parameter for sorting criteria in API schemas, enabling more complex query capabilities.
  • Bug Fixes

    • Improved robustness by defaulting to empty sorting criteria instead of null, reducing potential errors during searches.
  • Documentation

    • Updated method documentation to clarify the new plural parameter names for improved clarity and consistency.

@github-actions github-actions bot added product PR or Issue related to the DataHub UI/UX devops PR or Issue related to DataHub backend & deployment labels Jun 25, 2024
Copy link
Contributor

coderabbitai bot commented Aug 7, 2024

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

The recent changes enhance sorting capabilities across various classes in the metadata service. By transitioning from single SortCriterion parameters to List<SortCriterion>, the API now supports multiple sorting criteria, improving the flexibility and robustness of search operations. This update applies to key methods in the JavaEntityClient, EntityClient, and RestliEntityClient, along with corresponding JSON schema definitions.

Changes

Files Change Summary
metadata-io/src/main/java/com/linkedin/metadata/client/JavaEntityClient.java
metadata-service/restli-api/src/main/snapshot/com.linkedin.entity.entities.snapshot.json
metadata-service/restli-client-api/src/main/java/com/linkedin/entity/client/EntityClient.java
metadata-service/restli-client/src/main/java/com/linkedin/entity/client/RestliEntityClient.java
Transitioned from single SortCriterion parameters to List<SortCriterion>, enhancing sorting flexibility in key methods. Updated API documentation for clarity. Added sortCriteria in JSON schema to complement existing sorting options.
metadata-service/openapi-servlet/src/test/java/io/datahubproject/openapi/v3/controller/EntityControllerTest.java Updated test cases to encapsulate sorting criteria in a singleton list, improving clarity in sorting tests.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant JavaEntityClient
    participant EntityClient
    participant RestliEntityClient

    Client->>JavaEntityClient: search(entity, input, filter, sortCriteria)
    JavaEntityClient->>EntityClient: search(entity, input, filter, sortCriteria)
    EntityClient->>RestliEntityClient: search(entity, input, filter, sortCriteria)
    RestliEntityClient-->>EntityClient: Return SearchResult
    EntityClient-->>JavaEntityClient: Return SearchResult
    JavaEntityClient-->>Client: Return SearchResult
Loading

🐇 "With criteria in a list, oh what a delight,
Searches are swift, results shining bright.
No more just one, now multiple we seek,
Our queries are stronger, our options unique.
So hop along, friends, let’s explore and play,
With sorting so fine, it’s a beautiful day!" 🐇


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (4)
metadata-io/src/main/java/com/linkedin/metadata/search/cache/CachedSearchResult.java (1)

10-11: Consider adding serialVersionUID for Serializable class.

The class CachedSearchResult implements Serializable. It's a good practice to define a serialVersionUID to ensure consistency during the serialization and deserialization process.

private static final long serialVersionUID = 1L;
li-utils/src/main/java/com/datahub/util/RecordUtils.java (1)

81-87: LGTM! Consider handling large lists or null elements.

The new toJsonString method correctly handles a list of RecordTemplate objects. However, consider adding checks for null elements in the list and optimizing for large lists if performance becomes an issue.

if (recordTemplates == null || recordTemplates.isEmpty()) {
  return "";
}
for (RecordTemplate recordTemplate : recordTemplates) {
  if (recordTemplate != null) {
    json.append(toJsonString(recordTemplate));
  }
}
metadata-io/src/main/java/com/linkedin/metadata/search/LineageSearchService.java (2)

Line range hint 121-137: Update method documentation for searchAcrossLineage.

The method now accepts a list of SortCriterion. Ensure that the documentation comments reflect this change for clarity.

- * @param sortCriteria list of {@link SortCriterion} to be applied to search results

Line range hint 764-780: Update method documentation for scrollAcrossLineage.

The method now accepts a list of SortCriterion. Ensure that the documentation comments reflect this change for clarity.

- * @param sortCriteria list of {@link SortCriterion} to be applied to search results
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 900c259 and 378a325.

Files selected for processing (55)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/analytics/resolver/GetChartsResolver.java (1 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/analytics/resolver/GetMetadataAnalyticsResolver.java (1 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/auth/DebugAccessResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/auth/ListAccessTokensResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/container/ContainerEntitiesResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/domain/DomainEntitiesResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/domain/ListDomainsResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/group/ListGroupsResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/incident/EntityIncidentsResolver.java (3 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/ingest/execution/IngestionSourceExecutionRequestsResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/ingest/secret/ListSecretsResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/jobs/DataJobRunsResolver.java (3 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/jobs/EntityRunsResolver.java (3 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/ownership/ListOwnershipTypesResolver.java (1 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/post/ListPostsResolver.java (3 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/query/ListQueriesResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/AggregateAcrossEntitiesResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/GetQuickFiltersResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/SearchAcrossEntitiesResolver.java (3 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/SearchResolver.java (2 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/view/ListGlobalViewsResolver.java (1 hunks)
  • datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/view/ListMyViewsResolver.java (1 hunks)
  • datahub-graphql-core/src/main/resources/search.graphql (1 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/auth/ListAccessTokensResolverTest.java (2 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/container/ContainerEntitiesResolverTest.java (1 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/domain/DomainEntitiesResolverTest.java (1 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/domain/ListDomainsResolverTest.java (3 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/incident/EntityIncidentsResolverTest.java (2 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/ingest/execution/IngestionSourceExecutionRequestsResolverTest.java (2 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/ingest/secret/ListSecretsResolverTest.java (4 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/query/ListQueriesResolverTest.java (1 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/AggregateAcrossEntitiesResolverTest.java (3 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/GetQuickFiltersResolverTest.java (3 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/SearchAcrossEntitiesResolverTest.java (3 hunks)
  • datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/SearchResolverTest.java (6 hunks)
  • entity-registry/src/main/java/com/linkedin/metadata/aspect/GraphRetriever.java (2 hunks)
  • li-utils/src/main/java/com/datahub/util/RecordUtils.java (1 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/client/JavaEntityClient.java (15 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/graph/dgraph/DgraphGraphService.java (1 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/graph/elastic/ESGraphQueryDAO.java (3 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/graph/elastic/ElasticSearchGraphService.java (2 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/graph/neo4j/Neo4jGraphService.java (1 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/LineageSearchService.java (13 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/SearchService.java (9 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/SearchServiceSearchRetriever.java (1 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/cache/CachedSearchResult.java (1 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/client/CachingEntitySearchService.java (14 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/ElasticSearchService.java (9 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/query/ESSearchDAO.java (12 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/query/request/SearchRequestHandler.java (7 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/search/utils/ESUtils.java (4 hunks)
  • metadata-io/src/main/java/com/linkedin/metadata/timeseries/elastic/ElasticSearchTimeseriesAspectService.java (4 hunks)
  • metadata-io/src/test/java/com/linkedin/metadata/graph/GraphServiceTestBase.java (1 hunks)
  • metadata-io/src/test/java/com/linkedin/metadata/search/fixtures/SampleDataFixtureTestBase.java (3 hunks)
  • metadata-service/openapi-entity-servlet/src/main/java/io/datahubproject/openapi/v2/delegates/EntityApiDelegateImpl.java (2 hunks)
Files not processed due to max files limit (14)
  • metadata-service/openapi-servlet/src/main/java/io/datahubproject/openapi/controller/GenericEntitiesController.java
  • metadata-service/openapi-servlet/src/main/java/io/datahubproject/openapi/operations/elastic/OperationsController.java
  • metadata-service/openapi-servlet/src/main/java/io/datahubproject/openapi/v2/controller/TimeseriesController.java
  • metadata-service/openapi-servlet/src/test/java/io/datahubproject/openapi/v3/controller/EntityControllerTest.java
  • metadata-service/restli-api/src/main/idl/com.linkedin.entity.entities.restspec.json
  • metadata-service/restli-api/src/main/snapshot/com.linkedin.entity.entities.snapshot.json
  • metadata-service/restli-client-api/src/main/java/com/linkedin/entity/client/EntityClient.java
  • metadata-service/restli-client/src/main/java/com/linkedin/entity/client/RestliEntityClient.java
  • metadata-service/restli-servlet-impl/build.gradle
  • metadata-service/restli-servlet-impl/src/main/java/com/linkedin/metadata/resources/entity/EntityResource.java
  • metadata-service/restli-servlet-impl/src/main/java/com/linkedin/metadata/resources/restli/RestliConstants.java
  • metadata-service/restli-servlet-impl/src/test/java/mock/MockTimeseriesAspectService.java
  • metadata-service/services/src/main/java/com/linkedin/metadata/search/EntitySearchService.java
  • metadata-service/services/src/main/java/com/linkedin/metadata/timeseries/TimeseriesAspectService.java
Files skipped from review due to trivial changes (5)
  • entity-registry/src/main/java/com/linkedin/metadata/aspect/GraphRetriever.java
  • metadata-io/src/main/java/com/linkedin/metadata/graph/elastic/ESGraphQueryDAO.java
  • metadata-io/src/main/java/com/linkedin/metadata/graph/elastic/ElasticSearchGraphService.java
  • metadata-io/src/main/java/com/linkedin/metadata/graph/neo4j/Neo4jGraphService.java
  • metadata-io/src/main/java/com/linkedin/metadata/timeseries/elastic/ElasticSearchTimeseriesAspectService.java
Additional comments not posted (88)
metadata-io/src/main/java/com/linkedin/metadata/search/cache/CachedSearchResult.java (2)

15-18: Ensure proper error handling for compression.

The gzipCompress method is used without explicit error handling. Consider adding error handling to manage potential exceptions during compression.

try {
    this.searchResult = gzipCompress(toJsonString(lineageResult));
} catch (IOException e) {
    // Handle exception
}

20-22: Ensure proper error handling for decompression.

The gzipDecompress method is used without explicit error handling. Consider adding error handling to manage potential exceptions during decompression.

try {
    return toRecordTemplate(SearchResult.class, gzipDecompress(searchResult));
} catch (IOException e) {
    // Handle exception
}
metadata-io/src/main/java/com/linkedin/metadata/search/SearchServiceSearchRetriever.java (1)

46-46: Verify compatibility with multiple sorting criteria.

The change to use List.of(urnSort) suggests support for multiple sorting criteria. Ensure that the scrollAcrossEntities method and related logic can handle a list of sorting criteria.

Verification successful

The scrollAcrossEntities method supports multiple sorting criteria.

The method signature in the SearchService class includes a List<SortCriterion>, confirming compatibility with multiple sorting criteria. No further changes are needed.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the `scrollAcrossEntities` method can handle multiple sorting criteria.

# Test: Search for the method definition and ensure it supports a list of SortCriterion.
ast-grep --lang java --pattern $'scrollAcrossEntities($_, $_, $_, $_, List<SortCriterion> $_, $_, $_, $_)'

Length of output: 107


Script:

#!/bin/bash
# Search for the `scrollAcrossEntities` method definition in the codebase to verify its parameters.
rg 'scrollAcrossEntities' -A 5 --glob '*.java'

Length of output: 26916

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/auth/ListAccessTokensResolverTest.java (1)

50-50: Ensure test coverage for multiple sorting criteria.

The change to Mockito.any(List.class) reflects the new approach to handling sorting criteria. Ensure that the test cases cover scenarios with multiple sorting criteria to verify the functionality.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/post/ListPostsResolver.java (2)

21-23: Imports look good.

The addition of java.util.Collections and java.util.List is necessary for handling the list of sorting criteria.


54-58: Enhance flexibility with multiple sorting criteria.

The use of Collections.singletonList to wrap the SortCriterion allows for future extensibility to support multiple sorting criteria without significant code changes.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/ownership/ListOwnershipTypesResolver.java (1)

70-70: Adopt singleton list for sorting criteria.

The use of Collections.singletonList for DEFAULT_SORT_CRITERION aligns with the new approach to handle multiple sorting criteria, enhancing flexibility.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/container/ContainerEntitiesResolver.java (1)

96-96: Enhance robustness with Collections.emptyList().

Replacing null with Collections.emptyList() ensures that the method handles list inputs more robustly, reducing the risk of null pointer exceptions.

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/container/ContainerEntitiesResolverTest.java (1)

65-65: Update reflects a shift in expected behavior.

The change from Mockito.eq(null) to Mockito.eq(Collections.emptyList()) indicates that the method under test is now expected to handle an empty list instead of a null value. This change improves the robustness of the test by explicitly testing for an empty list scenario, which is generally safer and more predictable than handling null values.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/domain/ListDomainsResolver.java (2)

25-25: Ensure import statement is necessary.

The import statement for Collections has been added to accommodate the use of Collections.singletonList(). Ensure this import is necessary and used correctly in the code.


70-73: Enhance flexibility for sorting criteria.

The change to wrap the SortCriterion instance in Collections.singletonList() standardizes the input as a list, allowing for more flexible sorting options in the future. This is a forward-thinking modification that prepares the codebase for potential enhancements involving multiple sorting criteria.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/domain/DomainEntitiesResolver.java (2)

22-22: Ensure import statement is necessary.

The import statement for Collections has been added to accommodate the use of Collections.emptyList(). Ensure this import is necessary and used correctly in the code.


102-102: Improve method robustness with empty list.

Passing Collections.emptyList() instead of null improves the robustness of the method by preventing potential NullPointerExceptions. This change ensures safer handling of list parameters and aligns with best practices for null safety.

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/domain/DomainEntitiesResolverTest.java (1)

71-71: Good practice: Use an empty list instead of null.

Replacing Mockito.eq(null) with Mockito.eq(Collections.emptyList()) enhances the robustness of the test by avoiding potential null pointer issues. This aligns with best practices for handling collections.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/group/ListGroupsResolver.java (1)

68-71: Enhancement: Support for multiple sort criteria.

Wrapping the SortCriterion in a Collections.singletonList() prepares the method to handle multiple sorting criteria, enhancing flexibility. Ensure that any downstream components expecting a single SortCriterion are updated accordingly.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/view/ListGlobalViewsResolver.java (1)

77-77: Future-proofing: Handling multiple sort criteria.

Wrapping DEFAULT_SORT_CRITERION in a Collections.singletonList() allows the method to accommodate multiple sorting criteria, enhancing its robustness and flexibility. Verify that related components are compatible with this change.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/SearchResolver.java (1)

93-93: Good use of Collections.emptyList().

Replacing null with Collections.emptyList() enhances robustness by preventing potential null pointer exceptions.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/AggregateAcrossEntitiesResolver.java (1)

98-98: Improved robustness with Collections.emptyList().

Using Collections.emptyList() instead of null for the list parameter enhances the method's robustness and prevents null pointer exceptions.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/ingest/execution/IngestionSourceExecutionRequestsResolver.java (1)

80-83: Enhanced sorting flexibility with Collections.singletonList.

Wrapping SortCriterion in Collections.singletonList allows for easier expansion to multiple sorting criteria, aligning with the PR's goal of improving sorting flexibility.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/incident/EntityIncidentsResolver.java (2)

122-126: LGTM!

The method buildIncidentsSortCriteria now returns a list, which aligns with the new feature requirements for supporting multiple sort criteria.


64-70: LGTM! Verify the usage of buildIncidentsSortCriteria.

The changes to use a list of sort criteria are consistent with the new feature requirements.

Ensure that the _entityClient.filter method is compatible with receiving a list of sort criteria.

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/ingest/secret/ListSecretsResolverTest.java (2)

47-47: LGTM!

The use of Mockito.any(List.class) aligns with the updated handling of sorting criteria.


115-115: LGTM!

The use of Mockito.any(List.class) is consistent with the updated handling of sorting criteria.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/query/ListQueriesResolver.java (1)

Line range hint 64-78:
LGTM! Verify the usage of the list of sort criteria.

Encapsulating the SortCriterion within a List aligns with the new feature requirements.

Ensure that the _entityClient.search method is compatible with receiving a list of sort criteria.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/auth/ListAccessTokensResolver.java (1)

Line range hint 62-78: LGTM: Transition to a list of sort criteria enhances flexibility.

The change from a single SortCriterion to a List<SortCriterion> is well-executed and aligns with the goal of supporting multiple sorting criteria. This enhances the flexibility and future extensibility of the sorting mechanism.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/view/ListMyViewsResolver.java (1)

82-82: LGTM: Transition to a list of sort criteria enhances flexibility.

The change from a single SortCriterion to a List<SortCriterion> is well-executed and aligns with the goal of supporting multiple sorting criteria. This enhances the flexibility and future extensibility of the sorting mechanism.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/jobs/DataJobRunsResolver.java (2)

65-71: LGTM: Transition to a list of sort criteria enhances flexibility.

The change from a single SortCriterion to a List<SortCriterion> is well-executed and aligns with the goal of supporting multiple sorting criteria. This enhances the flexibility and future extensibility of the sorting mechanism.


127-131: LGTM: Method now returns a list of sort criteria.

The method buildTaskRunsSortCriteria now returns a List<SortCriterion>, which aligns with the enhanced sorting functionality. This change is consistent with the overall goal of supporting multiple sorting criteria.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/SearchAcrossEntitiesResolver.java (1)

70-87: Ensure comprehensive handling of sort criteria.

The logic for handling multiple sorting criteria is correctly implemented. However, ensure that the SortInput object is always properly validated before accessing its fields to avoid potential NullPointerExceptions. Additionally, consider logging the sort criteria for better traceability during debugging.

Verification successful

Null checks for SortInput are properly implemented.

The code correctly checks for null values before accessing fields of the SortInput object, preventing potential NullPointerExceptions. However, consider adding logging for sort criteria to enhance traceability during debugging.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of `SortInput` and ensure proper null checks are in place.

# Test: Search for `SortInput` usage. Expect: Proper null checks before field access.
rg --type java -A 5 $'input.getSortInput()'

Length of output: 2589

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/ingest/secret/ListSecretsResolver.java (1)

77-80: Verify handling of list-based sort criteria.

The change to use Collections.singletonList for SortCriterion is a step towards supporting multiple criteria. Ensure that all downstream code correctly handles a list of sort criteria, even if it currently only contains one element.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/jobs/EntityRunsResolver.java (1)

71-77: Confirm list handling in sorting logic.

The transition to using a list of SortCriterion objects is correctly implemented. Verify that the sorting logic throughout the codebase is compatible with this change, ensuring no assumptions of a single criterion exist.

Verification successful

Sorting logic is compatible with list-based sorting.

The transition to using a list of SortCriterion objects is well-supported across the codebase. Various components, including utility methods and resolvers, are designed to handle multiple sorting criteria, ensuring compatibility with the list-based sorting logic. No assumptions of a single criterion were found in the critical paths.

  • EntityResource class handles conversion to list-based criteria.
  • Utility methods like buildSortOrder in ESUtils support list-based sorting.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the sorting logic is compatible with a list of `SortCriterion`.

# Test: Search for sorting logic usage. Expect: Compatibility with list-based sorting.
rg --type java -A 5 $'List<SortCriterion>'

Length of output: 72904

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/analytics/resolver/GetMetadataAnalyticsResolver.java (1)

80-81: LGTM! But verify the method's usage.

Replacing null with Collections.emptyList() enhances code clarity and reduces the risk of NullPointerException. Ensure that the _entityClient.searchAcrossEntities method handles this change as expected.

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/domain/ListDomainsResolverTest.java (1)

51-54: LGTM! But verify the test coverage.

Wrapping SortCriterion in Collections.singletonList() enhances the flexibility of the test cases. Ensure that the tests still cover the intended functionality and edge cases.

Also applies to: 95-98

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/query/ListQueriesResolverTest.java (1)

72-75: LGTM! But verify the test coverage.

Wrapping SortCriterion in Collections.singletonList() aligns with the updated method signature and allows for multiple sorting criteria. Ensure that the test still validates the correct behavior and edge cases.

Verification successful

Test Coverage for Sorting Functionality Verified

The test files involving SortCriterion suggest that sorting functionality, including edge cases and correct behavior, is adequately covered. Although Collections.singletonList is not explicitly used, the tests ensure the intended sorting behavior is validated.

  • ListQueriesResolverTest.java
  • SearchResolverTest.java
  • EntityIncidentsResolverTest.java
  • ListDomainsResolverTest.java
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the test coverage for multiple sorting criteria.

# Test: Search for test methods that use `Collections.singletonList()`. Expect: Tests should cover multiple sorting criteria.
rg --type java -A 5 $'Collections.singletonList(new SortCriterion()'

Length of output: 191


Script:

#!/bin/bash
# Description: Verify the test coverage for multiple sorting criteria.

# Test: Search for test methods that use `Collections.singletonList(new SortCriterion())`. Expect: Tests should cover multiple sorting criteria.
rg --type java -A 5 $'Collections\\.singletonList\\(new SortCriterion\\('

Length of output: 71


Script:

#!/bin/bash
# Description: Search for test methods that involve `SortCriterion`.

# Test: Search for any usage of `SortCriterion` in test files to verify sorting functionality coverage.
rg --type java -A 5 'SortCriterion' datahub-graphql-core/src/test/java/

Length of output: 8539

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/SearchResolverTest.java (4)

61-61: Use of Collections.emptyList() for sort criteria is a good practice.

Replacing null with Collections.emptyList() enhances clarity and prevents null pointer exceptions.


102-102: Consistent use of Collections.emptyList() for sort criteria is beneficial.

This change aligns with best practices for handling optional list parameters.


134-134: Consistent application of Collections.emptyList() for sort criteria is commendable.

This change ensures uniform handling of list parameters across test cases.


Line range hint 175-186: Update to accept List<SortCriterion> enhances flexibility.

This change supports the new feature of handling multiple sort criteria, improving the test's alignment with functionality.

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/incident/EntityIncidentsResolverTest.java (1)

96-96: Use of Collections.singletonList() for sort criterion is appropriate.

This change aligns with the updated method signature, ensuring compatibility and correctness.

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/ingest/execution/IngestionSourceExecutionRequestsResolverTest.java (1)

49-49: Update to use List.class for sort criteria is necessary.

This change ensures the test is compatible with the updated method signature that supports multiple sorting criteria.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/search/GetQuickFiltersResolver.java (1)

111-111: Good practice: Use of Collections.emptyList() instead of null.

Replacing null with Collections.emptyList() enhances code robustness by preventing potential NullPointerExceptions. This is a commendable change.

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/auth/DebugAccessResolver.java (1)

185-185: Enhancement: Support for multiple sorting criteria.

Wrapping sortCriterion in Collections.singletonList() allows the method to handle a list of sorting criteria, increasing its flexibility. This change aligns with the PR's objective of supporting multi-field sorting.

metadata-io/src/main/java/com/linkedin/metadata/search/SearchService.java (1)

68-68: Enhancement: Transition to multiple sorting criteria.

The transition from a single SortCriterion to a list of SortCriterion objects in various methods enhances the search functionality's flexibility. This change is well-aligned with the PR's objective of supporting multi-field sorting.

Ensure that all invocations of these methods in the codebase are updated to pass a list of SortCriterion.

Also applies to: 80-80, 108-108, 136-136, 241-241, 253-253

Verification successful

Verification Complete: All Invocations Updated

The search results confirm that all invocations of the search, searchAcrossEntities, and scrollAcrossEntities methods are passing a list of SortCriterion. This aligns with the transition to support multiple sorting criteria.

  • Methods in EntityResource, RestliEntityClient, SearchService, and other relevant classes are correctly updated to use List<SortCriterion>.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all invocations of the updated methods are passing a list of SortCriterion.

# Test: Search for invocations of the `search` method. Expect: All invocations pass a list of SortCriterion.
rg --type java -A 2 $'search('

# Test: Search for invocations of the `searchAcrossEntities` method. Expect: All invocations pass a list of SortCriterion.
rg --type java -A 2 $'searchAcrossEntities('

# Test: Search for invocations of the `scrollAcrossEntities` method. Expect: All invocations pass a list of SortCriterion.
rg --type java -A 2 $'scrollAcrossEntities('

Length of output: 374


Script:

#!/bin/bash
# Description: Verify all invocations of the updated methods are passing a list of SortCriterion.

# Test: Search for invocations of the `search` method. Expect: All invocations pass a list of SortCriterion.
rg --type java -A 2 $'search\\('

# Test: Search for invocations of the `searchAcrossEntities` method. Expect: All invocations pass a list of SortCriterion.
rg --type java -A 2 $'searchAcrossEntities\\('

# Test: Search for invocations of the `scrollAcrossEntities` method. Expect: All invocations pass a list of SortCriterion.
rg --type java -A 2 $'scrollAcrossEntities\\('

Length of output: 129002


Script:

#!/bin/bash
# Extract lines where `search`, `searchAcrossEntities`, and `scrollAcrossEntities` are invoked with context to check if a list of SortCriterion is passed.

# Search for invocations of `search` method and check for list of SortCriterion.
rg --type java -A 5 -B 2 'search\(' | grep -E 'sortCriteria|sortCriterionList'

# Search for invocations of `searchAcrossEntities` method and check for list of SortCriterion.
rg --type java -A 5 -B 2 'searchAcrossEntities\(' | grep -E 'sortCriteria|sortCriterionList'

# Search for invocations of `scrollAcrossEntities` method and check for list of SortCriterion.
rg --type java -A 5 -B 2 'scrollAcrossEntities\(' | grep -E 'sortCriteria|sortCriterionList'

Length of output: 5528

metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/ElasticSearchService.java (5)

363-370: LGTM! Verify DAO method updates.

The change to accept a list of SortCriterion enhances flexibility. Ensure the esSearchDAO.scroll method is updated accordingly.


184-190: LGTM! Verify DAO method updates.

The change to accept a list of SortCriterion enhances flexibility. Ensure the esSearchDAO.filter method is updated accordingly.

Verification successful

DAO Method Update Verified

The filter method in ESSearchDAO has been successfully updated to handle a List<SortCriterion>, aligning with the changes in ElasticSearchService. No further action is needed.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the `esSearchDAO.filter` method handles List<SortCriterion>.

# Test: Search for the `filter` method in ESSearchDAO. Expect: Method signature with List<SortCriterion>.
ast-grep --lang java --pattern $'class ESSearchDAO {
  $$$
  SearchResult filter($_, $_, $_, List<SortCriterion> sortCriteria, $_, $_) {
    $$$
  }
  $$$
}'

Length of output: 158


Script:

#!/bin/bash
# Search for the ESSearchDAO class to verify its existence in the codebase.
rg 'class ESSearchDAO' -A 10

# Search for the filter method in the ESSearchDAO class to verify its existence and signature.
rg 'SearchResult filter' -A 5

Length of output: 5952


333-340: LGTM! Verify DAO method updates.

The change to accept a list of SortCriterion enhances flexibility. Ensure the esSearchDAO.scroll method is updated accordingly.

Verification successful

DAO Method Updated Correctly

The scroll method in ESSearchDAO has been updated to include List<SortCriterion> sortCriteria as a parameter. This confirms that the DAO layer is handling the new parameter correctly.

  • File: metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/query/ESSearchDAO.java
  • Method Signature: ScrollResult scroll(@Nonnull OperationContext opContext, @Nonnull List<String> entities, @Nonnull String input, @Nullable Filter postFilters, List<SortCriterion> sortCriteria, @Nullable String scrollId, @Nullable String keepAlive, int size)
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the `esSearchDAO.scroll` method handles List<SortCriterion>.

# Test: Search for the `scroll` method in ESSearchDAO. Expect: Method signature with List<SortCriterion>.
ast-grep --lang java --pattern $'class ESSearchDAO {
  $$$
  ScrollResult scroll($_, $_, $_, $_, List<SortCriterion> sortCriteria, $_, $_, $_) {
    $$$
  }
  $$$
}'

Length of output: 166


Script:

#!/bin/bash
# Search for the `scroll` method in ESSearchDAO to gather more context.
rg 'ScrollResult scroll' -A 10 --glob '**/ESSearchDAO.java'

Length of output: 1524


145-148: LGTM! Verify DAO method updates.

The change to accept a list of SortCriterion enhances flexibility. Ensure the esSearchDAO.search method is updated accordingly.


Line range hint 403-416: LGTM! Verify DAO method updates.

The change to accept a list of SortCriterion enhances flexibility. Ensure the esSearchDAO.explain method is updated accordingly.

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/GetQuickFiltersResolverTest.java (2)

118-118: Improved robustness by using Collections.emptyList().

Replacing null with Collections.emptyList() enhances code clarity and prevents null pointer exceptions.


304-304: Improved robustness by using Collections.emptyList().

Replacing null with Collections.emptyList() enhances code clarity and prevents null pointer exceptions.

metadata-io/src/main/java/com/linkedin/metadata/search/client/CachingEntitySearchService.java (5)

Line range hint 149-172: LGTM! Proper handling of List<SortCriterion>.

The method correctly checks if the list is not empty before serializing to JSON, ensuring proper caching behavior.


Line range hint 273-295: LGTM! Proper handling of List<SortCriterion>.

The method correctly checks if the list is not empty before serializing to JSON, ensuring proper caching behavior.


377-387: LGTM! Consistent handling of List<SortCriterion>.

The method signature update aligns with other service methods, ensuring consistent handling of multiple sorting criteria.


62-67: LGTM! Verify internal method updates.

The change to accept a list of SortCriterion enhances flexibility. Ensure the getCachedSearchResults method is updated accordingly.


130-135: LGTM! Verify internal method updates.

The change to accept a list of SortCriterion enhances flexibility. Ensure the getCachedScrollResults method is updated accordingly.

Verification successful

Verification Complete: getCachedScrollResults Method Updated

The getCachedScrollResults method in CachingEntitySearchService.java has been updated to accept a List<SortCriterion> as expected. This confirms that the internal handling of sort criteria is consistent with the changes.

  • Location: metadata-io/src/main/java/com/linkedin/metadata/search/client/CachingEntitySearchService.java
  • Method Signature:
    public ScrollResult getCachedScrollResults(
        @Nonnull OperationContext opContext,
        @Nonnull List<String> entities,
        @Nonnull String query,
        @Nullable Filter filters,
        List<SortCriterion> sortCriteria,
        @Nullable String scrollId,
        @Nullable String keepAlive,
        int size)
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the `getCachedScrollResults` method handles List<SortCriterion>.

# Test: Search for the `getCachedScrollResults` method. Expect: Method signature with List<SortCriterion>.
ast-grep --lang java --pattern $'class CachingEntitySearchService {
  $$$
  ScrollResult getCachedScrollResults($_, $_, $_, $_, List<SortCriterion> sortCriteria, $_, $_, $_) {
    $$$
  }
  $$$
}'

Length of output: 197


Script:

#!/bin/bash
# Search for the `getCachedScrollResults` method in the codebase without specifying parameter types.
ast-grep --lang java --pattern $'class CachingEntitySearchService {
  $$$
  ScrollResult getCachedScrollResults($$$) {
    $$$
  }
  $$$
}'

Length of output: 140


Script:

#!/bin/bash
# Search for occurrences of `getCachedScrollResults` across the entire codebase to determine its location and usage.
rg 'getCachedScrollResults' -A 5

Length of output: 2482

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/AggregateAcrossEntitiesResolverTest.java (1)

327-327: Approve changes for improved collection handling.

The replacement of Mockito.eq(null) with Mockito.eq(Collections.emptyList()) enhances the robustness of the tests by ensuring consistent handling of empty collections.

Also applies to: 400-400, 423-423

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/analytics/resolver/GetChartsResolver.java (1)

162-165: Approve change for flexible sorting criteria.

Wrapping SortCriterion in Collections.singletonList allows for future expansion to multiple sorting criteria, enhancing the method's flexibility.

Ensure that this change is integrated correctly with other parts of the codebase that may rely on this method.

Verification successful

Integration Verified: No External Dependencies Found

The searchForNewUsers method is only used within the GetChartsResolver.java file, indicating that the change to utilize Collections.singletonList for SortCriterion does not impact other parts of the codebase. The integration is correctly scoped to this file.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify integration of `searchForNewUsers` with other parts of the codebase.

# Test: Search for method calls and usages. Expect: Consistent handling of sorting criteria.
rg --type java -A 5 $'searchForNewUsers'

Length of output: 1949

datahub-graphql-core/src/test/java/com/linkedin/datahub/graphql/resolvers/search/SearchAcrossEntitiesResolverTest.java (1)

440-441: Approve changes for improved collection handling.

The replacement of Mockito.eq(null) with Mockito.eq(Collections.emptyList()) enhances the robustness of the tests by ensuring consistent handling of empty collections.

Also applies to: 488-488, 509-509

metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/query/request/SearchRequestHandler.java (3)

Line range hint 190-216:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in getSearchRequest look good and align with the PR objectives.

Ensure that ESUtils.buildSortOrder correctly handles a list of SortCriterion.

Verification successful

Verified: Handling of multiple sort criteria in ESUtils.buildSortOrder

The buildSortOrder method in ESUtils correctly processes a list of SortCriterion, ensuring compatibility with the changes made in getSearchRequest. The method handles potential null values by defaulting to an empty list, which is a good practice.

  • ESUtils.buildSortOrder implementation correctly handles a list of SortCriterion.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that `ESUtils.buildSortOrder` correctly handles a list of `SortCriterion`.

# Test: Search for the `buildSortOrder` method implementation and verify its handling of lists.
ast-grep --lang java --pattern $'public static void buildSortOrder($_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 958


Line range hint 288-306:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in getFilterRequest are consistent and necessary.

Ensure that ESUtils.buildSortOrder correctly handles a list of SortCriterion.

Verification successful

Verification Successful: ESUtils.buildSortOrder Handles Multiple Sort Criteria Correctly

The buildSortOrder method in ESUtils is designed to handle a list of SortCriterion, ensuring that multiple sorting criteria are processed correctly. It defaults to an empty list if sortCriteria is null, which is a robust approach.

  • ESUtils.buildSortOrder correctly processes a List<SortCriterion>, ensuring proper handling of multiple sort criteria.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that `ESUtils.buildSortOrder` correctly handles a list of `SortCriterion`.

# Test: Search for the `buildSortOrder` method implementation and verify its handling of lists.
ast-grep --lang java --pattern $'public static void buildSortOrder($_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 958


Line range hint 246-275:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in the overloaded getSearchRequest method are consistent with the overall design.

Ensure that ESUtils.buildSortOrder correctly handles a list of SortCriterion.

Verification successful

Verified: ESUtils.buildSortOrder correctly handles multiple sort criteria.

The method iterates over the sortCriteria list and applies each sorting criterion, ensuring that multiple sorting options are processed as intended. This confirms the correctness of the changes in the getSearchRequest method.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that `ESUtils.buildSortOrder` correctly handles a list of `SortCriterion`.

# Test: Search for the `buildSortOrder` method implementation and verify its handling of lists.
ast-grep --lang java --pattern $'public static void buildSortOrder($_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 958


Script:

#!/bin/bash
# Description: Verify the implementation details of the overloaded `buildSortOrder` method.

# Search for the overloaded `buildSortOrder` method implementation to confirm handling of multiple sort criteria.
rg 'void buildSortOrder\(' -A 10 --no-filename

Length of output: 1105

metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/query/ESSearchDAO.java (6)

Line range hint 231-260:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in the search method are appropriate and align with the PR objectives.

Ensure that the handling of sortCriteria is consistent and correct throughout the method.

Verification successful

Handling of sortCriteria is correct and consistent.

The sortCriteria is properly used in the search method of ESSearchDAO.java to construct search requests, supporting multiple sorting criteria as intended. The changes align with the PR objectives.

  • File: metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/query/ESSearchDAO.java
  • Lines: 241-247
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `search` method.

# Test: Search for the `search` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public SearchResult search($_, $_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 636


Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `search` method by searching for its usage.

# Find the implementation of the `search` method and check for `sortCriteria` usage.
ast-grep --lang java --pattern $'public SearchResult search($_, $_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }' || true

# Search for the usage of `sortCriteria` within the `search` method.
rg --context 5 'sortCriteria' --type java

Length of output: 272769


Line range hint 486-559:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in getScrollRequest are consistent and necessary.

Ensure that the handling of sortCriteria is consistent and correct throughout the method.


Line range hint 404-442:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in the aggregateByValue method are consistent and necessary.

Ensure that the handling of sortCriteria is consistent and correct throughout the method.


Line range hint 539-559:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in the explain method are consistent and necessary.

Ensure that the handling of sortCriteria is consistent and correct throughout the method.

Verification successful

Handling of sortCriteria is consistent and correct in the explain method.

The sortCriteria parameter is appropriately used in constructing the search request within the explain method, ensuring that multiple sorting criteria are handled correctly. No issues were found in its implementation.

  • ESSearchDAO.java: sortCriteria is passed to getScrollRequest for search request construction.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `explain` method.

# Test: Search for the `explain` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public ExplainResponse explain($_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 636


Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `explain` method.

# Test: Search for the `explain` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public ExplainResponse explain($_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 636


Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `explain` method.

# Test: Search for the `explain` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public ExplainResponse explain($_, $_, $_, List<SortCriterion> $_, $$$) { $$$ }'

Length of output: 616


Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `explain` method.

# Test: Search for the `explain` method implementation and verify its handling of `sortCriteria`.
rg 'public ExplainResponse explain' -A 20 | grep 'sortCriteria'

Length of output: 447


Script:

#!/bin/bash
# Description: Extract more lines from the `ESSearchDAO.java` to verify the handling of `sortCriteria`.

# Extract lines from the `ESSearchDAO.java` file to examine the handling of `sortCriteria`.
rg 'public ExplainResponse explain' -A 50 --no-filename

Length of output: 2342


Line range hint 442-486:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in the scroll method are consistent and necessary.

Ensure that the handling of sortCriteria is consistent and correct throughout the method.

Verification successful

Handling of sortCriteria is consistent and correct in ESSearchDAO.

The sortCriteria is appropriately passed as a parameter and used in constructing search requests across methods in the ESSearchDAO class. This supports the intended functionality for sorting.

  • Methods Involved:
    • search
    • filter
    • scroll
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `scroll` method.

# Test: Search for the `scroll` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public ScrollResult scroll($_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 628


Script:

#!/bin/bash
# Description: Corrected pattern to verify the handling of `sortCriteria` in the `scroll` method.

# Test: Search for the `scroll` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern 'public ScrollResult scroll($$$) { $$$ }'

Length of output: 3414


Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `ESSearchDAO` class.

# Test: Search for the usage of `sortCriteria` within the `ESSearchDAO` class.
ast-grep --lang java --pattern 'class ESSearchDAO { $$$ }'

Length of output: 60


Script:

#!/bin/bash
# Description: Search for methods in the `ESSearchDAO` class and verify the handling of `sortCriteria`.

# Find all methods within the ESSearchDAO class and check for usage of sortCriteria.
ast-grep --lang java --pattern 'class ESSearchDAO { $$$ }' --json | jq -r '.[] | .text' | grep -A 10 'sortCriteria'

Length of output: 115


Script:

#!/bin/bash
# Description: Search for method definitions and usage of sortCriteria in the ESSearchDAO class.

# Find all method definitions within the ESSearchDAO class.
ast-grep --lang java --pattern 'class ESSearchDAO { $_ }' --json | jq -r '.[] | .text' | grep -E 'public|private|protected'

# Search specifically for usage of sortCriteria in the ESSearchDAO class.
rg 'sortCriteria' --context 5 --glob '**/ESSearchDAO.java'

Length of output: 19302


Line range hint 273-292:
LGTM! Verify the handling of multiple sort criteria.

The changes to support multiple sorting criteria in the filter method are consistent and necessary.

Ensure that the handling of sortCriteria is consistent and correct throughout the method.

Verification successful

Handling of sortCriteria in filter Method is Correct

The filter method in ESSearchDAO correctly integrates sortCriteria into the search request construction, ensuring that multiple sorting criteria are applied as intended. The sortCriteria parameter is passed to the getFilterRequest method, which is responsible for building the search request with the specified sorting.

  • ESSearchDAO.java: filter method uses sortCriteria in getFilterRequest.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `filter` method.

# Test: Search for the `filter` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public SearchResult filter($_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

Length of output: 628


Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `filter` method.

# Test: Search for the `filter` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public SearchResult filter($_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }' --debug

# Additionally, search for the usage of `sortCriteria` within the method to ensure it's correctly handled.
rg 'sortCriteria' -A 10 --java

Length of output: 786


Script:

#!/bin/bash
# Description: Verify the handling of `sortCriteria` in the `filter` method.

# Test: Search for the `filter` method implementation and verify its handling of `sortCriteria`.
ast-grep --lang java --pattern $'public SearchResult filter($_, $_, $_, List<SortCriterion> sortCriteria, $$$) { $$$ }'

# Additionally, search for the usage of `sortCriteria` within the method to ensure it's correctly handled.
rg 'sortCriteria' -A 10

Length of output: 287475


Script:

#!/bin/bash
# Description: Locate the `filter` method implementation in the `ESSearchDAO` class and verify the usage of `sortCriteria`.

# Search for the `filter` method implementation in `ESSearchDAO.java`.
rg 'public SearchResult filter\(' -A 20 metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/query/ESSearchDAO.java

Length of output: 1092

datahub-graphql-core/src/main/resources/search.graphql (2)

1375-1375: Deprecation Notice for sortCriterion.

The sortCriterion field is marked as deprecated. Ensure that all dependent code is updated to use sortCriteria to avoid issues.


1378-1380: Introduction of sortCriteria.

The new sortCriteria field allows for multiple sorting criteria, enhancing search flexibility. Ensure that all relevant parts of the codebase are updated to utilize this new field.

metadata-io/src/main/java/com/linkedin/metadata/client/JavaEntityClient.java (8)

376-382: Update to search Method Signature.

The search method now accepts a list of SortCriterion instead of a single criterion. Ensure that this change is propagated to all calling code and that the method handles empty lists correctly.


413-413: Update to list Method Signature.

The list method now accepts a list of SortCriterion. Ensure that this change is reflected in all calling code and that the method handles empty lists appropriately.


Line range hint 424-436: Update to search Method Signature with Sort Criteria.

The search method now uses a list of SortCriterion. Verify that the implementation handles multiple sorting criteria correctly and that the documentation is updated to reflect this change.


456-459: Update to searchAcrossEntities Method Signature.

The searchAcrossEntities method now accepts a list of SortCriterion. Ensure that this change is handled in all related code and that the method processes multiple criteria as expected.


Line range hint 471-494: Update to searchAcrossEntities with Facets and Sort Criteria.

The method now supports multiple sorting criteria. Verify that the logic for handling these criteria is correctly implemented and that any relevant documentation is updated.


Line range hint 536-550: Update to searchAcrossLineage Method Signature.

The searchAcrossLineage method now takes a list of SortCriterion. Ensure that the method logic accommodates multiple criteria and that all calling code is updated accordingly.


Line range hint 566-584: Update to scrollAcrossLineage Method Signature.

The scrollAcrossLineage method now uses a list of SortCriterion. Verify that multiple criteria are processed correctly and that all related code is updated.


Line range hint 652-662: Update to filter Method Signature.

The filter method now accepts a list of SortCriterion. Ensure that the method handles multiple criteria appropriately and that all calling code is updated.

metadata-io/src/main/java/com/linkedin/metadata/graph/dgraph/DgraphGraphService.java (1)

791-791: Update to scrollRelatedEntities Method Signature.

The scrollRelatedEntities method now accepts a list of SortCriterion. Ensure that the method logic is updated to handle multiple criteria effectively.

metadata-io/src/main/java/com/linkedin/metadata/search/utils/ESUtils.java (2)

302-309: Ensure correct handling of null sortCriteria.

The method now accepts a list of SortCriterion, and it correctly defaults to an empty list if sortCriteria is null. This is a good practice to avoid null pointer exceptions.


Line range hint 321-365: Verify logic for handling multiple sort criteria.

The updated logic iterates over sortCriteria and applies each SortCriterion to the searchSourceBuilder. Ensure that this logic correctly handles cases where multiple criteria are provided and that the sort order is applied as expected.

Verification successful

Logic for Handling Multiple Sort Criteria Verified

The buildSortOrder method in ESUtils.java correctly iterates over the sortCriteria list and applies each SortCriterion to the searchSourceBuilder. The logic ensures that multiple sort criteria are handled as expected, with appropriate sort orders applied.

  • Location: metadata-io/src/main/java/com/linkedin/metadata/search/utils/ESUtils.java
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that multiple sort criteria are correctly applied in the codebase.

# Test: Search for usages of `buildSortOrder` to ensure multiple sort criteria are handled.
ast-grep --lang java --pattern $'buildSortOrder($_, $_, $_, $_)' 

Length of output: 339


Script:

#!/bin/bash
# Description: Locate and examine the implementation of the buildSortOrder method to verify handling of multiple sort criteria.
ast-grep --lang java --pattern $'void buildSortOrder($_, $_, $_, $_) { $$$ }'

Length of output: 544


Script:

#!/bin/bash
# Description: Correct the pattern to locate the buildSortOrder method definition.
ast-grep --lang java --pattern 'void buildSortOrder($_, $_, $_, $_)'

Length of output: 528


Script:

#!/bin/bash
# Description: Use rg to locate the buildSortOrder method definition in Java files.
rg 'void buildSortOrder' --glob '*.java' -A 10

Length of output: 2733

metadata-io/src/main/java/com/linkedin/metadata/search/LineageSearchService.java (4)

Line range hint 834-881: Review getScrollResultInBatches for handling sortCriteria.

The method now accepts sortCriteria. Ensure that it correctly applies these criteria when querying search results.

Verification successful

Handling of sortCriteria is verified and correct.

The sortCriteria parameter is appropriately passed to the _searchService.scrollAcrossEntities method, ensuring it is applied when querying search results. No issues found with its handling.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify `getScrollResultInBatches` method's handling of `sortCriteria`.

# Test: Search for the method implementation and ensure `sortCriteria` is used correctly.
ast-grep --lang java --pattern $'getScrollResultInBatches($_, $_, $_, $_, $_, $_, $_, $_)'

Length of output: 1077


Script:

#!/bin/bash
# Description: Check the usage of `sortCriteria` within the `getScrollResultInBatches` method.

# Search for the implementation of `getScrollResultInBatches` and extract lines around its usage of `sortCriteria`.
rg -A 10 'private LineageScrollResult getScrollResultInBatches' metadata-io/src/main/java/com/linkedin/metadata/search/LineageSearchService.java

Length of output: 528


Script:

#!/bin/bash
# Description: Extract more lines from the `getScrollResultInBatches` method to check the usage of `sortCriteria`.

# Extract additional lines from the `getScrollResultInBatches` method.
rg -A 50 'private LineageScrollResult getScrollResultInBatches' metadata-io/src/main/java/com/linkedin/metadata/search/LineageSearchService.java

Length of output: 2195


Line range hint 306-321: Ensure canDoLightning method handles sortCriteria correctly.

The method checks if sortCriteria is not empty. Verify that this condition is correctly used to determine whether the "lightning" approach can be applied.

Verification successful

canDoLightning method handles sortCriteria correctly. The method checks if sortCriteria is not empty, which is consistent with the logic for determining whether the "lightning" approach can be applied.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the `canDoLightning` method's handling of `sortCriteria`.

# Test: Search for the method implementation and ensure `sortCriteria` is used correctly.
ast-grep --lang java --pattern $'canDoLightning($_, $_, $_, $_)'

Length of output: 242


Script:

#!/bin/bash
# Search for the `canDoLightning` method definition to examine its implementation.
ast-grep --lang java --pattern 'private boolean canDoLightning($_, $_, $_, $_) { $$$ }'

Length of output: 566


Script:

#!/bin/bash
# Correct the pattern to search for the `canDoLightning` method definition.
ast-grep --lang java --pattern 'private boolean canDoLightning($_, $_, $_, $_)'

Length of output: 550


Script:

#!/bin/bash
# Use ripgrep to search for the `canDoLightning` method definition.
rg 'private boolean canDoLightning' --glob '*.java' -A 10

Length of output: 59


Script:

#!/bin/bash
# Broaden the search to find any method named `canDoLightning` in Java files.
rg 'boolean canDoLightning' --glob '*.java' -A 10

Length of output: 1317


Line range hint 536-569: Review getSearchResultInBatches for handling sortCriteria.

The method now accepts sortCriteria. Ensure that it correctly applies these criteria when querying search results.

Verification successful

Sort criteria are correctly applied in getSearchResultInBatches. The sortCriteria parameter is appropriately passed to the _searchService.searchAcrossEntities method, ensuring it is used in the search query. No further action is needed.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify `getSearchResultInBatches` method's handling of `sortCriteria`.

# Test: Search for the method implementation and ensure `sortCriteria` is used correctly.
ast-grep --lang java --pattern $'getSearchResultInBatches($_, $_, $_, $_, $_, $_, $_)'

Length of output: 1010


Script:

#!/bin/bash
# Search for the implementation of the `getSearchResultInBatches` method to verify the usage of `sortCriteria`.
ast-grep --lang java --pattern $'private LineageSearchResult getSearchResultInBatches($_, $_, $_, $_, $_, $_, $_) { $$$ }'

Length of output: 634


Script:

#!/bin/bash
# Search for the definition of the `getSearchResultInBatches` method to verify the usage of `sortCriteria`.
ast-grep --lang java --pattern $'private LineageSearchResult getSearchResultInBatches($_, $_, $_, $_, $_, $_, $_)'

Length of output: 618


Script:

#!/bin/bash
# Use rg to search for the definition of the `getSearchResultInBatches` method.
rg 'private LineageSearchResult getSearchResultInBatches\(' -A 20

Length of output: 2641


Script:

#!/bin/bash
# Extend the search to capture more lines of the `getSearchResultInBatches` method.
rg 'private LineageSearchResult getSearchResultInBatches\(' -A 50

Length of output: 6235


Line range hint 258-279: Verify logic for using sortCriteria in searchAcrossLineage.

The method now uses sortCriteria to determine the sorting behavior. Ensure the logic correctly handles multiple criteria and integrates well with the rest of the method.

metadata-service/openapi-entity-servlet/src/main/java/io/datahubproject/openapi/v2/delegates/EntityApiDelegateImpl.java (1)

632-643: Review scroll method for handling multiple sort criteria.

The method now constructs a list of SortCriterion from the sort parameter. Ensure that this logic correctly handles cases where multiple sort fields are provided.

Verification successful

The scroll method correctly handles multiple sorting criteria.

The method constructs a list of SortCriterion objects from the sort parameter, applying the specified or default sort order. This approach ensures that multiple sort fields are handled appropriately. No changes are needed.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify `scroll` method's handling of multiple sort criteria.

# Test: Search for the method implementation and ensure multiple sort criteria are handled correctly.
ast-grep --lang java --pattern $'scroll($_, $_, $_, $_, $_, $_, $_, $_)'

Length of output: 73


Script:

#!/bin/bash
# Search for the `scroll` method in the codebase to verify its implementation.
rg 'scroll\(' -A 10

Length of output: 24800


Script:

#!/bin/bash
# Extract more lines of the `scroll` method implementation to review handling of multiple sort criteria.
rg 'public ResponseEntity<S> scroll\(' -A 50 metadata-service/openapi-entity-servlet/src/main/java/io/datahubproject/openapi/v2/delegates/EntityApiDelegateImpl.java

Length of output: 2217

metadata-io/src/test/java/com/linkedin/metadata/search/fixtures/SampleDataFixtureTestBase.java (2)

219-219: LGTM! But verify the usage of buildSortOrder.

The changes correctly wrap the SortCriterion in a singleton list to match the updated method signature. This is a forward-compatible change that allows for multiple sorting criteria.

However, ensure that all calls to ESUtils.buildSortOrder in the codebase have been updated to use a list of SortCriterion.

Also applies to: 238-238

Verification successful

All occurrences of ESUtils.buildSortOrder are correctly using a list of SortCriterion.

The search confirmed that the method ESUtils.buildSortOrder is being called with a list, specifically using Collections.singletonList, which aligns with the expected usage pattern after the changes.

  • SampleDataFixtureTestBase.java contains the correct usage of ESUtils.buildSortOrder.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all calls to `ESUtils.buildSortOrder` use a list of `SortCriterion`.

# Test: Search for the method usage. Expect: Only occurrences with a list as the second argument.
rg --type java -A 1 $'ESUtils.buildSortOrder\\(.*\\,[^\\)]*\\)'

Length of output: 189


Script:

#!/bin/bash
# Description: Verify all calls to `ESUtils.buildSortOrder` use a list of `SortCriterion`.

# Test: Search for the method usage. Expect: Only occurrences with a list as the second argument.
rg --type java -A 1 'ESUtils\.buildSortOrder\(.*,\s*Collections\.singletonList\('

Length of output: 737


1962-1962: LGTM! But verify the usage of the updated search method.

The change to pass a list of SortCriterion is consistent with the updated method signature, allowing for multiple sorting criteria.

However, ensure that all calls to this search method in the codebase have been updated to use a list of SortCriterion.

metadata-io/src/test/java/com/linkedin/metadata/graph/GraphServiceTestBase.java (1)

2199-2199: Review the assertion logic change.

The change from maxDegree > 1 to maxDegree >= 1 broadens the acceptable range for maxDegree. Consider whether this change aligns with the intended test validation criteria and expected behavior of the graph's connectivity.

Verification successful

Assertion Logic Change Isolated to Base Test Class

The change in assertion logic from maxDegree > 1 to maxDegree >= 1 is isolated to the GraphServiceTestBase.java file. The other references to testHighlyConnectedGraphWalk in DgraphGraphServiceTest.java and Neo4jGraphServiceTest.java are commented out, indicating they are not currently active. Thus, the change does not impact other parts of the codebase or tests.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the impact of the assertion logic change in the `testHighlyConnectedGraphWalk` method.

# Test: Search for references to `testHighlyConnectedGraphWalk`. Expect: No breaking changes in related tests or documentation.
rg --type java -A 5 $'testHighlyConnectedGraphWalk'

Length of output: 1821

Comment on lines +137 to +141
private List<SortCriterion> buildTaskRunsSortCriteria() {
final SortCriterion sortCriterion = new SortCriterion();
sortCriterion.setField(CREATED_TIME_SEARCH_INDEX_FIELD_NAME);
sortCriterion.setOrder(SortOrder.DESCENDING);
return sortCriterion;
return Collections.singletonList(sortCriterion);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor suggestion: Simplify sort criteria construction.

Consider refactoring the buildTaskRunsSortCriteria method to allow for future extensions where multiple criteria might be added. This could involve parameterizing the method to accept additional criteria.

private List<SortCriterion> buildTaskRunsSortCriteria(SortCriterion... additionalCriteria) {
    List<SortCriterion> criteria = new ArrayList<>();
    criteria.add(new SortCriterion()
        .setField(CREATED_TIME_SEARCH_INDEX_FIELD_NAME)
        .setOrder(SortOrder.DESCENDING));
    Collections.addAll(criteria, additionalCriteria);
    return criteria;
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)
metadata-service/restli-client/src/main/java/com/linkedin/entity/client/RestliEntityClient.java (1)

652-652: The searchAcrossEntities method only uses the first sort criterion.

The implementation of the searchAcrossEntities method in RestliEntityClient currently uses only the first element of the sortCriteria list for sorting. This may not fully utilize the list of sort criteria as intended.

  • Consider updating the logic to handle multiple sort criteria if that is the intended functionality.
  • Ensure the method documentation reflects the current behavior of using only the first criterion.
Analysis chain

Verify the handling of multiple sort criteria.

The searchAcrossEntities method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify usage of `searchAcrossEntities` method to ensure it handles multiple sort criteria.

# Test: Search for the method usage. Expect: Correct handling of sortCriteria list.
rg --type java -A 5 $'searchAcrossEntities(.*List<SortCriterion> sortCriteria)'

Length of output: 80


Script:

#!/bin/bash
# Description: Locate the definition of `searchAcrossEntities` and examine its implementation.

# Search for the method definition in the codebase to understand how it handles the sortCriteria list.
ast-grep --lang java --pattern $'class $_ {
  $$$
  searchAcrossEntities($_, $_, $_, $_, $_, $_, $_, List<SortCriterion> sortCriteria) {
    $$$
  }
  $$$
}'

Length of output: 158


Script:

#!/bin/bash
# Description: Search for the `searchAcrossEntities` method definition and references in the codebase.

# Search for the method definition in all Java files to understand its implementation.
rg --type java -A 10 'searchAcrossEntities'

Length of output: 163796


Script:

#!/bin/bash
# Description: Extract the implementation of `searchAcrossEntities` in `RestliEntityClient`.

# Locate and display the implementation of the method to verify handling of sortCriteria.
rg --type java -A 30 'public SearchResult searchAcrossEntities\(' metadata-service/restli-client/src/main/java/com/linkedin/entity/client/RestliEntityClient.java

Length of output: 2141

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 378a325 and c2ff037.

Files selected for processing (4)
  • metadata-io/src/main/java/com/linkedin/metadata/client/JavaEntityClient.java (15 hunks)
  • metadata-service/restli-api/src/main/snapshot/com.linkedin.entity.entities.snapshot.json (7 hunks)
  • metadata-service/restli-client-api/src/main/java/com/linkedin/entity/client/EntityClient.java (10 hunks)
  • metadata-service/restli-client/src/main/java/com/linkedin/entity/client/RestliEntityClient.java (14 hunks)
Files skipped from review as they are similar to previous changes (1)
  • metadata-io/src/main/java/com/linkedin/metadata/client/JavaEntityClient.java
Additional comments not posted (21)
metadata-service/restli-client-api/src/main/java/com/linkedin/entity/client/EntityClient.java (9)

249-249: Verify the handling of multiple sort criteria.

The search method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


238-238: Verify the handling of multiple sort criteria.

The list method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


331-331: Verify the handling of multiple sort criteria.

The scrollAcrossEntities method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


359-359: Verify the handling of multiple sort criteria.

The searchAcrossLineage method with scrollId now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


439-439: Verify the handling of multiple sort criteria.

The filter method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


429-429: Verify the handling of multiple sort criteria.

The deleteEntityReferences method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


345-345: Verify the handling of multiple sort criteria.

The searchAcrossLineage method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


273-273: Verify the handling of multiple sort criteria.

The searchAcrossEntities method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


295-295: Verify the handling of multiple sort criteria.

The searchAcrossEntities method with facets now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.

metadata-service/restli-client/src/main/java/com/linkedin/entity/client/RestliEntityClient.java (5)

610-610: Verify the handling of multiple sort criteria.

The search method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


925-925: Verify the handling of multiple sort criteria.

The filter method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


597-597: Verify the handling of multiple sort criteria.

The list method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


753-753: Verify the handling of multiple sort criteria.

The searchAcrossLineage method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.


801-801: Verify the handling of multiple sort criteria.

The scrollAcrossLineage method now accepts a list of SortCriterion. Ensure the implementation correctly handles multiple criteria and updates any related logic.

metadata-service/restli-api/src/main/snapshot/com.linkedin.entity.entities.snapshot.json (7)

6824-6826: Addition of sortCriteria is consistent and correct.

The sortCriteria parameter is correctly added as an optional array of SortCriterion objects, complementing the existing sort parameter for multi-field sorting.


6877-6879: Addition of sortCriteria is consistent and correct.

The sortCriteria parameter is correctly added as an optional array of SortCriterion objects, aligning with the existing sort parameter for enhanced sorting capabilities.


6921-6923: Addition of sortCriteria is consistent and correct.

The sortCriteria parameter is correctly added as an optional array of SortCriterion objects, complementing the existing sort parameter for multi-field sorting.


6970-6972: Addition of sortCriteria is consistent and correct.

The sortCriteria parameter is correctly added as an optional array of SortCriterion objects, aligning with the existing sort parameter for enhanced sorting capabilities.


7015-7017: Addition of sortCriteria is consistent and correct.

The sortCriteria parameter is correctly added as an optional array of SortCriterion objects, complementing the existing sort parameter for multi-field sorting.


7056-7058: Addition of sortCriteria is consistent and correct.

The sortCriteria parameter is correctly added as an optional array of SortCriterion objects, aligning with the existing sort parameter for enhanced sorting capabilities.


7101-7103: Addition of sortCriteria is consistent and correct.

The sortCriteria parameter is correctly added as an optional array of SortCriterion objects, complementing the existing sort parameter for multi-field sorting.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between c2ff037 and 22567fe.

Files selected for processing (1)
  • metadata-service/openapi-servlet/src/test/java/io/datahubproject/openapi/v3/controller/EntityControllerTest.java (3 hunks)
Additional comments not posted (2)
metadata-service/openapi-servlet/src/test/java/io/datahubproject/openapi/v3/controller/EntityControllerTest.java (2)

99-99: LGTM: Use of Collections.singletonList for sorting criteria.

The change to encapsulate the sorting criteria within Collections.singletonList enhances clarity and aligns with the new feature supporting multiple sorting fields.


117-119: LGTM: Consistent use of Collections.singletonList for sorting criteria.

The update for descending order sorting criteria maintains consistency with the ascending order change, improving the test's structure.

@RyanHolstien RyanHolstien merged commit 06562f3 into datahub-project:master Aug 9, 2024
40 of 41 checks passed
@RyanHolstien RyanHolstien deleted the feat/supportMultiSort branch August 9, 2024 20:29
arosanda added a commit to infobip/datahub that referenced this pull request Sep 23, 2024
* feat(forms) Handle deleting forms references when hard deleting forms (datahub-project#10820)

* refactor(ui): Misc improvements to the setup ingestion flow (ingest uplift 1/2)  (datahub-project#10764)

Co-authored-by: John Joyce <john@Johns-MBP.lan>
Co-authored-by: John Joyce <john@ip-192-168-1-200.us-west-2.compute.internal>

* fix(ingestion/airflow-plugin): pipeline tasks discoverable in search (datahub-project#10819)

* feat(ingest/transformer): tags to terms transformer (datahub-project#10758)

Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>

* fix(ingestion/unity-catalog): fixed issue with profiling with GE turned on (datahub-project#10752)

Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>

* feat(forms) Add java SDK for form entity PATCH + CRUD examples (datahub-project#10822)

* feat(SDK) Add java SDK for structuredProperty entity PATCH + CRUD examples (datahub-project#10823)

* feat(SDK) Add StructuredPropertyPatchBuilder in python sdk and provide sample CRUD files (datahub-project#10824)

* feat(forms) Add CRUD endpoints to GraphQL for Form entities (datahub-project#10825)

* add flag for includeSoftDeleted in scroll entities API (datahub-project#10831)

* feat(deprecation) Return actor entity with deprecation aspect (datahub-project#10832)

* feat(structuredProperties) Add CRUD graphql APIs for structured property entities (datahub-project#10826)

* add scroll parameters to openapi v3 spec (datahub-project#10833)

* fix(ingest): correct profile_day_of_week implementation (datahub-project#10818)

* feat(ingest/glue): allow ingestion of empty databases from Glue (datahub-project#10666)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* feat(cli): add more details to get cli (datahub-project#10815)

* fix(ingestion/glue): ensure date formatting works on all platforms for aws glue (datahub-project#10836)

* fix(ingestion): fix datajob patcher (datahub-project#10827)

* fix(smoke-test): add suffix in temp file creation (datahub-project#10841)

* feat(ingest/glue): add helper method to permit user or group ownership (datahub-project#10784)

* feat(): Show data platform instances in policy modal if they are set on the policy (datahub-project#10645)

Co-authored-by: Hendrik Richert <hendrik.richert@swisscom.com>

* docs(patch): add patch documentation for how implementation works (datahub-project#10010)

Co-authored-by: John Joyce <john@acryl.io>

* fix(jar): add missing custom-plugin-jar task (datahub-project#10847)

* fix(): also check exceptions/stack trace when filtering log messages (datahub-project#10391)

Co-authored-by: John Joyce <john@acryl.io>

* docs(): Update posts.md (datahub-project#9893)

Co-authored-by: Hyejin Yoon <0327jane@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* chore(ingest): update acryl-datahub-classify version (datahub-project#10844)

* refactor(ingest): Refactor structured logging to support infos, warnings, and failures structured reporting to UI (datahub-project#10828)

Co-authored-by: John Joyce <john@Johns-MBP.lan>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* fix(restli): log aspect-not-found as a warning rather than as an error (datahub-project#10834)

* fix(ingest/nifi): remove duplicate upstream jobs (datahub-project#10849)

* fix(smoke-test): test access to create/revoke personal access tokens (datahub-project#10848)

* fix(smoke-test): missing test for move domain (datahub-project#10837)

* ci: update usernames to not considered for community (datahub-project#10851)

* env: change defaults for data contract visibility (datahub-project#10854)

* fix(ingest/tableau): quote special characters in external URL (datahub-project#10842)

* fix(smoke-test): fix flakiness of auto complete test

* ci(ingest): pin dask dependency for feast (datahub-project#10865)

* fix(ingestion/lookml): liquid template resolution and view-to-view cll (datahub-project#10542)

* feat(ingest/audit): add client id and version in system metadata props (datahub-project#10829)

* chore(ingest): Mypy 1.10.1 pin (datahub-project#10867)

* docs: use acryl-datahub-actions as expected python package to install (datahub-project#10852)

* docs: add new js snippet (datahub-project#10846)

* refactor(ingestion): remove company domain for security reason (datahub-project#10839)

* fix(ingestion/spark): Platform instance and column level lineage fix (datahub-project#10843)

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat(ingestion/tableau): optionally ingest multiple sites and create site containers (datahub-project#10498)

Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>

* fix(ingestion/looker): Add sqlglot dependency and remove unused sqlparser (datahub-project#10874)

* fix(manage-tokens): fix manage access token policy (datahub-project#10853)

* Batch get entity endpoints (datahub-project#10880)

* feat(system): support conditional write semantics (datahub-project#10868)

* fix(build): upgrade vercel builds to Node 20.x (datahub-project#10890)

* feat(ingest/lookml): shallow clone repos (datahub-project#10888)

* fix(ingest/looker): add missing dependency (datahub-project#10876)

* fix(ingest): only populate audit stamps where accurate (datahub-project#10604)

* fix(ingest/dbt): always encode tag urns (datahub-project#10799)

* fix(ingest/redshift): handle multiline alter table commands (datahub-project#10727)

* fix(ingestion/looker): column name missing in explore (datahub-project#10892)

* fix(lineage) Fix lineage source/dest filtering with explored per hop limit (datahub-project#10879)

* feat(conditional-writes): misc updates and fixes (datahub-project#10901)

* feat(ci): update outdated action (datahub-project#10899)

* feat(rest-emitter): adding async flag to rest emitter (datahub-project#10902)

Co-authored-by: Gabe Lyons <gabe.lyons@acryl.io>

* feat(ingest): add snowflake-queries source (datahub-project#10835)

* fix(ingest): improve `auto_materialize_referenced_tags_terms` error handling (datahub-project#10906)

* docs: add new company to adoption list (datahub-project#10909)

* refactor(redshift): Improve redshift error handling with new structured reporting system (datahub-project#10870)

Co-authored-by: John Joyce <john@Johns-MBP.lan>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* feat(ui) Finalize support for all entity types on forms (datahub-project#10915)

* Index ExecutionRequestResults status field (datahub-project#10811)

* feat(ingest): grafana connector (datahub-project#10891)

Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* fix(gms) Add Form entity type to EntityTypeMapper (datahub-project#10916)

* feat(dataset): add support for external url in Dataset (datahub-project#10877)

* docs(saas-overview) added missing features to observe section (datahub-project#10913)

Co-authored-by: John Joyce <john@acryl.io>

* fix(ingest/spark): Fixing Micrometer warning (datahub-project#10882)

* fix(structured properties): allow application of structured properties without schema file (datahub-project#10918)

* fix(data-contracts-web) handle other schedule types (datahub-project#10919)

* fix(ingestion/tableau): human-readable message for PERMISSIONS_MODE_SWITCHED error (datahub-project#10866)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* Add feature flag for view defintions (datahub-project#10914)

Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>

* feat(ingest/BigQuery): refactor+parallelize dataset metadata extraction (datahub-project#10884)

* fix(airflow): add error handling around render_template() (datahub-project#10907)

* feat(ingestion/sqlglot): add optional `default_dialect` parameter to sqlglot lineage (datahub-project#10830)

* feat(mcp-mutator): new mcp mutator plugin (datahub-project#10904)

* fix(ingest/bigquery): changes helper function to decode unicode scape sequences (datahub-project#10845)

* feat(ingest/postgres): fetch table sizes for profile (datahub-project#10864)

* feat(ingest/abs): Adding azure blob storage ingestion source (datahub-project#10813)

* fix(ingest/redshift): reduce severity of SQL parsing issues (datahub-project#10924)

* fix(build): fix lint fix web react (datahub-project#10896)

* fix(ingest/bigquery): handle quota exceeded for project.list requests (datahub-project#10912)

* feat(ingest): report extractor failures more loudly (datahub-project#10908)

* feat(ingest/snowflake): integrate snowflake-queries into main source (datahub-project#10905)

* fix(ingest): fix docs build (datahub-project#10926)

* fix(ingest/snowflake): fix test connection (datahub-project#10927)

* fix(ingest/lookml): add view load failures to cache (datahub-project#10923)

* docs(slack) overhauled setup instructions and screenshots (datahub-project#10922)

Co-authored-by: John Joyce <john@acryl.io>

* fix(airflow): Add comma parsing of owners to DataJobs (datahub-project#10903)

* fix(entityservice): fix merging sideeffects (datahub-project#10937)

* feat(ingest): Support System Ingestion Sources, Show and hide system ingestion sources with Command-S (datahub-project#10938)

Co-authored-by: John Joyce <john@Johns-MBP.lan>

* chore() Set a default lineage filtering end time on backend when a start time is present (datahub-project#10925)

Co-authored-by: John Joyce <john@ip-192-168-1-200.us-west-2.compute.internal>
Co-authored-by: John Joyce <john@Johns-MBP.lan>

* Added relationships APIs to V3. Added these generic APIs to V3 swagger doc. (datahub-project#10939)

* docs: add learning center to docs (datahub-project#10921)

* doc: Update hubspot form id (datahub-project#10943)

* chore(airflow): add python 3.11 w/ Airflow 2.9 to CI (datahub-project#10941)

* fix(ingest/Glue): column upstream lineage between S3 and Glue (datahub-project#10895)

* fix(ingest/abs): split abs utils into multiple files (datahub-project#10945)

* doc(ingest/looker): fix doc for sql parsing documentation (datahub-project#10883)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* fix(ingest/bigquery): Adding missing BigQuery types (datahub-project#10950)

* fix(ingest/setup): feast and abs source setup (datahub-project#10951)

* fix(connections) Harden adding /gms to connections in backend (datahub-project#10942)

* feat(siblings) Add flag to prevent combining siblings in the UI (datahub-project#10952)

* fix(docs): make graphql doc gen more automated (datahub-project#10953)

* feat(ingest/athena): Add option for Athena partitioned profiling (datahub-project#10723)

* fix(spark-lineage): default timeout for future responses (datahub-project#10947)

* feat(datajob/flow): add environment filter using info aspects (datahub-project#10814)

* fix(ui/ingest): correct privilege used to show tab (datahub-project#10483)

Co-authored-by: Kunal-kankriya <127090035+Kunal-kankriya@users.noreply.github.com>

* feat(ingest/looker): include dashboard urns in browse v2 (datahub-project#10955)

* add a structured type to batchGet in OpenAPI V3 spec (datahub-project#10956)

* fix(ui): scroll on the domain sidebar to show all domains (datahub-project#10966)

* fix(ingest/sagemaker): resolve incorrect variable assignment for SageMaker API call (datahub-project#10965)

* fix(airflow/build): Pinning mypy (datahub-project#10972)

* Fixed a bug where the OpenAPI V3 spec was incorrect. The bug was introduced in datahub-project#10939. (datahub-project#10974)

* fix(ingest/test): Fix for mssql integration tests (datahub-project#10978)

* fix(entity-service) exist check correctly extracts status (datahub-project#10973)

* fix(structuredProps) casing bug in StructuredPropertiesValidator (datahub-project#10982)

* bugfix: use anyOf instead of allOf when creating references in openapi v3 spec (datahub-project#10986)

* fix(ui): Remove ant less imports (datahub-project#10988)

* feat(ingest/graph): Add get_results_by_filter to DataHubGraph (datahub-project#10987)

* feat(ingest/cli): init does not actually support environment variables (datahub-project#10989)

* fix(ingest/graph): Update get_results_by_filter graphql query (datahub-project#10991)

* feat(ingest/spark): Promote beta plugin (datahub-project#10881)

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat(ingest): support domains in meta -> "datahub" section (datahub-project#10967)

* feat(ingest): add `check server-config` command (datahub-project#10990)

* feat(cli): Make consistent use of DataHubGraphClientConfig (datahub-project#10466)

Deprecates get_url_and_token() in favor of a more complete option: load_graph_config() that returns a full DatahubClientConfig.
This change was then propagated across previous usages of get_url_and_token so that connections to DataHub server from the client respect the full breadth of configuration specified by DatahubClientConfig.

I.e: You can now specify disable_ssl_verification: true in your ~/.datahubenv file so that all cli functions to the server work when ssl certification is disabled.

Fixes datahub-project#9705

* fix(ingest/s3): Fixing container creation when there is no folder in path (datahub-project#10993)

* fix(ingest/looker): support platform instance for dashboards & charts (datahub-project#10771)

* feat(ingest/bigquery): improve handling of information schema in sql parser (datahub-project#10985)

* feat(ingest): improve `ingest deploy` command (datahub-project#10944)

* fix(backend): allow excluding soft-deleted entities in relationship-queries; exclude soft-deleted members of groups (datahub-project#10920)

- allow excluding soft-deleted entities in relationship-queries
- exclude soft-deleted members of groups

* fix(ingest/looker): downgrade missing chart type log level (datahub-project#10996)

* doc(acryl-cloud): release docs for 0.3.4.x (datahub-project#10984)

Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>

* fix(protobuf/build): Fix protobuf check jar script (datahub-project#11006)

* fix(ui/ingest): Support invalid cron jobs (datahub-project#10998)

* fix(ingest): fix graph config loading (datahub-project#11002)

Co-authored-by: Pedro Silva <pedro@acryl.io>

* feat(docs): Document __DATAHUB_TO_FILE_ directive (datahub-project#10968)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* fix(graphql/upsertIngestionSource): Validate cron schedule; parse error in CLI (datahub-project#11011)

* feat(ece): support custom ownership type urns in ECE generation (datahub-project#10999)

* feat(assertion-v2): changed Validation tab to Quality and created new Governance tab (datahub-project#10935)

* fix(ingestion/glue): Add support for missing config options for profiling in Glue (datahub-project#10858)

* feat(propagation): Add models for schema field docs, tags, terms (datahub-project#2959) (datahub-project#11016)

Co-authored-by: Chris Collins <chriscollins3456@gmail.com>

* docs: standardize terminology to DataHub Cloud (datahub-project#11003)

* fix(ingestion/transformer): replace the externalUrl container (datahub-project#11013)

* docs(slack) troubleshoot docs (datahub-project#11014)

* feat(propagation): Add graphql API (datahub-project#11030)

Co-authored-by: Chris Collins <chriscollins3456@gmail.com>

* feat(propagation):  Add models for Action feature settings (datahub-project#11029)

* docs(custom properties): Remove duplicate from sidebar (datahub-project#11033)

* feat(models): Introducing Dataset Partitions Aspect (datahub-project#10997)

Co-authored-by: John Joyce <john@Johns-MBP.lan>
Co-authored-by: John Joyce <john@ip-192-168-1-200.us-west-2.compute.internal>

* feat(propagation): Add Documentation Propagation Settings (datahub-project#11038)

* fix(models): chart schema fields mapping, add dataHubAction entity, t… (datahub-project#11040)

* fix(ci): smoke test lint failures (datahub-project#11044)

* docs: fix learning center color scheme & typo (datahub-project#11043)

* feat: add cloud main page (datahub-project#11017)

Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>

* feat(restore-indices): add additional step to also clear system metadata service (datahub-project#10662)

Co-authored-by: John Joyce <john@acryl.io>

* docs: fix typo (datahub-project#11046)

* fix(lint): apply spotless (datahub-project#11050)

* docs(airflow): example query to get datajobs for a dataflow (datahub-project#11034)

* feat(cli): Add run-id option to put sub-command (datahub-project#11023)

Adds an option to assign run-id to a given put command execution. 
This is useful when transformers do not exist for a given ingestion payload, we can follow up with custom metadata and assign it to an ingestion pipeline.

* fix(ingest): improve sql error reporting calls (datahub-project#11025)

* fix(airflow): fix CI setup (datahub-project#11031)

* feat(ingest/dbt): add experimental `prefer_sql_parser_lineage` flag (datahub-project#11039)

* fix(ingestion/lookml): enable stack-trace in lookml logs (datahub-project#10971)

* (chore): Linting fix (datahub-project#11015)

* chore(ci): update deprecated github actions (datahub-project#10977)

* Fix ALB configuration example (datahub-project#10981)

* chore(ingestion-base): bump base image packages (datahub-project#11053)

* feat(cli): Trim report of dataHubExecutionRequestResult to max GMS size (datahub-project#11051)

* fix(ingestion/lookml): emit dummy sql condition for lookml custom condition tag (datahub-project#11008)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* fix(ingestion/powerbi): fix issue with broken report lineage (datahub-project#10910)

* feat(ingest/tableau): add retry on timeout (datahub-project#10995)

* change generate kafka connect properties from env (datahub-project#10545)

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* fix(ingest): fix oracle cronjob ingestion (datahub-project#11001)

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* chore(ci): revert update deprecated github actions (datahub-project#10977) (datahub-project#11062)

* feat(ingest/dbt-cloud): update metadata_endpoint inference (datahub-project#11041)

* build: Reduce size of datahub-frontend-react image by 50-ish% (datahub-project#10878)

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* fix(ci): Fix lint issue in datahub_ingestion_run_summary_provider.py (datahub-project#11063)

* docs(ingest): update developing-a-transformer.md (datahub-project#11019)

* feat(search-test): update search tests from datahub-project#10408 (datahub-project#11056)

* feat(cli): add aspects parameter to DataHubGraph.get_entity_semityped (datahub-project#11009)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* docs(airflow): update min version for plugin v2 (datahub-project#11065)

* doc(ingestion/tableau): doc update for derived permission (datahub-project#11054)

Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* fix(py): remove dep on types-pkg_resources (datahub-project#11076)

* feat(ingest/mode): add option to exclude restricted (datahub-project#11081)

* fix(ingest): set lastObserved in sdk when unset (datahub-project#11071)

* doc(ingest): Update capabilities (datahub-project#11072)

* chore(vulnerability): Log Injection (datahub-project#11090)

* chore(vulnerability): Information exposure through a stack trace (datahub-project#11091)

* chore(vulnerability): Comparison of narrow type with wide type in loop condition (datahub-project#11089)

* chore(vulnerability): Insertion of sensitive information into log files (datahub-project#11088)

* chore(vulnerability): Risky Cryptographic Algorithm (datahub-project#11059)

* chore(vulnerability): Overly permissive regex range (datahub-project#11061)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* fix: update customer data (datahub-project#11075)

* fix(models): fixing the datasetPartition models (datahub-project#11085)

Co-authored-by: John Joyce <john@ip-192-168-1-200.us-west-2.compute.internal>

* fix(ui): Adding view, forms GraphQL query, remove showing a fallback error message on unhandled GraphQL error (datahub-project#11084)

Co-authored-by: John Joyce <john@ip-192-168-1-200.us-west-2.compute.internal>

* feat(docs-site): hiding learn more from cloud page (datahub-project#11097)

* fix(docs): Add correct usage of orFilters in search API docs (datahub-project#11082)

Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>

* fix(ingest/mode): Regexp in mode name matcher didn't allow underscore (datahub-project#11098)

* docs: Refactor customer stories section (datahub-project#10869)

Co-authored-by: Jeff Merrick <jeff@wireform.io>

* fix(release): fix full/slim suffix on tag (datahub-project#11087)

* feat(config): support alternate hashing algorithm for doc id (datahub-project#10423)

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>

* fix(emitter): fix typo in get method of java kafka emitter (datahub-project#11007)

* fix(ingest): use correct native data type in all SQLAlchemy sources by compiling data type using dialect (datahub-project#10898)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* chore: Update contributors list in PR labeler (datahub-project#11105)

* feat(ingest): tweak stale entity removal messaging (datahub-project#11064)

* fix(ingestion): enforce lastObserved timestamps in SystemMetadata (datahub-project#11104)

* fix(ingest/powerbi): fix broken lineage between chart and dataset (datahub-project#11080)

* feat(ingest/lookml): CLL support for sql set in sql_table_name attribute of lookml view (datahub-project#11069)

* docs: update graphql docs on forms & structured properties (datahub-project#11100)

* test(search): search openAPI v3 test (datahub-project#11049)

* fix(ingest/tableau): prevent empty site content urls (datahub-project#11057)

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat(entity-client): implement client batch interface (datahub-project#11106)

* fix(snowflake): avoid reporting warnings/info for sys tables (datahub-project#11114)

* fix(ingest): downgrade column type mapping warning to info (datahub-project#11115)

* feat(api): add AuditStamp to the V3 API entity/aspect response (datahub-project#11118)

* fix(ingest/redshift): replace r'\n' with '\n' to avoid token error redshift serverless… (datahub-project#11111)

* fix(entiy-client): handle null entityUrn case for restli (datahub-project#11122)

* fix(sql-parser): prevent bad urns from alter table lineage (datahub-project#11092)

* fix(ingest/bigquery): use small batch size if use_tables_list_query_v2 is set (datahub-project#11121)

* fix(graphql): add missing entities to EntityTypeMapper and EntityTypeUrnMapper (datahub-project#10366)

* feat(ui): Changes to allow editable dataset name (datahub-project#10608)

Co-authored-by: Jay Kadambi <jayasimhan_venkatadri@optum.com>

* fix: remove saxo (datahub-project#11127)

* feat(mcl-processor): Update mcl processor hooks (datahub-project#11134)

* fix(openapi): fix openapi v2 endpoints & v3 documentation update

* Revert "fix(openapi): fix openapi v2 endpoints & v3 documentation update"

This reverts commit 573c1cb.

* docs(policies): updates to policies documentation (datahub-project#11073)

* fix(openapi): fix openapi v2 and v3 docs update (datahub-project#11139)

* feat(auth): grant type and acr values custom oidc parameters support (datahub-project#11116)

* fix(mutator): mutator hook fixes (datahub-project#11140)

* feat(search): support sorting on multiple fields (datahub-project#10775)

* feat(ingest): various logging improvements (datahub-project#11126)

* fix(ingestion/lookml): fix for sql parsing error (datahub-project#11079)

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>

* feat(docs-site) cloud page spacing and content polishes (datahub-project#11141)

* feat(ui) Enable editing structured props on fields (datahub-project#11042)

* feat(tests): add md5 and last computed to testResult model (datahub-project#11117)

* test(openapi): openapi regression smoke tests (datahub-project#11143)

* fix(airflow): fix tox tests + update docs (datahub-project#11125)

* docs: add chime to adoption stories (datahub-project#11142)

* fix(ingest/databricks): Updating code to work with Databricks sdk 0.30 (datahub-project#11158)

* fix(kafka-setup): add missing script to image (datahub-project#11190)

* fix(config): fix hash algo config (datahub-project#11191)

* test(smoke-test): updates to smoke-tests (datahub-project#11152)

* fix(elasticsearch): refactor idHashAlgo setting (datahub-project#11193)

* chore(kafka): kafka version bump (datahub-project#11211)

* readd UsageStatsWorkUnit

* fix merge problems

* change logo

---------

Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: John Joyce <john@Johns-MBP.lan>
Co-authored-by: John Joyce <john@ip-192-168-1-200.us-west-2.compute.internal>
Co-authored-by: dushayntAW <158567391+dushayntAW@users.noreply.github.com>
Co-authored-by: sagar-salvi-apptware <159135491+sagar-salvi-apptware@users.noreply.github.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Kevin Chun <kevin1chun@gmail.com>
Co-authored-by: jordanjeremy <72943478+jordanjeremy@users.noreply.github.com>
Co-authored-by: skrydal <piotr.skrydalewicz@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: sid-acryl <155424659+sid-acryl@users.noreply.github.com>
Co-authored-by: Julien Jehannet <80408664+aviv-julienjehannet@users.noreply.github.com>
Co-authored-by: Hendrik Richert <github@richert.li>
Co-authored-by: Hendrik Richert <hendrik.richert@swisscom.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: Felix Lüdin <13187726+Masterchen09@users.noreply.github.com>
Co-authored-by: Pirry <158024088+chardaway@users.noreply.github.com>
Co-authored-by: Hyejin Yoon <0327jane@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: cburroughs <chris.burroughs@gmail.com>
Co-authored-by: ksrinath <ksrinath@users.noreply.github.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
Co-authored-by: Kunal-kankriya <127090035+Kunal-kankriya@users.noreply.github.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: ipolding-cais <155455744+ipolding-cais@users.noreply.github.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Shubham Jagtap <132359390+shubhamjagtap639@users.noreply.github.com>
Co-authored-by: haeniya <yanik.haeni@gmail.com>
Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Gabe Lyons <gabe.lyons@acryl.io>
Co-authored-by: 808OVADOZE <52988741+shtephlee@users.noreply.github.com>
Co-authored-by: noggi <anton.kuraev@acryl.io>
Co-authored-by: Nicholas Pena <npena@foursquare.com>
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
Co-authored-by: ethan-cartwright <ethan.cartwright.m@gmail.com>
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
Co-authored-by: Nadav Gross <33874964+nadavgross@users.noreply.github.com>
Co-authored-by: Patrick Franco Braz <patrickfbraz@poli.ufrj.br>
Co-authored-by: pie1nthesky <39328908+pie1nthesky@users.noreply.github.com>
Co-authored-by: Joel Pinto Mata (KPN-DSH-DEX team) <130968841+joelmataKPN@users.noreply.github.com>
Co-authored-by: Ellie O'Neil <110510035+eboneil@users.noreply.github.com>
Co-authored-by: Ajoy Majumdar <ajoymajumdar@hotmail.com>
Co-authored-by: deepgarg-visa <149145061+deepgarg-visa@users.noreply.github.com>
Co-authored-by: Tristan Heisler <tristankheisler@gmail.com>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
Co-authored-by: Davi Arnaut <davi.arnaut@acryl.io>
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: amit-apptware <132869468+amit-apptware@users.noreply.github.com>
Co-authored-by: Sam Black <sam.black@acryl.io>
Co-authored-by: Raj Tekal <varadaraj_tekal@optum.com>
Co-authored-by: Steffen Grohsschmiedt <gitbhub@steffeng.eu>
Co-authored-by: jaegwon.seo <162448493+wornjs@users.noreply.github.com>
Co-authored-by: Renan F. Lima <51028757+lima-renan@users.noreply.github.com>
Co-authored-by: Matt Exchange <xkollar@users.noreply.github.com>
Co-authored-by: Jonny Dixon <45681293+acrylJonny@users.noreply.github.com>
Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>
Co-authored-by: Pinaki Bhattacharjee <pinakipb2@gmail.com>
Co-authored-by: Jeff Merrick <jeff@wireform.io>
Co-authored-by: skrydal <piotr.skrydalewicz@acryl.io>
Co-authored-by: AndreasHegerNuritas <163423418+AndreasHegerNuritas@users.noreply.github.com>
Co-authored-by: jayasimhankv <145704974+jayasimhankv@users.noreply.github.com>
Co-authored-by: Jay Kadambi <jayasimhan_venkatadri@optum.com>
Co-authored-by: David Leifker <david.leifker@acryl.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops PR or Issue related to DataHub backend & deployment product PR or Issue related to the DataHub UI/UX release-0.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants