Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'change_feed_mode' to 'query_items_change_feed' API #38105

Conversation

allenkim0129
Copy link
Contributor

@allenkim0129 allenkim0129 commented Oct 25, 2024

Description

In this PR, we added support for change_feed_mode for query_items_change_feed API. For the change_feed_mode argument, users can use either string values(LatestVersion or AllVersionsAndDeletes) or Enum values(ChangeFeedMode.LATEST_VERSIONor ChangeFeedMode.ALL_VERSIONS_AND_DELETES). The Enum can be imported from change_feed_state.py like the following:

from azure.cosmos._change_feed.change_feed_state import ChangeFeedMode

...
create or set "created_container"
... 

# query change feed with 'ALL_VERSIONS_AND_DELETES' mode
query_iterable = created_container.query_items_change_feed(
    change_feed_mode=ChangeFeedMode.ALL_VERSIONS_AND_DELETES,
)

Currently the default value for the change_feed_mode is LATEST_VERSION mode. If the change_feed_mode argument were not passed, query_items_change_feed will use LATEST_VERSION by default. This mode was the only mode supported before we added the change_feed_mdoe argument. Now users can choose either option by passing change_feed_mode

ChangeFeedMode.LATEST_VERSION

With this mode, we return all items of the latest/current version from a container. Even if some items were updated after creation, it only returns the latest version of all created items from a specified point of time, which could be passed by the following arguments. (By default, the point in time will be Now):

  • is_start_from_beginning: Query all items created from the beginning of container creation. (is_start_from_beginning and start_time are exclusive, and cannot be used together.)
  • start_time: Query items created from the start time. The start time can be [datetime, 'Now', or 'Beginning']. (is_start_from_beginning and start_time are exclusive, and cannot be used together.)
  • continuation: Query all items created from a continuation token, which was stored from previous query run.

example:

# query change feed from beginning
query_iterable = created_container.query_items_change_feed(
    is_start_from_beginning=True,
    change_feed_mode=ChangeFeedMode.LATEST_VERSION,
)

# query change feed from Now
query_iterable = created_container.query_items_change_feed(
    start_time="Now",
    change_feed_mode=ChangeFeedMode.LATEST_VERSION,
)

# query change feed from a continuation token
query_iterable = created_container.query_items_change_feed(
    continuation=continuation_token_1,
    change_feed_mode=ChangeFeedMode.LATEST_VERSION,
)

ChangeFeedMode.ALL_VERSIONS_AND_DELETES

With this option, we return all items of all versions including creates, updates, deletes. If an item was deleted , the deleted items will be included, and the metadata will specify that the item was deleted. Currently, we backend server only supports continuation token to specify the beginning time. If is_start_from_beginning or start_time were used, exceptions will be risen with error message. If continuation token was not passed, the point in time will be set to Now.

example:

# query change feed from a continuation token
query_iterable = created_container.query_items_change_feed(
    continuation=continuation_token_1,
    change_feed_mode=ChangeFeedMode.ALL_VERSIONS_AND_DELETES,
)

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@allenkim0129 allenkim0129 marked this pull request as ready for review October 28, 2024 18:26
@allenkim0129 allenkim0129 requested review from annatisch and a team as code owners October 28, 2024 18:26
@azure-sdk
Copy link
Collaborator

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-cosmos

@jalauzon-msft jalauzon-msft removed their request for review November 22, 2024 19:59
Copy link
Member

@FabianMeiswinkel FabianMeiswinkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me except for the comment in ChangeFeedStateV2

Copy link
Member

@FabianMeiswinkel FabianMeiswinkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM-Thanks

@FabianMeiswinkel FabianMeiswinkel merged commit 5218e0f into Azure:main Nov 23, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.