Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't explicitly create a collection #932

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ndepal
Copy link

@ndepal ndepal commented Dec 23, 2021

Problem

We have various mongo servers that we sync with a central server using mongo-connector with the mongo_doc_manager.
They all sync into the same collection on the central server.
When a new system is initialized, it has nothing in the DB and mongo-connector is running.
When the first entry is made into the local DB, there is an oplog entry to create the collection. When mongo-connector tries to replay that command on the target DB, pymongo throws an exception because the collection already exists on the central server.

Note: this problem does not present itself if mongo-connector is started (without an existing oplog.timestamp) after the first entry in the local collection has already been made.

Solution

This problem is easily avoided by not trying to explicitly create the collection, effectively ignoring the create entry in the oplog.
This does not cause any problems because mongo creates collections automatically whenever a document is inserted into a collection that does not yet exist. The only reason to explicitly create a collection is if special options are specified as per the documentation:

Normally collection creation is automatic. This method should only be used to specify options on creation. CollectionInvalid will be raised if the collection already exists.

Since mongo_doc_manager does not specify any options in the create_collection() call, that call should not be made.

…y when a document is inserted. Trying to explicity create a collection that exists causes an exception in pymongo.
Copy link

@Imran-imtiaz48 Imran-imtiaz48 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review:
The code snippet efficiently handles MongoDB operations by performing conditional checks to rename, create, or drop collections based on the provided inputs. The renameCollection command is appropriately used to rename collections if both a and b are specified. The logic for creating collections is also well-implemented; it verifies if the create key exists in the document and maps the collection using map_collection before creating it. Similarly, for dropping collections, the presence of the drop key is checked, and the appropriate collection is dropped if the mapping is successful.
Recommendation:
While the code is functional and handles MongoDB operations as intended, consider consolidating and refactoring the repetitive code blocks to improve readability and maintainability. For instance, you could extract the common operations for renaming, creating, and dropping collections into separate methods or utility functions. This approach would reduce redundancy and make it easier to manage changes or enhancements in the future. Additionally, ensure that error handling and logging are implemented to capture any potential issues during these operations, which would enhance the robustness of the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants