Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extremely easy to wipe out a container #91

Open
onionhammer opened this issue Oct 4, 2023 · 2 comments
Open

Extremely easy to wipe out a container #91

onionhammer opened this issue Oct 4, 2023 · 2 comments

Comments

@onionhammer
Copy link

If you set source and sink to the same settings, the container will be wiped - this should never happen, since the job is already "done" before it begins.

@bowencode
Copy link
Collaborator

What is your goal in using the same source and sink? Since data transfers happen asynchronously using streaming but container re-creation is essentially a one-time synchronous operation which needs to happen before the first write it won't be possible to directly read and then write into a re-created empty container.

I would first suggest that you don't use the RecreateContainer setting for this scenario since that is what controls deleting and recreating the container. Beyond that other settings would depend on the intended outcome. For example, if you want to add duplicates for all of the records in the container you could use a query that leaves out "id" to autogenerate new IDs per record.

If you're hoping to read all of the data out of a container, delete it, and then repopulate back into a new instance of that container (I'm not sure why) you would need an intermediate step to store all of the data elsewhere, like a local JSON file that you could write to and then read back in. I haven't tried this but you may be able to do this in a single settings file with multiple Operations since they run sequentially.

@onionhammer
Copy link
Author

It was accidental. Logically, I don't believe erasing the source should be the intended outcome

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants