Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Transaction Deletion record for large volumes (backport #39717) #40574

Merged
merged 30 commits into from
Mar 21, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Mar 20, 2024

Transaction Deletion Record - tool used to clear all transactions related to a company, timesout when there are large number of transactions. To fix this, tool will be redesign to run in background/

Overview

Data clearing will be performed as a series of consecutive tasks, with each task give an ample timeout limit of 25mins.
Each task is executed only after the previous one is completed.
The last task - 'Delete Transactions', will clear transaction data in batches of 5000, with each batch having timeout of 25mins.

Safegaurds

Validation is added to prevent new docs from being created while the Deletion job is running.
Screenshot 2024-03-20 at 4 03 00 PM

todo:

  • Make 'Company Master -> Manager -> Delete Transactions' work
  • Test Cases

Screenshots

image

Video

tdr.mov

This is an automatic backport of pull request #39717 done by [Mergify](https://mergify.com).

(cherry picked from commit 0d65d87)

# Conflicts:
#	erpnext/setup/doctype/transaction_deletion_record/transaction_deletion_record.py
(cherry picked from commit 8944ab8)
(cherry picked from commit 6a77d86)

# Conflicts:
#	erpnext/setup/doctype/transaction_deletion_record_item/transaction_deletion_record_item.py
(cherry picked from commit b12ca65)
(cherry picked from commit b98a5e4)
(cherry picked from commit 98afb4d)
(cherry picked from commit 3046365)
(cherry picked from commit eea260b)
(cherry picked from commit 4a55240)
(cherry picked from commit 0455d0c)
(cherry picked from commit 5fe0b20)
(cherry picked from commit 5a3afea)

# Conflicts:
#	erpnext/setup/doctype/company/company.js
#	erpnext/setup/doctype/company/company.py
(cherry picked from commit 8130957)

# Conflicts:
#	erpnext/setup/demo.py
#	erpnext/setup/doctype/transaction_deletion_record/test_transaction_deletion_record.py
(cherry picked from commit 02c522b)
Copy link
Contributor Author

mergify bot commented Mar 20, 2024

Cherry-pick of 0d65d87 has failed:

On branch mergify/bp/version-14-hotfix/pr-39717
Your branch is up to date with 'origin/version-14-hotfix'.

You are currently cherry-picking commit 0d65d878de.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   erpnext/setup/doctype/transaction_deletion_record/transaction_deletion_record.json

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   erpnext/setup/doctype/transaction_deletion_record/transaction_deletion_record.py

Cherry-pick of 6a77d86 has failed:

On branch mergify/bp/version-14-hotfix/pr-39717
Your branch is ahead of 'origin/version-14-hotfix' by 4 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 6a77d86a53.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   erpnext/setup/doctype/transaction_deletion_record/transaction_deletion_record.js
	modified:   erpnext/setup/doctype/transaction_deletion_record/transaction_deletion_record.json
	modified:   erpnext/setup/doctype/transaction_deletion_record/transaction_deletion_record.py
	modified:   erpnext/setup/doctype/transaction_deletion_record_item/transaction_deletion_record_item.json

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   erpnext/setup/doctype/transaction_deletion_record_item/transaction_deletion_record_item.py

Cherry-pick of 5a3afea has failed:

On branch mergify/bp/version-14-hotfix/pr-39717
Your branch is ahead of 'origin/version-14-hotfix' by 24 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 5a3afea8c7.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   erpnext/setup/doctype/transaction_deletion_record/transaction_deletion_record.py

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   erpnext/setup/doctype/company/company.js
	both modified:   erpnext/setup/doctype/company/company.py

Cherry-pick of 8130957 has failed:

On branch mergify/bp/version-14-hotfix/pr-39717
Your branch is ahead of 'origin/version-14-hotfix' by 26 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 81309576b0.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   erpnext/setup/demo.py
	both modified:   erpnext/setup/doctype/transaction_deletion_record/test_transaction_deletion_record.py

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@ruthra-kumar
Copy link
Member

Linter failure is unrelated

@ruthra-kumar ruthra-kumar merged commit 67481ad into version-14-hotfix Mar 21, 2024
9 of 10 checks passed
@ruthra-kumar ruthra-kumar deleted the mergify/bp/version-14-hotfix/pr-39717 branch March 21, 2024 04:48
@frappe-pr-bot
Copy link
Collaborator

🎉 This PR is included in version 14.66.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants