Merged
Conversation
Bumps [json](https://github.com/douglascrockford/JSON-java) from 20201115 to 20210307. - [Release notes](https://github.com/douglascrockford/JSON-java/releases) - [Commits](https://github.com/douglascrockford/JSON-java/commits) --- updated-dependencies: - dependency-name: org.json:json dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
dependabot
bot
added
dependencies
Merged
lvca
added a commit
that referenced
this pull request
Jan 6, 2026
Fix for issue #3091 (Fix for Edge Index Corruption) ## Solution Summary Implemented a general fix that **prevents RID reuse within the same transaction**, eliminating the root cause of index corruption. ## Root Cause When an edge was deleted and its RID immediately reused within the same transaction with different property values, the index entries from the deleted record were not properly removed before the RID was assigned to a new record. This caused stale index entries pointing to the reused RID. ## The Fix ### 1. Track Deleted RIDs in Transaction Context **File**: `engine/src/main/java/com/arcadedb/database/TransactionContext.java` Added a `Set<RID>` to track all RIDs deleted in the current transaction: ```java private final Set<RID> deletedRecordsInTx = new HashSet<>(); public void addDeletedRecord(final RID rid) { deletedRecordsInTx.add(rid); } public boolean isDeletedInTransaction(final RID rid) { return deletedRecordsInTx.contains(rid); } ``` The set is cleared in the `reset()` method when the transaction commits or rolls back. ### 2. Mark RIDs as Deleted **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1007` When a record is deleted, add its RID to the transaction's deleted set: ```java // POINTER = 0 MEANS DELETED page.writeUnsignedInt(PAGE_RECORD_TABLE_OFFSET + positionInPage * INT_SERIALIZED_SIZE, 0); // Track deleted RID to prevent reuse within the same transaction database.getTransaction().addDeletedRecord(rid); ``` ### 3. Prevent Reuse During Allocation **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1588-1597` Modified the `getFreeSpaceInPage()` method to skip RIDs deleted in the current transaction: ```java for (int i = 0; i < pageAnalysis.totalRecordsInPage; i++) { final int recordPositionInPage = getRecordPositionInPage(pageAnalysis.page, i); if (recordPositionInPage == 0) { // Check if this position was deleted in the current transaction final RID potentialRID = new RID(database, file.getFileId(), ((long) pageAnalysis.page.getPageId().getPageNumber()) * maxRecordsInPage + i); if (!database.getTransaction().isDeletedInTransaction(potentialRID)) { // REUSE THE FIRST AVAILABLE POSITION FROM DELETED RECORD (from previous transactions only) if (pageAnalysis.availablePositionIndex == -1) pageAnalysis.availablePositionIndex = i; } } else if (recordPositionInPage > pageAnalysis.lastRecordPositionInPage) pageAnalysis.lastRecordPositionInPage = recordPositionInPage; } ``` ### 4. Supporting Fixes **File**: `engine/src/main/java/com/arcadedb/database/DocumentIndexer.java:line 416` - Moved `unsetDirty()` call to AFTER property extraction for index deletion **File**: `engine/src/main/java/com/arcadedb/database/LocalDatabase.java:line 1031` - Added handling to load full edge when deleting a `LightEdge` (which returns null for all properties) ## Benefits of This Approach 1. **General Solution**: Prevents RID reuse within ANY transaction, not just for edges 2. **Prevents Multiple Issues**: Eliminates a whole class of potential bugs related to RID reuse 3. **Simple and Clean**: Minimal code changes, easy to understand and maintain 4. **No Performance Impact**: HashSet lookup is O(1), and the set is cleared on transaction completion 5. **Cleaner Transaction Semantics**: RIDs allocated in a transaction remain stable until commit ## Test Results The test case `EdgeIndexCorruptionTest` now passes: **Before Fix:** ``` After Transaction #3: - Edge #9:0: from_id=trs_2 (REUSED RID!) - Edge #9:1: from_id=trs_1 Query for from_id='trs_1' returned 2 edges (WRONG - stale index entry for #9:0) ``` **After Fix:** ``` After Transaction #3: - Edge #9:1: from_id=trs_2 - Edge #9:2: from_id=trs_1 (RID #9:0 NOT reused!) Query for from_id='trs_1' returned 1 edge (CORRECT!) ``` ## Files Modified 1. **engine/src/main/java/com/arcadedb/database/TransactionContext.java** - Added `deletedRecordsInTx` Set - Added `addDeletedRecord()` and `isDeletedInTransaction()` methods - Clear deleted set in `reset()` method 2. **engine/src/main/java/com/arcadedb/engine/LocalBucket.java** - Call `addDeletedRecord()` when record is deleted - Check `isDeletedInTransaction()` before reusing RID in `getFreeSpaceInPage()` 3. **engine/src/main/java/com/arcadedb/database/DocumentIndexer.java** - Moved `unsetDirty()` to after property extraction 4. **engine/src/main/java/com/arcadedb/database/LocalDatabase.java** - Added `LightEdge` import - Handle `LightEdge` by loading full edge before index deletion 5. **engine/src/test/java/com/arcadedb/graph/EdgeIndexCorruptionTest.java** - New test case that reproduces and validates the fix ## Backward Compatibility This fix is **fully backward compatible**: - No API changes - No schema changes - No data migration required - Existing databases work without modification ## Performance Considerations - **Memory**: O(N) where N is the number of deletions in a transaction (typically small) - **CPU**: O(1) HashSet operations for add/check - **Cleanup**: Automatic on transaction commit/rollback The performance impact is negligible as: 1. Most transactions don't delete many records 2. HashSet operations are very fast 3. The set is cleared after each transaction ## Future Enhancements This fix could be extended to: 1. Track all RID allocations in the transaction for rollback purposes 2. Provide transaction-level RID allocation statistics 3. Detect and warn about high RID churn within transactions
This was referenced Jan 27, 2026
robfrank
pushed a commit
that referenced
this pull request
Feb 11, 2026
Fix for issue #3091 (Fix for Edge Index Corruption) ## Solution Summary Implemented a general fix that **prevents RID reuse within the same transaction**, eliminating the root cause of index corruption. ## Root Cause When an edge was deleted and its RID immediately reused within the same transaction with different property values, the index entries from the deleted record were not properly removed before the RID was assigned to a new record. This caused stale index entries pointing to the reused RID. ## The Fix ### 1. Track Deleted RIDs in Transaction Context **File**: `engine/src/main/java/com/arcadedb/database/TransactionContext.java` Added a `Set<RID>` to track all RIDs deleted in the current transaction: ```java private final Set<RID> deletedRecordsInTx = new HashSet<>(); public void addDeletedRecord(final RID rid) { deletedRecordsInTx.add(rid); } public boolean isDeletedInTransaction(final RID rid) { return deletedRecordsInTx.contains(rid); } ``` The set is cleared in the `reset()` method when the transaction commits or rolls back. ### 2. Mark RIDs as Deleted **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1007` When a record is deleted, add its RID to the transaction's deleted set: ```java // POINTER = 0 MEANS DELETED page.writeUnsignedInt(PAGE_RECORD_TABLE_OFFSET + positionInPage * INT_SERIALIZED_SIZE, 0); // Track deleted RID to prevent reuse within the same transaction database.getTransaction().addDeletedRecord(rid); ``` ### 3. Prevent Reuse During Allocation **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1588-1597` Modified the `getFreeSpaceInPage()` method to skip RIDs deleted in the current transaction: ```java for (int i = 0; i < pageAnalysis.totalRecordsInPage; i++) { final int recordPositionInPage = getRecordPositionInPage(pageAnalysis.page, i); if (recordPositionInPage == 0) { // Check if this position was deleted in the current transaction final RID potentialRID = new RID(database, file.getFileId(), ((long) pageAnalysis.page.getPageId().getPageNumber()) * maxRecordsInPage + i); if (!database.getTransaction().isDeletedInTransaction(potentialRID)) { // REUSE THE FIRST AVAILABLE POSITION FROM DELETED RECORD (from previous transactions only) if (pageAnalysis.availablePositionIndex == -1) pageAnalysis.availablePositionIndex = i; } } else if (recordPositionInPage > pageAnalysis.lastRecordPositionInPage) pageAnalysis.lastRecordPositionInPage = recordPositionInPage; } ``` ### 4. Supporting Fixes **File**: `engine/src/main/java/com/arcadedb/database/DocumentIndexer.java:line 416` - Moved `unsetDirty()` call to AFTER property extraction for index deletion **File**: `engine/src/main/java/com/arcadedb/database/LocalDatabase.java:line 1031` - Added handling to load full edge when deleting a `LightEdge` (which returns null for all properties) ## Benefits of This Approach 1. **General Solution**: Prevents RID reuse within ANY transaction, not just for edges 2. **Prevents Multiple Issues**: Eliminates a whole class of potential bugs related to RID reuse 3. **Simple and Clean**: Minimal code changes, easy to understand and maintain 4. **No Performance Impact**: HashSet lookup is O(1), and the set is cleared on transaction completion 5. **Cleaner Transaction Semantics**: RIDs allocated in a transaction remain stable until commit ## Test Results The test case `EdgeIndexCorruptionTest` now passes: **Before Fix:** ``` After Transaction #3: - Edge #9:0: from_id=trs_2 (REUSED RID!) - Edge #9:1: from_id=trs_1 Query for from_id='trs_1' returned 2 edges (WRONG - stale index entry for #9:0) ``` **After Fix:** ``` After Transaction #3: - Edge #9:1: from_id=trs_2 - Edge #9:2: from_id=trs_1 (RID #9:0 NOT reused!) Query for from_id='trs_1' returned 1 edge (CORRECT!) ``` ## Files Modified 1. **engine/src/main/java/com/arcadedb/database/TransactionContext.java** - Added `deletedRecordsInTx` Set - Added `addDeletedRecord()` and `isDeletedInTransaction()` methods - Clear deleted set in `reset()` method 2. **engine/src/main/java/com/arcadedb/engine/LocalBucket.java** - Call `addDeletedRecord()` when record is deleted - Check `isDeletedInTransaction()` before reusing RID in `getFreeSpaceInPage()` 3. **engine/src/main/java/com/arcadedb/database/DocumentIndexer.java** - Moved `unsetDirty()` to after property extraction 4. **engine/src/main/java/com/arcadedb/database/LocalDatabase.java** - Added `LightEdge` import - Handle `LightEdge` by loading full edge before index deletion 5. **engine/src/test/java/com/arcadedb/graph/EdgeIndexCorruptionTest.java** - New test case that reproduces and validates the fix ## Backward Compatibility This fix is **fully backward compatible**: - No API changes - No schema changes - No data migration required - Existing databases work without modification ## Performance Considerations - **Memory**: O(N) where N is the number of deletions in a transaction (typically small) - **CPU**: O(1) HashSet operations for add/check - **Cleanup**: Automatic on transaction commit/rollback The performance impact is negligible as: 1. Most transactions don't delete many records 2. HashSet operations are very fast 3. The set is cleared after each transaction ## Future Enhancements This fix could be extended to: 1. Track all RID allocations in the transaction for rollback purposes 2. Provide transaction-level RID allocation statistics 3. Detect and warn about high RID churn within transactions (cherry picked from commit a81163a)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps json from 20201115 to 20210307.
Release notes
Sourced from json's releases.
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot mergewill merge this PR after your CI passes on it@dependabot squash and mergewill squash and merge this PR after your CI passes on it@dependabot cancel mergewill cancel a previously requested merge and block automerging@dependabot reopenwill reopen this PR if it is closed@dependabot closewill close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)