Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(Dgraph): update reverse index when updating single UID predicates. #5868

Merged
merged 12 commits into from
Jul 15, 2020

Conversation

martinmr
Copy link
Contributor

@martinmr martinmr commented Jul 7, 2020

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738


This change is Reviewable

Docs Preview: Dgraph Preview

@martinmr martinmr changed the title fix: update reverse index when updating single UID predicates. fix(Dgraph): update reverse index when updating single UID predicates. Jul 7, 2020
@martinmr martinmr marked this pull request as ready for review July 7, 2020 22:57
@martinmr martinmr requested a review from a team July 8, 2020 22:13
Copy link
Contributor

@parasssh parasssh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 4 unresolved discussions (waiting on @manishrjain, @martinmr, and @vvbalaji-dgraph)


posting/index.go, line 219 at r1 (raw file):

	// entries for this key in the index are removed.
	pred, ok := schema.State().Get(ctx, t.Attr)
	isSingleUidUpdate := ok && !pred.GetList() && pred.GetValueType() == pb.Posting_UID &&

Wouldn't we need to fix for when object/value is UID list as well? e.g. A -> [B,C,D] and C is deleted?


posting/index.go, line 228 at r1 (raw file):

				hex.Dump(dataKey))
		}
		err = dataList.Iterate(txn.StartTs, 0, func(p *pb.Posting) error {

We expect only one item in the list, correct?


systest/mutations_test.go, line 2015 at r1 (raw file):

	testutil.CompareJSON(t, `{"me":[{"name":"Alice","best_friend": {"name": "Bob"}}]}`,
		string(resp.GetJson()))

Add check for reverse to make sure Bob has the reverse edge.


systest/mutations_test.go, line 2040 at r1 (raw file):

	require.NoError(t, err)
	testutil.CompareJSON(t, `{"reverse":[{"name":"Carol","~best_friend": [{"name": "Alice"}]}]}`,
		string(resp.GetJson()))

Other related Test cases could be:

  1. When Bob is deleted, the edge and reverse edge should be deleted.
  2. When Alice is deleted, the edge and reverse edge should be deleted.

Copy link
Contributor Author

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 4 unresolved discussions (waiting on @manishrjain, @parasssh, and @vvbalaji-dgraph)


posting/index.go, line 219 at r1 (raw file):

Previously, parasssh wrote…

Wouldn't we need to fix for when object/value is UID list as well? e.g. A -> [B,C,D] and C is deleted?

No. This is just for the single uid. Overwriting a single uid predicate means the previous value needs to be deleted. We were doing that but not deleting the reverse edge.


posting/index.go, line 228 at r1 (raw file):

Previously, parasssh wrote…

We expect only one item in the list, correct?

Yes. I am just using iterate because it's the proper way to handle the values.


systest/mutations_test.go, line 2015 at r1 (raw file):

Previously, parasssh wrote…

Add check for reverse to make sure Bob has the reverse edge.

Done.


systest/mutations_test.go, line 2040 at r1 (raw file):

Previously, parasssh wrote…

Other related Test cases could be:

  1. When Bob is deleted, the edge and reverse edge should be deleted.
  2. When Alice is deleted, the edge and reverse edge should be deleted.

Not sure if this cases are relevant to this case. Deletion is handled the same for uid and [uid] predicates. The difference with single uid predicates is that writing to the list means that the previous value has to be deleted (along with the reverse index).

Copy link
Contributor

@parasssh parasssh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: minor comments. I'd like Pawan to bless it as well.

Reviewable status: 0 of 2 files reviewed, 3 unresolved discussions (waiting on @manishrjain, @martinmr, @parasssh, and @vvbalaji-dgraph)


posting/index.go, line 219 at r1 (raw file):

Previously, martinmr (Martin Martinez Rivera) wrote…

No. This is just for the single uid. Overwriting a single uid predicate means the previous value needs to be deleted. We were doing that but not deleting the reverse edge.

Ok. With a list, it just extends the list and so issue doesn't exist there.


systest/mutations_test.go, line 2040 at r1 (raw file):

Previously, martinmr (Martin Martinez Rivera) wrote…

Not sure if this cases are relevant to this case. Deletion is handled the same for uid and [uid] predicates. The difference with single uid predicates is that writing to the list means that the previous value has to be deleted (along with the reverse index).

I didn't mean tests for [uid] preds. I meant deleting Alice and Bob with your current test schema of single predicate (with reverse) to make sure reverse edges are not left dangling. Feel free to resolve.

Copy link
Contributor Author

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 3 unresolved discussions (waiting on @manishrjain, @parasssh, and @vvbalaji-dgraph)


posting/index.go, line 219 at r1 (raw file):

Previously, parasssh wrote…

Ok. With a list, it just extends the list and so issue doesn't exist there.

Yeah, that's the case.


systest/mutations_test.go, line 2040 at r1 (raw file):

Previously, parasssh wrote…

I didn't mean tests for [uid] preds. I meant deleting Alice and Bob with your current test schema of single predicate (with reverse) to make sure reverse edges are not left dangling. Feel free to resolve.

I added a test to verify the reverse edge is gone when I remove the original edge.

Copy link
Contributor

@pawanrawal pawanrawal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm

Reviewed 2 of 2 files at r2.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @manishrjain, @martinmr, @parasssh, and @vvbalaji-dgraph)


posting/index.go, line 214 at r2 (raw file):

		return err
	}
	x.AssertTrue(plist != nil)

Do we need this to be an assert or should this be an error that can be returned to the user?

Copy link
Contributor Author

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @manishrjain, @parasssh, @pawanrawal, and @vvbalaji-dgraph)


posting/index.go, line 214 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

Do we need this to be an assert or should this be an error that can be returned to the user?

Done.

@martinmr martinmr merged commit 5b70fe8 into master Jul 15, 2020
@martinmr martinmr deleted the martinmr/uid-rev branch July 15, 2020 20:13
martinmr added a commit that referenced this pull request Jul 15, 2020
#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

(cherry picked from commit 5b70fe8)
martinmr added a commit that referenced this pull request Jul 15, 2020
#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

(cherry picked from commit 5b70fe8)
parasssh pushed a commit that referenced this pull request Jul 16, 2020
#5868) (#6005)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

(cherry picked from commit 5b70fe8)
parasssh pushed a commit that referenced this pull request Jul 16, 2020
#5868) (#6015)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

(cherry picked from commit 5b70fe8)
dna2github pushed a commit to dna2fork/dgraph that referenced this pull request Jul 18, 2020
dgraph-io#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738
parasssh pushed a commit that referenced this pull request Aug 28, 2020
#5868) (#6006)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

(cherry picked from commit 5b70fe8)
danielmai pushed a commit that referenced this pull request Oct 16, 2020
#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

(cherry picked from commit 5b70fe8)
danielmai pushed a commit that referenced this pull request Oct 22, 2020
#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738
danielmai pushed a commit that referenced this pull request Oct 22, 2020
#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738
danielmai pushed a commit that referenced this pull request Oct 22, 2020
#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738
danielmai added a commit that referenced this pull request Oct 27, 2020
#6748)

Fixes the test OverwriteUidPredicatesReverse added in the previous PR #6746.

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

(cherry picked from commit 5b70fe8)

* Remove context arguments.

* Optimize computing reverse reindexing (#4755)

We used to compute reverse count indexes twice while reindexing.
First, while computing reverse edges, and then later while explicitly
computing reverse count index. Now, we use a different function
while reindexing reverse edges and do not compute the reverse
count indexes there any more.

* fix(Dgraph): update reverse index when updating single UID predicates. (#5868)

When updating a single uid predicate with a reverse index, the existing entry in the
reverse index should be deleted first.

Fixes DGRAPH-1738

Co-authored-by: Martin Martinez Rivera <martinmr@dgraph.io>
Co-authored-by: Aman Mangal <aman@dgraph.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants