Skip to content

Fb dataset update rewrite #6552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: develop
Choose a base branch
from
Open

Conversation

labkey-klum
Copy link
Contributor

@labkey-klum labkey-klum commented Apr 10, 2025

Rationale

Dataset updateRows had previously been implemented as a single row delete followed by an insert row. This PR attempts to rewrite this code to look like more typical QueryUpdateService.updateRow implementations.

This work was focused on a few main areas and tasks:

  • Refactor the helpers in DatasetDataIteratorBuilder so they could be used outside of a DataIterator context. These helpers were responsible for : LSID, subject ID, sequence number, and participant sequence number generation.
  • Add unit tests and selenium tests to validate the integrity of insert and update operations around these special fields
  • Swap in the new helpers into DatasetDataIteratorBuilder
  • Rewrite DatasetUpdateService.updateRow to use the new helpers as well as Table.update to update the row.

Copy link

WARNING: This PR appears to have the default title generated by GitHub. Please use something more descriptive.

@labkey-klum labkey-klum requested a review from labkey-adam April 16, 2025 16:47
@labkey-klum labkey-klum marked this pull request as ready for review April 16, 2025 16:47
Copy link

WARNING: This PR appears to have the default title generated by GitHub. Please use something more descriptive.

@labkey-klum labkey-klum requested a review from XingY April 16, 2025 17:45
@labkey-klum
Copy link
Contributor Author

@mbellew I'm going to need your opinion on the test failure I'm seeing in DatasetUpdateService.TestCase.updateRowTest. It was previously working because of the the code in SimpleTranslator.MissingValueConvertColumn, but no longer does because updates are no longer handled through the insert code.

It also looks like other data types don't support this on update either. Do you think the old behavior is still required, should there be code in DefaultQueryUpdateService to deal with MV columns?

@labkey-klum labkey-klum removed the request for review from XingY May 8, 2025 16:57
return null;

Object o = it.get(i);
if (null == o)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JdbcType.DOUBLE.convert(o)

return null;

Object o = it.get(i);
return null == o ? "" : o.toString();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Objects.toString(o,"")

@labkey-klum labkey-klum requested a review from labkey-chrisj May 12, 2025 19:31
Copy link
Contributor

@labkey-chrisj labkey-chrisj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests look good to go

try (DbScope.Transaction transaction = StudyService.get().getDatasetSchema().getScope().ensureTransaction())
{
String lsid = keyFromMap(oldRow);
checkDuplicateUpdate(lsid);// Make sure we've found the original participant before doing the update
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment is for the next line, not checkDuplicateUpdate, a merge error I believe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants