Skip to content

Commit

Permalink
apacheGH-41198: [C#] Fix concatenation of union arrays (apache#41226)
Browse files Browse the repository at this point in the history
### Rationale for this change

Fixes concatenation of union arrays.

### What changes are included in this PR?

* Re-enables union array concatenation tests that were disabled in apache#41197 after making union array comparisons more thorough in the `ArrowReaderVerifier`
* Updates the union array concatenation logic to account for array lengths when concatenating the type and offset buffers, and fixes how the base offset is calculated.
* Fixes creating the type buffers for the array concatenation tests.

### Are these changes tested?

Yes, this uses the existing `ArrowArrayConcatenatorTests` tests.

### Are there any user-facing changes?

Yes, this is a user-facing bug fix.
* GitHub Issue: apache#41198

Authored-by: Adam Reeve <adreeve@gmail.com>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
  • Loading branch information
adamreeve authored and tolleybot committed May 4, 2024
1 parent 82f5a0f commit 351ae76
Showing 1 changed file with 1 addition and 11 deletions.
12 changes: 1 addition & 11 deletions csharp/test/Apache.Arrow.Tests/ArrowArrayConcatenatorTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,6 @@ public void TestStandardCases()
{
foreach ((List<IArrowArray> testTargetArrayList, IArrowArray expectedArray) in GenerateTestData())
{
if (expectedArray is UnionArray)
{
// Union array concatenation is incorrect. See https://github.com/apache/arrow/issues/41198
continue;
}

IArrowArray actualArray = ArrowArrayConcatenator.Concatenate(testTargetArrayList);
ArrowReaderVerifier.CompareArrays(expectedArray, actualArray);
}
Expand Down Expand Up @@ -410,15 +404,11 @@ public void Visit(UnionType type)

for (int j = 0; j < dataList.Count; j++)
{
bool includeInResult = IncludeInResult(i, j);
byte index = (byte)Math.Min(j % 3, 1);
int? intValue = (index == 1) ? dataList[j] : null;
string stringValue = (index == 1) ? null : dataList[j]?.ToString();
typeBuilder.Append(index);
if (includeInResult)
{
typeResultBuilder.Append(index);
}
typeResultBuilder.Append(index);

if (isDense)
{
Expand Down

0 comments on commit 351ae76

Please sign in to comment.