Fix: Align sort_merge_join filter output with join schema to fix right-anti panic

[kumarUjjawal]

Which issue does this PR close?

• Closes Panic when having right anti join with filter in sort_merge_join and different number of columns between each side #18787.

Rationale for this change

Sort-merge joins assumed both inputs had the same width when applying filter results. With different column counts (like right-anti joins) we built batches that no longer matched the output schema, so Arrow panicked during concat_batches.

What changes are included in this PR?

• Fix the null-row and filtered-row builders in SortMergeJoinStream to slice the streamed vs buffered columns using the correct widths, keeping every batch consistent with self.schema.
• Merge the semi/anti projection branch so it always projects the streamed-side columns expected in the output.

Are these changes tested?

Manual testing and ran previous tests.

Are there any user-facing changes?

[kumarUjjawal]

hmm, why is the test failing?

[rluvaton]

Looks like this include fixes created by, let's wait for it to be merged

• fix: Pick correct columns in Sort Merge Equijoin #18772

Also, once merged can you update the fuzz tests for join:
similar to what he added here but add the for loop to the other tests as well:
test_right_semi_join_1k_filtered, test_right_anti_join_1k_filtered, test_right_anti_join_1k_binary, test_right_anti_join_1k_binary_filtered
https://github.com/apache/datafusion/pull/18772/files#diff-f8330244c576c142337679ecbeeb99e9016c2c7c54d7bae492144374e138a0b7

[rluvaton]

also, when fixing a bug, we require tests to be added that failed on main and pass on this pr

[kumarUjjawal]

also, when fixing a bug, we require tests to be added that failed on main and pass on this pr

Sure will do.

[rluvaton]

I don't think we should add it as we can lose columns or hide bugs if we the rest of the columns are not left

[kumarUjjawal]

Right, should I Drop the .take(left_columns_length) and add a sanity check like assert_eq_or_internal_err!(..., left_columns_length, "...")) before extending right_columns.

[rluvaton]

Yes