Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the stage EncryptedAddDummyRowExec is meaningless #220

Open
Oliver001 opened this issue Apr 29, 2021 · 3 comments
Open

the stage EncryptedAddDummyRowExec is meaningless #220

Oliver001 opened this issue Apr 29, 2021 · 3 comments

Comments

@Oliver001
Copy link

Instead of inserting a blank line into a data block, we can pass it through the parameters of the function NonObliviousSortMergeJoin.

#215

@octaviansima
Copy link
Collaborator

Meaningless how? Inserting it as a parameter to NonObliviousSortMergeJoin means modifying the Flatbuffers definition, the corresponding serialization of equi-joins, and involves adding additional functionality to the C++ code to actually write output according to a certain schema. Using a dummy row allows us to use an already existing functionality.

@Oliver001
Copy link
Author

we can still use this code to generate a block, and pass it through a parameter to NonObliviousSortMergeJoin.

val nullRowsBlock = Utils.encryptInternalRowsFlatbuffers(
            Seq(InternalRow.fromSeq(Seq.fill(buildPlan.output.length)(null))),
            buildPlan.output.map(_.dataType),
            useEnclave = true,
            isDummyRows = true
          ) // the code to generate a null row
// Block(enclave.NonObliviousSortMergeJoin(eid, joinExprSer, block.bytes)) // the origin line
Block(enclave.NonObliviousSortMergeJoin(eid, joinExprSer, block.bytes, nullRowsBlock.bytes)) // add a parameter

The Flatbuffers definition is no need to change. And no needed to adding additional functionality.
Only should we modify the definition of non_oblivious_sort_merge_join and read the null row just like

non_oblivious_sort_merge_join(uint8_t *join_expr, size_t join_expr_length,
                                   uint8_t *input_rows, size_t input_rows_length,
                                   uint8_t *null_row, size_t null_row_length, // the line added
                                   uint8_t **output_rows, size_t *output_rows_length)
// add tow line to get the null row
RowReader null_row_r(BufferRefView<tuix::EncryptedBlocks>(null_row, null_row_length));
const tuix::Row *null_row = null_row_r.peek();

If we want to implement the shuffleHashJoin, this maybe a more gentle way.

@octaviansima
Copy link
Collaborator

@Oliver001 sounds good to me! If you want to open a PR I'm happy to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants