Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort by #72

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Sort by #72

wants to merge 1 commit into from

Conversation

return02
Copy link
Contributor

@return02 return02 commented Feb 27, 2020

Implement sort by transform by a very simple range_partitioner.

This algorithm is almost the same with Apache Spark: partition all the data into ordered partitions and sort them separately.

There're still some work to be done:

  • Find a better algorithm for building range_bounds.
  • implement descending.
  • use binary search in method get_partitions().
  • perhaps F: SerFunc(&Self::Item) -> K + Clone is better than F: SerFunc(Self::Item) -> K.
  • test corner case.

I have rebased master in my branch and I still don't know what changed in Cargo.lock(because it's in .gitignore).

@iduartgomez
Copy link
Collaborator

It will compile with whatever last versions based on the Cargo.toml manifest, which is fine as a library. As a tip, in case you didn't know, if you don't want to, is not necessary to close the PR, next time you can do the following:

git checkout <working branch>
git rebase native_spark/master # add this repo as upstream  
git push origin <working branch> -f

Do you want this work to be reviewed and merged or will be doing the other working items before merge?

@return02
Copy link
Contributor Author

It will compile with whatever last versions based on the Cargo.toml manifest, which is fine as a library. As a tip, in case you didn't know, if you don't want to, is not necessary to close the PR, next time you can do the following:

git checkout <working branch>
git rebase native_spark/master # add this repo as upstream  
git push origin <working branch> -f

Do you want this work to be reviewed and merged or will be doing the other working items before merge?

I'm not familiar with git push -f that I had to close the old PR . Next time I'll do what you mentioned.

Please do not merge this PR before finishing other work item.

Actually, I have no good idea about good algorithm for building range_bounds. I'd like to discuss this question so that I created this PR although it's not complete.

@return02 return02 closed this Feb 28, 2020
@return02 return02 reopened this Feb 28, 2020
@iduartgomez
Copy link
Collaborator

Sorry I haven't been able to look too much into this as I was fixing other stuff, will check out tomorrow hopefully.

But as I understand, it works right now but you are not happy about how efficient it is, ¿correct?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants