Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

serialize: base64: allow LF in addition to CRLF and optimize slightly #19594

Merged
merged 3 commits into from
Dec 9, 2014

Conversation

Arcterus
Copy link
Contributor

@Arcterus Arcterus commented Dec 6, 2014

It is useful to have configurable newlines in base64 as the standard
leaves that for the implementation to decide. GNU base64 apparently
uses LF, which meant in uutils we had to manually convert the CRLF to
LF. This made the program very slow for large inputs.

[breaking-change]

Arcterus added a commit to Arcterus/coreutils that referenced this pull request Dec 6, 2014
Arcterus added a commit to Arcterus/coreutils that referenced this pull request Dec 6, 2014
@@ -14,6 +14,7 @@

pub use self::FromBase64Error::*;
pub use self::CharacterSet::*;
pub use self::Newline::*;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you leave this out? The other two reexports were just added to make the initial namespacing transition easier, but we don't want to keep them around forever.

@sfackler
Copy link
Member

sfackler commented Dec 6, 2014

What do the benchmark numbers look like before and after this?

@Arcterus
Copy link
Contributor Author

Arcterus commented Dec 6, 2014

I have only benchmarked this using GNU time and base64 from uutils.

Setup for benchmarking:

cd path/to/uutils
FILES=$(find . -name '*.rs')
FILES="$FILES $FILES $FILES $FILES"
FILES="$FILES $FILES $FILES $FILES"
FILES="$FILES $FILES $FILES $FILES"

Original:

[arcterus@Alex-Razer coreutils]$ time for a in {1..10}; do cat $FILES | build/base64; done > /dev/null

real    0m7.068s
user    0m3.630s
sys 0m4.137s

New:

[arcterus@Alex-Razer coreutils]$ time for a in {1..10}; do cat $FILES | build/base64; done > /dev/null

real    0m5.895s
user    0m3.560s
sys 0m2.880s

New (using LF instead of CRLF, so we don't have to replace CRLF with LF ourselves):

[arcterus@Alex-Razer coreutils]$ time for a in {1..10}; do cat $FILES | build/base64; done > /dev/null

real    0m3.402s
user    0m1.930s
sys 0m2.013s

@Arcterus
Copy link
Contributor Author

Arcterus commented Dec 9, 2014

r?

@alexcrichton
Copy link
Member

needs a rebase

It is useful to have configurable newlines in base64 as the standard
leaves that for the implementation to decide.  GNU `base64` apparently
uses LF, which meant in `uutils` we had to manually convert the CRLF to
LF.  This made the program very slow for large inputs.

[breaking-change]
@Arcterus
Copy link
Contributor Author

Arcterus commented Dec 9, 2014

Rebased.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Dec 9, 2014
It is useful to have configurable newlines in base64 as the standard
leaves that for the implementation to decide.  GNU `base64` apparently
uses LF, which meant in `uutils` we had to manually convert the CRLF to
LF.  This made the program very slow for large inputs.

[breaking-change]
@bors bors merged commit a119ad8 into rust-lang:master Dec 9, 2014
cnd pushed a commit to uutils/coreutils that referenced this pull request Dec 10, 2014
base64: fix build (assuming rust-lang/rust#19594 is merged)
jbcrail pushed a commit to jbcrail/coreutils that referenced this pull request Apr 29, 2015
jbcrail pushed a commit to jbcrail/coreutils that referenced this pull request Apr 29, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants