Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring net/parser.rs up to modern up to date with modern rust patterns #72369

Merged
merged 1 commit into from
Jul 1, 2020

Conversation

Lucretiel
Copy link
Contributor

@Lucretiel Lucretiel commented May 20, 2020

The current implementation of IP address parsing is very unidiomatic; it's full of if / return / is_some / is_none instead of ?, loop with manual index tracking; etc. Went through and did and cleanup to try to bring it in line with modern sensibilities.

The obvious concern with making changes like this is "make sure you understand why it's written that way before changing it". Looking through the commit history for this file, there are several much smaller commits that make similar changes (For instance, 3024c14, 4f3ab49, 79f8764), and there don't seem to be any commits in the history that indicate that this lack of idiomaticity is related to specific performance needs (ie, there aren't any commits that replace a for loop with a loop and a manual index count). In fact, the basic shape of the file is essentially unchanged from its initial commit back in 2015.

Made the following changes throughout the IP address parser:

  • Replaced all uses of is_some() / is_none() with ?.
  • "Upgraded" loops wherever possible; ie, replace while with for, etc.
    • Removed all cases of manual index tracking / incrementing.
  • Renamed several single-character variables with more expressive names.
  • Replaced several manual control flow segments with equivalent adapters (such as Option::filter).
  • Removed read_seq_3; replaced with simple sequences of ?.
  • Parser now reslices its state when consuming, rather than carrying a separate state and index variable.
  • read_digit now uses char::to_digit.
  • Added comments throughout, especially in the complex IPv6 parsing logic.
  • Added comprehensive local unit tests for the parser to validate these changes.

@rust-highfive
Copy link
Collaborator

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 20, 2020
@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
##[section]Starting: Linux mingw-check
##[section]Starting: Initialize job
Agent name: 'Hosted Agent'
Agent machine name: 'fv-az578'
Current agent version: '2.168.2'
##[group]Operating System
16.04.6
LTS
LTS
##[endgroup]
##[group]Virtual Environment
Environment: ubuntu-16.04
Version: 20200512.2
Included Software: https://github.com/actions/virtual-environments/blob/ubuntu16/20200512.2/images/linux/Ubuntu1604-README.md
##[endgroup]
Agent running as: 'vsts'
Prepare build directory.
Set build variables.
Download all required tasks.
Download all required tasks.
Downloading task: Bash (3.163.2)
Checking job knob settings.
   Knob: AgentToolsDirectory = /opt/hostedtoolcache Source: ${AGENT_TOOLSDIRECTORY} 
   Knob: AgentPerflog = /home/vsts/perflog Source: ${VSTS_AGENT_PERFLOG} 
Start tracking orphan processes.
##[section]Finishing: Initialize job
##[section]Starting: Configure Job Name
==============================================================================
---
========================== Starting Command Output ===========================
[command]/bin/bash --noprofile --norc /home/vsts/work/_temp/535e1d46-5b66-4eed-ac64-2ebb9b04e02c.sh

##[section]Finishing: Disable git automatic line ending conversion
##[section]Starting: Checkout rust-lang/rust@refs/pull/72369/merge to s
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
---
##[command]git remote add origin https://github.com/rust-lang/rust
##[command]git config gc.auto 0
##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
##[command]git config --get-all http.proxy
##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/72369/merge:refs/remotes/pull/72369/merge
---
 ---> 3adb0605cc65
Step 6/7 : ENV RUN_CHECK_WITH_PARALLEL_QUERIES 1
 ---> Using cache
 ---> 28dbc326cb7f
Step 7/7 : ENV SCRIPT python3 ../x.py test src/tools/expand-yaml-anchors &&            python3 ../x.py check --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu &&            python3 ../x.py build --stage 0 src/tools/build-manifest &&            python3 ../x.py test --stage 0 src/tools/compiletest &&            python3 ../x.py test src/tools/tidy &&            python3 ../x.py doc --stage 0 src/libstd &&            /scripts/validate-toolstate.sh
 ---> 537a01811900
Successfully built 537a01811900
Successfully tagged rust-ci:latest
Built container sha256:537a018119009dc218456238dec90b5530050db1e2a1e166550c218003f6159d
---
    Checking panic_unwind v0.0.0 (/checkout/src/libpanic_unwind)
error[E0425]: cannot find value `p` in this scope
   --> src/libstd/net/parser.rs:116:21
    |
116 |             *slot = p.read_separator('.', i, |p| p.read_number(10, 3, 0x100))? as u8;

error: aborting due to previous error

For more information about this error, try `rustc --explain E0425`.
---
  local time: Wed May 20 04:03:32 UTC 2020
  network time: Wed, 20 May 2020 04:03:32 GMT
== end clock drift check ==

##[error]Bash exited with code '1'.
##[section]Finishing: Run build
##[section]Starting: Checkout rust-lang/rust@refs/pull/72369/merge to s
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
Author       : Microsoft
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
==============================================================================
Cleaning any cached credential from repository: rust-lang/rust (GitHub)
##[section]Finishing: Checkout rust-lang/rust@refs/pull/72369/merge to s
Cleaning up task key
Start cleaning up orphan processes.
Terminate orphan process: pid (3683) (python)
##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

src/libstd/net/parser.rs Outdated Show resolved Hide resolved
@Elinvynia Elinvynia added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 3, 2020
@Dylan-DPC-zz
Copy link

r? @dtolnay maybe?

@rust-highfive rust-highfive assigned dtolnay and unassigned cramertj Jun 23, 2020
@dtolnay dtolnay added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Jun 23, 2020
Copy link
Member

@dtolnay dtolnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks great. I didn't review too closely but it passes this fuzz test so I have reasonable confidence that the behavior matches the original.
https://gist.githubusercontent.com/rust-play/8d17b0e3789782ac09d7387c41943d65/raw/dc9728b304d72ba64553b1ea361559ef9798dd94/playground.rs

@dtolnay
Copy link
Member

dtolnay commented Jun 29, 2020

@bors r+

@bors
Copy link
Contributor

bors commented Jun 29, 2020

📌 Commit e02b3fb00bfe8baf185017db4cff1939a06814d5 has been approved by dtolnay

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 29, 2020
@bors
Copy link
Contributor

bors commented Jun 29, 2020

⌛ Testing commit e02b3fb00bfe8baf185017db4cff1939a06814d5 with merge 6b9a1b4007948c3e6b72eb81b83a04913dede9c7...

@bors
Copy link
Contributor

bors commented Jun 29, 2020

💥 Test timed out

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 29, 2020
src/libstd/net/parser.rs Outdated Show resolved Hide resolved
Made the following changes throughout the IP address parser:
- Replaced all uses of `is_some()` / `is_none()` with `?`.
- "Upgraded" loops wherever possible; ie, replace `while` with `for`, etc.
    - Removed all cases of manual index tracking / incrementing.
- Renamed several single-character variables with more expressive names.
- Replaced several manual control flow segments with equivalent adapters (such as `Option::filter`).
- Removed `read_seq_3`; replaced with simple sequences of `?`.
- Parser now reslices its state when consuming, rather than carrying a separate state and index variable.
- `read_digit` now uses `char::to_digit`.
- Removed unnecessary casts back and forth between u8 and u32
- Added comments throughout, especially in the complex IPv6 parsing logic.
- Added comprehensive local unit tests for the parser to validate these changes.
@dtolnay
Copy link
Member

dtolnay commented Jun 30, 2020

@bors r+

@bors
Copy link
Contributor

bors commented Jun 30, 2020

📌 Commit 3ab7ae3 has been approved by dtolnay

@bors bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 30, 2020
@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jun 30, 2020
Manishearth added a commit to Manishearth/rust that referenced this pull request Jul 1, 2020
Bring net/parser.rs up to modern up to date with modern rust patterns

The current implementation of IP address parsing is very unidiomatic; it's full of `if` / `return` / `is_some` / `is_none` instead of `?`, `loop` with manual index tracking; etc. Went through and did and cleanup to try to bring it in line with modern sensibilities.

The obvious concern with making changes like this is "make sure you understand why it's written that way before changing it". Looking through the commit history for this file, there are several much smaller commits that make similar changes (For instance, rust-lang@3024c14, rust-lang@4f3ab49, rust-lang@79f8764), and there don't seem to be any commits in the history that indicate that this lack of idiomaticity is related to specific performance needs (ie, there aren't any commits that replace a `for` loop with a `loop` and a manual index count). In fact, the basic shape of the file is essentially unchanged from its initial commit back in 2015.

Made the following changes throughout the IP address parser:
- Replaced all uses of `is_some()` / `is_none()` with `?`.
- "Upgraded" loops wherever possible; ie, replace `while` with `for`, etc.
    - Removed all cases of manual index tracking / incrementing.
- Renamed several single-character variables with more expressive names.
- Replaced several manual control flow segments with equivalent adapters (such as `Option::filter`).
- Removed `read_seq_3`; replaced with simple sequences of `?`.
- Parser now reslices its state when consuming, rather than carrying a separate state and index variable.
- `read_digit` now uses `char::to_digit`.
- Added comments throughout, especially in the complex IPv6 parsing logic.
- Added comprehensive local unit tests for the parser to validate these changes.
bors added a commit to rust-lang-ci/rust that referenced this pull request Jul 1, 2020
…arth

Rollup of 17 pull requests

Successful merges:

 - rust-lang#72071 (Added detailed error code explanation for issue E0687 in Rust compiler.)
 - rust-lang#72369 (Bring net/parser.rs up to modern up to date with modern rust patterns)
 - rust-lang#72445 (Stabilize `#[track_caller]`.)
 - rust-lang#73466 (impl From<char> for String)
 - rust-lang#73548 (remove rustdoc warnings)
 - rust-lang#73649 (Fix sentence structure)
 - rust-lang#73678 (Update Box::from_raw example to generalize better)
 - rust-lang#73705 (stop taking references in Relate)
 - rust-lang#73716 (Document the static keyword)
 - rust-lang#73752 (Remap Windows ERROR_INVALID_PARAMETER to ErrorKind::InvalidInput from Other)
 - rust-lang#73776 (Move terminator to new module)
 - rust-lang#73778 (Make `likely` and `unlikely` const, gated by feature `const_unlikely`)
 - rust-lang#73805 (Document the type keyword)
 - rust-lang#73806 (Use an 'approximate' universal upper bound when reporting region errors)
 - rust-lang#73828 (Fix wording for anonymous parameter name help)
 - rust-lang#73846 (Fix comma in debug_assert! docs)
 - rust-lang#73847 (Edit cursor.prev() method docs in lexer)

Failed merges:

r? @ghost
@bors bors merged commit 33f8ce2 into rust-lang:master Jul 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants