Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HeaderMap: C string to string_view conversion follow ups #6580

Open
dnoe opened this issue Apr 15, 2019 · 16 comments
Open

HeaderMap: C string to string_view conversion follow ups #6580

dnoe opened this issue Apr 15, 2019 · 16 comments

Comments

@dnoe
Copy link
Contributor

dnoe commented Apr 15, 2019

Description:

These are various places where we need to follow up to complete removal of C style strings in the header map.

  1. HeaderString::find(const char* str) can probably be eliminated entirely by converting call sites to use getStringView().find().
  2. StringUtil::atoull currently requires creation of some temporary std::string objects because it expects C strings, and also returns a C style string. Most call sites ignore the return value, so they can be converted to use absl::SimpleAtoi or a new StringUtil function that takes and returns string views.
  3. UuidUtils::uuidModBy() should be migrated to take absl::string_view
  4. Span::setOperation() should be migrated to take absl::string_view

Context:
#6494
#6564

Action item for CVE-2019-9900

@dnoe
Copy link
Contributor Author

dnoe commented Apr 15, 2019

Once this is all done, we can reclaim the NUL byte: #6581

@dnoe
Copy link
Contributor Author

dnoe commented Apr 15, 2019

Span::setTag could also use some love.

@dnoe
Copy link
Contributor Author

dnoe commented Apr 15, 2019

GzipFilter::isContentTypeAllowed can use heterogeneous lookup to elide an std::string construction if CaseUnorderedSet is updated to use a Swiss Set.

@mattklein123 mattklein123 added this to the 1.11.0 milestone Apr 15, 2019
@mattklein123 mattklein123 added the no stalebot Disables stalebot from closing an issue label Apr 15, 2019
dnoe added a commit that referenced this issue Apr 16, 2019
…d() (#6603)

Description: Removed the HeaderString::find function, migrating all callers to getStringView().find() which is equivalent.

Risk Level: Low
Testing: bazel test //test/...
Part of #6580
@dnoe
Copy link
Contributor Author

dnoe commented Apr 24, 2019

Looks like StringUtil::atoll can be replaced entirely by calls to absl::SimpleAtoi since all uses are base 10 conversions.

lizan pushed a commit that referenced this issue Apr 25, 2019
Description: `absl::SimpleAtoi` takes `absl::string_view` argument and can easily replace all of the calls to `StringUtil::atoull` that are using Base 10 conversion. This eliminates a significant number of places where a `std::string` needed to be constructed from `string_view` to obtain a C string for `StringUtil:atoull`.
    
Risk Level: Low
Testing: `bazel test //test/...`
Docs Changes: N/A
Release Notes: N/A
Part of: Issue #6580

Signed-off-by: Dan Noé <dpn@google.com>
htuch pushed a commit that referenced this issue Apr 25, 2019
Description: absl::SimpleAtoi takes absl::string_view argument and can replace all of the calls to StringUtil::atoll that are using Base 10 conversion, which turns out to be all of them. This PR removes StringUtil::atoll entirely and migrates all callers to absl::SimpleAtoi.

Risk Level: Low
Testing: bazel test //test/...
Docs Changes: N/A
Release Notes: N/A
Part of: Issue #6580

Signed-off-by: Dan Noé <dpn@google.com>
@dnoe
Copy link
Contributor Author

dnoe commented May 6, 2019

The remaining fruit left hanging:

  1. StringUtil::atoull is still used for Base 16 hex conversions. Unfortunately absl::SimpleAtoi only does base 10. It looks like there are no remaining non-hex users of StringUtil::atoull, but there are a number of hex ones. Since atoull is just a relatively thin wrapper around std::strtoull which expects a C string this fruit hangs high.
  2. UuidUtils::uuidModBy() is one of the hex conversions, and will be an easy follow up once 1) is done.

lizan pushed a commit that referenced this issue May 7, 2019
Description: Convert `Span::setTag` and `Span::setOperation` to  take `absl::string_view` arguments. This avoids unnecessary string construction when the value string originates from a header. Zipkin ultimately requires string construction in some cases, but we can directly convert to an Opentracing `string_view` flavor without any additional construction.
Risk Level: Low
Testing: `bazel test //test/...`
Docs Changes: N/A
Release Notes: N/A
Part of: Issue #6580

Signed-off-by: Dan Noé <dpn@google.com>
@mattklein123
Copy link
Member

@dnoe one quick thought here: Maybe rename StringUtil::atoull to hexToUint or something just so we prevent new uses of base 10 conversion?

@dnoe
Copy link
Contributor Author

dnoe commented May 13, 2019

I like this idea. It would enable us to write a custom hexToUint which only has to support hex, and then if any future other odd base use cases show up then the user can construct a string and use c_str() or roll their own conversion function if it's performance critical. I suspect those future use cases are extremely rare given none exist today.

@htuch
Copy link
Member

htuch commented May 13, 2019

@mattklein123 @dnoe one thought I had was to switch everyone to safe variants of StringUtil::atoull etc. that take a bound and clamp to this. This is to prevent potential wire level attacks where header values etc. are consumed opaquely and used for memory allocation or to attempt overflow attacks. By making the only methods we export for this safe-by-construction, we would have confidence in avoiding an entire class of attacks. We could then lint for the unsafe variants.

@mattklein123
Copy link
Member

Yes agreed.

@dnoe
Copy link
Contributor Author

dnoe commented May 14, 2019

Definitely. I'm just wondering if we can skip the generic atoull functionality for now and create a safe variant that takes a bound but only does hex. It can be refactored later if conversion from other bases is required but right now it looks like 10 and 16 are the only two being used and base 10 conversion is covered by absl::SimpleAtoi.

I guess we might need to think about extensions we aren't aware of here - when we get to this it's probably worth polling the community in case someone is using octal or whatever.

@htuch
Copy link
Member

htuch commented May 14, 2019

@dnoe I'm hoping we can also avoid direct reliance on absl::SimpleAtoi. Basically, anywhere we do ASCII to integer conversion, this is reasonable place to be defensive and go through wrapped variants that do bounds checks IMHO. Maybe there will be some performance exemptions, but we can deal with these on a case-by-case basis.

@mattklein123 mattklein123 modified the milestones: 1.11.0, 1.12.0 Jul 3, 2019
@mattklein123 mattklein123 removed this from the 1.12.0 milestone Oct 10, 2019
@mattklein123 mattklein123 removed the no stalebot Disables stalebot from closing an issue label Dec 12, 2020
@github-actions
Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Jan 11, 2021
@htuch
Copy link
Member

htuch commented Jan 11, 2021

Is there any work here remaining? Is this just down to StringUtil:atoull? Would making its argument a string_view get rid of C strings outside this one util?

@github-actions github-actions bot removed the stale stalebot believes this issue/PR has not been touched recently label Jan 11, 2021
@mattklein123
Copy link
Member

Is there any work here remaining? Is this just down to StringUtil:atoull? Would making its argument a string_view get rid of C strings outside this one util?

I'm not sure of current status. This issue is pretty old so my preference would be to assign an owner and do a fresh pass if we are going to keep this open.

@github-actions
Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Feb 22, 2021
@htuch htuch added area/security help wanted Needs help! and removed stale stalebot believes this issue/PR has not been touched recently labels Feb 24, 2021
@htuch
Copy link
Member

htuch commented Feb 24, 2021

Marking help wanted, I think we should at least cleanup the atoull.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants