Skip to content

Conversation

@elsloo
Copy link
Contributor

@elsloo elsloo commented Mar 17, 2023

  • Initial attempt to backport the fix (Fixes silent header duplication that occurs upon successful revalidation when duplicate headers are present #9527) for the duplicate header bug on revalidations to 9.2.x.
  • Inner loop's auto spot2 iterator contains a member variable, _block that references the memory address of the response headers, not the cached headers
    • Method called on the cached_headers object uses a memory address, not a named header, and leads to all response headers being deleted from the duplicate header forward
    • Once response headers are deleted, the first duplicate header is written to the cached headers as a new and additional field which leads to duplication in the cached object's headers, then the current iteration ends
    • The iterator of the outer loop then attempts to increment forward one step via the overridden ++ operator on spot, and due to header slot checks via MIMEField::is_live() within MIMEHdrImpl::iterator::step(), _slot is incremented to limit and iteration of the outer loop halts, because all response headers from that point forward were deleted in the prior iteration
  • This change moves away from the address-based approach to delete the header and instead uses the header name
    • Using the header name instead of relying on header alignment also fixes a secondary issue that could arise if response header ordering does not match the cached object's header ordering
    • Using the header name is slightly less efficient due to having to call find_header, however, this is necessary to ensure the cached headers are correctly removed before the response headers are merged into the cached object
    • Using a slightly less efficient approach that occurs only on successful revalidations that also contain duplicate headers should be acceptable given the tradeoff is allowing duplication if response header ordering differs from the cached object
  • Due to the existing logic, if a cached header is not in the response, it will remain in the cached object's headers

Thanks to Masakazu and Leif for providing the unit test and helping with the fix, respectively.

…s upon successful revalidation when duplicate headers are present

- Initial attempt to backport the fix for the duplicate header bug on revalidations to 9.2.x.
- Inner loop's auto `spot2` iterator contains a member variable, `_block` that references the memory address of the response headers, not the cached headers
  - Method called on the `cached_headers` object uses a memory address, not a named header, and leads to all response headers being deleted from the duplicate header forward
  - Once response headers are deleted, the first duplicate header is written to the cached headers as a new and *additional* field which leads to duplication in the cached object's headers, then the current iteration ends
  - The iterator of the outer loop then attempts to increment forward one step via the overridden `++` operator on `spot`, and due to header slot checks via `MIMEField::is_live()` within `MIMEHdrImpl::iterator::step()`, `_slot` is incremented to `limit` and iteration of the outer loop halts, because all response headers from that point forward were deleted in the prior iteration
- This change moves away from the address-based approach to delete the header and instead uses the header name
  - Using the header name instead of relying on header alignment also fixes a secondary issue that could arise if response header ordering does not match the cached object's header ordering
  - Using the header name is slightly less efficient due to having to call `find_header`, however, this is necessary to ensure the cached headers are correctly removed before the response headers are merged into the cached object
  - Using a slightly less efficient approach that occurs only on successful revalidations that also contain duplicate headers should be acceptable given the tradeoff is allowing duplication if response header ordering differs from the cached object
- Due to the existing logic, if a cached header is *not* in the response, it will remain in the cached object's headers

Thanks to Masakazu and Leif for providing the unit test and helping with the fix, respectively.

Co-Authored-By: Masakazu Kitajo <maskit@apache.org>
Co-Authored-By: Leif Hedstrom <zwoop@apache.org>
@elsloo elsloo added the Bug label Mar 17, 2023
@elsloo elsloo requested review from bryancall and zwoop as code owners March 17, 2023 20:46
@maskit maskit added the Backport Marked for backport for an LTS patch release label Mar 17, 2023
@bneradt
Copy link
Contributor

bneradt commented Apr 3, 2023

@elsloo : can you please re-merge this to resolve conflicts.

@zwoop zwoop added this to the 9.2.1 milestone Apr 5, 2023
zwoop added a commit that referenced this pull request Apr 5, 2023
…evalidation when duplicate headers are present (#9527)"

This reverts commit c7ed799.

Instead, we'll do #9534
@zwoop zwoop merged commit 05c4d25 into apache:9.2.x Apr 5, 2023
masaori335 pushed a commit to masaori335/trafficserver that referenced this pull request Sep 26, 2023
… that occurs upon successful revalidation when duplicate headers are present (apache#9534)

* Fixes silent header duplication and early loop termination that occurs upon successful revalidation when duplicate headers are present

- Initial attempt to backport the fix for the duplicate header bug on revalidations to 9.2.x.
- Inner loop's auto `spot2` iterator contains a member variable, `_block` that references the memory address of the response headers, not the cached headers
  - Method called on the `cached_headers` object uses a memory address, not a named header, and leads to all response headers being deleted from the duplicate header forward
  - Once response headers are deleted, the first duplicate header is written to the cached headers as a new and *additional* field which leads to duplication in the cached object's headers, then the current iteration ends
  - The iterator of the outer loop then attempts to increment forward one step via the overridden `++` operator on `spot`, and due to header slot checks via `MIMEField::is_live()` within `MIMEHdrImpl::iterator::step()`, `_slot` is incremented to `limit` and iteration of the outer loop halts, because all response headers from that point forward were deleted in the prior iteration
- This change moves away from the address-based approach to delete the header and instead uses the header name
  - Using the header name instead of relying on header alignment also fixes a secondary issue that could arise if response header ordering does not match the cached object's header ordering
  - Using the header name is slightly less efficient due to having to call `find_header`, however, this is necessary to ensure the cached headers are correctly removed before the response headers are merged into the cached object
  - Using a slightly less efficient approach that occurs only on successful revalidations that also contain duplicate headers should be acceptable given the tradeoff is allowing duplication if response header ordering differs from the cached object
- Due to the existing logic, if a cached header is *not* in the response, it will remain in the cached object's headers

Thanks to Masakazu and Leif for providing the unit test and helping with the fix, respectively.

Co-Authored-By: Masakazu Kitajo <maskit@apache.org>
Co-Authored-By: Leif Hedstrom <zwoop@apache.org>

* Additional imports to fix the Debian build.

---------

Co-authored-by: Masakazu Kitajo <maskit@apache.org>
Co-authored-by: Leif Hedstrom <zwoop@apache.org>
(cherry picked from commit 05c4d25)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backport Marked for backport for an LTS patch release Bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants