Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct timestamp handling for all fields and options #42

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

martinling
Copy link

@martinling martinling commented Nov 8, 2024

Up to release 2.0.0, the variable timestamp formats in PcapNg were not handled at all.

PR #37 added support for variable resolution for the timestamp field in ExpandedPacketBlock, but this only addressed one aspect of the timestamping issues.

  • There are also timestamps in the InterfaceStatisticsBlock, the IsbStartTime and IsbEndTime options, and the obsolete but still supported PacketBlock, all of which are defined to follow the same encoding/decoding rules.
  • The timestamp resolution is not the only thing affecting encoding/decoding. There is also the IfTsOffset option, which defines an offset in seconds that must be added or subtracted to relate timestamps to absolute time.

The previous fix also introduced some new problems:

  • The EnhancedPacketBlock type became no longer constructable, because of the new private write_ts_resolution field, as observed in v3.0.0-rc1 feedback #39. This complicates constructing new packets to save to a file.
  • A PcapError::TimestampTooBig variant was added, but could never be returned, because the paths that lead to this failure were only able to return std::io::Error.

This PR introduces a broader solution that addresses all of the above problems. The fundamental issue is that correctly encoding or decoding a block requires knowledge of the protocol state.

Currently, that state is maintained in the section, interfaces and ts_resolutions fields on PcapNgParser and PcapNgWriter. The first step in this PR is to centralise that state into a common PcapNgState type. All work necessary to update that state is centralised into its update_from_block method.

All APIs that convert to and from Block are modified to require a reference to a PcapNgState. This allows encoding and decoding to be state-dependent. Timestamp encoding and decoding can then be correctly implemented wherever timestamps appear, by calling into helper methods on PcapNgState in the from_slice and write_to methods of each block type. This eliminates the need to fix things up outside those methods.

All timestamps which appear in blocks and options are now represented as a Duration.

This approach requires some one-off API changes, but provides a full solution, and also allows for correctly implementing any other state-dependent encoding/decoding of other blocks and options in the future.

Block::from_slice(), Block::write_to(), and all conversions from
RawBlock to Block require a PcapNgState, because without state
information the encoding is ambiguous.

This was previously handled in an ad-hoc manner in the reader, writer
and parser but is now centralised into the PcapNgState type.

The previous functions for adjusting EPB timestamps are removed as they
are no longer necessary. Timestamps are now converted as required in
from_slice() and write_to(), according to the current PcapNgState.
This field follows the same rules as the EPB timestamp.
This field follows the same rules as the EPB timestamp.
This is necessary to return InvalidInterfaceId and TimestampTooBig.
@martinling martinling mentioned this pull request Nov 8, 2024
Copy link

@caspermeijn caspermeijn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like an interesting improvement.

  • It is a big PR, which makes it difficult to review. Does it make sense to split it into smaller PRs to make reviewing easier?
  • I would like to see some tests that verify the behavior, especially the endianess.

@@ -170,7 +126,7 @@ pub enum EnhancedPacketOption<'a> {
}

impl<'a> PcapNgOption<'a> for EnhancedPacketOption<'a> {
fn from_slice<B: ByteOrder>(code: u16, length: u16, mut slice: &'a [u8]) -> Result<Self, PcapError> {
fn from_slice<B: ByteOrder>(_state: &PcapNgState, _interface_id: Option<u32>, code: u16, length: u16, mut slice: &'a [u8]) -> Result<Self, PcapError> {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like that this function takes a state and interface_id, but doesn't use it. I see that it is needed because this is a trait. But, I don't see a better solution.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty common to have unused parameters when implementing traits. The trait has to include all parameters that might be necessary, and in the general case, decoding an option correctly may require knowledge of the state as well as which interface is involved.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I understand. In particular, the optional interface_id is a bit awkward in a context where there is no interface defined. As I said, I don't see a better solution.

src/errors.rs Show resolved Hide resolved
src/pcapng/blocks/interface_statistics.rs Show resolved Hide resolved
src/pcapng/blocks/interface_statistics.rs Outdated Show resolved Hide resolved
src/pcapng/state.rs Show resolved Hide resolved
src/pcapng/state.rs Show resolved Hide resolved
@martinling
Copy link
Author

martinling commented Nov 15, 2024

It is a big PR, which makes it difficult to review. Does it make sense to split it into smaller PRs to make reviewing easier?

My suggestion would be to review it commit-by-commit.

I have already taken care to break it down into as many logical steps as possible, and the tests should pass at each commit.

I would like to see some tests that verify the behavior, especially the endianess.

It passes all the existing tests including those added in #37. But it would be good to have some more test cases as well, particularly some covering new features like the IfTsOffset handling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants