Skip to content

Commit 6f8f523

Browse files
AlanQuatermainLukasa
authored andcommitted
Implementation of HPACK encoding and header tables (#10)
Motivation: This is the beginning of removing the dependency on `nghttp2`. Modifications: I've added a new library target, `NIOHPACK`, which contains all the necessary pieces to implement HPACK encoding as specified in RFC 7541. There are unit tests exercising all the code and some additional edge cases around integer encoding, along with some performance tests for the Huffman encode/decode process. The latter currently takes longer to decode than to encode, which is worrying, but this is only really noticeable when encoding > 16KB strings as a rule; therefore I've decided not to pull out the optimization hammer just yet. Included in the unit tests are the inputs and outputs from all the examples in RFC 7541 and RFC 7540. I've tried to match the coding style from the repo, but some of my own idioms may have crept in here & there. Please let me know if you see anything wrong in that regard. Currently the static header table and Huffman encoding tables are implemented as Swift `let`s. For the static header table that's ok (only 62 entries), but the Huffman table is huge. I'm debating whether it's worth having that in a plain C file somewhere, but I'm not sure what the tradeoffs are vs. performance. Right now it seems to only affect compilation time, but I may be wrong. Result: In general, nothing will change: nothing in NIOHTTP2 is using this library yet. That part is what I'm working on now.
1 parent b7085b6 commit 6f8f523

34 files changed

+9857
-3
lines changed

.gitmodules

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[submodule "hpack-test-case"]
2+
path = hpack-test-case
3+
url = git@github.com:http2jp/hpack-test-case

.mailmap

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
11
Johannes Weiß <johannesweiss@apple.com>
22
<cbenfield@apple.com> <lukasaoz@gmail.com>
33
<cbenfield@apple.com> <lukasa@apple.com>
4+
<jimdovey@mac.com> <jdovey@linkedin.com>
5+
<tomer@apple.com> <tomer.doron@gmail.com>

CONTRIBUTORS.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,10 @@ needs to be listed here.
1212
### Contributors
1313

1414
- Cory Benfield <cbenfield@apple.com>
15+
- Jim Dovey <jimdovey@mac.com>
1516
- Johannes Weiß <johannesweiss@apple.com>
17+
- Tim <0xTim@users.noreply.github.com>
18+
- tomer doron <tomer@apple.com>
1619

1720
**Updating this list**
1821

Package.swift

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@ let package = Package(
1919
name: "swift-nio-http2",
2020
products: [
2121
.executable(name: "NIOHTTP2Server", targets: ["NIOHTTP2Server"]),
22-
.library(name: "NIOHTTP2", targets: ["NIOHTTP2"])
22+
.library(name: "NIOHTTP2", targets: ["NIOHTTP2"]),
2323
],
2424
dependencies: [
25-
.package(url: "https://github.com/apple/swift-nio.git", from: "1.7.0"),
25+
.package(url: "https://github.com/apple/swift-nio.git", from: "1.9.0"),
2626
.package(url: "https://github.com/apple/swift-nio-nghttp2-support.git", from: "1.0.0"),
2727
],
2828
targets: [
@@ -31,7 +31,11 @@ let package = Package(
3131
dependencies: ["NIOHTTP2"]),
3232
.target(name: "NIOHTTP2",
3333
dependencies: ["NIO", "NIOHTTP1", "NIOTLS", "CNIONghttp2"]),
34+
.target(name: "NIOHPACK",
35+
dependencies: ["NIO", "NIOConcurrencyHelpers", "NIOHTTP1"]),
3436
.testTarget(name: "NIOHTTP2Tests",
3537
dependencies: ["NIO", "NIOHTTP1", "NIOHTTP2"]),
38+
.testTarget(name: "NIOHPACKTests",
39+
dependencies: ["NIOHPACK"])
3640
]
3741
)
Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
//===----------------------------------------------------------------------===//
2+
//
3+
// This source file is part of the SwiftNIO open source project
4+
//
5+
// Copyright (c) 2017-2018 Apple Inc. and the SwiftNIO project authors
6+
// Licensed under Apache License v2.0
7+
//
8+
// See LICENSE.txt for license information
9+
// See CONTRIBUTORS.txt for the list of SwiftNIO project authors
10+
//
11+
// SPDX-License-Identifier: Apache-2.0
12+
//
13+
//===----------------------------------------------------------------------===//
14+
15+
import NIO
16+
17+
/// Implements the dynamic part of the HPACK header table, as defined in
18+
/// [RFC 7541 § 2.3](https://httpwg.org/specs/rfc7541.html#dynamic.table).
19+
struct DynamicHeaderTable {
20+
public static let defaultSize = 4096
21+
22+
/// The actual table, with items looked up by index.
23+
private var storage: HeaderTableStorage
24+
25+
/// The length of the contents of the table.
26+
var length: Int {
27+
return self.storage.length
28+
}
29+
30+
/// The size to which the dynamic table may currently grow. Represents
31+
/// the current maximum length signaled by the peer via a table-resize
32+
/// value at the start of an encoded header block.
33+
///
34+
/// - note: This value cannot exceed `self.maximumTableLength`.
35+
var allowedLength: Int {
36+
get {
37+
return self.storage.maxSize
38+
}
39+
set {
40+
self.storage.setTableSize(to: newValue)
41+
}
42+
}
43+
44+
/// The maximum permitted size of the dynamic header table as set
45+
/// through a SETTINGS_HEADER_TABLE_SIZE value in a SETTINGS frame.
46+
var maximumTableLength: Int {
47+
didSet {
48+
if self.allowedLength > maximumTableLength {
49+
self.allowedLength = maximumTableLength
50+
}
51+
}
52+
}
53+
54+
/// The number of items in the table.
55+
var count: Int {
56+
return self.storage.count
57+
}
58+
59+
init(allocator: ByteBufferAllocator, maximumLength: Int = DynamicHeaderTable.defaultSize) {
60+
self.storage = HeaderTableStorage(allocator: allocator, maxSize: maximumLength)
61+
self.maximumTableLength = maximumLength
62+
self.allowedLength = maximumLength // until we're told otherwise, this is what we assume the other side expects.
63+
}
64+
65+
/// Subscripts into the dynamic table alone, using a zero-based index.
66+
subscript(i: Int) -> HeaderTableEntry {
67+
return self.storage[i]
68+
}
69+
70+
func view(of index: HPACKHeaderIndex) -> ByteBufferView {
71+
return self.storage.view(of: index)
72+
}
73+
74+
// internal for testing
75+
func dumpHeaders() -> String {
76+
return self.storage.dumpHeaders(offsetBy: StaticHeaderTable.count)
77+
}
78+
79+
// internal for testing -- clears the dynamic table
80+
mutating func clear() {
81+
self.storage.purge(toRelease: self.storage.length)
82+
}
83+
84+
/// Searches the table for a matching header, optionally with a particular value. If
85+
/// a match is found, returns the index of the item and an indication whether it contained
86+
/// the matching value as well.
87+
///
88+
/// Invariants: If `value` is `nil`, result `containsValue` is `false`.
89+
///
90+
/// - Parameters:
91+
/// - name: The name of the header for which to search.
92+
/// - value: Optional value for the header to find. Default is `nil`.
93+
/// - Returns: A tuple containing the matching index and, if a value was specified as a
94+
/// parameter, an indication whether that value was also found. Returns `nil`
95+
/// if no matching header name could be located.
96+
func findExistingHeader<Name: Collection, Value: Collection>(named name: Name, value: Value?) -> (index: Int, containsValue: Bool)? where Name.Element == UInt8, Value.Element == UInt8 {
97+
// looking for both name and value, but can settle for just name if no value
98+
// has been provided. Return the first matching name (lowest index) in that case.
99+
guard let value = value else {
100+
// no `first` on AnySequence, just `first(where:)`
101+
return self.storage.firstIndex(matching: name).map { ($0, false) }
102+
}
103+
104+
// If we have a value, locate the index of the lowest header which contains that
105+
// value, but if no value matches, return the index of the lowest header with a
106+
// matching name alone.
107+
var firstNameMatch: Int? = nil
108+
for index in self.storage.indices(matching: name) {
109+
if firstNameMatch == nil {
110+
// record the first (most recent) index with a matching header name,
111+
// in case there's no value match.
112+
firstNameMatch = index
113+
}
114+
115+
if self.storage.view(of: self.storage[index].value).matches(value) {
116+
// this entry has both the name and the value we're seeking
117+
return (index, true)
118+
}
119+
}
120+
121+
// no value matches -- but did we find a name?
122+
if let index = firstNameMatch {
123+
return (index, false)
124+
} else {
125+
// no matches at all
126+
return nil
127+
}
128+
}
129+
130+
/// Appends a header to the table. Note that if this succeeds, the new item's index
131+
/// is always zero.
132+
///
133+
/// This call may result in an empty table, as per RFC 7541 § 4.4:
134+
/// > "It is not an error to attempt to add an entry that is larger than the maximum size;
135+
/// > an attempt to add an entry larger than the maximum size causes the table to be
136+
/// > emptied of all existing entries and results in an empty table."
137+
///
138+
/// - Parameters:
139+
/// - name: A collection of UTF-8 code points comprising the name of the header to insert.
140+
/// - value: A collection of UTF-8 code points comprising the value of the header to insert.
141+
/// - Returns: `true` if the header was added to the table, `false` if not.
142+
mutating func addHeader<Name: Collection, Value: Collection>(named name: Name, value: Value) throws where Name.Element == UInt8, Value.Element == UInt8 {
143+
do {
144+
try self.storage.add(name: name, value: value)
145+
} catch let error as RingBufferError.BufferOverrun {
146+
// ping the error up the stack, with more information
147+
throw NIOHPACKErrors.FailedToAddIndexedHeader(bytesNeeded: self.storage.length + error.amount,
148+
name: name, value: value)
149+
}
150+
}
151+
152+
/// Appends a header to the table. Note that if this succeeds, the new item's index
153+
/// is always zero.
154+
///
155+
/// This call may result in an empty table, as per RFC 7541 § 4.4:
156+
/// > "It is not an error to attempt to add an entry that is larger than the maximum size;
157+
/// > an attempt to add an entry larger than the maximum size causes the table to be
158+
/// > emptied of all existing entries and results in an empty table."
159+
///
160+
/// - Parameters:
161+
/// - name: A contiguous collection of UTF-8 bytes comprising the name of the header to insert.
162+
/// - value: A contiguous collection of UTF-8 bytes comprising the value of the header to insert.
163+
/// - Returns: `true` if the header was added to the table, `false` if not.
164+
mutating func addHeader<Name: ContiguousCollection, Value: ContiguousCollection>(nameBytes: Name, valueBytes: Value) throws where Name.Element == UInt8, Value.Element == UInt8 {
165+
do {
166+
try self.storage.add(nameBytes: nameBytes, valueBytes: valueBytes)
167+
} catch let error as RingBufferError.BufferOverrun {
168+
// convert the error to something more useful/meaningful to client code.
169+
throw NIOHPACKErrors.FailedToAddIndexedHeader(bytesNeeded: self.storage.length + error.amount,
170+
name: nameBytes, value: valueBytes)
171+
}
172+
}
173+
}

0 commit comments

Comments
 (0)