Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dhcp protocol update #1254

Merged
merged 94 commits into from
Feb 6, 2025
Merged
Show file tree
Hide file tree
Changes from 69 commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
f1a9a74
WIP: add an async dhcp client
etene Jan 14, 2025
353efde
add StrEnum and async sock recv compat for python < 3.11
etene Jan 15, 2025
3d009de
remove typing.Self to restore compat with older pythons
etene Jan 15, 2025
3f34271
dnsmasq fixture: ensure consistent output language (LANG=C)
etene Jan 15, 2025
e2ce512
dhcp client: indent lease files
etene Jan 15, 2025
14eaf41
dhcp client: split send/recv into separate tasks
etene Jan 15, 2025
2176061
add ETHERTYPE_IP compat for python < 3.12
etene Jan 15, 2025
e1d2a4f
typing fixes and compat
etene Jan 15, 2025
52937a2
dhcp client: replace match/case with if/else
etene Jan 15, 2025
b6b752a
dhcp client: use Optional everywhere for py39 compat
etene Jan 15, 2025
e0566b0
improve dnsmasq fixture & handling of timeouts in dhcp client tests
etene Jan 16, 2025
2a45a26
dhcp client: add basic NAK support
etene Jan 16, 2025
ba3d62a
dhcp client: handle expired leases on loading
etene Jan 16, 2025
e941c98
dhcp client: fix coroutine warnings in timer callbacks
etene Jan 16, 2025
091056d
dhcp client: fix logic error when loading lease
etene Jan 16, 2025
1e195c4
dhcp client: add comments, improve waiting for state
etene Jan 17, 2025
417a8d9
dhcp client: add a udhcpd fixture and a test with it
etene Jan 17, 2025
c5679a0
properly do unicast & broadcast for REQUESTs according to RFC
etene Jan 20, 2025
d9128e8
dhcp: Merge pull request #1250 from etene/async-dhcp-client
svinota Jan 21, 2025
5994076
dhcp: fix & improve docstrings after review
etene Jan 22, 2025
ea56043
dhcp client: add randomized retransmission interval
etene Jan 22, 2025
9144502
dhcp: fix lease loading logic, improve docstrings
etene Jan 22, 2025
e1f5810
dhcp: use an enum for dhcp options
etene Jan 22, 2025
4b855c8
dhcp: typing + udhcpd fixture fixes
etene Jan 23, 2025
5bf05f4
dhcp: fix logic error when sending REQUESTs after init-reboot
etene Jan 23, 2025
298050e
dhcp: properly set bootp flags
etene Jan 23, 2025
000963e
dhcp: fix DHCPRELEASE messages
etene Jan 23, 2025
304e666
dhcp: add missing options
etene Jan 23, 2025
ce2da1e
dhcp: use enums instead of magic numbers in encoding/decoding code
etene Jan 23, 2025
8512964
dhcp: make dnsmasq fixture more verbose
etene Jan 24, 2025
93a777b
wip: dhcp: add unit test for packet decoding + small fixes
etene Jan 24, 2025
ca9d3ff
wip: move test_raw to test_dhcp
etene Jan 24, 2025
2804a0d
dhcp: add --authoritative option to the dnsmasq fixture
etene Jan 24, 2025
a75b881
dhcp client: define mtu dhcp option
BrianBaboch Jan 24, 2025
c8b132a
dhcp: prevent the client from getting stuck
etene Jan 27, 2025
d5ff980
dhcp: improve & move some tests
etene Jan 27, 2025
4447788
dhcp: add a decoding test for an android dhcp client request
etene Jan 27, 2025
03178b5
dhcp: add a decoding test for a wii dhcp discover message
etene Jan 27, 2025
7bfffdb
dhcp: fix string option parsing bug
etene Jan 27, 2025
b492168
dhcp: set 'secs' in messages, fix its endianness
etene Jan 27, 2025
ff80277
dhcp client: convert dotted decimal mask to CIDR
BrianBaboch Jan 24, 2025
6ce20a4
dhcp client: routers property should return a list of routers
BrianBaboch Jan 24, 2025
b3a25ee
dhcp client: add a default gateway property
BrianBaboch Jan 24, 2025
289aa80
dhcp client: add broadcast and mtu properties
BrianBaboch Jan 24, 2025
50df3db
dhcp client: fix typing and use get to avoid exceptions
BrianBaboch Jan 24, 2025
4c47a4b
dhcp client: implement hook to configure IP
BrianBaboch Jan 24, 2025
fb3d7db
dhcp client: we want unbound the hooks in reverse order
BrianBaboch Jan 24, 2025
b265592
dhcp client: implement a hook to add default route
BrianBaboch Jan 24, 2025
4a9578c
dhcp client: define dhcp exceptions
BrianBaboch Jan 27, 2025
2c658c6
dhcp client: raise DHCPOptionMissingError if subnet mask option is no…
BrianBaboch Jan 27, 2025
7eeb28a
Merge pull request #1257 from BrianBaboch/add-hooks-and-other-improve…
etene Jan 28, 2025
558a395
dhcp: small post-merge fixes & comments
etene Jan 28, 2025
2dbd1be
dhcp: refactor Options & Parameters, add missing values
etene Jan 28, 2025
68925d8
dhcp: add a unit test for lease renewing
etene Jan 28, 2025
b713e5c
dhcp: add EnumType compat for py311, linter fixes
etene Jan 29, 2025
bb030b5
dhcp: fix enum mess, try to find a workaround that's valid for all py…
etene Jan 29, 2025
3fe4d5a
dhcp client: send queued messages before stopping
etene Jan 29, 2025
062c1be
WIP: dhcp: refactor transaction id handling
etene Jan 29, 2025
1939fd5
dhcp client: allow to specify xid
svinota Jan 29, 2025
d684cb4
format: fix linting
svinota Jan 29, 2025
27e8cd0
dhcp xid: fix annotation for python < 3.10
svinota Jan 29, 2025
634ee39
ci: fix dhcp unit test fixture for python < 3.10
svinota Jan 29, 2025
1643463
dhcp: allow monkeypatching xids for tests
etene Jan 29, 2025
4b9d731
dhcp: rewrite hooks to be more flexible
etene Jan 30, 2025
1ca5815
ci: fast/offline nox sessions
svinota Jan 30, 2025
26cd5f7
dhcp client: add an option to write a pidfile
etene Jan 31, 2025
85a4bcc
dhcp client: allow trying once to get a lease then exiting, like dhcl…
etene Jan 31, 2025
d1862ae
dhcp: use pyroute2 instead of iproute2 calls in veth test fixture
etene Feb 1, 2025
ecb8bb1
dhcp: add a parser test for a washing machine REQUEST
etene Feb 1, 2025
3254c13
dhcp: clean up stuff, add comments, FIXME & TODOs to prepare for firs…
etene Feb 3, 2025
493f573
dhcp: add unit test for client hooks
etene Feb 3, 2025
100679c
dhcp: add test for the default gw client hooks
etene Feb 3, 2025
580b2e6
dhcp: fix (and skip) default gw hook test
etene Feb 4, 2025
a431a72
dhcp: add a unit test for NAK in init-reboot state
etene Feb 4, 2025
36fc51b
dhcp client: make sending a RELEASE configurable
etene Feb 4, 2025
c09b762
dhcp client: handle interface state changes without crashing
etene Feb 4, 2025
7d2bd75
dhcp: add unit test for request timeout, fix linter
etene Feb 5, 2025
086ab1f
dhcp: use SystemRandom for xids
etene Feb 5, 2025
12b4b64
dhcp: use time.time instead of datetime for lease timestamps
etene Feb 5, 2025
65122d3
dhcp: improve option & parameter enum declarations
etene Feb 5, 2025
d6a7125
dhcp: add test, improve fixtures
etene Feb 5, 2025
aa0ff5f
dhcp: fix tests for older python versions
etene Feb 5, 2025
8deea4a
dhcp: factorize dhcp option access in leases
etene Feb 5, 2025
4c0be0b
dhcp: refactor cli main()/run_client()
etene Feb 5, 2025
1828465
dhcp: fix python compat for cli.py
etene Feb 5, 2025
b036527
dhcp: add a test for a flapping interface + option to disable hooks
etene Feb 5, 2025
cae84a2
dhcp: add test for the client pidfile
etene Feb 5, 2025
a50a02c
dhcp: fix linter, more TimeoutError fixes (?) for older pythons
etene Feb 5, 2025
073d67e
dhcp: add options to lease, add hook timeout, try to fix flaky test
etene Feb 5, 2025
deab017
dhcp: typing & style fixes
etene Feb 5, 2025
c5a8f20
dhcp: add wrong xid test, improve & test cli errors
etene Feb 6, 2025
2f2976e
dhcp: add a test for unexpected DHCP message, some client refactoring
etene Feb 6, 2025
ea077f6
dhcp: add missing test pcaps
etene Feb 6, 2025
e7b5783
dhcp: don't crash ip addr hooks when server doesn't provide a broadca…
etene Feb 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,5 @@ benchmark.log
venv
.venv
.nox*
tests/*.db
tests/*.json
16 changes: 12 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,20 @@ python ?= $(shell util/find_python.sh)
platform := $(shell uname -s)
releaseTag ?= $(shell git describe --tags --abbrev=0)
releaseDescription := $(shell git tag -l -n1 ${releaseTag} | sed 's/[0-9. ]\+//')
noxboot ?= ~/.venv-boot

define nox
{\
which nox 2>/dev/null || {\
${python} -m venv ~/.venv-boot/;\
. ~/.venv-boot/bin/activate;\
pip install --upgrade pip;\
pip install nox;\
test -d ${noxboot} && \
{\
. ${noxboot}/bin/activate;\
} || {\
${python} -m venv ${noxboot};\
. ${noxboot}/bin/activate;\
pip install --upgrade pip;\
pip install nox;\
};\
};\
nox $(1) -- '${noxconfig}';\
}
Expand Down Expand Up @@ -46,6 +52,8 @@ clean:
@rm -rf lab/_build
@rm -rf docs/html
@rm -rf docs/man
@rm -f tests/*.db
@rm -f tests/*.json
@rm -rf dist build MANIFEST
@rm -f docs-build.log
@rm -rf pyroute2.egg-info
Expand Down
86 changes: 49 additions & 37 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,18 @@
]


def load_global_config():
if sys.argv[-2] == '--' and len(sys.argv[-1]):
return json.loads(sys.argv[-1])
return {}


global_config = load_global_config()
if global_config.get('fast'):
nox.options.reuse_venv = 'yes'
nox.options.no_install = True


def add_session_config(func):
'''Decorator to load the session config.

Expand All @@ -55,12 +67,7 @@ def my_session_func(session, config):
'''

def wrapper(session):
if session.posargs and len(session.posargs[0]) > 0:
config = json.loads(session.posargs[0])
else:
config = {}
session.debug(f'session config: {config}')
return func(session, config)
return func(session, global_config)

wrapper.__name__ = func.__name__
wrapper.__doc__ = func.__doc__
Expand Down Expand Up @@ -151,14 +158,21 @@ def setup_venv_minimal(session, config):
return tmpdir


def setup_venv_common(session, flavour='dev'):
session.install('--upgrade', 'pip')
session.install('-r', f'requirements.{flavour}.txt')
session.install('.')
def setup_venv_common(session, flavour='dev', config=None):
if config is None:
config = {}
if not config.get('fast'):
session.install('--upgrade', 'pip')
session.install('-r', f'requirements.{flavour}.txt')
session.install('.')
return os.path.abspath(session.create_tmp())


def setup_venv_dev(session):
def setup_venv_dev(session, config=None):
if config is None:
config = {}
if config.get('fast'):
return os.getcwd()
tmpdir = setup_venv_common(session)
session.run('cp', '-a', 'tests', tmpdir, external=True)
session.run('cp', '-a', 'examples', tmpdir, external=True)
Expand All @@ -185,8 +199,8 @@ def setup_venv_repo(session):
return tmpdir


def setup_venv_docs(session):
tmpdir = setup_venv_common(session, 'docs')
def setup_venv_docs(session, config=None):
tmpdir = setup_venv_common(session, flavour='docs', config=config)
session.run('cp', '-a', 'docs', tmpdir, external=True)
session.run('cp', '-a', 'examples', tmpdir, external=True)
[
Expand All @@ -209,9 +223,10 @@ def test_platform(session):


@nox.session(python='python3.10')
def docs(session):
@add_session_config
def docs(session, config):
'''Generate project docs.'''
tmpdir = setup_venv_docs(session)
tmpdir = setup_venv_docs(session, config)
cwd = os.path.abspath(os.getcwd())
# man pages
session.chdir(f'{tmpdir}/docs/')
Expand All @@ -232,9 +247,11 @@ def docs(session):


@nox.session
def linter(session):
@add_session_config
def linter(session, config):
'''Run code checks and linters.'''
session.install('pre-commit')
if not config.get('fast'):
session.install('pre-commit')
session.run('pre-commit', 'run', '-a')


Expand Down Expand Up @@ -262,36 +279,31 @@ def integration(session, config):
session.run(*options('test_integration', config))


def test_common(session, config, module):
setup_linux(session)
workspace = setup_venv_dev(session, config)
path = f'{workspace}/tests/mocklib'
if config.get('fast'):
path += f':{workspace}'
session.chdir('tests')
session.run(
*options(module, config),
env={'WORKSPACE': workspace, 'SKIPDB': 'postgres', 'PYTHONPATH': path},
)


@nox.session(python=['python3.9', 'python3.10', 'python3.11', 'python3.12'])
@add_session_config
def linux(session, config):
'''Run Linux functional tests. Requires root to run all the tests.'''
setup_linux(session)
workspace = setup_venv_dev(session)
session.run(
*options('test_linux', config),
env={
'WORKSPACE': workspace,
'SKIPDB': 'postgres',
'PYTHONPATH': f'{workspace}/tests/mocklib',
},
)
test_common(session, config, 'test_linux')


@nox.session(python=['python3.10', 'python3.11', 'python3.12'])
@add_session_config
def core(session, config):
'''Run Linux tests in asyncio.'''
setup_linux(session)
workspace = setup_venv_dev(session)
session.run(
*options('test_core', config),
env={
'WORKSPACE': workspace,
'SKIPDB': 'postgres',
'PYTHONPATH': f'{workspace}/tests/mocklib',
},
)
test_common(session, config, 'test_core')


@nox.session
Expand Down
19 changes: 19 additions & 0 deletions pyroute2/compat.py
svinota marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
'''Compatibility with older but supported Python versions'''

try:
from enum import StrEnum # noqa: F401
except ImportError:
from enum import Enum

class StrEnum(str, Enum):
'''Same as enum, but members are also strings.'''


try:
from socket import ETHERTYPE_IP
except ImportError:
# ETHERTYPE_* are new in python 3.12
ETHERTYPE_IP = 0x800


__all__ = ('StrEnum', 'ETHERTYPE_IP')
44 changes: 21 additions & 23 deletions pyroute2/dhcp/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,18 +105,7 @@ class array8(option):
from pyroute2.common import basestring
from pyroute2.protocols import msg

BOOTREQUEST = 1
BOOTREPLY = 2

DHCPDISCOVER = 1
DHCPOFFER = 2
DHCPREQUEST = 3
DHCPDECLINE = 4
DHCPACK = 5
DHCPNAK = 6
DHCPRELEASE = 7
DHCPINFORM = 8

from .enums.dhcp import MessageType, Option

if not hasattr(array, 'tobytes'):
# Python2 and Python3 versions of array differ,
Expand Down Expand Up @@ -194,7 +183,7 @@ def decode(self):
isinstance(value, basestring)
and self.policy['format'] == 'string'
):
value = value[: value.find(b'\x00')]
value = value.lstrip(b"\x00")
self.value = value
else:
# remember current offset as msg.decode() will advance it
Expand All @@ -216,8 +205,8 @@ class dhcpmsg(msg):
_decode_map = {}

def _register_options(self):
for option in self.options:
code, name, fmt = option[:3]
for code, fmt in self.options:
name = code.name.lower()
self._decode_map[code] = self._encode_map[name] = {
'name': name,
'code': code,
Expand All @@ -232,10 +221,10 @@ def decode(self):
code = struct.unpack('B', self.buf[self.offset : self.offset + 1])[
0
]
if code == 0:
if code == Option.PAD:
self.offset += 1
continue
if code == 255:
if code == Option.END:
return self
# code is unknown -- bypass it
if code not in self._decode_map:
Expand All @@ -262,19 +251,25 @@ def encode(self):
self._register_options()
# put message type
options = self.get('options') or {
'message_type': DHCPDISCOVER,
'parameter_list': [1, 3, 6, 12, 15, 28],
'message_type': MessageType.DISCOVER,
'parameter_list': [1, 3, 6, 12, 15, 28], # FIXME
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some coming PRs I have to change to parameter values from the enums

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could do it but i'm starting to think maybe the parameter list should be left empty at this level by default, since there are already default parameters set at the client & cli levels, and one might very well want to send DHCP messages without parameters for testing purposes.

}

self.buf += (
self.uint8(code=53, value=options['message_type']).encode().buf
self.uint8(code=Option.MESSAGE_TYPE, value=options['message_type'])
.encode()
.buf
)
self.buf += (
self.client_id({'type': 1, 'key': self['chaddr']}, code=61)
self.client_id(
{'type': 1, 'key': self['chaddr']}, code=Option.CLIENT_ID
)
.encode()
.buf
)
self.buf += self.string(code=60, value='pyroute2').encode().buf
self.buf += (
self.string(code=Option.VENDOR_ID, value='pyroute2').encode().buf
)

for name, value in options.items():
if name in ('message_type', 'client_id', 'vendor_id'):
Expand All @@ -294,7 +289,7 @@ def encode(self):
)
self.buf += option.encode().buf

self.buf += self.none(code=255).encode().buf
self.buf += self.none(code=Option.END).encode().buf
return self

class none(option):
Expand All @@ -321,3 +316,6 @@ class array8(option):

class client_id(option):
fields = (('type', 'uint8'), ('key', 'l2addr'))

class message_type(option):
policy = {'format': 'B', 'decode': MessageType}
Loading