loggerd: extract `write_encode_data` and eliminate recursion from `handle_encoder_msg` #33219

deanlee · 2024-08-07T19:46:14Z

Changes:

Refactored handle_encoder_msg: Moved the encoding packet writing logic to a new function, write_encode_data, improving code clarity and separation of concerns while preserving the existing writing behavior.
Eliminated Recursive Calls: Replaced recursive calls within handle_encoder_msg with direct calls to write_encode_data for writing queued packets in the new segment. This change prevents potential crashes by handling queued packets safely, even if they are out of sync with the encoderd.
Improved Memory Management: Replaced raw pointers with std::unique_ptr for managing Message objects, improving memory safety.
Removed Redundant Parameter: Removed the service.name parameter from handle_encoder_msg, as it duplicates the re.publish_name value already available through the re parameter.

Resolves #28857

* manual matrix exp * remove control

…3123) Export fingerprints

…ai#33108) * ensure proper resource management with io.BytesIO * improve

* Add Support for Decompressing ZST Log Files * 2 space and check magic number * match BZ2 --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Filter out non-relevant services in most_messages_valid check * Set as comprehension * Update segments * Update ref commit * New check for hyundai2

* Use livePose instead of llk * Update process replay sockets * Fix import * Fix calib * Fix field name * Dont store device_from_calib * Update ref commit

qlog_size: use zstd

* Use livePose * Replace it in process replay * Add liveCalibration to messages * Update ref commit

* test * update refs to zst * update

fix multiprocessing issue with can_replay.py on mac

* unlock numpy * remove old cache * sheel * lock * remove temp * depends * export * put this back * use __version__ * move this --------- Co-authored-by: quebec <quebec@quebec.attlocal.net>

* md * Update README.md * rename * exclude * Update README.md

* ford and gm * clean that up * also this * honda * temp fix * move into selfdrive.car * clean up * more

use helper

* deprecate busTime * bump * do car can + more * forgot some * bump opendbc * fix that too * bump

This reverts commit 7824074.

* this should be common * super! * bump * and frame is common * bump

* use new opendbc api * export pandad_python * merge master * merge master * bump opendbc * bump opendbc * improve func * keep interface unchanged * fix test_car_interfaces * bump opendbc * bump opendbc * fix test_models * the interface now has to convert from can capnp to list, so we should include this time * goes from ~210 to ~240 mean ms real time * remoe busTime * lowercase sendcan * consistent msgtype * bump * bump * not used in lat_mpc * space * bump to master --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>

This reverts commit 0dddc97.

* improve * update readme * stdin * better * allow non interactive * specify dir

* remove hexdump * uv lock * cleanup

* add the test fix * bump * not sure why mypy didn't catch this

This reverts commit ac13000.

* minimal * uv.lock * remove anisotropic filtering * new wheel

* state what we support * more * wording

* Wellcome comment just for first timers * Update auto_pr_review.yaml https://github.com/actions/first-interaction/blob/main/README.md

* controlsd: deprecate alertBlinkingRate * simplify * update refs

* setup selfdriveState * little more * update refs * migration * all too slow

add controlsState back

* personality-alerts * Update events.py * Update car.capnp * no AudibleAlert * shorter * no VisualAlert * one liner * smaller * normal * what am I doing wrong? * it works, but is it messy? * arg all * lil more * update that --------- Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

Show lead car in on-road screenshots

…parsing (commaai#33425) Migrate Old ControlState Event to New Format During Log File Parsing

…ommaai#33426) check selfdrive_state

…ommaai#33408) add offroad alerts

remove copy initialization of dialogs

* athena: use dongle ID on /persist/ when available * comment * test * cleanup

add body

…rRenderer` class (commaai#33375) * Refactor Driver Monitor Updating and Rendering add comments * rebase master * rename dmon to driver_monitoring

github-actions · 2024-09-02T01:57:29Z

This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.

maxime-desroches · 2024-09-04T06:23:27Z

This PR was closed because of a git history rewrite.
Please read #33399 for what to do with your fork and your PRs.

maxime-desroches and others added 30 commits July 29, 2024 11:52

Remove control usage (commaai#33103)

01c0756

* manual matrix exp * remove control

[bot] Fingerprints: add missing FW versions from new users (commaai#3…

606890c

…3123) Export fingerprints

uploader.py: ensure proper resource management with io.BytesIO (comma…

2728c95

…ai#33108) * ensure proper resource management with io.BytesIO * improve

update mac_setup.sh brew install (commaai#33131)

7d6ff19

replay: add support for decompressing ZST log files (commaai#32910)

ade1372

* Add Support for Decompressing ZST Log Files * 2 space and check magic number * match BZ2 --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>

qlog: remove mag and less CAN

43dcddc

process_replay: regenerate locationd logs (commaai#33134)

6e185f4

* Filter out non-relevant services in most_messages_valid check * Set as comprehension * Update segments * Update ref commit * New check for hyundai2

paramsd: use livePose (commaai#33099)

84cff4f

* Use livePose instead of llk * Update process replay sockets * Fix import * Fix calib * Fix field name * Dont store device_from_calib * Update ref commit

qlog_size.py: use zstd (commaai#33137)

11cb2d3

qlog_size: use zstd

torqued: use livePose (commaai#33136)

86d8d1d

* Use livePose * Replace it in process replay * Add liveCalibration to messages * Update ref commit

process replay: use zst (commaai#33138)

dfd3875

* test * update refs to zst * update

fix multiprocessing issue with can_replay.py on macOS (commaai#33139)

4e73b76

fix multiprocessing issue with can_replay.py on mac

add roadmap to readme

0b29db5

unpin numpy (commaai#33132)

9536dd0

* unlock numpy * remove old cache * sheel * lock * remove temp * depends * export * put this back * use __version__ * move this --------- Co-authored-by: quebec <quebec@quebec.attlocal.net>

docs: fix README.md (commaai#33142)

086dcfe

* md * Update README.md * rename * exclude * Update README.md

test_onroad: lower lower log size bound

0fe143e

cars: remove some external imports (commaai#33133)

0739d79

* ford and gm * clean that up * also this * honda * temp fix * move into selfdrive.car * clean up * more

CI: increase shm size to 2G

76fd5b0

Ford: add long controls todo

661ef03

tester present msgs: use helper (commaai#33147)

b6d124d

use helper

deprecate busTime (commaai#33146)

42f2601

* deprecate busTime * bump * do car can + more * forgot some * bump opendbc * fix that too * bump

bump

7824074

Revert "bump"

80f9278

This reverts commit 7824074.

CarController: call to super's init (commaai#33148)

1b6ac2d

* this should be common * super! * bump * and frame is common * bump

Revert "update to new opendbc API (commaai#32009)"

ac13000

This reverts commit 0dddc97.

setup.sh: retry + timing (commaai#33145)

bf003f2

* improve * update readme * stdin * better * allow non interactive * specify dir

Remove hexdump package (commaai#33155)

1dd8bdc

* remove hexdump * uv lock * cleanup

get_fw_versions: test the full stack (commaai#33156)

ebff7ca

* add the test fix * bump * not sure why mypy didn't catch this

Reapply "update to new opendbc API (commaai#32009)" (commaai#33151)

7c11234

This reverts commit ac13000.

maxime-desroches and others added 20 commits August 30, 2024 20:08

reduce metadrive size (commaai#33391)

629dcdc

* minimal * uv.lock * remove anisotropic filtering * new wheel

joystick: perform deadzone first

fc42504

joystickd: works with game controller on device (commaai#33414)

dae88e1

* state what we support * more * wording

Bot comment only for first timers (commaai#33412)

0a1b6c7

* Wellcome comment just for first timers * Update auto_pr_review.yaml https://github.com/actions/first-interaction/blob/main/README.md

move PID controller to common/ (commaai#33419)

f59a1bf

ui: gc experiments

42f8875

controlsd: deprecate alertBlinkingRate (commaai#33422)

f4e101c

* controlsd: deprecate alertBlinkingRate * simplify * update refs

controlsd: split out state/alerts into selfdriveState (commaai#33421)

8705180

* setup selfdriveState * little more * update refs * migration * all too slow

ci: add controlsState back to ui report (commaai#33424)

ff4cce2

add controlsState back

update test_alerts

519b666

ci: display lead car indicator in on-road screenshots (commaai#33416)

100ca65

Show lead car in on-road screenshots

replay: migrate old ControlState event to new format during log file …

6741877

…parsing (commaai#33425) Migrate Old ControlState Event to New Format During Log File Parsing

cabana: update event Handling from CONTROLS_STATE to SELFDRIVE_STATE (c…

fdf6c90

…ommaai#33426) check selfdrive_state

ci: integrate offroad alerts and update notifications to ui report (c…

db9263d

…ommaai#33408) add offroad alerts

ui: remove copy initialization of dialogs (commaai#33427)

dc06813

remove copy initialization of dialogs

athena: use dongle ID on /persist/ when available (commaai#33259)

91db49b

* athena: use dongle ID on /persist/ when available * comment * test * cleanup

ci: add body to ui report (commaai#33429)

fefb033

add body

carEvent -> onroadEvent (commaai#33432)

c481f11

ui: refactor driver monitor updating and rendering into `DriverMonito…

af80a13

…rRenderer` class (commaai#33375) * Refactor Driver Monitor Updating and Rendering add comments * rebase master * rename dmon to driver_monitoring

github-actions bot closed this Sep 2, 2024

adeebshihadeh reopened this Sep 2, 2024

AlexandreSato and others added 2 commits September 1, 2024 20:38

Fix small typos on run.py (commaai#33434)

352d0b6

refactor handle_encoder_msg

fce44ac

deanlee force-pushed the loggerd_refactor branch from 668d359 to fce44ac Compare September 2, 2024 06:26

github-actions bot removed the stale label Sep 3, 2024

adeebshihadeh closed this Sep 4, 2024

adeebshihadeh force-pushed the master branch from 29af572 to 0058ea5 Compare September 4, 2024 05:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

loggerd: extract `write_encode_data` and eliminate recursion from `handle_encoder_msg` #33219

loggerd: extract `write_encode_data` and eliminate recursion from `handle_encoder_msg` #33219

deanlee commented Aug 7, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024

maxime-desroches commented Sep 4, 2024

loggerd: extract write_encode_data and eliminate recursion from handle_encoder_msg #33219

loggerd: extract write_encode_data and eliminate recursion from handle_encoder_msg #33219

Conversation

deanlee commented Aug 7, 2024 • edited Loading

github-actions bot commented Sep 2, 2024

maxime-desroches commented Sep 4, 2024

loggerd: extract `write_encode_data` and eliminate recursion from `handle_encoder_msg` #33219

loggerd: extract `write_encode_data` and eliminate recursion from `handle_encoder_msg` #33219

deanlee commented Aug 7, 2024 •

edited

Loading