Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(framework) Add SuperExec logcatcher #3584

Merged
merged 86 commits into from
Sep 19, 2024
Merged

feat(framework) Add SuperExec logcatcher #3584

merged 86 commits into from
Sep 19, 2024

Conversation

chongshenng
Copy link
Contributor

@chongshenng chongshenng commented Jun 11, 2024 •
edited
Loading

Issue

Description

Logs from the subprocess that launches the ServerApp needs caught in the SuperExec service.

Related issues/PRs

The companion PR is the following.

Blocks:

Proposal

Explanation

The logging mechanism is split into two parts:

  • Capture stdout and stderr to memory
  • Stream logs from memory as the response to the client request

In-memory logs

Logstreams can be requested at any stage of a run via flwr log. Each call to StartRun starts log capturing for a run_id. Logs for a run_id are stored in self.runs[run_id].logs.

Checklist

  • Implement proposed change
  • Make CI checks pass
  • Ping maintainers on Slack (channel #contributions)

Any other comments?

@chongshenng chongshenng changed the title feat(framework) Add logcatcher to SuperExec feat(framework) Add logcatcher to SuperExec Jun 11, 2024
@chongshenng chongshenng mentioned this pull request Jun 11, 2024
Merged
3 tasks
@chongshenng chongshenng self-assigned this Jun 13, 2024
@chongshenng chongshenng marked this pull request as ready for review June 14, 2024 13:10
@chongshenng chongshenng changed the title feat(framework) Add logcatcher to SuperExec feat(framework) Add SuperExec logcatcher Jun 14, 2024
yan-gao-GY and others added 19 commits September 19, 2024 10:36
@yan-gao-GY @chongshenng
refactor(examples) Update get/set parameters functions for FlowerTune…
ef486a8 
… LLM example (#4219)
@chongshenng @jafermarq
refactor(examples) Update quickstart-pandas example (#4007)
8628b1a 
Co-authored-by: jafermarq <javier@flower.ai>
@charlesbvll @jafermarq
@danieljanes @chongshenng
feat(examples) Update vertical-fl example (#3716)
24d0353 
Co-authored-by: jafermarq <javier@flower.ai>
Co-authored-by: Daniel J. Beutel <daniel@flower.ai>
@mohammadnaseri @jafermarq @chongshenng
refactor(examples) Update tensorflow-privacy example (#4204)
d994741 
Co-authored-by: jafermarq <javier@flower.ai>
@mohammadnaseri @chongshenng
refactor(examples:skip) Update quickstart-tensorflow example (#4227)
624c868
@ray-ruisun @chongshenng
docs(framework) Add new blockchain example to FAQ (#3993)
0135f60
@yan-gao-GY @chongshenng
refactor(benchmarks) Update llm leaderboard Readmes (#4228)
97e94ff
@mohammadnaseri @panh99
@danieljanes @chongshenng
refactor(framework) Migrate ID handling from sint64 to uint64 (#4170)
ba1c577 
Co-authored-by: Heng Pan <pan@flower.ai>
Co-authored-by: Daniel J. Beutel <daniel@flower.ai>
@chongshenng @jafermarq @charlesbvll
docs(framework) Update Quickstart Tutorial documentation for TensorFl…
5869a18 
…ow with `flwr run` (#3338)

Co-authored-by: Javier <jafermarq@users.noreply.github.com>
Co-authored-by: Charles Beauville <charles@flower.ai>
@chongshenng @jafermarq
docs(framework) Update quickstart tutorial documentation for Hugging …
f47380f 
…Face with `flwr run` (#3341)

Co-authored-by: Javier <jafermarq@users.noreply.github.com>
@jafermarq @chongshenng
docs(framework) Update fastai quickstart tutorial (#4151)
75d779c
@jafermarq @chongshenng
docs(framework) Update pytorch-lightning quickstart tutorial (#4152)
bd7ba5f
@adam-narozniak @jafermarq @chongshenng
feat(datasets) Add SizePartitioner (#4111)
fa0ae67 
Co-authored-by: jafermarq <javier@flower.ai>
@jafermarq @chongshenng
refactor(examples) Update advanced-pytorch example (#4130)
f110cb0 
Co-authored-by: Chong Shen Ng <chong.shen@flower.ai>
@jafermarq @chongshenng
refactor(examples) Make quickstart-huggingfaceuse a smaller LM (#4206)
93977af
@danieljanes @chongshenng
fix(*:skip) Update .editorconfig (#4238)
7f0f5c4
@Robert-Steiner @flwrmachine @chongshenng
feat(framework:skip) Add gcc in Docker compose setup (#4187)
ed2f70c 
Signed-off-by: Robert Steiner <robert@flower.ai>
Co-authored-by: Flower <148336023+flwrmachine@users.noreply.github.com>
@mohammadnaseri @chongshenng
refactor(framework) Add further unit tests to sint64 to uint64 conver…
0710884 
…sion utils (#4237)
@chongshenng
Address comments
8fcd519
@danieljanes danieljanes merged commit ebb78ae into main Sep 19, 2024
32 checks passed
@danieljanes danieljanes deleted the add-superexec-logcatcher branch September 19, 2024 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanertopal tanertopal tanertopal left review comments

@danieljanes danieljanes danieljanes approved these changes

@Robert-Steiner Robert-Steiner Awaiting requested review from Robert-Steiner

@Moep90 Moep90 Awaiting requested review from Moep90

@jafermarq jafermarq Awaiting requested review from jafermarq

Assignees

@chongshenng chongshenng

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
10 participants
@chongshenng @tanertopal @danieljanes @jafermarq @yan-gao-GY @adam-narozniak @charlesbvll @mohammadnaseri @ray-ruisun @Robert-Steiner