Skip to content
This repository has been archived by the owner on Mar 3, 2023. It is now read-only.

Clean up executor logging in LocalScheduler #1288

Merged
merged 3 commits into from
Aug 25, 2016

Conversation

billonahill
Copy link
Contributor

Executor log files in LocalScheduler currently look like this:

heron-executor-d7e5809a-started.stderr
heron-executor-dc47dc90-started.stderr
heron-executor-edd18eeb-started.stderr
heron-executor-effb081c-started.stderr
heron-executor.stdout

Two issues:

  1. Every run of the executor makes a new stderr file which causes file bloat of mostly 0-byte sized files. Each run shares the same stdout file though, which is inconsistent.
  2. When running multiple executors, they all log to the same stdout file, which makes the file contents difficult to read.

This changes executor logging to look more like the logging of the other processes. If you have two executors you'd see something like this:

heron-executor-0.stderr
heron-executor-0.stdout
heron-executor-1.stderr
heron-executor-1.stdout

If one fails and is restarted, it continues to log to the same file as the previous instance.

@billonahill
Copy link
Contributor Author

Ping. Any comments on this one?

@objmagic
Copy link
Contributor

LGTM

👍

@billonahill billonahill merged commit 94f52da into apache:master Aug 25, 2016
nicknezis pushed a commit that referenced this pull request Sep 14, 2020

* Change to not reassign logFileUuid

* Fix unit test now that ShellUtils method signature changed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants