-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
verbose output #46
base: master
Are you sure you want to change the base?
verbose output #46
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The bones look reasonable to me, although there are a few patterns of nits (I've only commented once for each of the patterns). Longer term, I think we want to write TAP or some such for the validation output, but this PR will be useful regardless of whether validation output stays in logrus or not.
|
||
logLevel, err := logrus.ParseLevel(v.logLevelString) | ||
if err != nil { | ||
logrus.Fatalf("%s", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logrus.Fatalf("%s", err)
→ logrus.Fatal(err)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer to only Fatalf
, because it can format the output message. Some needn't be formatted, but it has no harm. And it is unnecessary to use 2 methods as Fatalf
and Fatal
. Unitary method is clearer for next implementation.
And I hold same opinion for other levels(Infof
to Info
, Debugf
to Debug
...).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The logrus and fmt maintainers feel like it's useful to include both formatted and unformatted printers. I think it's less surprising to use the unformatted interface when you want the default format, but it's not a big deal either way.
cmd.Flags().StringVar( | ||
&v.logLevelString, "log-level", "info", | ||
`Log level (panic, fatal, error, warn, info, or debug)`, | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--log-level
should be documented in the associated man page (e.g. oci-create-runtime-bundle.1.md
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added as 2d1f26d
v.stdout.Printf("commit: %s", gitCommit) | ||
v.stdout.Printf("spec: %s", specs.Version) | ||
logrus.Infof("commit: %s", gitCommit) | ||
logrus.Infof("spec: %s", specs.Version) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are supposed to go to stdout, so we should leave them alone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, but I think it is easy and clear to use fmt.Printf()
directly here, WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fmt.Printf
works for me. It means that there's no way to capture the stream generated by the command (which you can currently do by setting v.stdout
to something other than os.Stdout
), but this is in cmd/
and not in a library, so I think that decreased flexibility is fine.
} | ||
os.Exit(1) | ||
logrus.Fatalf("both src and dest must be provided") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logrus.Fatal("both src and dest must be provided")
(Fatalf
→ Fatal
).
@@ -26,6 +26,9 @@ runtime-spec-compatible `dest/config.json`. | |||
**--type** | |||
Type of the file to unpack. If unset, oci-create-runtime-bundle will try to auto-detect the type. One of "imageLayout,image" | |||
|
|||
**--log-level** | |||
Log level to create runtime bundle. To be one of panic, fatal, error, warn, info, or debug. If unset, the level is identical to info. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should follow runtime-tools and make the default level error
. That covers the stuff that leads to non-zero exit codes. Folks interested in additional detail can bump up --log-level
, and more restricted default allows us to be more generous about the amount of warn- and info-level logging we put in (without making the default command too noisy).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wking
I've researched runtime tool repo, but found it has only debug messages, without info message(so far) need feed back to consumers.
But for image tools repo, there are messages, like 5beb020#diff-da07f373046a0a24e67b18699dc15a70R88 and 5beb020#diff-8c24ca9b63b3598ef3f4cc64741f433eR117, need be shown to consumers.
Later we maybe add more information, which consumer does care about, so I think default info level is better.
I'm not persistent to it, but could you please consider it? Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wking , WDYT on above comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't had time to look over the logging in both repos; maybe tomorrow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All right, thanks so much.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
runtime-tools has a warning in-flight with opencontainers/runtime-tools#256. I still think the default should be error
. I agree that the info messages you linked to are useful output, but I'd rather handle them by printing TAP to stdout instead of by using logrus to conditionally print them to stderr. On the other hand, TAP is a big shift from where we are now; I'm fine handling that change in a follow-up PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wking
Fixed it in d527098 and 0c2f6f2, PTAL.
Currently I suppose image tool to use fmt.Printf()
directly as stdout (for it should be unconditional), logrus.Errorf()
as stderr, and other conditional level message via logrus, which is adopted in docker/docker. But, I doubt a little if it is prefect behavior. Could you please present your understanding about TAP? Thanks a lot!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filed the TAP thoughts in #60.
glide.lock
Outdated
@@ -25,6 +25,8 @@ imports: | |||
version: e02fc20de94c78484cd5ffb007f8af96be030a45 | |||
- name: github.com/xeipuuv/gojsonschema | |||
version: d5336c75940ef31c9ceeb0ae64cf92944bccb4ee | |||
- name: github.com/Sirupsen/logrus | |||
version: 3ec0642a7fb6488f65b06f9040adc67e3990296a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logrus cuts tagged releases. Can we use 0.10.0 here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wking
I fix it in 5deb77e, to use 0.10.0.
BTW, if we really need glide to fetch the vendor code? I notice that runtime-tools repo(and many popular repos) put package code into Godeps
folder. I know that the benifit of glide is to reserve the necessary code accurately. But if we really want to take it, to have to install glide on each host? Maybe we should only import third party code into vendor
or Godeps
directly(follow up PR). WDYT?
if v.version { | ||
v.stdout.Printf("commit: %s", gitCommit) | ||
v.stdout.Printf("spec: %s", specs.Version) | ||
fmt.Printf("commit: %s\n", gitCommit) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Printf
→ Println
(then you can drop the \n
addition).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, nevermind. I'm fine with Printf
, since Println
doesn't support a format string.
|
On Sun, Oct 30, 2016 at 11:20:43PM -0700, xiekeyang wrote:
Errorf vs. Error seems the same as Fatalf vs. Fatal to me. But I'm |
@opencontainers/image-tools-maintainers |
cmd/oci-image-validate/main.go
Outdated
if err := cmd.Usage(); err != nil { | ||
v.stderr.Println(err) | ||
logrus.Errorf("%s", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logrus.WithError(err).Errorf("usage failed")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed in 9ae60ce.
To use logrus.WithError(err).Errorf()
to instead logrus.Errorf()
, on related positions.
@xiekeyang Make sure that logging is going to |
I assume that if err := cmd.Usage(); err != nil {
logrus.WithError(err).Errorf("usage failed")
} And below use err := v.validatePath(arg)
if err != nil {
fmt.Printf("error: validation failed: %s",err)
} Is that what you want to say? If so, I'm not very clear about their difference. |
@xiekeyang Yes, that is effectively what I am saying. However, the output to |
@stevvooe |
@stevvooe |
@xiekeyang Simply put, I would print all of logrus to |
This is not really what I said or meant. :/ There isn't really a point to using both the standard logger and and logrus. They are redundant. What I said was that you need to differentiate program output, ie. validation, and status of the program with Let's say we have output, tagged with the stream (this is an example, don't add it), it would look like this:
Typically, we use a logger to write to This distinction is the hallmark of a well structured unix tool. It allows us to pipe program output somewhere, while monitoring stderr for behavior. I hope this clarifies the requirements here. (if anyone else has a link explaining this concept, it would be helpful). |
And update the PR description at #46 (comment) |
rebased, PTAL. |
@@ -0,0 +1,62 @@ | |||
package logger |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the purpose of this package?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is an example of a context-integrated logger that is threadsafe: https://github.com/docker/containerd/blob/master/log/context.go#L28.
However, please let me know what the intent of this package and we can get it into shape.
I'm in progress for this PR. Excuse me that I might only partly understand you. docker/containerd and docker/distribution are using individual, and public package to organize log, and provide an instance. |
I'm in progress for this PR. Excuse me that I might only partly understand you. docker/containerd and docker/distribution are using individual, and public package to organize log, and provide an instance. |
@xiekeyang I don't understand why you need a leveled logger in a command line tool. It makes no sense. |
@stevvooe You might just want |
Don't use this as an example: this is very new code and half of it is a daemon. |
@stevvooe OK. So I use it as |
@stevvooe @opencontainers/image-tools-maintainers
|
|
||
cmd := newBundleCmd(stdout, stderr) | ||
ctx := context.Background() | ||
ctx = context.WithLogger(ctx, context.G(ctx)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this reads in a stuttered way. Seems like it could just be a oneliner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vbatts You are right. It is updated to 548a4ae and eb76ffb
The changeset is:
eb76ffb#diff-bd4694bbf0ec2644693b1ef195e3d330R49
eb76ffb#diff-8c24ca9b63b3598ef3f4cc64741f433eR54
eb76ffb#diff-cfb3ced180edd25cfc0a028c1a6e900dR48
Changed logentry
to LogEntry
as public variable. And then in */main.go
it is applied as,
ctx := context.WithLogger(context.Background(), LogEntry)
That improves declaration of context
to oneline, and keep context resource usage unchanged.
i like the idea of using context for sharing the logger |
I also really like it, because it means that the CAS and other functions in |
context/context.go
Outdated
@@ -0,0 +1,31 @@ | |||
// Copyright 2016 The Linux Foundation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this package totally new or did you find it from somewhere else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it is totally new. I intend that context
package will include more functionalities besides logger
, so I define a widely used name as context
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I refer this framework to docker/containerd
and docker/distribution
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't follow the distribution pattern. We'll probably rip that out in favor the containerd/swarmkit approach.
@opencontainers/image-tools-maintainers It is updated, PTAL, thanks. |
Seems like a good starting point. Thanks @xiekeyang |
@vbatts I still don't understand why we are using a logger for user consumed output. This still seems like the wrong direction. |
context/logger.go
Outdated
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package context |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd recommend against calling this package context
. It causes future problems when you try to work with both the context package and other packages. This package provides logging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@stevvooe If logger
or logging
is feasible name? I worry that in future this might include more functionalities besides logging to use context value to, so that should we use generic name?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that referred to docker/distribution
? 😄
glide.lock
Outdated
- name: golang.org/x/net | ||
version: d379faa25cbdc04d653984913a2ceb43b0bc46d7 | ||
subpackages: | ||
- context |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd recommend we use the stdlib package for new interfaces.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@stevvooe Do you mean import context
directly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't do that. Currently on openSUSE we haven't switched to Go 1.8 because of issues building on aarch64
(amongst other things). So using import "context"
breaks for non-x86_64
architectures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cyphar context
was added in Go 1.7. If we export interfaces that use golang.org/x/net/context
, we will be stuck using that package forever.
context/logger.go
Outdated
type ( | ||
loggerKey struct{} | ||
|
||
stderrHook struct{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the role of the stderr hook? Can't we just set the logging output to go to stderr?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@stevvooe
In this design, I set default logger.Out to stdout, because I want to print info and debug to stdout.
LogEntry.Logger.Out = os.Stdout
I create hook because I can fire the output to stderr, by hook.Fire()
function. And I trigger it when level = [fatal,panic,error,warn], as hook.Level()
is doing. Hook is necessary for logger.
About if we should use logger, I mostly agree with @vbatts. If we use logrus directly without logger, maybe the applicability will be a little narrow (although I can not figure out all usage now).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logging and debug should always go to stderr
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wow, Excuse me I was not aware of it. If it is a popular way for client tools? Or could you please give me some more explain?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logging and debug should always go to stderr.
Wow, Excuse me I was not aware of it. If it is a popular way for client tools? Or could you please give me some more explain?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Thu, Mar 16, 2017 at 04:41:49PM -0700, xiekeyang wrote:
If it is a popular way for client tools?
POSIX suggests:
… standard output (for writing conventional output), and standard error (for writing diagnostic output).
The Linux man pages have:
Under normal circumstances every UNIX program has three streams opened for it when it starts up, one for input, one for output, and one for printing diagnostic or error messages… the output stream is referred to as "standard output"; and the error stream is referred to as "standard error".
So I think “normal output to stdout and logging/debug to stderr” is a fairly well-established pattern. Deciding what is “normal output” and what is “logging/debug” may require more clarity around our intended output, but I'd guess most things which need tunable verbosity are debugging logs which should be going to stderr.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
printing out a report or a consumable json, is not the as a logging. These
don't have to be blocking each other.
…On Wed, Mar 15, 2017 at 2:53 PM, Stephen Day ***@***.***> wrote:
@vbatts <https://github.com/vbatts> I still don't understand why we are
using a logger for user consumed output. This still seems like the wrong
direction.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#46 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEF6WFZNbRNLYrJe48-_WcphCTGTa8wks5rmDOYgaJpZM4KdECk>
.
|
I'm rebasing it. For the code uses |
@opencontainers/image-tools-maintainers Rebased, PTAL. |
@stevvooe This should be mostly looking well and reasonable to your suggestion.
PTAL, cc @opencontainers/image-tools-maintainers |
This package provides contexted logging. Create logrus logger, and add it to context value. It help to print output to stderr. Signed-off-by: xiekeyang <xiekeyang@huawei.com>
Verbose output utilizes public logger pakcage. It create command context with which logger can be retrieved. Signed-off-by: xiekeyang <xiekeyang@huawei.com>
rebased, PTAL. ping @stevvooe |
"github.com/pkg/errors" | ||
) | ||
|
||
// ValidateLayout walks through the given file tree and validates the manifest | ||
// pointed to by the given refs or returns an error if the validation failed. | ||
func ValidateLayout(src string, refs []string, out *log.Logger) error { | ||
return validate(newPathWalker(src), refs, out) | ||
func ValidateLayout(ctx logger.Context, src string, refs []string) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't make sense. This should just use a regular context. Don't define a new Context
type.
Please follow the pattern in containerd
. It is well-healed and does exactly what is needed here.
) | ||
|
||
// Context is a copy of Context from the stdlib context package. | ||
type Context interface { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not define a new context type.
close #14 and #22 .
This provides a contexted logger instance, which is exported from
logrus
, to print the information to stderr and stdout. It level the information throughdebug
option.Signed-off-by: xiekeyang xiekeyang@huawei.com