Suppress "INFO: Unable to fetch credentials" #41
Comments
|
I'll take a look. |
|
@grzm I don't think adding a fixed logging config in resources is flexible enough since this gets baked into the binary and is hard to override by users. Also passing Java properties at pod compile time doesn't affect the runtime. The property has to be set at runtime. On the pod side I added a function: (defn set-logging-config! [s]
(let [s (.readConfiguration (java.util.logging.LogManager/getLogManager) (string->stream s))]
(.println System/err (str s))))So in a script you can do: (require '[babashka.pods :as pods])
(pods/load-pod ["clojure" "-M" "-m" "pod.babashka.aws"])
(require '[pod.babashka.aws :as aws])
(aws/set-logging-config! "java.util.logging.ConsoleHandler.level = TRACE")
(defn ex [_]
(let [s3 (aws/client {:api :s3})]
(prn (aws/invoke s3 {:op :ListBuckets})))
(let [sts (aws/client {:api :sts})]
(prn (aws/invoke sts {:op :GetCallerIdentity}))))
(ex nil)This has an effect on the logging, although with TRACE I would expect to see more, not less logs and now I see less logs. I added the above in a branch called |
|
@borkdude I agree on the flexibility point: I don't know the ins and outs of how pods work to know what the options are. What prevents the pod from picking up the |
|
Pods run in their own operating system process, so any properties passed to bb are not automatically passed to the pod. |
|
Also the bb classpath has no effect on pods, since they run in their own process. |
|
We could make other functions like: read the config from this property file etc, but we'll have to do this using bespoke functions I think. But we already have some code that automatically picks up on some properties when you create a client. So we might also do it like that. |
|
Would it make sense to have something like clojure.tools.deps' I suppose this might only make sense for a pod to opt into this: it's not clear what :extra-paths would mean for a Rust-based pod for example (if that makes sense). |
|
One note is that TRACE is not a valid java.util.logging.Level value: https://docs.oracle.com/javase/9/docs/api/java/util/logging/Level.html However, that's not the issue as using a valid enum doesn't change the no-logging behavior. |
|
@grzm Perhaps you could try the similar thing in a regular clj script and see what the behavior there is? |
|
Believe me, I have been :) The success I've had so far is showing what it's not: (defn logger-levels []
(let [log-manager (LogManager/getLogManager)]
(->> (.getLoggerNames log-manager)
(enumeration-seq)
(map (juxt identity #(-> (.getLevel (.getLogger log-manager %)))))
(into (sorted-map)))))
(defn log-manager-prop-vals [prop-names]
(let [log-manager (LogManager/getLogManager)]
(->> prop-names
(map (juxt identity #(.getProperty log-manager %)))
(into (sorted-map)))))
(set-logging-config! "java.util.logging.ConsoleHandler.level = ALL")
(pprint/pprint {:top-level (log-manager-prop-vals ["java.util.logging.ConsoleHandler.level"])})
(defn ex [_]
(pprint/pprint {:ex (log-manager-prop-vals ["java.util.logging.ConsoleHandler.level"])})
(pprint/pprint {:before (logger-levels)})
(let [sts (aws/client {:api :sts})]
(prn (keys (aws/invoke sts {:op :GetCallerIdentity}))))
(pprint/pprint {:after (logger-levels)}))
Calling as-is, with And with commenting it out: Riddles, mysteries, and enigmas. And on a Sunday! |
|
Well, I'm not sure what's going on. Short answer, this works: (defn set-logging-level!
"Sets the log level of a java.util.logging Logger. With no logger-name, the global logger is updated."
([lvl]
(set-logging-level! "" lvl))
([logger-name lvl]
(some-> (.getLogger (java.util.logging.LogManager/getLogManager) logger-name)
(.setLevel (java.util.logging.Level/parse lvl)))))
(pprint/pprint {:before-set (logger-levels)})
(set-logging-level! (System/getenv "X_LOGGER_LEVEL"))
(pprint/pprint {:top-level (log-manager-prop-vals ["java.util.logging.ConsoleHandler.level"])})
(pprint/pprint {:after-set (logger-levels)})
(defn ex [_]
(let [sts (aws/client {:api :sts})]
(prn (keys (aws/invoke sts {:op :GetCallerIdentity}))))
(pprint/pprint {:after-invoke (logger-levels)}))Setting level to Setting level to The global logger is the logger with only an empty string for a name. Works in the pod, too. It's not as flexible as passing in the entire logging configuration, but I haven't been able to get that to work. I came across some notes saying
(defn update-logging-configuration [s]
(let [lm (LogManager/getLogManager)]
(.updateConfiguration lm (string->stream s)
(reify java.util.function.Function
(apply [_ _k]
(reify java.util.function.BiFunction
(apply [_ _o n] n)))))))I know I'd find |
|
Maybe |
|
@grzm OK, let's proceed with just a function to tweak the log level like you proposed, with a sane default. PR welcome. |
|
Some more context from Slack:
So it's still useful to tweak the log level or other logging settings. More info here: https://github.com/grzm/aws-api-logging-ex Response from @grzm:
|
The aws-api pod is quite noisy, logging INFO messges to STDERR.
This is because the aws-api client logs at INFO level when it tries to find valid credentials:
https://github.com/cognitect-labs/aws-api/blob/fb41cdad69bd04ff5ba36890e1f667fc1f955ebc/src/cognitect/aws/credentials.clj#L130
This can be really noisy when using multiple AWS services (as shown
above). One solution would be to redirect stderr to /dev/null, but
that would also mean suppressing all other uses of stderr, including
more serious messages.
The aws-api client uses clojure.tools.logging, and falls back to
java.util.logging as there is no other logging implementation.
https://github.com/clojure/tools.logging/blob/2472b6f84853075cb58082753ec39f49d1716dc2/src/main/clojure/clojure/tools/logging/impl.clj#L245-L256
As expected, the aws-api client exhibits the same behavior using JVM Clojure:
(Though, it's only half as noisy: the bb equivalent echoes twice as
much. I haven't dug into why that might be.)
I'm able to set the log level for using a
logging.propertiesfileusing JVM Clojure. See https://github.com/grzm/aws-api-logging-ex for
a running example.
I added
resources/logging.propertiesto the aws-api pod, updateddeps to include the additional JVM property, and recompiled. However,
it doesn't have the desired effect: the
Unable to fetch credentialslog messages are still echoed.
Is there a way to get the aws-api pod to pick up the
logging.properties file? Is this something that needs to be compiled
with the pod or does it need to be specified when the bb script is
called? If the latter, it doesn't seem to have the desired effect,
either.
Maybe it's not getting passed to the pod somehow? I'd be happy
just having it compiled in and applied to the pod itself, if that's an
easier option.
The text was updated successfully, but these errors were encountered: