Ability to write logs from alternative languages

[pyhedgehog]

In JavaScript hooks one can just use console.log to write entry to hook log.
Is it correct, that console.log just writes to stderr stream?
If yes, than why it doesn't works in hooks on other languages?
I've wrote a demo:
Source: https://gist.github.com/pyhedgehog/66cdff7f2d4bcaf792b78515f4e994bf
Test: https://hook.io/pyhedgehog/pytestlog
Result: https://hook.io/pyhedgehog/pytestlog/logs

[Marak]

So this is something that needs to be fixed.

In order to make this work we need to make modifications to the way python process is spawned.

It shouldn't be that complicated, but you have to understand how it's currently working.

see:

https://github.com/bigcompany/hook.io/blob/master/lib/resources/hook/stderr/index.js
https://github.com/bigcompany/hook.io/blob/master/lib/resources/hook/spawnService.js#L92
https://github.com/bigcompany/hook.io/blob/master/lib/resources/hook/spawnService.js#L291

most importantly, we need to modify run-python binary to conform to this kind of API:

https://github.com/bigcompany/hook.io/blob/master/bin/run-hook#L285
https://github.com/bigcompany/hook.io/blob/master/bin/run-hook#L385

Right now it looks like all stdout and stderr are being sent to response for python. We'll want to pipe stderr to logs like it works for JS.

[pyhedgehog]

Thank you. So we have such ability:
https://hook.io/pyhedgehog/testpylog/source
https://hook.io/pyhedgehog/testpylog/logs
All we need - add interface.

[Marak]

@pyhedgehog - Is better we make this integrated as part of run-python binary?

Make nice debug type method for python scripts?

[Marak]

Maybe stderr should auto-encode into JSON message for logging API if no JSON message is found?

That way it would be supported for all languages easily?

[Marak]

( no need to make json object in code )

[pyhedgehog]

I think that this should be installed as logging handler.
stderr should be left intact to allow someone who don't want to use sdk to choose different approach.

[Marak]

That sounds like best practice for python @pyhedgehog

Can you make pull request to run-python and run-python3 binaries to include logging handle?

I think maybe it's easy to do?

[pyhedgehog]

I think such changes should be installed explicitly by micro-service author. Thats what I mean by "allow someone to choose different approach". There are "Explicit is better than implicit" statement in The Zen of Python.
I think that anyone who wants this functionality will write something like:

import hookio
Hook = hookio.install_sdk(Hook)

And anyone who don't want (i.e. wants only datastore), will use:

import hookio
Hook = hookio.install_sdk(Hook, inject_logging=False)
val = Hook.sdk.datastore.get(Hook.params['name'])

or ever:

import hookio
sdk = hookio.createClient()
val = sdk.datastore.get(Hook['params']['name'])

[Marak]

Node.js has console.log method, so for JavaScript, we provide a console.log which pipes to service logs.

You tell me Python has logging handler. Well, we should provide a logging handler which pipes to service logs?

Sugar syntax for logs is expected platform feature.

[pyhedgehog]

Felicitous remark. I will consider it.

[pyhedgehog]

Default functionality of logging module is different than console.log in nodejs.
By default sending to logging does nothing and you must implicitly configure where to send logs.
This can be something as easy as logging.basicConfig() that sends all logs to stderr, but you can precisely select which loggers should be processed by which handlers (log targets) and sent different filters and formatters for them.
Another issue is that alternative languages are supported already a half a year (since 17 Sep 2015) and this will introduce incompatibility (at least theoretically).

[Marak]

Don't worry about incompabitilties, python needs logs.

present solution if you can, or i will try to implement a default logging class for every python service that sends to stderr

[pyhedgehog]

I've prepared piece of code that can be inserted to run-hook-python. But I'm still unsure that it should be enabled by default. Python is about choices and standard logging also has several configurable options.

Here is:

• An example hook: https://hook.io/pyhedgehog/hookiologging236 (will just print "OK")
• Logs of this hook: https://hook.io/pyhedgehog/hookiologging236/logs
• Source of this hook: https://gist.github.com/pyhedgehog/99dae38d1cca86d6fb3ed0d6a4c3cab2

As you can see there are two possible logging modes:

• When format argument is passed to hookioLoggingConfig() it writes formatted strings as a payload.
• When (by default) format is None it writes complete log record data (at least json-able part) and this record can be "revived" on client side:

import hookio, json, logging
logging.basicConfig()
sdk = hookio.createClient()
l1 = sdk.logs.read("pyhedgehog/hookiologging236")
l2 = map(json.loads, l1)
l3 = [logging.makeLogRecord(json.loads(o['data'])) for o in l2]
for o in reversed(l3):
    logging.getLogger(o.name).handle(o)