Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --log-format json #948

Merged
merged 4 commits into from
Dec 8, 2022
Merged

Add --log-format json #948

merged 4 commits into from
Dec 8, 2022

Conversation

rukai
Copy link
Member

@rukai rukai commented Dec 1, 2022

Adds a flag that sets shotover to write its tracing logs as json instead of the default human readable format.
e.g.

{"timestamp":"2022-12-01T04:13:17.880070Z","level":"INFO","fields":{"message":"Starting Shotover 0.1.7"},"target":"shotover_proxy::runner"}
{"timestamp":"2022-12-01T04:13:17.880153Z","level":"INFO","fields":{"configuration":"Config { main_log_level: \"warn,shotover_proxy=warn\", observability_interface: \"0.0.0.0:9001\" }"},"target":"shotover_proxy::runner"}
{"timestamp":"2022-12-01T04:13:17.880182Z","level":"INFO","fields":{"topology":"Topology { sources: {\"cassandra_prod\": Cassandra(CassandraConfig { listen_addr: \"127.0.0.1:9042\", connection_limit: None, hard_connection_limit: None, tls: None, timeout: None })}, chain_config: {\"main_chain\": [DebugForceEncode(DebugForceEncodeConfig { encode_requests: true, encode_responses: true }), CassandraSinkSingle(CassandraSinkSingleConfig { address: \"127.0.0.1:9043\", tls: None, connect_timeout_ms: 3000, read_timeout: None })]}, source_to_chain_mapping: {\"cassandra_prod\": \"main_chain\"} }"},"target":"shotover_proxy::runner"}
{"timestamp":"2022-12-01T04:13:17.880403Z","level":"INFO","fields":{"message":"Loaded chains [\"main_chain\"]"},"target":"shotover_proxy::config::topology"}
{"timestamp":"2022-12-01T04:13:17.880469Z","level":"INFO","fields":{"message":"Starting Cassandra source on [127.0.0.1:9042]"},"target":"shotover_proxy::sources::cassandra_source"}
{"timestamp":"2022-12-01T04:13:17.880566Z","level":"INFO","fields":{"message":"Loaded sources [[\"cassandra_prod\"]] and linked to chains"},"target":"shotover_proxy::config::topology"}
{"timestamp":"2022-12-01T04:13:17.880634Z","level":"INFO","fields":{"message":"accepting inbound connections"},"target":"shotover_proxy::server"}
{"timestamp":"2022-12-01T04:13:17.981697Z","level":"INFO","fields":{"message":"Negotiating protocol version: rejecting version 66 (configure the client to use a supported version by default to improve connection time)"},"target":"shotover_proxy::codec::cassandra","span":{"id":2,"source":"CassandraSource","name":"connection"},"spans":[{"id":2,"source":"CassandraSource","name":"connection"}]}
{"timestamp":"2022-12-01T04:13:17.982215Z","level":"INFO","fields":{"message":"Negotiating protocol version: rejecting version 65 (configure the client to use a supported version by default to improve connection time)"},"target":"shotover_proxy::codec::cassandra","span":{"id":3,"source":"CassandraSource","name":"connection"},"spans":[{"id":3,"source":"CassandraSource","name":"connection"}]}
{"timestamp":"2022-12-01T04:13:19.396892Z","level":"INFO","fields":{"message":"Negotiating protocol version: rejecting version 66 (configure the client to use a supported version by default to improve connection time)"},"target":"shotover_proxy::codec::cassandra","span":{"id":7,"source":"CassandraSource","name":"connection"},"spans":[{"id":7,"source":"CassandraSource","name":"connection"}]}
{"timestamp":"2022-12-01T04:13:19.397418Z","level":"INFO","fields":{"message":"Negotiating protocol version: rejecting version 65 (configure the client to use a supported version by default to improve connection time)"},"target":"shotover_proxy::codec::cassandra","span":{"id":8,"source":"CassandraSource","name":"connection"},"spans":[{"id":8,"source":"CassandraSource","name":"connection"}]}
{"timestamp":"2022-12-01T04:13:19.446729Z","level":"INFO","fields":{"message":"Negotiating protocol version: rejecting version 66 (configure the client to use a supported version by default to improve connection time)"},"target":"shotover_proxy::codec::cassandra","span":{"id":12,"source":"CassandraSource","name":"connection"},"spans":[{"id":12,"source":"CassandraSource","name":"connection"}]}
{"timestamp":"2022-12-01T04:13:19.447337Z","level":"INFO","fields":{"message":"Negotiating protocol version: rejecting version 65 (configure the client to use a supported version by default to improve connection time)"},"target":"shotover_proxy::codec::cassandra","span":{"id":13,"source":"CassandraSource","name":"connection"},"spans":[{"id":13,"source":"CassandraSource","name":"connection"}]}
{"timestamp":"2022-12-01T04:13:19.503113Z","level":"WARN","fields":{"message":"Server-side warning: `USE <keyspace>` with prepared statements is considered to be an anti-pattern due to ambiguity in non-qualified table names. Please consider removing instances of `Session#setKeyspace(<keyspace>)`, `Session#execute(\"USE <keyspace>\")` and `cluster.newSession(<keyspace>)` from your code, and always use fully qualified table names (e.g. <keyspace>.<table>). Keyspace used: test_prepare_statements, statement keyspace: test_prepare_statements, statement id: 2418a6606d64ccd02a7805fc4416a352","log.target":"datastax::internal::core::Decoder::decode_warnings","log.module_path":"datastax::internal::core::Decoder::decode_warnings","log.file":"decoder.cpp","log.line":134},"target":"datastax::internal::core::Decoder::decode_warnings"}
{"timestamp":"2022-12-01T04:13:19.614296Z","level":"ERROR","fields":{"message":"UHO H H\nYEET"},"target":"lib::cassandra_int_tests"}
{"timestamp":"2022-12-01T04:13:19.615360Z","level":"INFO","fields":{"message":"source CassandraSource was shutdown"},"target":"shotover_proxy::server"}
{"timestamp":"2022-12-01T04:13:19.615503Z","level":"INFO","fields":{"message":"Shotover was shutdown cleanly."},"target":"shotover_proxy::runner"}

My original motivation of this change is to be able to make assertions on tracing logs in our integration tests.

However users will want to be able to process shotovers logs programmatically so this is an important feature for shotover in its own right.
As such it should be reviewed for that purpose, as a user exposed feature.

To test it out you can run:

  • cargo run uses --log-format human by default
  • cargo run --log-format json

Feel free to bikeshed arg naming.

Due to a currently poor API from tracing the reload handle type currently includes whether we are using json formatting or not.
So in order to handle it generically we need to wrap each case in an enum and then pass that into our web server.

Copy link
Member

@benbromhead benbromhead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't get too hung up on the web log control stuff we have at the moment, it's largely an "interesting" feature. We can start to work on it's idiosyncrasies when we get some proper user feedback

@conorbros conorbros enabled auto-merge (squash) December 8, 2022 05:11
@conorbros conorbros merged commit 071a6e2 into shotover:main Dec 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants