Kafka Packaging rework 2 #1
Conversation
| PIDFile=/run/kafka/kafka10-pt.pid | ||
| Environment=KAFKA_HOME=/usr/lib/kafka CONFIG_HOME=/etc/kafka | ||
| ExecStart=/usr/lib/kafka/bin/kafka-server-start.sh /etc/kafka/server.properties | ||
| ExecStop=/usr/lib/kafka/bin/kafka-server-stop.sh |
There was a problem hiding this comment.
For the ZK systemd script, I ended up adding this:
Restart=always
RestartSec=3
so it'll get restarted if it crashes. I think we're doing that now with god and it's all good. Do we want that with Kafka as well? Could see args on both sides so just raising the question more than anything.
| @@ -29,18 +49,17 @@ mkdir -p build/etc/init | |||
| mkdir -p build/etc/kafka | |||
| mkdir -p build/var/log/kafka | |||
|
|
|||
| cp ${origdir}/kafka-broker.default build/etc/default/kafka-broker | |||
| cp ${origdir}/kafka-broker.upstart.conf build/etc/init/kafka-broker.conf | |||
|
|
|||
| tar zxf ${origdir}/${src_package} | |||
| cd kafka_${scala_version}-${version} | |||
| mv config/log4j.properties config/server.properties ../build/etc/kafka | |||
There was a problem hiding this comment.
Posting this here so I remember to ask but it's prob a salt thing... Is Kafka logging working okay for prod? I know there were issues in staging.
There was a problem hiding this comment.
I don't actually know. I didn't remember any clear complaints. :-/
There was a problem hiding this comment.
Ah yeah, it's potentially confusing. So I do see something in: https://staging.papertrailapp.com/groups/1/events?q=host%3Akafka%20-program%3A(cron%20rsyslog%20dhclient%20systemd%20snapteld%20clear-aoagent%20salt-minion) but I'm guessing that's not all of it as I can't seem to track down where the logs specified in log4j.properties are going:
log4j.appender.kafkaAppender.File=${kafka.logs.dir}/server.log
I assumed kafka.logs.dir = /var/log/kafka, but that's empty. If we can find out where it lives, we can monitor them with remote_syslog2 or send them into syslog and rsyslog will forward 'em over.
Having said all that, perhaps there's just nothing to log and so no files existing is normal?
Ok. This time should be better. I pushed the systemd unit into the package, and eventually the upstart script should be updated to reflect what Librato has in salt. Also add a patch to support generating a pidfile directory to avoid using dbus for finding the pid of the kafka process.
TODOs: