Build from source #71

gerhard · 2019-02-20T18:14:19Z

Add make ezs so that we can build .ez files for all deps & plugin with a single command.

The resulting files will be placed in ./tmp:

ls -l tmp/
total 992
-rw-r--r--  1 gerhard  staff   12548 20 Feb 18:02 accept-0.3.4.ez
-rw-r--r--  1 gerhard  staff  211232 20 Feb 18:02 prometheus-4.2.2.ez
-rw-r--r--  1 gerhard  staff   14053 20 Feb 18:02 prometheus_cowboy-0.1.7.ez
-rw-r--r--  1 gerhard  staff   21638 20 Feb 18:02 prometheus_httpd-2.1.10.ez
-rw-r--r--  1 gerhard  staff   10708 20 Feb 18:02 prometheus_process_collector-1.4.0.ez
-rw-r--r--  1 gerhard  staff  223943 20 Feb 18:02 prometheus_rabbitmq_exporter-3.7.20190220.ez

I've changed the version of this plugin to use the date for the patch part, feel free to change back if you disagree.

Notice that I've deleted the prometheus_rabbitmq_exporter.app.src, this is now generated by erlang.mk and dropped in ebin/prometheus_rabbitmq_exporter.app - does it look correct @deadtrickster?

Want to integrate with Docker & testing before merging.

This started as #63, see the discussion for more context.

@deadtrickster

The resulting files will be placed in ./tmp, e.g. ls -l tmp/ total 992 -rw-r--r-- 1 gerhard staff 12548 20 Feb 18:02 accept-0.3.4.ez -rw-r--r-- 1 gerhard staff 211232 20 Feb 18:02 prometheus-4.2.2.ez -rw-r--r-- 1 gerhard staff 14053 20 Feb 18:02 prometheus_cowboy-0.1.7.ez -rw-r--r-- 1 gerhard staff 21638 20 Feb 18:02 prometheus_httpd-2.1.10.ez -rw-r--r-- 1 gerhard staff 10708 20 Feb 18:02 prometheus_process_collector-1.4.0.ez -rw-r--r-- 1 gerhard staff 223943 20 Feb 18:02 prometheus_rabbitmq_exporter-3.7.20190220.ez I've changed the version of this plugin to use the date for the patch part, feel free to change back if you disagree. Notice that I've deleted the prometheus_rabbitmq_exporter.app.src, this is now generated by erlang.mk and dropped in ebin/prometheus_rabbitmq_exporter.app - does it look correct @deadtrickster? Want to integrate with Docker & testing before merging. This started as #63, see the discussion for more context.

@deadtrickster

This is what happens when I use `./rebar3 archive` instead of zip. Let me know if you can figure it out @deadtrickster - I'm happy to stick with zip.

This reverts commit 72ad2f2.

gerhard · 2019-02-20T18:15:57Z

cc @dcorbacho

tsloughter · 2019-02-20T19:49:09Z

We prefer it if rebar3 is not committed to repos.

deadtrickster · 2019-02-20T19:55:25Z

I'm ok with the date, wondering why, though.

Does patched rebar3 archive work for you?

@tsloughter

Thanks @tsloughter! You might want to check the following repos for rebar3: * https://github.com/deadtrickster/accept * https://github.com/deadtrickster/prometheus.erl * https://github.com/deadtrickster/prometheus-cowboy * https://github.com/deadtrickster/prometheus_process_collector

gerhard · 2019-02-21T17:19:24Z

I'm ok with the date, wondering why, though.

The last release, prometheus_rabbitmq_exporter v3.7.2.4, indicates to me that it requires RabbitMQ v3.7.2. But does it work with RabbitMQ v3.7.12? What about RabbitMQ v3.7.1? By using a date, the indication is that it works with any RabbitMQ version published on or after this date. Is that a fair assumption to make? Can you think of a better approach?

Maybe @michaelklishin can suggest a better versioning schema. @dumbbell might also be able to help.

Does patched rebar3 archive work for you?

No, it doesn't. 72ad2f2 uses rebar3_archive_plugin v0.0.2 as captured in rebar.config & fails as captured in the log file.

deadtrickster · 2019-02-21T17:46:13Z

But does it work with RabbitMQ v3.7.12? What about RabbitMQ v3.7.1?

Hmm, I explained that in the README and issues, guess not clear, but - it works up to most recent. Really simple.

I just realized you tried to use rebar to build the plugin. Never tried that yet.

@deadtrickster

The resulting Docker images are only tagged with the RabbitMQ minor version, since prometheus_rabbitmq_exporter is expected to work with the latest stable RabbitMQ, regardless which that is at time of building. When these images are built, since we are using the `--pull` flag, we will always get the latest FROM image version at time of building. Without any automation, manually updating this version to the latest upstream feels inefficient and quiet boring - that would explain the version gaps in https://hub.docker.com/r/deadtrickster/rabbitmq_prometheus/tags, so I'm thinking to not use the exact RabbitMQ version in the tag for this image. I am on-board with using the PROJECT_VERSION (e.g. 3.7.2.4) if you think that is better @deadtrickster. Deleted the pure variant, it's just a duplicate of Dockerfile since #54, prometheus_process_collector ends up in both image variants.

gerhard · 2019-02-25T17:06:47Z

@deadtrickster this is now ready for you. I've tested both deadtrickster/rabbitmq_prometheus:3.7 & deadtrickster/rabbitmq_prometheus:3.7-alpine Docker image, prometheus_rabbitmq_exporter works as expected:

gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 15672 (#0)
> GET /api/metrics HTTP/1.1
> Host: localhost:15672
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-encoding: identity
< content-length: 144853
< content-type: text/plain; version=0.0.4
< date: Mon, 25 Feb 2019 17:05:48 GMT
< server: Cowboy
<
{ [16219 bytes data]
* Connection #0 to host localhost left intact

deadtrickster · 2019-02-26T07:59:32Z

Thanks, that's amazing. do you think

gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics

is a good candidate for Travis/Circle?

deadtrickster · 2019-02-26T08:10:53Z

hmm, ln -h is bsd specific?

gerhard · 2019-02-26T08:31:38Z

gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics
is a good candidate for Travis/Circle?

I almost added a make target that runs an acceptance test on both containers, but then didn't want to extend the PR too far. Yes, adding it to your CI of choice is a great idea. I would use curl's --fail --fail-early flags.

hmm, ln -h is bsd specific?

Yes, it is. -n has the same effect and is cross-platform:

LN(1)                     BSD General Commands Manual                    LN(1)

NAME
     link, ln -- make links

SYNOPSIS
     ln [-Ffhinsv] source_file [target_file]
     ln [-Ffhinsv] source_file ... target_dir

...
     -h    If the target_file or target_dir is a symbolic link, do not follow it.  This is most useful with the -f option, to replace a symlink which may point to a directory.

...

     -n    Same as -h, for compatibility with other ln implementations.

deadtrickster · 2019-02-26T08:52:00Z

so the sequence is make ezs && make docker_build && make docker_push/run. Or I'm missing the shortcut?

gerhard · 2019-02-26T09:35:01Z

Single command, multiple targets: make distclean up app ezs docker_build docker_run

gerhard · 2019-02-26T09:39:15Z

Will try on the Linux workstation later on today and add a few improvements. I've only made it work. Will make it right next & eventually make it fast 😉

gerhard · 2019-02-27T09:43:32Z

This should be easy to reproduce and guaranteed to work:

docker run --interactive --tty --volume prometheus_rabbitmq_exporter:/prometheus_rabbitmq_exporter --workdir /prometheus_rabbitmq_exporter --name prometheus_rabbitmq_exporter elixir:1.8.1 bash
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

elixir -v
Erlang/OTP 21 [erts-10.2.4] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [hipe]
Elixir 1.8.1 (compiled with Erlang/OTP 21)

git clone https://github.com/deadtrickster/prometheus_rabbitmq_exporter.git
cd prometheus_rabbitmq_exporter
apt update && apt install -y zip # required to build prometheus_rabbitmq_exporter*.ez

time make ezs
real	3m6.491s
user	1m50.850s
sys	0m38.550s
# most of the time is spent downloading the rabbit repository

ls -lh tmp/*.ez
-rw-r--r-- 1 root root  13K Feb 27 12:57 tmp/accept-0.3.5.ez
-rw-r--r-- 1 root root 205K Feb 27 12:57 tmp/prometheus-4.2.2.ez
-rw-r--r-- 1 root root  14K Feb 27 12:57 tmp/prometheus_cowboy-0.1.7.ez
-rw-r--r-- 1 root root  21K Feb 27 12:57 tmp/prometheus_httpd-2.1.10.ez
-rw-r--r-- 1 root root  18K Feb 27 12:57 tmp/prometheus_process_collector-1.4.3.ez
-rw-r--r-- 1 root root 219K Feb 27 12:58 tmp/prometheus_rabbitmq_exporter-3.7.2.4.ez

Based on the above, to build all ezs on a Linux system with Erlang 21.2 & Elixir 1.8 installed (wget & zip are also required), all ezs can be built with a single command: make ezs

I've tested the ezs produced:

docker run --detach --volumes-from prometheus_rabbitmq_exporter --name test_prometheus_rabbitmq_exporter rabbitmq:3.7-management

docker exec --interactive --tty test_prometheus_rabbitmq_exporter bash

cp /prometheus_rabbitmq_exporter/prometheus_rabbitmq_exporter/tmp/*.ez /plugins/
chown -fR rabbitmq:rabbitmq /plugins/*
rabbitmq-plugins list
Listing plugins with pattern ".*" ...
 Configured: E = explicitly enabled; e = implicitly enabled
 | Status: * = running on rabbit@3340a91c770c
 |/
[  ] prometheus_rabbitmq_exporter      3.7.2.4
[  ] rabbitmq_amqp1_0                  3.7.12
[  ] rabbitmq_auth_backend_cache       3.7.12
[  ] rabbitmq_auth_backend_http        3.7.12
[  ] rabbitmq_auth_backend_ldap        3.7.12
[  ] rabbitmq_auth_mechanism_ssl       3.7.12
[  ] rabbitmq_consistent_hash_exchange 3.7.12
[  ] rabbitmq_event_exchange           3.7.12
[  ] rabbitmq_federation               3.7.12
[  ] rabbitmq_federation_management    3.7.12
[  ] rabbitmq_jms_topic_exchange       3.7.12
[E*] rabbitmq_management               3.7.12
[e*] rabbitmq_management_agent         3.7.12
[  ] rabbitmq_mqtt                     3.7.12
[  ] rabbitmq_peer_discovery_aws       3.7.12
[  ] rabbitmq_peer_discovery_common    3.7.12
[  ] rabbitmq_peer_discovery_consul    3.7.12
[  ] rabbitmq_peer_discovery_etcd      3.7.12
[  ] rabbitmq_peer_discovery_k8s       3.7.12
[  ] rabbitmq_random_exchange          3.7.12
[  ] rabbitmq_recent_history_exchange  3.7.12
[  ] rabbitmq_sharding                 3.7.12
[  ] rabbitmq_shovel                   3.7.12
[  ] rabbitmq_shovel_management        3.7.12
[  ] rabbitmq_stomp                    3.7.12
[  ] rabbitmq_top                      3.7.12
[  ] rabbitmq_tracing                  3.7.12
[  ] rabbitmq_trust_store              3.7.12
[e*] rabbitmq_web_dispatch             3.7.12
[  ] rabbitmq_web_mqtt                 3.7.12
[  ] rabbitmq_web_mqtt_examples        3.7.12
[  ] rabbitmq_web_stomp                3.7.12
[  ] rabbitmq_web_stomp_examples       3.7.12

rabbitmq-plugins enable prometheus_rabbitmq_exporter
Enabling plugins on node rabbit@3340a91c770c:
prometheus_rabbitmq_exporter
The following plugins have been configured:
  prometheus_rabbitmq_exporter
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@3340a91c770c...
The following plugins have been enabled:
  prometheus_rabbitmq_exporter

started 1 plugins.

apt update && apt install -y curl
curl -sv -o /dev/null localhost:15672/api/metrics
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 15672 (#0)
> GET /api/metrics HTTP/1.1
> Host: localhost:15672
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-encoding: identity
< content-length: 144892
< content-type: text/plain; version=0.0.4
< date: Wed, 27 Feb 2019 13:02:25 GMT
< server: Cowboy
<
{ [102235 bytes data]
* Connection #0 to host localhost left intact

deadtrickster · 2019-02-27T10:53:29Z

I added make test. It work, but I'm not sure I manager test container in the best possible way. Also, despite @ it still produces some noise :-)

gerhard added 4 commits February 20, 2019 18:03

Capture rebar archive failure

72ad2f2

This is what happens when I use `./rebar3 archive` instead of zip. Let me know if you can figure it out @deadtrickster - I'm happy to stick with zip.

Revert "Capture rebar archive failure"

7b7980b

This reverts commit 72ad2f2.

Remove redundant comments from Makefile

7cf2cd7

gerhard added 2 commits February 21, 2019 18:25

Preserve the original versioning schema

e4adbbf

gerhard marked this pull request as ready for review February 25, 2019 17:01

deadtrickster merged commit e37018b into deadtrickster:master Feb 26, 2019

gerhard deleted the build_from_source branch February 26, 2019 08:32

gerhard mentioned this pull request Feb 27, 2019

Improve make test target #72

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build from source #71

Build from source #71

gerhard commented Feb 20, 2019 •

edited

Loading

gerhard commented Feb 20, 2019

tsloughter commented Feb 20, 2019

deadtrickster commented Feb 20, 2019

gerhard commented Feb 21, 2019 •

edited

Loading

deadtrickster commented Feb 21, 2019

gerhard commented Feb 25, 2019

deadtrickster commented Feb 26, 2019

deadtrickster commented Feb 26, 2019

gerhard commented Feb 26, 2019 •

edited

Loading

deadtrickster commented Feb 26, 2019

gerhard commented Feb 26, 2019 •

edited

Loading

gerhard commented Feb 26, 2019

gerhard commented Feb 27, 2019 •

edited

Loading

deadtrickster commented Feb 27, 2019

Build from source #71

Build from source #71

Conversation

gerhard commented Feb 20, 2019 • edited Loading

gerhard commented Feb 20, 2019

tsloughter commented Feb 20, 2019

deadtrickster commented Feb 20, 2019

gerhard commented Feb 21, 2019 • edited Loading

deadtrickster commented Feb 21, 2019

gerhard commented Feb 25, 2019

deadtrickster commented Feb 26, 2019

deadtrickster commented Feb 26, 2019

gerhard commented Feb 26, 2019 • edited Loading

deadtrickster commented Feb 26, 2019

gerhard commented Feb 26, 2019 • edited Loading

gerhard commented Feb 26, 2019

gerhard commented Feb 27, 2019 • edited Loading

deadtrickster commented Feb 27, 2019

gerhard commented Feb 20, 2019 •

edited

Loading

gerhard commented Feb 21, 2019 •

edited

Loading

gerhard commented Feb 26, 2019 •

edited

Loading

gerhard commented Feb 26, 2019 •

edited

Loading

gerhard commented Feb 27, 2019 •

edited

Loading