Home
'master' of this repository is the latest version of the sts-agent. When we want updates from the upstream dd-agent, we merge those from the upstream master into the master branch in this repository.
By convention, we git remote add upstream git@github.com:DataDog/dd-agent.
Current differences between the dd-agent and the sts-agent can be seen via git diff remotes/upstream/master master. (do git fetch --all to refresh your data)
Branch references can be found throughout the build files in docker-sts-agent-build-deb-x64 and sts-agent-omnibus. Here we maintain a list of branch references which have to be updated when forking a feature branch and merging.
- docker-sts-agent-build-deb-x64/deb-x86/Dockerfile
- docker-sts-agent-build-deb-x64/deb-x86/docker_build.sh
- sts-agent-omnibus/sotware/datadog-agent.rb:23
The differences between sts-agent and the upstream dd-agent is that we want customers to come to us for support first instead of bothering upstream, so in most customer-facing outputs we want to show 'sts' and 'StackState' rather than 'dd' and 'Datadog'. Changes include:
- /etc/dd-agent should be /etc/sts-agent
- /opt/datadog-agent should be /opt/stackstate-agent
- /var/log/datadog should be /var/log/stackstate
- config file should be stackstate.conf
- url config parameter should be sts_url
Automating these updates has proven hard to automate reliably, so our current strategy is to perform these changes manually, aided by some scripts to make detecting new violations easier.
To create a Debian package, you need the docker-sts-agent-build-deb-x64 docker container.
- Clone (https://github.com/StackVista/docker-sts-agent-build-deb-x64)
- Build
docker build --no-cache -t docker-sts-agent-build-deb-x64 .
- Add
stsbuild_rsato the directory. Should be an rsa key with access to the stackvista github repo's in the current directory - Set permission
chmod 600 ./stsbuild_rsa
- Run the builder (two strategies, for osx or linux users)
Run the builder for linux users
docker run -e OMNIBUS_SOFTWARE_BRANCH=master
-e OMNIBUS_BRANCH=withConnBeat
-e AGENT_BRANCH=withConnBeat
-e PROJECT_DIR=sts-agent-omnibus
-v <local_dir>/pkg:/sts-agent-omnibus/pkg
-v <local_dir>/cache:/var/cache/omnibus
-v <local_dir>/sts-build-keys:/sts-build-keys
docker-sts-agent-build-deb-x64
Run the builder for osx users
There seems to be an issue with volume sharing with docker OSX and compiling datadog, that results in instable compilation or hanging builds. To avoid this problem, use docker container data sharing to keep te cache:
- Create a cache container
docker create -v /var/cache/omnibus --name omnicache debian:wheezy /bin/true
docker run -e OMNIBUS_SOFTWARE_BRANCH=master
-e OMNIBUS_BRANCH=withConnBeat
-e AGENT_BRANCH=withConnBeat
-e PROJECT_DIR=sts-agent-omnibus
-v *<customdir>*/pkg:/sts-agent-omnibus/pkg
-v *<customdir>*/sts-build-keys:/sts-build-keys
--volumes-from omnicache
docker-sts-agent-build-deb-x64
Once you have docker-sts-agent-build-deb-x64, you can use it to build for example the 'withConnBeat' branch of the agent (replace local paths with paths valid on your machine):
This currently always builds a '0.0.0' version of the package.
Demo plugin that emits one event and one gauge per set interval.
~/.datadog-agent/checks.d/eventspammer.py
from checks import AgentCheck
import time
class EventSpammer(AgentCheck):
def check(self, instance):
self.log.info("emitting dummy gauge")
self.gauge('eventspammer.gauge', 1)
self.log.info("emitting dummy event")
self.event({
'timestamp': int(time.time()),
'source_type_name': "event_spammer_source",
'api_key' : "ABCDEFGHI",
'msg_title': "test title",
'msg_text': "test msg",
'tags': [
'test:tag',
'tag:1'
]
})
~/.datadog-agent/conf.d/eventspammer.yaml
init_config:
min_collection_interval: 5
instances:
[{}]
You may want to restart datadog agent.
HTTP POST to: /intake/?api=<api_key>
Note: truncated processes and metrics
{
"agentVersion": "5.8.0",
"apiKey": "5f98193e83ece68c811df22174859355",
"collection_timestamp": 1467037580.595086,
"cpuIdle": 38.0,
"cpuStolen": 0,
"cpuSystem": 54.0,
"cpuUser": 8.0,
"cpuWait": 0,
"events": {
"eventspammer": [
{
"api_key": "ABCDEFGHI",
"msg_text": "test msg",
"msg_title": "test title",
"source_type_name": "event_spammer_source",
"tags": [
"test:tag",
"tag:1"
],
"timestamp": 1467037584
}
]
},
"external_host_tags": {},
"host-tags": {},
"internalHostname": "mac-wytze",
"ioStats": {
"disk0": {
"system.io.bytes_per_s": 660602.88
},
"disk2": {
"system.io.bytes_per_s": 0.0
},
"disk3": {
"system.io.bytes_per_s": 0.0
},
"disk4": {
"system.io.bytes_per_s": 0.0
}
},
"memBuffers": null,
"memCached": null,
"memPageTables": null,
"memPhysFree": 649.0625,
"memPhysPctUsable": 0.41200000000000003,
"memPhysTotal": null,
"memPhysUsable": 6744.97265625,
"memPhysUsed": 14711.33984375,
"memShared": null,
"memSlab": null,
"memSwapCached": null,
"memSwapFree": 786.0,
"memSwapPctFree": null,
"memSwapTotal": null,
"memSwapUsed": 1262.0,
"metrics": [
[
"system.disk.total",
1467037584,
487374848.0,
{
"device_name": "/dev/disk1",
"hostname": "mac-wytze",
"type": "gauge"
}
],
[
"system.disk.free",
1467037584,
413437364.0,
{
"device_name": "/dev/disk1",
"hostname": "mac-wytze",
"type": "gauge"
}
],
[
"eventspammer.gauge",
1467037584,
1,
{
"hostname": "mac-wytze",
"type": "gauge"
}
],
[
"system.net.packets_out.count",
1467037584,
470.4736842105263,
{
"device_name": "en0",
"hostname": "mac-wytze",
"type": "gauge"
}
],
[
"system.net.packets_in.count",
1467037584,
554.4736842105264,
{
"device_name": "en0",
"hostname": "mac-wytze",
"type": "gauge"
}
]
],
"os": "mac",
"processes": {
"apiKey": "5f98193e83ece68c811df22174859355",
"host": "mac-wytze",
"processes": [
[
"wytzehazenberg",
"80987",
"0.0",
"0.1",
"2503228",
"9880",
"s004",
"S+",
"2:12PM",
"0:23.20",
"vim"
],
[
"wytzehazenberg",
"79040",
"0.0",
"0.2",
"2527308",
"37844",
"??",
"S",
"2:09PM",
"0:15.82",
"/opt/datadog-agent/embedded/bin/python /opt/datadog-agent/agent/agent.py foreground --use-local-forwarder"
],
[
"wytzehazenberg",
"79039",
"0.0",
"0.2",
"2540112",
"29924",
"??",
"S",
"2:09PM",
"0:11.55",
"/opt/datadog-agent/embedded/bin/python /opt/datadog-agent/agent/ddagent.py"
],
[
"wytzehazenberg",
"79038",
"0.0",
"0.2",
"2530504",
"32852",
"??",
"S",
"2:09PM",
"0:11.14",
"/opt/datadog-agent/embedded/bin/python /opt/datadog-agent/agent/dogstatsd.py --use-local-forwarder"
],
[
"wytzehazenberg",
"79035",
"0.0",
"0.0",
"2478400",
"5540",
"??",
"Ss",
"2:09PM",
"0:01.99",
"/opt/datadog-agent/embedded/bin/python /opt/datadog-agent/bin/supervisord -c /opt/datadog-agent/etc/supervisor.conf"
],
[
"wytzehazenberg",
"58859",
"0.0",
"0.0",
"2435856",
"680",
"s009",
"S+",
"1:32PM",
"0:00.05",
"tail -f ./collector.log"
]
]
},
"python": "2.7.11 (default, May 16 2016, 13:19:07) \n[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]",
"resources": {},
"service_checks": [
{
"check": "datadog.agent.check_status",
"host_name": "mac-wytze",
"id": 2091,
"message": null,
"status": 0,
"tags": [
"check:ntp"
],
"timestamp": 1467037584.680259
},
{
"check": "datadog.agent.check_status",
"host_name": "mac-wytze",
"id": 2092,
"message": null,
"status": 0,
"tags": [
"check:disk"
],
"timestamp": 1467037584.681405
},
{
"check": "datadog.agent.check_status",
"host_name": "mac-wytze",
"id": 2093,
"message": null,
"status": 0,
"tags": [
"check:eventspammer"
],
"timestamp": 1467037584.682277
},
{
"check": "datadog.agent.check_status",
"host_name": "mac-wytze",
"id": 2094,
"message": null,
"status": 0,
"tags": [
"check:network"
],
"timestamp": 1467037584.709078
},
{
"check": "datadog.agent.up",
"host_name": "mac-wytze",
"id": 2095,
"message": null,
"status": 0,
"tags": null,
"timestamp": 1467037584.709122
}
],
"system.load.1": 3.31,
"system.load.15": 2.25,
"system.load.5": 2.62,
"system.load.norm.1": 0.41375,
"system.load.norm.15": 0.28125,
"system.load.norm.5": 0.3275,
"system.uptime": 862103.6049060822,
"uuid": "f8c341c7a50a53759d8726c8c918c0a2"
}
HTTP POST: /api/v1/series/?api_key=<api_key>
{
"series": [
{
"device_name": null,
"host": "mac-wytze",
"interval": 10.0,
"metric": "datadog.dogstatsd.packet.count",
"points": [
[
1467037580.0,
0
]
],
"tags": null,
"type": "gauge"
},
{
"device_name": null,
"host": "mac-wytze",
"interval": 10.0,
"metric": "datadog.dogstatsd.serialization_status",
"points": [
[
1467037593.243472,
0.1
]
],
"tags": [
"status:success"
],
"type": "rate"
}
]
}
The following output is generated by executing the following command multiple times on a running StatsD instance with the datadog backend installed;
echo "statsd_test:1|c" | nc -u -w0 127.0.0.1 8125
The StatsD backend can be found here: https://github.com/DataDog/statsd-datadog-backend/. The backend sends StatsD's output directly to the API.
Backend tag used: statsd:testtag
HTTP POST: /api/v1/series?api_key=<api_key>
{
"series": [
{
"host": "mac-wytze",
"metric": ".statsd.bad_lines_seen",
"points": [
[
1467105304,
0
]
],
"tags": [
"statsd:testtag"
],
"type": "gauge"
},
{
"host": "mac-wytze",
"metric": ".statsd.packets_received",
"points": [
[
1467105304,
1.5
]
],
"tags": [
"statsd:testtag"
],
"type": "gauge"
},
{
"host": "mac-wytze",
"metric": ".statsd.metrics_received",
"points": [
[
1467105304,
1.5
]
],
"tags": [
"statsd:testtag"
],
"type": "gauge"
},
{
"host": "mac-wytze",
"metric": ".statsd_test",
"points": [
[
1467105304,
1.5
]
],
"tags": [
"statsd:testtag"
],
"type": "gauge"
},
{
"host": "mac-wytze",
"metric": ".statsd.timestamp_lag",
"points": [
[
1467105304,
0
]
],
"tags": [
"statsd:testtag"
],
"type": "gauge"
}
]
}