Money Configuration

Money uses the Typesafe Config Library for configuring the system. Money follows the recommendations set forth by the config library; namely it provides "sane" default values for everything that can be configured, and allows you to override the default configuration by providing a file named application.conf on the classpath of your application.

Core configuration

• application-name - set this to the name you want reported for your application. It is best not to use spaces in the application name. This value is reported on each Span as an app-name Note. If you see app-name=unknown, it indicates that you have not configured the application-name.
• override-environment-variable - Defaults to MONEY_ENV. Money provides a built in facility that allows you to have different configuration files for different environments. This value is set to a string that is used to lookup the application configuration file. For example, if you had a test environment and a prod environment, you would have an application.test.conf and application.prod.conf files both available on your classpath. When the environment variable MONEY_ENV=test, then Money will load with the application.test.conf file. When the environment variable is MONEY_ENV=prod, then Money will use the overrides in the application.prod.conf file. More information can be found below.
• enabled - true or false, depending on whether you want money turned on or not. If this is set to false, then all requests through the Money system will be no-op.
• span-timeout - this is an important configuration flag, that specifies how long a single Span is allowed to be open before it is shut down. Spans that timeout are not emitted; but a log entry will be made indicating that the span timed out. Also, a JMX property will be updated indicating a timed out span. This value should be set to the shortest acceptable time duration; otherwise, under heavy volume you can experience increased memory pressure due to lots of spans hanging out waiting to timeout.
• stopped-span-timeout - this is how long a span will remain open to accept additional notes after it has been Stopped. We keep spans open for a short time to allow fire-and-forget type of requests to complete and log data. We also keep spans open for Scala Futures, which can stop a span multiple times depending upon how they are used. This should also be kept to the smallest possible value. The default is 100 milliseconds, but it should be only 5 or 10 milliseconds.
• mdc.enabled - true or false, indicates whether or not you want to use the SLF4J MDC value. If enabled, then you can include the current Trace Context in every log line. This is very helpful when correlating log entries with span data.
• akka.loglevel - the log level for the Money Akka system. This is by default set to INFO, and it must be at least WARN in order to get Span Data. The default is usually acceptable.
• emitters - emitters will allow you to configure the individual emitters that will be handling trace data. They are included in their own section below.

Example Configuration File:

money {
  application-name = "my-app"
  enabled = true
  span-timeout = 5 seconds
  stopped-span-timeout = 5 ms

  mdc {
    enabled = true
  }

  akka {
    loglevel = INFO
    log-config-on-start = off
  }
}

Emitters

Emitters are the method by which Money will send (or emit) data. Money comes with a few out-of-the-box Emitters that you can use.

The Emitter Configuration is an Array of emitters, so you can have 0 or 100 different emitters

Log Emitter

The log emitter is our standard setup that we use by default. This allows us to get data into Heka, LogStash, or Splunk very easily. There is no configuration needed other than enabling it in the configuration file.

All logging in Money is performed via SLF4J. You need to configure your application as appropriate to log data, we typically use a non-blocking logger like Logback.

Configuration Example

money {
  emitter {
    emitters = [
      {
        name = "log-emitter"
        class-name = "com.comcast.money.emitters.LogEmitter"
        subscriptions = [Trace]
        configuration = {
          emitter = "com.comcast.money.emitters.LogEmitter"
        }
      }
    ]
  }
}

Span Metrics Emitter

The Span Metrics Emitter calculates aggregate data for each unique span-name in the application. Even in large applications, the total number of unique span names (operations) is below 1000. The Span Metrics Emitter exposes the metrics it collects via JMX. All metrics are captured using Dropwizard Metrics.

The Span Metrics Emitter calculates the following aggregates

• latency histogram - calculates the distribution of latency (duration) of each span name - See Histogram
• error rate - calculates the rate of errors for a particular span - See Meters

Configuration Example

money {
  emitter {
    emitters = [
      {
        name = "span-metrics-emitter"
        class-name = "com.comcast.money.metrics.SpanMetricsCollector"
        subscriptions = [Trace]
        configuration = {
        }
      }
    ]
  }
}

Graphite Emitter

The Graphite Emitter supports sending metrics to a Graphite server.

• Configuration Example *

         {
             name = "graphite-emitter"
             class-name = "com.comcast.money.emitters.GraphiteEmitter"
             subscriptions = [Trace, Metric]
             configuration = {
                 app-name = "inventory-management"
                 port = 2003
                 hostname = "graphite"
                 enabled = true
                 emitterPoolSize = 1
                 metric {
                   emitter = "com.comcast.money.emitters.GraphiteMetricEmitter"
                 }
             }
         }