Bunyan-dynamo

A Bunyan stream to Amazon's DynamoDB, allowing you to store your application's logs in Amazon's document storage.

Getting Started

Install bunyan-dynamo using npm and save it as a dependency.

npm install bunyan-dynamo --save

Use bunyan-dynamo like any other Bunyan stream.

var bunyan = require('bunyan'),
	BunyanDynamo = require('bunyan-dynamo');

var bunyanDynamoOptions = {
	"aws": {										// Accepts all AWS SDK options.
		"accessKeyId": "MyAccessKeyId",				// AWS access key ID.
        "secretAccessKey": "MySecretAccessKey"		// AWS secret access key.
    },
    "tableName": "myAppLogs",						// Name of the database table.
    "tableHashKey": "id",							// Partial key for database.
    "tableHashType": BunyanDynamo.TYPE_STRING 		// Hash key data type.
}

var bunyanOptions = {
	name: "MyAppLogger",							// Name of the bunyan logger.
	serializers: bunyan.stdSerializers,				// Bunyan serializers.
	level: 'info',									// Bunyan log level.
	type: 'raw',									// Bunyan type.
    stream: new BunyanDynamo(bunyanDynamoOptions) 	// Bunyan-Dynamo stream.
}

var log = bunyan.createLogger(bunyanOptions);

log.info("Logging to DynamoDB.");

Here is what the record created in DynamoDB looks like.

Time

Time is stored as a number in milliseconds since January 1, 1970, 00:00:00 UTC (zulu time). Amazon's SDK may return this value as a string, so use the following to convert time into an object.

var time = "1466190023000";           // This is the time value returned from DynamoDB using amazon's SDK.

console.log(new Date(Number(time)));  // Fri Jun 17 2016 15:00:23 GMT-0400 (EDT)

Options

Property	Type	Default	Description
aws	Object	{}	Options passed directly into the aws-sdk for DynamoDB.
aws.accessKeyId	String	undefined	Your AWS access key ID.
aws.apiVersion	String	2012-08-10	The AWS API version of DynamoDB to use.
aws.maxRetries	Number	15	Maximum amount of retries to attempt with a request to AWS.
aws.profile	String	default	AWS credential profile to use.
aws.region	Number	us-east-1	AWS region to send service requests to.
aws.secretAccessKey	String	undefined	Your AWS secret access key.
batchSize	Number	25	Number of log messages to send at a time to the DynamoDB service.
debug	Boolean	false	When enabled, additional log messages will be displayed and configurations used to help debug the module.
enableHostname	Boolean	true	When enabled, the hostname will be included as an attribute in each item saved to the DynamoDB table.
sendInterval	Number	5000	How often, in milliseconds, to send log messages to the DynamoDB service. Default send interval in debug and trace mode is 1000.
tableName	String	<APP_NAME>_<HOSTNAME>_<PORT>	Name of the database table, which must be unique. If not defined the module will attempt to create a unique name using the application's name, hostname, and port. See environment variables.
tableHashKey	String	id	Name of the partial key for the database table.
tableHashType	String	S	Data type of the hash key. (S stands for String)
tableRangeKey	String	time	Name of the sort key for the database table.
tableRangeType	String	N	Data type of the range key. (N stands for Number)
tableReadCapacity	Number	5	AWS read capacity for the table.
tableWriteCapacity	Number	5	AWS write capacity for the table.
trace	Boolean	false	When enabled, debug mode and trace messages will be displayed and additional configurations used to help debug the module.

Environment Variables

The module can take advantage of some optional environment variables.

Property	Type	Default	Description
APP_NAME	String	undefined	Name of the application. Used to uniquely generate a table name if one has not already been defined in the options object.
PORT	Number	undefined	Port the application is listening on. Used to uniquely generate a table name if one has not already been defined in the options object.

AWS Credentials

Credentials can either be passed in as options or configured any other way allowed by the amazon sdk

Static Properties

Method	Type	Description
DYNAMO_TYPE_ARRAY_BINARY	String	Amazon's DynamoDB data type identifier for an array of binary data.
DYNAMO_TYPE_ARRAY_MAP	String	Amazon's DynamoDB data type identifier for an array of map data.
DYNAMO_TYPE_ARRAY_NUMBER	String	Amazon's DynamoDB data type identifier for an array of numbers.
DYNAMO_TYPE_ARRAY_STRING	String	Amazon's DynamoDB data type identifier for an array of strings.
DYNAMO_TYPE_BINARY	String	Amazon's DynamoDB data type identifier for binary data.
DYNAMO_TYPE_BOOLEAN	String	Amazon's DynamoDB data type identifier for a boolean variable.
DYNAMO_TYPE_MAP	String	Amazon's DynamoDB data type identifier for map data.
DYNAMO_TYPE_NULL	String	Amazon's DynamoDB data type identifier for null data.
DYNAMO_TYPE_NUMBER	String	Amazon's DynamoDB data type identifier for a number.
DYNAMO_TYPE_STRING	String	Amazon's DynamoDB data type identifier for a string.

Example

var BunyanDynamo = require('bunyan-dynamo');
console.log("The DynamoDB string type identifier is %s.", BunyanDynamo.DYNAMO_TYPE_STRING); // 

Example Output

The DynamoDB string type identifier is S.

Instance Methods

Get Config

Returns the current bunyan-dynamo configuration object.

Example

var BunyanDynamo = require('bunyan-dynamo'),
    bunyanDynamoStream = new BunyanDynamo();

console.log(bunyanDynamoStream.getConfig());

Example Output

{ 
    aws: { 
        apiVersion: '2012-08-10', 
        maxRetries: 15, 
        region: 
        'us-east-1' 
    },
    enableHostname: true,
    batchSize: 25,
    sendInterval: 5000,
    tableName: 'MyAppName_MyHostName_3000',
    tableHashKey: 'id',
    tableHashType: 'S',
    tableRangeKey: 'time',
    tableRangeType: 'N',
    tableReadCapacity: 5,
    tableWriteCapacity: 5
}

Set Config

Set or change the configuration options.

Example

var BunyanDynamo = require('bunyan-dynamo'),
    bunyanDynamoStream = new BunyanDynamo();

bunyanDynamoStream.setConfig({
    batchSize: 50,
    sendInterval: 60000,
    tableName: "MyTableName"
});

console.log(bunyanDynamoStream.getConfig());

Example Output

{ 
    aws: { 
        apiVersion: '2012-08-10', 
        maxRetries: 15, 
        region: 
        'us-east-1' 
    },
    enableHostname: true,
    batchSize: 50,
    sendInterval: 60000,
    tableName: 'MyTableName',
    tableHashKey: 'id',
    tableHashType: 'S',
    tableRangeKey: 'time',
    tableRangeType: 'N',
    tableReadCapacity: 5,
    tableWriteCapacity: 5
}