Theory of Operation

If you ever tried to do analytics, you will have encountered two problems: getting information and showing information. Between these two problem lies some kind of processing to gather and aggregate the incoming data into useful information for the user.

For example, analytics for visits on a web site typically involves getting data from the logs or using some quick web process. Then, you must process this information so the display process is as fast as possible. For a large amount of information this process can be very slow so big sites do this asynchronously, this is why most analytics services are not in real time.

Trackoid tries to solve this problem and do analytics in real time by taking advantage of a Document Oriented Database such as MongoDB. Please visit MongoDB site to know more about this excellent database. MongoDB is heavily optimized for write operations, in a “fire and forget” philosophy. This coupled with a clever document schema would permit us forget about the aggregation process so the data is immediately available for displaying.

Trackoid also takes advantage of the Document model for the database, storing all track analysis information inside the models themselves so when you read the model, you also have the analysis information at the same time.

Upsert operations

Trackoid uses “upsert” operations for all it’s writes. Those are special update commands of the MongoDB database which are equivalents of “INSERT … ON DUPLICATE KEY UPDATE” SQL statements or “find_or_create” ActiveModel, etc. In short, they are atomic updates who creates the record if it is not already in the database. For a document database it has another implications: since it’s an schema less database, upserts operations also atomically creates a new field or part of it.

Using MongoDB and upsert operations enables Trackoid to be used in a highly scalable web application. You can have many processes hitting Trackoid without ever losing a bit.

Document schema

MongoDB is a schema less database. Trackoid is able then to insert a field into each tracked model. This field is two level Hash which stores day, hour & count.

Example JSON document:

{
  visits_data : {
    "15456"=>{"22"=>7}
    "15458"=>{"8"=>1, "11"=>2, "12"=>3, "13"=>2, "14"=>1, "22"=>3}
  }
}

Example upsert command issued to MongoDB

document.inc("views_data.15564.0", 1)