intro.tex

% !TeX root = sr.tex
\section{Introduction}

\subsection{Measuring Storage Resource Consumption}

% storeage resource intro
Measuring consumption of storage resources is distinctively different from
the measurement of the consumption of computing resources in batch jobs. 
On a computing resource it is quite easy to continuously collect CPU 
usage information with a high time resolution - the operating system itself does 
it and an accounting system can access these data. Additionally,
the resource usage data are usually strictly increasing, e.g. 
CPU time or wall clock time.
A collection of storage resource consumption is much harder
to achieve. The usage of each participant will vary over time and only few 
systems do permanently record the storage use of the single users or 
groups with a fine granularity in time. 
Usage snapshot are taken from time to time only
resulting in a rough consumption estimation especially in environments
with a high fluctuation rate. 
Nevertheless, there is a need to do storage accounting both for the provider
of storage resources and the users. It is the base for billing and for 
the development of a storage infrastructure.

% While jobs typically have exclusive access
% to a set of resources (CPU cores) for a given amount of time, a storage
% resource is almost always shared
% among several parties, with each of these participants using a part of the
% resource. Furthermore, the usage of each participant will vary over time, and as
% data tend to be long-lived there is no fixed start and end time of the resource
% consumption.

% integrals and spikes
%To exactly measure the amount of consumed resource\ldots

\subsection{About this document}
In the following the StAR format 
(\textbf{St}orage \textbf{A}ccounting \textbf{R}ecord) is described 
that allows to report the usage of 
storage resources. StAR tries to define a standardized way to exchange storage
consumption data. Though it was developed in the context of distributed storage
in a Grid environment it is not specific to such a storage infrastructure.
Due to StAR's generic approach it can be used in any storage environment.

This document does not address in detail how the StAR records should be used, 
how the accounting data are aggregated 
nor does it attempt to dictate the format in which the accounting records are 
stored at a local site. 
Furthermore, nothing is said regarding the communication mechanisms
employed to exchange the records, i.e. transport layer, framing, 
authentication, integrity, etc.


\subsection{Record Structure \& Content}

The structure of the format described in this document can be split into
logical parts, each describing an aspect of the resource consumption.
The parts are
\begin{description}

\item[Resource:] Fields describing the system the resource was consumed on. They 
can specify a certain subsystem of the storage system.

\item[Consumption Details:] Fields describing what the data is consuming, e.g. 
storage classes, directory path, etc.

\item[Identity:] Describes the person or group accountable for the resource
consumption.

\item[Resource Consumption:] Fields describing how much of the described
resource has been used.

\end{description}

Please note, that these logical sections are not necessarily directly reflected
in record format. They are merely a good mental model to have in mind.

Most of the information in the record is common to all files, e.g. resource or
identity description. Certain fields are aggregates over the consumed
resources. This includes the consumption itself, but can also be consumption
details, such as number of files.

The record is not intended to be used for describing intricate information
about the consumption. E.g., file names, per-file data or application
metadata should not be included in the record. Such details are out of scope
for the record and are not important for accounting of resource consumption.

To see examples of records, see section~\ref{sec:record-examples}.


% this might be a seperate section sometime, for now it is here
\subsection{Including Additional Information}

It is allowed to add additional fields with information in the record, e.g.
for a more accurate description of the data that has consumed the resources.

If any user or group information is added it must be added under the
SubjectIdentity block. This makes it possible to automatically remove user
and group information.