-
Notifications
You must be signed in to change notification settings - Fork 22
Shard Management Commands (OLAP)
[Table of Contents](https://github.com/dell-oss/Doradus/wiki/OLAP Databases: Table-of-Contents) | Previous | Next
OLAP REST Commands: Shard Management Commands
Doradus OLAP provides the following commands to list, merge, and delete shards.
The list of all known shards of a given application can be obtained with the following REST command:
GET /{application}/_shards
Each shard that has at least one batch posted are listed, even if the shard has not yet been merged. A typical response in XML:
<result>
<application name="Email">
<shards>
<value>2004-11-16</value>
<value>2005-08-03</value>
<value>2005-10-10</value>
<value>2005-11-01</value>
<value>2005-12-20</value>
<value>2006-01-17</value>
<value>2006-02-01</value>
</shards>
</application>
</result>
In JSON:
{"result": {
"Email": {
"shards": [
"2004-11-16",
"2005-08-03",
"2005-10-10",
"2005-11-01",
"2005-12-20",
"2006-01-17",
"2006-02-01"
]
}
}}
Statistics for a specific shard can be retrieved with any of the following REST commands:
GET /{application}/_shards/{shard}
GET /{application}/_statistics/{shard}
GET /{application}/_statistics/{shard}?{params}
Statistics will be available for the given {shard}
only if at least one batch has been loaded for it and the shard has been merged. The first command shown returns general information for each table and field. A typical response in XML is shown::
<stats memory="0.984 MB" storage="1.654 MB" documents="28455">
<tables>
<table documents="370" memory="0.010 MB" name="Domain">
<numFields>
<field type="BOOLEAN" min="0" max="1" min_pos="1" bits="1" memory="0.000 MB" name="IsInternal"/>
</numFields>
<textFields>
<field valuesCount="370" doclistSize="370" isSingleValued="true" memory="0.001 MB" name="Name"/>
</textFields>
<linkFields>
<field linkedTableName="Address" inverseLink="Domain" doclistSize="1884" isSingleValued="false" memory="0.009 MB" name="Addresses"/>
</linkFields>
</table>
<table documents="6030" memory="0.272 MB" name="Message">
...
</table>
...
</tables>
</stats>
In JSON:
{"stats": {
"memory": "0.984 MB",
"storage": "1.654 MB",
"documents": "28455",
"tables": {
"Domain": {
"documents": "370",
"memory": "0.010 MB",
"numFields": {
"IsInternal": {"type": "BOOLEAN", "min": "0", "max": "1", "min_pos": "1", "bits": "1", "memory": "0.000 MB"}
},
"textFields": {
"Name": {"valuesCount": "370", "doclistSize": "370", "isSingleValued": "true", "memory": "0.001 MB"}
},
"linkFields": {
"Addresses": {"linkedTableName": "Address", "inverseLink": "Domain", "doclistSize": "1884", "isSingleValued": "false", "memory":"0.009 MB"}
}
},
"Message": {
...
},
...
}
}}
The command response provides statistics about data contained in the given shard such as the total number of objects, the number of objects per table, the name value range of scalar fields, and so forth.
The second shard statistics command returns more similar information as the first command plus additional details about the virtual files (segments) stored within the shard. The third statistics command dumps the contents of a specific virtual file. These commands are intended for debugging purposes.
All batches that have been loaded for a given shard are merged with the following REST command:
POST /{application}/_shards/{shard}[?expire-date=<timestamp>]
where {application}
is the owning application's name and {shard}
is the shard name. The command requires no input entity. It merges all segments synchronously and returns when the merge is complete.
The Merge Shard command optionally assigns an expiration date to the shard via the expire-date
parameter. The given <timestamp>
must be in the format of a timestamp (YYYY-MM-DD HH:MM:SS
); trailing elements can be omitted. Example:
POST /Email/_shards/2014-01-01?expire-date=2015-01-01
If the expire-date
is not given, the shard will have no expiration date, even if it was previously assigned one. If the command is successful, a 200 OK
response is returned, and all updates will be visible to queries.
Note that only one process can submit a merge request for a given shard at a time. If a merge request is submitted for a shard that is already undergoing a merge request, the second request immediately receives an error.
A shard can be explicitly deleted with the following command:
DELETE /{application}/_shards/{shard}
All of the shard's data is removed, even if it was previously assigned an expire-date
that has not yet passed. If the request is successful, a 200 OK
response is returned.
Technical Documentation
[Doradus OLAP Databases](https://github.com/dell-oss/Doradus/wiki/Doradus OLAP Databases)
- Architecture
- OLAP Database Overview
- OLAP Data Model
- Doradus Query Language (DQL)
- OLAP Object Queries
- OLAP Aggregate Queries
- OLAP REST Commands
- Architecture
- Spider Database Overview
- Spider Data Model
- Doradus Query Language (DQL)
- Spider Object Queries
- Spider Aggregate Queries
- Spider REST Commands
- [Installing and Running Doradus](https://github.com/dell-oss/Doradus/wiki/Installing and Running Doradus)
- [Deployment Guidelines](https://github.com/dell-oss/Doradus/wiki/Deployment Guidelines)
- [Doradus Configuration and Operation](https://github.com/dell-oss/Doradus/wiki/Doradus Configuration and Operation)
- [Cassandra Configuration and Operation](https://github.com/dell-oss/Doradus/wiki/Cassandra Configuration and Operation)