Skip to content

Commit

Permalink
Merge pull request #21 from Tencent/dev_1.0
Browse files Browse the repository at this point in the history 
merge dev_1.0 to master
  • Loading branch information
@shanbaoyin
shanbaoyin authored Jun 18, 2019
2 parents 38fd119 + 9256210 commit c366bcc
Show file tree
Hide file tree
Showing 79 changed files with 9,188 additions and 3,988 deletions.
4 changes: 2 additions & 2 deletions docs-en/install-en.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ change directory to `src/thirdParty`, issue commands: `chmod +x thirdparty.sh; .

## Compilation

Issue command: `make release` in `src/` directory, then go to directory of each service, issue the following commands:
Issue command: `make release;make` in `src/` directory, then go to directory of each service, issue the following commands:
```
make; make tar
make tar
```
to generate a Tars release package for each service.

Expand Down
148 changes: 102 additions & 46 deletions docs-en/server_config_example-en.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ This document describes and examples the configuration of each module of DCache.
# ConfigServer Configuration

```
# database infomation
# database infomation of db_dcache_relation
<Main>
<DB>
dbhost=
Expand Down Expand Up @@ -283,7 +283,7 @@ This document describes and examples the configuration of each module of DCache.
ProxyMaxSilentTime=1800
# interval for clearing Proxies
ClearProxyInterval=1800
# 轮询迁移数据库的时间
# interval for polling unstart transfer task in db
TransferInterval=3
# number of threads used for polling
TimerThreadSize=20
Expand All @@ -305,7 +305,9 @@ This document describes and examples the configuration of each module of DCache.
MaxTransferThreadEachGroup=8
</Transfer>
<Switch>
# 自动切换超时的检测间隔(秒)
# whether to enable auto switch
enable = N
# interval for checking whether need to switch (seconds)
SwitchCheckInterval= 10
# timeout for automatic switching (seconds)
SwitchTimeOut=60
Expand Down Expand Up @@ -345,17 +347,20 @@ This document describes and examples the configuration of each module of DCache.
```
<Main>
<DB>
Sql=CREATE TABLE `${TABLE}` (`stattime` timestamp NOT NULL default CURRENT_TIMESTAMP,`f_date` date NOT NULL default '1970-01-01',`f_tflag` varchar(8) NOT NULL default '',`app_name` varchar(20) default NULL,`module_name` varchar(50) default NULL,`group_name` varchar(100) default NULL,`idc_area` varchar(10) default NULL,`server_status` varchar(10) default NULL,`master_name` varchar(128) NOT NULL default '',`master_ip` varchar(16) default NULL,`set_name` varchar(15) NOT NULL default '',`set_area` varchar(15) NOT NULL default '',`set_id` varchar(15) NOT NULL default '',`value1` varchar(255) default NULL,`value2` varchar(255) default NULL,`value3` varchar(255) default NULL,`value4` varchar(255) default NULL,`value5` varchar(255) default NULL,`value6` varchar(255) default NULL,`value7` varchar(255) default NULL,`value8` varchar(255) default NULL,`value9` varchar(255) default NULL,`value10` varchar(255) default NULL,`value11` varchar(255) default NULL,`value12` varchar(255) default NULL,`value13` varchar(255) default NULL,`value14` varchar(255) default NULL,`value15` varchar(255) default NULL,`value16` varchar(255) default NULL,`value17` varchar(255) default NULL,`value18` varchar(255) default NULL,`value19` varchar(255) default NULL,`value20` varchar(255) default NULL,`value21` varchar(255) default NULL,`value22` varchar(255) default NULL,`value23` varchar(255) default NULL,`value24` varchar(255) default NULL,`value25` varchar(255) default NULL,`value26` varchar(255) default NULL,`value27` varchar(255) default NULL,`value28` varchar(255) default NULL,`value29` varchar(255) default NULL,`value30` varchar(255) default NULL,`value31` varchar(255) default NULL,`value32` varchar(255) default NULL,`value33` varchar(255) default NULL,`value34` varchar(255) default NULL,`value35` varchar(255) default NULL,KEY(`f_date`,`f_tflag`,`master_name`,`master_ip`),KEY `IDX_MASTER_NAME` (`master_name`),KEY `IDX_MASTER_IP` (`master_ip`),KEY `IDX_TIME` (`stattime`),KEY `IDX_F_DATE` (`f_date`))ENGINE\=MyISAM
TbNamePre=t_property_realtime
AppName=dcache_idc5min_147
#maximum properties to monitoring
PropertyFieldNum=48
TbNamePre=t_property_realtime
AppName=dcache_idc5min_147
#database to stroe monitoring data
<property>
dbhost=
dbname=taf_property_147
dbname=db_dcache_property
dbuser=
dbpass=
dbport=
charset=gbk
</property>
#database of db_dcache_relation
<relation>
charset=gbk
dbname=db_dcache_relation
Expand All @@ -366,47 +371,62 @@ This document describes and examples the configuration of each module of DCache.
</relation>
</DB>
<HashMap>
InsertInterval=1
#interval to dump memory data to database(minutes)
InsertInterval=5
</HashMap>
<NameMap>
BakCenterError = property1
BinLogErr = property2
BinLogSyn = property3
CacheError = property4
Chunks/OnceElement = property5
BackUpObjAdapter.connectRate = property6
BackUpObjAdapter.queue = property7
BinLogObjAdapter.connectRate = property8
BinLogObjAdapter.queue = property9
CacheObjAdapter.connectRate = property10
CacheObjAdapter.queue = property11
RouterClientObjAdapter.connectRate = property12
RouterClientObjAdapter.queue = property13
WCacheObjAdapter.connectRate = property14
WCacheObjAdapter.queue = property15
asyncqueue = property16
memsize = property17
DataMemUsedRatio = property18
DbError = property19
DbException = property20
DirtyNum = property21
DirtyRatio = property22
ElementCount = property23
Exception = property24
HitCount = property25
MemSize = property26
getBatchCount = property27
setBatchCount = property28
MKMemUsedRatio = property29
eraseCount = property30
eraseCountUnexpire = property31
asyncqueue0 = property32
Jmem0DataUsedRatio = property33
Jmem1DataUsedRatio = property34
ColdDataRatio = property35
expireCount = property36
OnlyKeyCount = property37
BigChunk = property38
# value1 to value20 for obj and tars server fixed property
CacheObjAdapter.queue = value1
WCacheObjAdapter.queue = value2
BinLogObjAdapter.queue = value3
RouterClientObjAdapter.queue = value4
ControlAckObjAdapter.queue = value5
BackUpObjAdapter.queue = value6
sendrspqueue = value7
asyncqueue = value8
memsize = value9
reservedPro10 = value10
reservedPro11 = value11
reservedPro12 = value12
reservedPro13 = value13
reservedPro14 = value14
reservedPro15 = value15
reservedPro16 = value16
reservedPro17 = value17
reservedPro18 = value18
reservedPro19 = value19
reservedPro20 = value20
# value21 for dcache property
CacheMemSize_MB = value21
DataMemUsage = value22
CacheHitRatio = value23
TotalCountOfRecords = value24
ProportionOfDirtyRecords = value25
CountOfDirtyRecords = value26
CountOfOnlyKey = value27
M/S_ReplicationErr = value28
M/S_ReplicationLatency = value29
CacheError = value30
ProgramException = value31
ReadRecordCount = value32
WriteRecordCount = value33
evictedRecordCount = value34
evictedCountOfUnexpiredRecord = value35
expiredRecordCount = value36
ChunkUsedPerRecord = value37
MaxMemUsageOfJmem = value38
ProportionOfColdData = value39
DbError = value40
DbException = value41
BackupError = value42
MKV_MainkeyMemUsage = value43
</NameMap>
</Main>
Expand Down Expand Up @@ -434,3 +454,39 @@ This document describes and examples the configuration of each module of DCache.
RouterObj = DCache.RouterServer.RouterObj
</Main>
```

# OptServer Configuration

```
<Main>
# admin registry obj
AdminRegObj = tars.tarsAdminRegistry.AdminRegObj
<DB>
<tars>
charset = utf8
dbname = db_tars
dbhost =
dbport =
dbuser =
dbpass =
</tars>
<relation>
charset = utf8
dbname = db_dcache_relation
dbhost =
dbport =
dbuser =
dbpass =
</relation>
</DB>
<Release>
ThreadCount = 5
</Release>
<Uninstall>
Timeout = 20
BakPath = /data/dcacheuninstall/
ThreadCount = 2
</Uninstall>
</Main>
```

Binary file modified docs/images/KV_service_conf.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/create_KV_module.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/images/install_and_release_kv.png
View file
Binary file not shown.
Binary file modified docs/images/install_dcache_app.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/tars_add_tmplate.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
55 changes: 42 additions & 13 deletions docs/install.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

## <a id = "1"></a> 1. 依赖环境
DCache是基于[Tars](https://github.com/TarsCloud/Tars)框架开发，所以编译之前请先安装Tars开发环境和管理平台，安装步骤请参考[Tars的install文档](https://github.com/TarsCloud/Tars/blob/master/Install.zh.md)。安装完Tars管理平台后，在浏览器中访问管理平台主页，如下图：
DCache是基于[Tars](https://github.com/TarsCloud/Tars)框架（版本v1.6.0以上）开发，所以编译之前请先安装Tars开发环境和管理平台，安装步骤请参考[Tars的install文档](https://github.com/TarsCloud/Tars/blob/master/Install.zh.md)。安装完Tars管理平台后，在浏览器中访问管理平台主页，如下图：

![Tars管理平台主页](images/tars_mainPage.png)

Expand All @@ -29,7 +29,7 @@ DCache是基于[Tars](https://github.com/TarsCloud/Tars)框架开发，所以编

### 2.2 编译

在`src/`目录下执行：`make release`，然后再分别进入以下目录：
在`src/`目录下执行：`make release;make`，然后再分别进入以下目录：

- src/OptServer/
- src/ConfigServer/
Expand All @@ -41,10 +41,30 @@ DCache是基于[Tars](https://github.com/TarsCloud/Tars)框架开发，所以编

执行:
```
make; make tar
make tar
```
即可生成各服务的发布包。

### 2.3 创建模板

在Tars的Web平台创建DCache.Cache模板，后续部署DCache模块时会用到该模板。

![创建模板](images/tars_add_tmplate.png)

新增模板DCache.Cache，父模板选择tars.default，模板内容填入：

```xml
<tars>
<application>
<client>
property=DCache.PropertyServer.PropertyObj
</client>
</application>
</tars>
```




## <a id = "3"></a> 3. 公共服务部署
在正式安装DCache应用前需要部署和安装公共服务：OptServer、ConfigServer和PropertyServer。
Expand Down Expand Up @@ -88,7 +108,7 @@ make; make tar

![添加配置文件内容](images/add_opt_conf.png)

其中文件名称填“DCacheOptServer.conf”，文件内容的填写请参考[DCache服务配置说明](optserver.md)。
其中文件名称填“DCacheOptServer.conf”，文件内容的填写请参考[DCache服务配置说明](server_config_example.md)。

**step5：** 重启OptServer。

Expand Down Expand Up @@ -124,7 +144,7 @@ make; make tar
该步骤参考安装OptServer的**step5**即可。



### <a id = "3.3"></a> 3.3 安装PropertyServer

安装PropertyServer的步骤和安装OptServer的步骤类似，除了不需要创建数据库外。
Expand All @@ -149,7 +169,7 @@ make; make tar

## <a id = "4"></a> 4. DCache管理平台安装

DCache管理界面是以模块的形式加载到Tars管理平台，最终会和Tars管理平台使用同一个地址，具体的模块安装步骤请参考[DCache管理平台安装](https://github.com/TarsCloud/TarsWeb/tree/dcache-alpha)
DCache管理界面是以模块的形式加载到Tars管理平台，最终会和Tars管理平台使用同一个地址，具体的模块安装步骤请参考[DCache管理平台安装](https://github.com/TarsCloud/TarsWeb)

## <a id = "5"></a> 5. 创建一个DCache应用

Expand Down Expand Up @@ -191,7 +211,7 @@ Cache发布包的上传和Proxy发布包上传步骤类似，只不过Cache有

![安装DCache应用](images/install_dcache_app.png)

根据上图，依次点击“DCache”和“运维管理”，其中“应用”和“管理员”必填，可自定义，然后点击“创建应用”，得到下图：
根据上图，依次点击“服务创建”，自定义“应用”名称，然后点击“创建应用”，得到下图：

![创建proxy和router服务](images/create_proxy&router.png)

Expand All @@ -218,11 +238,19 @@ Cache发布包的上传和Proxy发布包上传步骤类似，只不过Cache有

**注意：** <font color=red>共享内存key必须是唯一的，不能在服务部署机器上已存在，否则会造成服务拉起失败，可使用**ipcs**命令确认。</font>

必要信息填写完毕，点击“下一步”进入“安装发布”步骤，如下图：
Cache服务的模板默认会选择DCache.Cache，如果模板DCache.Cache不存在，可创建该模板或者选择其他可用的模板，模板中必须配置property，这样才能查看服务的特性监控数据。

![安装发布](images/install_and_release_kv.png)
```xml
<tars>
<application>
<client>
property=DCache.PropertyServer.PropertyObj
</client>
</application>
</tars>
```

再次确认信息填写无误，点击“安装发布”，等待服务发布完成。刷新管理平台主页，左侧目录树出现此模块信息，如下图：
必要信息填写完毕，点击“下一步”进入“安装发布”步骤，等待服务发布完成。刷新管理平台主页，左侧目录树出现此模块信息，如下图：

![安装成功](images/install_kv_succ.png)

Expand All @@ -231,8 +259,8 @@ Cache发布包的上传和Proxy发布包上传步骤类似，只不过Cache有
步骤和[部署和发布KVCache](#5.3)类似，参考即可。

### <a id = "5.5"></a> 5.5 Cache配置管理


![Cache配置管理](images/cache_config.png)

按照上图箭头依次点击，可添加配置项。
Expand All @@ -242,4 +270,5 @@ Cache发布包的上传和Proxy发布包上传步骤类似，只不过Cache有

![模块配置管理](images/add_conf_for_module.png)

按照上图箭头依次点击，可在该页面上修改和添加配置。该页面的配置管理分两种类型：针对模块所有节点的配置管理和针对模块特定节点的配置管理。
按照上图箭头依次点击，可在该页面上修改和添加配置。该页面的配置管理分两种类型：针对模块所有节点的配置管理和针对模块特定节点的配置管理。如果节点配置和模块配置有重叠的配置项，那么节点配置将覆盖模块配置。
**注意：** <font color=red>修改配置后，需要重启服务才能生效。</font>
Loading

0 comments on commit c366bcc

Please sign in to comment.