Skip to content

Template Object

Jump to bottom
mytrygithub edited this page Jul 10, 2023 · 3 revisions

[中文版 (Chinese)]

1. background

Many times we will define multiple DB, multiple CF(ColumnFamily), and these DB/CF options are much the same, are written repeatedly in json/yaml, will be very verbose, and it is likely to cause inconsistencies due to negligence.

With RocksDB, such duplication can be resolved by extracting common code.

So, in the SidePlugin configuration, we added a new feature that allows you to define the template parameter, define the common configuration in the template, and then define the different configuration items separately.

2. template parameter

The template parameter is currently supported by Options, DBOptions, ColumnFamilyOptions, and DcompactEtcd, as examples:

DBOptions:
  dbo:
    create_if_missing: true
    create_missing_column_families: true
    max_background_compactions: 40
    max_subcompactions: 1
    max_level1_subcompactions: 7
    inplace_update_support: false
    WAL_size_limit_MB: 0
    statistics: "${stat}"
  dbo2:
    template: "$dbo"
    max_level1_subcompactions: 3 # override this param

3. Work with predefined objects

In Ingest Predefine Object, user code can inject Options and other objects created with code via 'SidePluginRepo::Put', but in this case, the entire object needs to be fully configured by user code. For example, the compaction_filter/merge_operator object in CFOptions needs to be set by the user in advance. Once injected, this object can no longer be changed, which significantly limits flexibility.

Now, we can combine templates and puts: we put an Option as a template, and then the other options are modified based on that template.

yaml configuration

DBOptions:
  dbo:
    create_if_missing: true
    create_missing_column_families: true
CFOptions:
  default:
    template: "$template_cfo" # has been ingested by put
    table_factory: dispatch
  cfo1:
    template: "$template_cfo" # has been ingested by put
    memtable_factory: "${cspp}"
    table_factory: dispatch
  cfo2:
    template: "$template_cfo" # has been ingested by put
    memtable_factory: "${cspp}"   
databases:
  db1:
    method: DB::Open
    params:
      db_options: "$dbo"
      column_families:
        default: "$default"
        cf1: "$cfo1"
      dyna_cf_opt: "$cfo2"

C++ demo

  SidePluginRepo repo;
  repo.Put("template_cfo", CreateMyColumnFamilyOptions());
  repo.ImportAutoFile("config.yaml"); // omit error check
  // now CFOptions objects have been initialized with "template_cfo"
  // and modified by their "params"

Java demo

  SidePluginRepo repo = new SidePluginRepo();
  repo.put("template_cfo", createMyColumnFamilyOptions());
  repo.importAutoFile("config.yaml"); // omit error check
  // now CFOptions objects have been initialized with "template_cfo"
  // and modified by their "params"

4. Documents

Clone this wiki locally