Skip to content

Commit

Permalink
odbcBackend: Apply suggestions from review
Browse files Browse the repository at this point in the history
- Apply suggestions from the review from @kodebach
  • Loading branch information
flo91 committed Jun 13, 2023
1 parent 6ca5452 commit 5ee8bb7
Show file tree
Hide file tree
Showing 12 changed files with 360 additions and 505 deletions.
9 changes: 4 additions & 5 deletions doc/help/kdb-mountOdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
<mt key column name> <mt metakey column name> <mt metavalue column name> <timeout (s)> <mountpoint>`<br>

- Where `data source name` is the name of the ODBC data source as defined in `odbc.ini` (usually stored at `/etc/unixODBC/`).
- Only ODBC drivers that support **outer joins** are suited for the Elektra ODBC backend.
- `user name`: The username that should be used for connecting to the ODBC data source. If no username is needed, or it is already specified in the ini-file for the configuration of the datasource, `""` can be provided for this argument.
- `password`: The same rules as for `user name` also apply to this argument.
- `table name`: The name of the table in the data source where the keys and their values should be stored.
Expand All @@ -21,8 +22,6 @@
> All columns in the data source that are specified via one of the described arguments, should be of type `TEXT`.
> The tables can have additional columns. These are not processed by the ODBC backend, but should support NULL or DEFAULT values.
> Only ODBC drivers that support **outer joins** are suited for the Elektra ODBC backend.
## DESCRIPTION

This command allows a user to persistently mount a new _backend_ that refers to an ODBC data source.
Expand All @@ -32,8 +31,8 @@ More about mounting is explained in [elektra-mounting(7)](elektra-mounting.md).

## IMPORTANT

This command writes into the `/etc` directory to make the mounting persistent.
As such it requires root permissions.
This command modifies `system:/elektra/mountpoints`.
Depending on the location of that part of the KDB, you may need root privileges to execute this command.
Use `kdb file system:/elektra/mountpoints` to find out where exactly it will write to.

> There is no special command for unmounting ODBC data sources.
Expand All @@ -58,7 +57,7 @@ Use `kdb file system:/elektra/mountpoints` to find out where exactly it will wri

## Examples

To mount the sample SQLite data source:<br>
To mount the sample SQLite data source which is available at /src/plugins/backend_odbc/sampleDb/elektraDB.db:<br>
`kdb mountOdbc Selektra "" "" elektraKeys keyName keyValue metaKeys keyName metaKeyName metaKeyValue "" user:/odbcSqlite`

To specify all arguments:<br>
Expand Down
11 changes: 6 additions & 5 deletions doc/man/man1/kdb-mountOdbc.1
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@
.br
.IP "\(bu" 4
Where \fBdata source name\fR is the name of the ODBC data source as defined in \fBodbc\.ini\fR (usually stored at \fB/etc/unixODBC/\fR)\.
.IP "\(bu" 4
Only ODBC drivers that support \fBouter joins\fR are suited for the Elektra ODBC backend\.
.IP "" 0

.IP "\(bu" 4
\fBuser name\fR: The username that should be used for connecting to the ODBC data source\. If no username is needed, or it is already specified in the ini\-file for the configuration of the datasource, \fB""\fR can be provided for this argument\.
.IP "\(bu" 4
Expand All @@ -34,13 +38,10 @@ Where \fBdata source name\fR is the name of the ODBC data source as defined in \
.IP "" 4
All columns in the data source that are specified via one of the described arguments, should be of type \fBTEXT\fR\. The tables can have additional columns\. These are not processed by the ODBC backend, but should support NULL or DEFAULT values\.
.IP "" 0
.IP "" 4
Only ODBC drivers that support \fBouter joins\fR are suited for the Elektra ODBC backend\.
.IP "" 0
.SH "DESCRIPTION"
This command allows a user to persistently mount a new \fIbackend\fR that refers to an ODBC data source\. The concept is the same as for mounting with the file\-based backend\. So it is recommended to also read to man page for kdb\-mount(1) \fIkdb\-mount\.md\fR\. More about mounting is explained in elektra\-mounting(7) \fIelektra\-mounting\.md\fR\.
.SH "IMPORTANT"
This command writes into the \fB/etc\fR directory to make the mounting persistent\. As such it requires root permissions\. Use \fBkdb file system:/elektra/mountpoints\fR to find out where exactly it will write to\.
This command modifies \fBsystem:/elektra/mountpoints\fR\. Depending on the location of that part of the KDB, you may need root privileges to execute this command\. Use \fBkdb file system:/elektra/mountpoints\fR to find out where exactly it will write to\.
.IP "" 4
There is no special command for unmounting ODBC data sources\. You can just use the kdb\-umount(1) \fIkdb\-umount\.md\fR command the same way as with mountpoints for files\.
.IP "" 0
Expand All @@ -67,7 +68,7 @@ Give debug information\. Prints additional information in case of errors/warning
\fB\-f\fR, \fB\-\-force\fR
Create the mountpoint even if the data source can not be reached at mount\-time\.
.SH "Examples"
To mount the sample SQLite data source:
To mount the sample SQLite data source which is available at /src/plugins/backend_odbc/sampleDb/elektraDB\.db:
.br
\fBkdb mountOdbc Selektra "" "" elektraKeys keyName keyValue metaKeys keyName metaKeyName metaKeyValue "" user:/odbcSqlite\fR
.P
Expand Down
111 changes: 10 additions & 101 deletions doc/tutorials/odbc-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,9 @@ The tutorial covers the following steps:
### Database scheme

The basic database scheme is quite simple.
You just need a table that contains two columns which can store a text of arbitrary length.
Usually, the SQL data type `TEXT` is used for that purpose.
You just need a table that contains two columns which can store a text value.
Usually, the SQL data types `TEXT`, `CHAR` and `VARCHAR` are used for that purpose.
They must contain valid UTF-8 sequences.
One column is used for storing the **names** of the keys, the other one for storing their **values**.
The column where the key-names are stored should be defined as the _primary key_ (PK) for that table.

Expand All @@ -43,17 +44,16 @@ Otherwise, lots of Elektra's features are not supported.-->
Next, we have to create a table for storing metadata.
The table for the metadata needs at least three columns:

- A column with the _key-name_ which must be defined as a _foreign key_ (FK) to the PK of the table where the keys are stored.
- A column with the _key-name_ which should be defined as a _foreign key_ (FK) to the PK of the table where the keys are stored.
- A column for the _name_ of the metakeys.
- A column for the _value_ of the metakeys.

> The column for the **key-name** and the column for the **metakey-name** together form the PK of that table.
> The column for the **key-name** and the column for the **metakey-name** together should form the PK of that table.
Again, all these columns should be defined to store the data type `TEXT`.
If you want to use `CHAR` or `VARCHAR` columns, it's your responsibility to make sure that the stored values don't exceed the maximum defined length.
Again, all these columns should be defined to store the data types `TEXT`, `CHAR` or `VARCHAR` column.

It is allowed that the tables contain more columns, these additional columns are not processed by Elektra,
_must not_ be part of a PK and _must_ support NULL-values (if you want to add new keys in a future version of Elektra).
_must not_ be part of a PK and _must_ support NULL- or DEFAULT-values (if you want to add new keys in a future version of Elektra).
Currently, as only read support is implemented, columns that don't support NULL values are also possible, but not recommended.

The following ER-diagram shows the described scheme:
Expand Down Expand Up @@ -129,7 +129,7 @@ If you want to use PostgreSQL, a bit more initial work is necessary.
It is recommended to set up a user account, create a database and then create the tables using similar SQL statements as given above.
You can run the PostgreSQL instance locally on the same computer where you use Elektra, but also on another node in the network.
With this approach, it is possible to provide a centralized configuration storage for a whole network.
As PostgreSQL natively supports transactions, it is also possible to share the same tables between multiple Elektra clients.
As PostgreSQL natively supports transactions and multiple clients, it is also possible to share the same tables between multiple Elektra clients.
Another option is to save the configurations for different users in separate tables.
When a user logs in on any client PC in the network, the table with the matching configuration data can then, with some scripting, be mounted via Elektra.

Expand Down Expand Up @@ -169,40 +169,6 @@ Now we have to create or edit two configuration files for unixODBC:
- **odbc.ini:** for configuring the actual data sources, which can then be mounted into Elektra's KDB

On most systems, these files should be stored at `/etc/unixODBC/`.

As we are working with Elektra here, it would be a nice option, to also use it for creating and managing the `.ini` files for the ODBC configuration.
Unfortunately, the current Elektra plugins for .ini-files don't support the exact syntax that is used by the unixODBC configuration files.
But we can still use the more generic [line plugin](/src/plugins/line/README.md) to create the files.
Of course, you can also use a text editor to edit the files.

> As you can see in the following listings, you need the permission to write to the `/etc/unixODBC/` ini-files.
> In most cases, this means that the `kdb set` command to store the vales must be executed with root privileges, e.g. by using `sudo`.
> The line-plugin ignores the key-name given in `kdb set` and just adds the given key-value as the last line to the given file.
> The keys can be accessed via the line-number in the file, similar to [arrays](./arrays.md).
> Therefore, we use these indices also as key-names in the following `kdb set` commands.
### odbcinst.ini

At first, we tell unixODBC where it can find the ODBC drivers we want to use.
We store this information in the `/etc/unixODBC/odbcinst.ini` file:

```bash
kdb mount /etc/unixODBC/odbcinst.ini system:/conf/unixODBC/odbcinst/#0/current line
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#0 "[SQLite]"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#1 "Description=SQLite ODBC Driver"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#2 "Driver=/usr/local/lib/libsqlite3odbc.so"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#3 "Setup=/usr/local/lib/libsqlite3odbc.so"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#4 "FileUsage=1"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#5 "Threading=2"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#6 ""
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#7 "[Postgresql]"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#8 "Description=General ODBC for PostgreSQL"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#9 "Driver=/usr/local/lib/psqlodbca.so"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#10 "Setup=/usr/lib/libodbcpsqlS.so"
sudo kdb set system:/conf/unixODBC/odbcinst/#0/current/#11 "Setup64=/usr/lib64/libodbcpsqlS.so"
```

The content of the `/etc/unixODBC/odbcinst.ini` file should look like this:

```ini
Expand Down Expand Up @@ -238,31 +204,6 @@ The setting `FileUsage=1` indicates to unixODBC that the driver is file-based.
### odbc.ini

After the driver settings, we have to define the actual ODBC data sources in `/etc/unixODBC/odbc.ini`.

```bash
kdb mount /etc/unixODBC/odbc.ini system:/conf/unixODBC/odbc/#0/current line
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#1 "[Selektra]"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#2 "Description=SQLite Database for Elektra"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#3 "Database=/home/user/elektraOdbc.db"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#4 "Timeout=3500"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#5 ""
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#6 "[Pelektra]"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#7 "Description=Postgresql"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#8 "Driver=Postgresql"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#9 "Trace=No"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#10 "Database=elektraDB"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#11 "Servername=localhost"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#12 "Username=elektraUser"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#13 "Password=elektra"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#14 "Port=5432"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#15 "Protocol=6.4"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#16 "ReadOnly=No"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#17 "RowVersioning=No"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#18 "ShowSystemTables=No"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#19 "ShowOidColumn=No"
sudo kdb set system:/conf/unixODBC/odbc/#0/current/#20 "FakeOidIndex=No"
```

This content of the file should look like this:

```ini
Expand Down Expand Up @@ -320,46 +261,14 @@ Otherwise, the `backend_odbc` plugin gets automatically excluded.
Now, we can finally mount our ODBC data sources into the global Key Database (KDB) of Elektra.
Currently, a separate command `kdb mountOdbc` exists for this purpose.
Unfortunately, in contrast to the well-known `kdb mount` for the file-based backend, currently no additional plugins are supported.
The situation is not ideal and may change in the future.
One option is to provide a general mounting command for all types of backends.

However, for now we use the `kdb mountOdbc` command.
If you just type the command without additional arguments, you get some information about how the command works and which arguments are expected.

> Only `system:/` and `user:/` namespaces are supported for ODBC mountpoints.
To fully define a mountpoint for an ODBC data source, we need 12 arguments, 11 for defining the data source,
and the last argument is the path where the mountpoint should be created in the KDB.

The usage-message tells us which arguments are needed and in which order they must be given:

```sh
Usage: kdb mountOdbc <data source name> <user name> <password> <table name>
<key column name> <value column name> <meta table name> <mt key column name>
<mt metakey column name> <mt metavalue column name> <timeout (s)> <mountpoint>
```

If you have read the first section about the expected scheme of the used databases, the arguments should be self-explanatory.
Nevertheless, here is a listing that describes the different arguments:

- **\<data source name\>:** The name of the data source as defined in `/etc/unixODBC/odbc.ini`.
- **\<user name\>:** The name of the user that should be used for connecting to the data source.
- If your data source doesn't need a username, or you have defined a username in the `odbc.ini` file, you can just pass an empty string `""` here.
- **\<password\>:** The password that should be used to connect to the data source.
- As with the username, you can pass `""` if no password is needed or the password is already set in the `odbc.ini` file.
- **\<table name\>:** The name of the table where the key-names and -values are stored.
- **\<key column name\>:** The name of the column where the key-names are stored. This must be the PK of the table.
- **\<value column name\>:** The name of the column where the key-values are stored.
- **\<meta table name\>:** The name of the table where the metadata for the keys is stored.
- **\<mt key column name\>:** The column where the name of the key to which the metadata belongs to is stored.
- This must be a foreign key (FK) to the PK `key column` of the table for the (non-meta) keys.
- **\<mt metakey column name\>:** The name of the column in the meta-table where the names of the metakeys are stored.
- This column together with the key-column form the PK of the meta-table.
- **\<mt metaval column name\>:** The name of the column in the meta-table where the values of the metakeys are stored.
- **\<timeout (s)\>:** The timeout (in seconds) after a connection attempt to the data source should be aborted. If you specify `""`, a default value (a few seconds) is used, if you specify `0`, the timeout is disable and the application can potentially wait forever. Use this option with care.
- **\<mountpoint\>:** The place in the KDB where the new mountpoint should be created (e.g. `user:/odbcData`).

Finally, we can create the mountpoint for our SQLite database:
Finally, we can create the mountpoint for our SQLite database using the `kdb mountOdbc` command.
Further details about the arguments of the command are available in its [man page](/doc/help/kdb-mountOdbc.md).

```sh
kdb mountOdbc Selektra "" "" elektraKeys keyName keyValue metaKeys keyName metaKeyName metaKeyValue "" user:/odbcSqlite
Expand Down
16 changes: 7 additions & 9 deletions src/plugins/backend_odbc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

## Introduction

This plugin is a backend plugin that stores and retrieves data using ODBC (Open Database Connectivity) data sources.
This plugin is a backend plugin that retrieves data using ODBC (Open Database Connectivity) data sources.
It was tested with unixODBC on Linux, but should also work with iODBC and Microsoft ODBC (on Windows).

> If you want to use it with one of the latter two ODBC implementations, feel free to update this documentation with your experiences!
Expand All @@ -37,10 +37,9 @@ In the language of ER-modelling, the metatable can therefore by considered a **w
> If you want to use metadata, the ODBC driver for your data source has to support **outer joins**.
> This implies that currently, only ODBC drivers with support for outer joins are supported by the ODBC backend.
The tables may also contain other columns, but they are not processed by this plugin and must support NULL- or DEFAULT-values,
if you want to add tuples to the table via Elektra (e.g. by calling `kdb set <key> <value>`).
The tables may also contain other columns, but they are not processed by this plugin and must support NULL- or DEFAULT-values, if you want to add tuples to the table via Elektra (e.g. by calling `kdb set <key> <value>`) in future versions.

The following properties are available:
The following properties must be set when mounting an ODBC data source into Elektra:

- DataSourceName: The name of the ODBC data source (as in the ODBC driver manager)
- UserName: Name of the user that should be used to connect to the data source
Expand All @@ -54,16 +53,15 @@ The following properties are available:
- MetaTableKeyColName: The name of the column in the MetaTable where the FK to the table with the keys is stored
- MetaTableMetaKeyColName: The name of the column where the name of the metakeys is stored
- MetaTableMetaValColName: The name of the column where the value of the metakeys is stored
- Timeout: The number of seconds to wait after a connection attempt is considered as failed (0 = wait indefinitely)

## Mounting

There is a new command for the kdb-tool: `kdb mountOdbc`.
Please be aware that this command, in contrast to the classic `kdb mount` for the file-backend,
Please be aware that this plugin, in contrast to the classic `kdb mount` for the file-backend,
currently does not support adding other plugins to the mountpoint.
This should change when the new tooling and mounting library are finished.

For unmounting, there is no new command necessary.
Just use the well-knows `kdb umount <mountpoint>`, exactly like for the file-based backend.
Just use the well-known `kdb umount <mountpoint>`, exactly like for the file-based backend.

For more information about the ODBC backend, there is a tutorial available at
[/doc/tutorials/odbc-backend.md](/doc/tutorials/odbc-backend.md).
For more information about the ODBC backend, there is a tutorial available at [/doc/tutorials/odbc-backend.md](/doc/tutorials/odbc-backend.md).
Loading

0 comments on commit 5ee8bb7

Please sign in to comment.