Adding a separator in the CSV extension

[xabi12]

Hello everyone, is it possible to add a CSV separator extension ? Thanks for your feedback.

[utelle]

What exactly do you mean by "CSV separator extension"? In what CSV functionality are you interested?

SQLite3 Multiple Ciphers already includes the CSV extension that is provided by SQLite; this extension allows to select data from CSV files just as from ordinary SQL tables.

Additionally, the SQLite shell (which is also included in SQLite3 Multiple Ciphers) allows to import data from CSV files into tables (.import command), and to export query results into CSV files (output mode csv).

If you are missing certain features from those SQLite components, please ask on the SQLite forum.

[xabi12]

Hi, generally I mean to be able to import data from the sql query level. (without command-line shell help)
Currently the csv extension allows you to import files where fields are separated by a ",".
In the csv.c file I see that you can't specify the separator parameter. The separator is fixed ","
It would be nice if you could specify the "separator" parameter (separator="|").
The extension would be universal, you could import files regardless of the separator.
filed1,field2,field3,field4,field5
filed1|field2|field3|field4|field5
filed1;field2;field3;field4;field5

CREATE VIRTUAL TABLE temp.csv1 USING csv(
filename = "../http.log",
separator="|",
schema = "CREATE TABLE x(date,ipaddr,url,referrer,userAgent)"
);

CREATE VIRTUAL TABLE temp.csv2 USING csv(
columns=5,
separator="|",
filename = "../http.log"
);

CREATE VIRTUAL TABLE temp.csv3 USING csv(
separator="|",
data="filed1|field2|field3|field4|field5"
);

Greetings

[utelle]

Looking at this post on the SQLite forum you are obviously not the only one interested in additional processing options for CSV import. The post points forward to another thread on the old SQLite mailing list and offers an alternative implementation of the CSV extension.

I will consider to add the alternative implementation to this repository. However, due to the differences in the extension names and their functionality I can't simply replace the CSV extension by the new one without possibly breaking existing code relying on the original CSV extension.

[xabi12]

It seems to me that in the current version csv is not used (or very rarely due to lack of choice of separator). Your library is excellent but unfortunately I miss it a lot. I hope you will add another implementation :)
Currently I'm fighting with changing your code myself because I don't know c++. Thanks to .

[utelle]

It seems to me that in the current version csv is not used (or very rarely due to lack of choice of separator).

Unfortunately, I haven't got any feedback, which SQLite extensions are actually in use by consumers of the SQLite3 Multiple Ciphers resp wxSQLite3 library. Therefore, I'm reluctant to simply drop an extension that was included in earlier releases. However, it is no big deal to add the enhanced CSV extension along with the original one. Developers can always configure the build to omit certain extensions, if they are not needed.

Your library is excellent but unfortunately I miss it a lot. I hope you will add another implementation :)

I'll do that in the near future.

Currently I'm fighting with changing your code myself because I don't know c++.

Well, SQLite itself is written in C. Only the wxSQLite3 library is written in C++, but will not be affected by the addition of another extension.

[utelle]

Commit 33e7026 adds the VSV extension, which allows a better configuration of the data import than the CSV extension. Currently, both extensions will be enabled.

[utelle]

The project wxSQLite3 has been updated, too. Therefore I'm closing this issue.

[xabi12]

Hi, thanks for the very quick reaction, you're great :)