Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to insert rows if table has many columns #45

Closed
santolex opened this issue Sep 27, 2020 · 6 comments
Closed

Unable to insert rows if table has many columns #45

santolex opened this issue Sep 27, 2020 · 6 comments
Assignees
Labels
bug Something isn't working

Comments

@santolex
Copy link

I use ClickHouseBulkCopy to insert rows and it throws some http exception.
In my opinion the problem is that query is too long about 22k symbols long.

@santolex
Copy link
Author

In this line query string is created and it's too long for http URL in my case.

@DarkWanderer DarkWanderer self-assigned this Sep 29, 2020
@DarkWanderer DarkWanderer added the bug Something isn't working label Sep 29, 2020
@DarkWanderer
Copy link
Owner

Do you have a reproduction example I can put in a test? (fix is already in, but I'd like to cover it)

@santolex
Copy link
Author

santolex commented Oct 7, 2020

Here you are.

var host = "localhost";
var port = 8123;
var tblName = "default.test_long_insert_tbl";
int columnCount = 600;

var con =new ClickHouseConnection($"Host={host};Port={port}");

//Generating create tbl statement with a lot of columns 
var query = $"CREATE TABLE IF NOT EXISTS {tblName}(\n";
var columns = Enumerable.Range(1, columnCount)
    .Select(x => $" some_loooooooooooooonnnnnnnnnnnngggggggg_column_name_{x} Int32");
query += string.Join(",\n", columns);
query += ")\n ENGINE = MergeTree()\n ORDER BY (some_loooooooooooooonnnnnnnnnnnngggggggg_column_name_1)";

//Create tbl in db
await con.ExecuteStatementAsync(query);

var bulkCopy = new ClickHouseBulkCopy(con) {DestinationTableName = tblName};

var rowToInsert = new[] {Enumerable.Range(1, columnCount).Select(x => (object) x).ToArray()};
await bulkCopy.WriteToServerAsync(rowToInsert);

DarkWanderer added a commit that referenced this issue Oct 13, 2020
@DarkWanderer
Copy link
Owner

Fix released in 2.2.2.294

@DarkWanderer
Copy link
Owner

I had to pull 2.2.2.294 from search on NuGet.org temporarily as the new way of handling requests turned out to be not compatible with ClickHouse 20.3. If you use a recent version of ClickHouse, 2.2.2.294 is still available to install manually (by changing version in .csproj PackageReference tag or in packages.config)

Reopening this to track fix in next version

(@santolex )

@DarkWanderer DarkWanderer reopened this Oct 16, 2020
DarkWanderer added a commit that referenced this issue Oct 24, 2020
* Backward-compatible fix for "wide table" bulk copy issue (#45)
* Added feature flags for tests (DateTime64, Decimal, IPv6) to disable them on incompatible version
* Switched feature support flag methods to internal
@DarkWanderer
Copy link
Owner

Closing again, as has been properly fixed in 2.3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants