Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] interface for obtaining table information throw exception #5110

Open
2 tasks done
hawk9821 opened this issue Feb 19, 2025 · 0 comments · May be fixed by #5126
Open
2 tasks done

[Bug] interface for obtaining table information throw exception #5110

hawk9821 opened this issue Feb 19, 2025 · 0 comments · May be fixed by #5126
Labels
bug Something isn't working

Comments

@hawk9821
Copy link
Contributor

hawk9821 commented Feb 19, 2025 •
edited
Loading

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

0.7.0-incubating
1.0.1

Compute Engine

JavaAPI

Minimal reproduce step

step1. create table using version 0.7.0-incubating

package com.hawk.paimon;

import org.apache.paimon.catalog.Catalog;
import org.apache.paimon.catalog.CatalogContext;
import org.apache.paimon.catalog.CatalogFactory;
import org.apache.paimon.catalog.Identifier;
import org.apache.paimon.fs.Path;
import org.apache.paimon.schema.Schema;
import org.apache.paimon.table.Table;
import org.apache.paimon.types.DataTypes;

import java.util.List;
import java.util.concurrent.TimeUnit;

public class CreateTable {

    public static String tableName = "paimon_version_07";
    public static void main(String[] args) {
        CatalogContext catalogContext = CatalogContext.create(new Path("file:///tmp/paimon_version"));
        try (Catalog catalog = CatalogFactory.createCatalog(catalogContext)){
            Identifier identifier = Identifier.create("test",tableName);
            List<String> databases = catalog.listDatabases();
            if (!databases.contains(identifier.getDatabaseName())){
                catalog.createDatabase(identifier.getDatabaseName(), false);
            }
            List<String> tables = catalog.listTables(identifier.getDatabaseName());
            if (!tables.contains(identifier.getObjectName())){
                Schema.Builder schemaBuilder = Schema.newBuilder();
                schemaBuilder.column("id", DataTypes.SMALLINT());
                schemaBuilder.column("name", DataTypes.STRING());
                schemaBuilder.column("age", DataTypes.TINYINT());
                // schemaBuilder.primaryKey("id");
                schemaBuilder.comment("changelog table");
                catalog.createTable(identifier, schemaBuilder.build(), false);
            }
            TimeUnit.SECONDS.sleep(1);
            Table table = catalog.getTable(identifier);
            System.out.println("===============================\n" + table.rowType());
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

step2. run the above code again using version 1.0.1 throw exception

Caused by: java.lang.RuntimeException: You should define a 'bucket-key' for bucketed append mode.
	at org.apache.paimon.schema.SchemaValidation.validateBucket(SchemaValidation.java:581)
	at org.apache.paimon.schema.SchemaValidation.validateTableSchema(SchemaValidation.java:99)
	at org.apache.paimon.table.AbstractFileStoreTable.copyInternal(AbstractFileStoreTable.java:351)
	at org.apache.paimon.table.AbstractFileStoreTable.copy(AbstractFileStoreTable.java:295)
	at org.apache.paimon.table.FileStoreTableFactory.createWithoutFallbackBranch(FileStoreTableFactory.java:127)
	at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:90)
	at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:80)
	at org.apache.paimon.catalog.AbstractCatalog.getDataOrFormatTable(AbstractCatalog.java:423)
	at org.apache.paimon.catalog.AbstractCatalog.getTable(AbstractCatalog.java:415)
	at org.apache.paimon.catalog.CachingCatalog.getTable(CachingCatalog.java:254)
	at com.hawk.paimon.CreateTable.main(CreateTable.java:37)

What doesn't meet your expectations?

Primary key table is ok

APPendOnly table throw exception

version 0.8 and above is ok

Anything else?

because the Schemaserializer #deserialize method performs the following logic when serializing version 0.7 of the appendOnly table schema

if (version <= 1 && !options.containsKey(CoreOptions.BUCKET.key())) {
          options.put(CoreOptions.BUCKET.key(), "1");
 }

Be changed to

if (schema.primaryKeys().isEmpty() && schema.bucketKeys().isEmpty() && !(schema.version() == TableSchema.PAIMON_07_VERSION && bucket == 1)) {
                throw new RuntimeException(
                        "You should define a 'bucket-key' for bucketed append mode.");
            }

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@hawk9821 hawk9821 added the bug Something isn't working label Feb 19, 2025
@hawk9821 hawk9821 linked a pull request Feb 20, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Bug][core] obtain table information Interface version compatibility optimization hawk9821/paimon
1 participant
@hawk9821