input exceeds maximum allowed size

[michbens]

Hello
I have a schema with a very big enum. Then I got the following error when compiling the schema:
"input exceeds maximum allowed size"
It seems the enum type is limited to 1215 elements.
Is there a way to increase this limitation ?
Thanks

[mikkelfj]

I don't have a lot to go by. Enums have different types, and none that are limited by a 1215 value. There shouldn't be any artificial limitations. Errors are generally due to physical limits.

Can you identify the exact source location of the error and a fragment of the schema that triggers it?
For example locating source lines with the text and set a breakpoint or an assert, then see the parser structs offset to find the likely text segment triggering the error, assuming it happens during parsing.
It might also happen later in semantics.c where the schema is validated after parsing, prior to code generation, or it could happen during code generation.

[michbens]

It happens in flatcc_parse_file(), line 374. Here:

    if (!buf && !is_root) {
        inpath = P->referer_path;
        inpath_len = fb_find_basename(inpath, strlen(inpath));
        checkmem((path = fb_create_join_path_n(inpath, inpath_len, filename, filename_len, "", 1)));
        if (!(buf = fb_read_file(path, P->opts.max_schema_size, &size))) {
            free(path);
            path = 0;
            if (size > P->opts.max_schema_size && P->opts.max_schema_size > 0) {
                fb_print_error(P, "4. input exceeds maximum allowed size %d (%d)\n", size, P->opts.max_schema_size);
                goto done;
            }
        }
    }

When the max_schema_size is 65536

[mikkelfj]

It's literally the first option in config/config.h, and yes, that is a very low limit. Not sure why it is so low by default, but when parsing include files it can add up. You can compile flatcc with -DFLATCC_MAX_SCHEMA_SIZE 100000
or something.

[mikkelfj]

Or you can set it to 0 to disable the limit, provided that is correctly implemented in all checks.

[michbens]

I directly change it in config.h
I didn't find a way to change is in build command lines.
Thanks

[mikkelfj]

There is no build option per se, but all flags can be overriden by using the -D compile flag to define a symbol. config.h checks if it is defined already only defines its own value otherwise. To set the -D flag, you need to modify CFLAGS in CMakeList.txt, but it may be just as easy to do it in config.h, depending on the setup.

[mikkelfj]

if you have enums with assigned values, for example an enum of int16 (short), the maximum value will be 2**15. If you have assigned explicit values to some enum names, those that follow will be incremented, and you could hit the ceiling that way, if you assigned a large value. Or, you might simply be trying to assign a value to an enum that is outside of its types range of values.

[michbens]

This is not the case

[mikkelfj]

It's in src/compiler/flatcc.c
There are a few places, but apparently there is a maximum schema file size, probably to prevent DoS abuse if used in a server context. This opts limit should be easy to change. I have not investigated, and it depends on the exact location of the error, but likely it can be set in config/config.h

[mikkelfj]

I just increased FLATCC_MAX_SCHEMA_SIZE to 1000000 (1 MB) on master branch.

[michbens]

Thank you