Out of IncrementingUuidGenerator capacity

[tester12345132]

👓 What did you see?

• It looks like IncrementingUuidGenerator cannot close/reuse sessionId´s if testcases have a short execution time
• CucumberException: Out of IncrementingUuidGenerator capacity. Please reuse existing instances or use another UuidGenerator implementation instead.

Stacktrace: 
Exception in thread "main" java.util.ServiceConfigurationError: io.cucumber.core.eventbus.UuidGenerator: Provider io.cucumber.core.eventbus.IncrementingUuidGenerator could not be instantiated
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:582)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:804)
	at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:722)
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1395)
	at io.cucumber.core.runtime.UuidGeneratorServiceLoader.loadSingleUuidGeneratorOrDefault(UuidGeneratorServiceLoader.java:60)
	at io.cucumber.core.runtime.UuidGeneratorServiceLoader.loadUuidGenerator(UuidGeneratorServiceLoader.java:46)
	at io.cucumber.core.runtime.Runtime$Builder.build(Runtime.java:180)
	at io.cucumber.core.cli.Main.run(Main.java:85)
	at ta.TestRunner.main(TestRunner.java:59)
Caused by: io.cucumber.core.exception.CucumberException: Out of IncrementingUuidGenerator capacity. Please reuse existing instances or use another UuidGenerator implementation instead.
	at io.cucumber.core.eventbus.IncrementingUuidGenerator.<init>(IncrementingUuidGenerator.java:102)
	at jdk.internal.reflect.GeneratedConstructorAccessor87.newInstance(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:780)
	... 7 more

✅ What did you expect to see?

I would expect, that some other UUidGenerator would be used, if MAX_SESSION_ID from IncrementingUuidGenerator is out of capacity or the existing instances are reused

📦 Which tool/library version are you using?

io.cucumber, cucumber-junit, v7.19.0
org.seleniumhq.selenium, selenium-java, v.4.17.0

🔬 How could we reproduce it?

Not sure about steps to reproduce, any calls against an api should do the trick

📚 Any additional context?

• Workaround: Use io.cucumber, cucumber-junit v7.11.2 (last version before changes on UUID generator mentioned below)
• Found this issue: [Core] Add faster UUID generator selectable through SPI #2703 merged in 04/2023
• Some testcases have > 100 Steps
• Some testcases are very quick < 200 ms
• Nearly no break in between test executions
• Before the Exception there are two big AND fast testcases (111 Steps - 0,354s and 52 Steps - 0,229s)
• Steps and times in log

Executions.txt

[mpkorstanje]

The IncrementingUuidGenerator is designed to support a single Cucumber run with 256 sessions per class loader (one session per worker thread, one for main) and approximately 4.6*10^18 events per session. Both of these seem like reasonable parameters.

But from your stacktrace and log output it looks like you are running Cucumber multiple times. Once for each test case. Used in this way the IncrementingUuidGenerator can only support up to 256 runs. So you are operating somewhat out of the design parameters. Could you explain why? That might help inform a solution as I would normally expect a single Cucumber run to run all scenarios.

From your stack trace it also looks like you are not using cucumber-junit. While your project might have a dependency on cucumber-junit, but in your problem are only using functionality from cucumber-core. Could you demonstrate exactly how you run Cucumber?

[mpkorstanje]

Workaround: Use io.cucumber, cucumber-junit v7.11.2 (last version before changes on UUID generator mentioned below)

This I don't understand.

The IncrementingUuidGenerator was introduced in v7.12.0 and is not used by default. You must actively choose to use it. By rolling back this far, before its introduction, you are actively choosing not to use it.

So it seems unlikely that this problem was introduced by a change in Cucumber. Have you been adding test cases and are now exceeding 256 cases?

[jkronegg]

I confirm that IncrementingUuidGenerator is not used by default. To enable it, you need to add the following into your cucumber.properties:

cucumber.uuid-generator=io.cucumber.core.eventbus.IncrementingUuidGenerator

Alternatively, you may have a custom META-INF/services/io.cucumber.core.eventbus.UuidGenerator file which contains:

io.cucumber.core.eventbus.IncrementingUuidGenerator

The default Cucumber META-INF file contains both the RandomUuidGenerator and the IncrementingUuidGenerator, and the UuidGeneratorServiceLoader uses the RandomUuidGenerator by default.

Are you in one of these configuration ?

The IncrementingUuidGenerator was designed to get the highest performance from Cucumber. The typical use-case is when you have very short duration test scenarios (e.g. about 1 millisecond): in this situation, the Cucumber framework overhead starts to be bigger in percentage, so IncrementingUuidGenerator helps to reduce the framework impact. However, when having long-running test scenarios (>100 ms), I don't think you'll get a true advantage with IncrementingUuidGenerator.

Changing the IncrementingUuidGenerator logic with a more complex approach which reuse the session ids would probably decrease the performance, which is against the original idea.

[Piscenois]

Hi,

Same problem without configuring any custom UUID generator :

java.util.ServiceConfigurationError: io.cucumber.core.eventbus.UuidGenerator: Provider io.cucumber.core.eventbus.IncrementingUuidGenerator could not be instantiated
Caused by: io.cucumber.core.exception.CucumberException: Out of IncrementingUuidGenerator capacity. Please reuse existing instances or use another UuidGenerator implementation instead.

It doesn't happen with 7.0.0.
To give you context, I'm upgrading Cucumber version (from version 1.2.6 to 7.20.0) following your guidelines. I change only some code for TypeRegistry and @CucumberContextConfiguration.
The problem appears when I launch all the tests (maven command on Jenkins). With IntelliJ and only one test, it works.

[mpkorstanje]

@Piscenois looks like you are running into #2930. Can you try upgrading to 7.19.0 instead.

[jkronegg]

I reproduced the issue with the following unit test in UuidGeneratorServiceLoaderTest:

@Test
void test_case_2_robustness() {
    Options options = () -> null;
    UuidGeneratorServiceLoader loader = new UuidGeneratorServiceLoader(
            UuidGeneratorServiceLoaderTest.class::getClassLoader,
            options);
    UuidGenerator actual = null;
    for (int i=0; i<1000; i++) {
        actual = loader.loadUuidGenerator();
    }
    assertThat(actual, instanceOf(RandomUuidGenerator.class));
}

The error is:

java.util.ServiceConfigurationError: io.cucumber.core.eventbus.UuidGenerator: Provider io.cucumber.core.eventbus.IncrementingUuidGenerator could not be instantiated

	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813)
	at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729)
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
	at io.cucumber.core.runtime.UuidGeneratorServiceLoader.loadSingleUuidGeneratorOrDefault(UuidGeneratorServiceLoader.java:60)
	at io.cucumber.core.runtime.UuidGeneratorServiceLoader.loadUuidGenerator(UuidGeneratorServiceLoader.java:46)
	at io.cucumber.core.runtime.UuidGeneratorServiceLoaderTest.test_case_2_robustness(UuidGeneratorServiceLoaderTest.java:81)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: io.cucumber.core.exception.CucumberException: Out of IncrementingUuidGenerator capacity. Please reuse existing instances or use another UuidGenerator implementation instead.
	at io.cucumber.core.eventbus.IncrementingUuidGenerator.<init>(IncrementingUuidGenerator.java:102)
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789)
	... 8 more

I think this will be solved by #2931

[Piscenois]

@Piscenois looks like you are running into #2930. Can you try upgrading to 7.19.0 instead.

Thank you @mpkorstanje ! It works fine. 👍