-
Notifications
You must be signed in to change notification settings - Fork 566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Concat array enhancement #2508
Concat array enhancement #2508
Conversation
97e2fc2
to
edff274
Compare
Signed-off-by: Daniel Kec <daniel.kec@oracle.com>
The code looks quite unconventional to me and the state transitions are somewhat hard to follow. In general, Here is a test that passes on master with the static void race(Runnable r1, Runnable r2, ExecutorService exec) {
AtomicInteger sync = new AtomicInteger(2);
CountDownLatch cdl = new CountDownLatch(1);
exec.submit(() -> {
if (sync.decrementAndGet() != 0) {
while (sync.get() != 0) ;
}
try {
r1.run();
} finally {
cdl.countDown();
}
});
if (sync.decrementAndGet() != 0) {
while (sync.get() != 0) ;
}
r2.run();
try {
Assert.assertTrue(cdl.await(5, TimeUnit.SECONDS));
} catch (InterruptedException ex) {
Assert.fail("", ex);
}
}
@Test
public void switchOverRequestRace() {
var exec = Executors.newSingleThreadExecutor();
try {
for (int i = 0; i < 10_000; i++) {
System.out.printf("Round %d%n", i + 1);
TestSubscriber<Object> ts = new TestSubscriber<>(0L);
AtomicReference<Flow.Subscriber<? super Integer>> sref = new AtomicReference<>();
Multi.concatArray(new Flow.Publisher<Integer>() {
@Override
public void subscribe(Flow.Subscriber<? super Integer> subscriber) {
subscriber.onSubscribe(EmptySubscription.INSTANCE);
sref.set(subscriber);
}
}, Multi.range(1, 100)).subscribe(ts);
race(() -> {
sref.get().onComplete();
}, () -> {
var s = ts.getSubcription();
for (int j = 0; j < 100; j++) {
s.request(1);
}
}, exec);
ts.awaitDone(5, TimeUnit.SECONDS);
ts.assertItemCount(100);
}
} finally {
exec.shutdown();
}
} |
Thanks. The problem with the original code is the lack of atomicity in state transitions around the same place, which can also be shown to lead to problems (not conformant to the specification). Consequently, an attempt to capture all state transitions in a single atomic counter. |
produced++; // assert: matching request(1) has been done by nextSource() | ||
this.subscription = subscription; | ||
// assert: requested == SEE_OTHER | ||
REQUESTED.setOpaque(this, p0); // assert: p0 is guaranteed to be a value of requested never seen before |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two things need doing:
long oldProduced = produced;
on the line before change of REQUESTED
, then use oldProduced
to the end of this method.
This addresses the case raised by @akarnok - concurrent request
results in concurrent onNext
, which updates produced
.
and
REQUESTED.setVolatile(this, p0)
to replace setOpaque
- opaque doesn't guarantee ordering with other stores.
I even remember considering the race condition this test shows. It really only shows https://github.com/reactive-streams/reactive-streams-jvm/blob/006202eb877c54cf64b258a8a78328e4fb38cab6/tck/src/main/java/org/reactivestreams/tck/PublisherVerification.java#L244 - this is the test that |
103 tests for concurrent access violation which Of course, you can ensure no request is delivered until the |
That is not the proof of the absence of the race in Even without such a test it is easy to see from the code of the We have a prototype that works around this issue with spec conformance of the |
…/onComplete concurrently with onSubscribe in response to concurrent request(). Signed-off-by: Daniel Kec <daniel.kec@oracle.com>
} | ||
|
||
LASTTHREADCOMPLETING.setOpaque(this, current); | ||
VarHandle.storeStoreFence(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reading this again, strictly speaking this fence is not needed, because we have stronger fences between the stores targeted by this fence. But no need to change the code as the cost of this fence on the target platforms is zero.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Appears to have only cosmetic changes compared to the internal proposal.
* Upgrade Netty to 4.1.58 (#2678) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Added overall timeout to evictable cache (#2659) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Fix copyright year for commits broken by squashing. (#2687) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Concat array enhancement (#2508) * Concat array enhancement Signed-off-by: Daniel Kec <daniel.kec@oracle.com> * Update Jackson to 2.12.1 (#2690) * Update Jackson to 2.12.1 * Upgrade to latest Junit5 to get fix for junit-team/junit5#2198 * Manage junit4 version * PokemonService template fixed in SE Database Archetype. (#2701) Signed-off-by: Tomas Kraus <Tomas.Kraus@oracle.com> * Fixed different output in DbClient SE archetype (#2703) Signed-off-by: Tomas Kraus <Tomas.Kraus@oracle.com> * Fix TODO application: (#2708) - WebSecurity needs to be passed config.get("security") to take the "security.web-server" configuration - Added outbound configuration for the google login - Upgraded cassandra driver to fix issues with old guava dependencies - Removed metrics to avoid issues with cassandra driver. Fixes #2707 * Update k8s descriptors to avoid using deprecated APIs. (#2719) * Separate execution of DataChunkReleaseTest in its own VM to prevent leak messages in other test's logs. (#2716) Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Changes in this commit: (#2727) 1. Upgrade to Jersey 2.33 2. Configuration via system properties for the Jersey Client API. Any response in an exception will be mapped to an empty one to prevent data leaks. See eclipse-ee4j/jersey#4641. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Properly release underlying buffer before passing it to WebSocket handler (#2715) * Properly release underlying buffer before passing it to handler. * Releases data chunks after passing them to Tyrus without any copying. Reports an error and closes connection if Tyrus is unable to handle the data. Finally, fixed a problem related to subscription requests. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Removed unused logger. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed checkstyle. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fix issue with null value in JSON. (#2723) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Upgrade grpc to v1.35.0 (#2713) * Upgrade grpc to v1.35.0 * Update copyright * Upgrades OCI SDK to version 1.31.0 (#2699) * Updated OCI to 1.31.0 Signed-off-by: Laird Nelson <laird.nelson@oracle.com> * Fix null array values in HOCON/JSON config parser. (#2731) Resolves #2720 (follow-up) * Performance improvements to queue(s) management in Webserver (#2704) * Initial patch. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed some type params and improved comments. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * More cleanup and make sure to fail publisher on an error condition. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Suppress warnings. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Call clearQueues on every new request for proper cleanup of keep-alive connections. Some copyright fixes. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed checkstyle issues. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Force logging of LEAK error even if finalize does not get called on a DataChunk. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Upgrade Weld (#2668) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Rest client async header propagation with usage of Helidon Context (#2735) Rest client header propagation with usage of Helidon Context Signed-off-by: David Kral <david.k.kral@oracle.com> * Allow override of Jersey property via config (#2737) * Allow the default value of property jersey.config.client.ignoreExceptionResponse to be overridden via config. New test. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed copyright year. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * New implementation of LazyValue (#2738) * New implementation of LazyValue that lazily initializes a Semaphore instead of eagerly creating a ReentrantLock. Makes use of volatile guarantees and atomicity of VarHandle updates. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * New test for LazyValueImpl. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Reduced sleep time in test. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Update CHANGELOG for 2.2.1 release (#2743) * 2.2.1 THIRD_PARTY_LICENSES update (#2746) * Update THIRD_PARTY_LICENSES * Support async invocations using optional synthetic SimplyTimed behavior (#2745) * Add support for async invocations for optional inferred SimplyTimed behavior on JAX-RS endpoints Signed-off-by: tim.quinn@oracle.com <tim.quinn@oracle.com> * Do not attempt to access the request context in Fallback callback. If used together with Retry, it is possible for the fallback to be called in a fresh thread for which there is no current request scope. Instead just use the original value obtained in this class' constructor. Updated functional test (with some class renaming) to cover this use case. (#2748) Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fix for native image. (#2753) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Fixed checkstyle issues. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> Co-authored-by: Tomas Langer <tomas.langer@gmail.com> Co-authored-by: Daniel Kec <daniel.kec@oracle.com> Co-authored-by: Joe DiPol <joe.dipol@oracle.com> Co-authored-by: Tomáš Kraus <tomas.kraus@oracle.com> Co-authored-by: Romain Grecourt <romain.grecourt@oracle.com> Co-authored-by: Jonathan Knight <jk@thegridman.com> Co-authored-by: Laird Nelson <laird.nelson@oracle.com> Co-authored-by: David Král <david.k.kral@oracle.com> Co-authored-by: Tim Quinn <tim.quinn@oracle.com>
* Fault Tolerance 3.0 Support (#2680) * Initial changes to implement new metrics layer. Moving from complex names to simpler names and tags. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * More metric updates. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Migration of most unit tests to new metrics. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Completed migration of metrics test. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * New exception to discern timeouts during retries. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Implementation of retry metrics. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Cleanup metrics between tests. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Several changes related to execution of FT 3.0 TCKs. Adjusted initial size of executors and fixed a few other problems. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Copyright and checkstyle updates. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed copyright year. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed typos and some cleanup. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Created exclude file as a workaround for a sportbugs' bug. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Updated copyright year. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * MicroProfile Opentracing 2.0 (#2676) * Microprofile Opentracing uprgated to 2.0 * Unused dependences removed * Obsolete excludes removed * Sync up of microprofile-4.0 with master branch (#2757) * Upgrade Netty to 4.1.58 (#2678) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Added overall timeout to evictable cache (#2659) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Fix copyright year for commits broken by squashing. (#2687) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Concat array enhancement (#2508) * Concat array enhancement Signed-off-by: Daniel Kec <daniel.kec@oracle.com> * Update Jackson to 2.12.1 (#2690) * Update Jackson to 2.12.1 * Upgrade to latest Junit5 to get fix for junit-team/junit5#2198 * Manage junit4 version * PokemonService template fixed in SE Database Archetype. (#2701) Signed-off-by: Tomas Kraus <Tomas.Kraus@oracle.com> * Fixed different output in DbClient SE archetype (#2703) Signed-off-by: Tomas Kraus <Tomas.Kraus@oracle.com> * Fix TODO application: (#2708) - WebSecurity needs to be passed config.get("security") to take the "security.web-server" configuration - Added outbound configuration for the google login - Upgraded cassandra driver to fix issues with old guava dependencies - Removed metrics to avoid issues with cassandra driver. Fixes #2707 * Update k8s descriptors to avoid using deprecated APIs. (#2719) * Separate execution of DataChunkReleaseTest in its own VM to prevent leak messages in other test's logs. (#2716) Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Changes in this commit: (#2727) 1. Upgrade to Jersey 2.33 2. Configuration via system properties for the Jersey Client API. Any response in an exception will be mapped to an empty one to prevent data leaks. See eclipse-ee4j/jersey#4641. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Properly release underlying buffer before passing it to WebSocket handler (#2715) * Properly release underlying buffer before passing it to handler. * Releases data chunks after passing them to Tyrus without any copying. Reports an error and closes connection if Tyrus is unable to handle the data. Finally, fixed a problem related to subscription requests. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Removed unused logger. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed checkstyle. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fix issue with null value in JSON. (#2723) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Upgrade grpc to v1.35.0 (#2713) * Upgrade grpc to v1.35.0 * Update copyright * Upgrades OCI SDK to version 1.31.0 (#2699) * Updated OCI to 1.31.0 Signed-off-by: Laird Nelson <laird.nelson@oracle.com> * Fix null array values in HOCON/JSON config parser. (#2731) Resolves #2720 (follow-up) * Performance improvements to queue(s) management in Webserver (#2704) * Initial patch. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed some type params and improved comments. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * More cleanup and make sure to fail publisher on an error condition. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Suppress warnings. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Call clearQueues on every new request for proper cleanup of keep-alive connections. Some copyright fixes. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed checkstyle issues. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Force logging of LEAK error even if finalize does not get called on a DataChunk. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Upgrade Weld (#2668) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Rest client async header propagation with usage of Helidon Context (#2735) Rest client header propagation with usage of Helidon Context Signed-off-by: David Kral <david.k.kral@oracle.com> * Allow override of Jersey property via config (#2737) * Allow the default value of property jersey.config.client.ignoreExceptionResponse to be overridden via config. New test. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed copyright year. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * New implementation of LazyValue (#2738) * New implementation of LazyValue that lazily initializes a Semaphore instead of eagerly creating a ReentrantLock. Makes use of volatile guarantees and atomicity of VarHandle updates. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * New test for LazyValueImpl. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Reduced sleep time in test. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Update CHANGELOG for 2.2.1 release (#2743) * 2.2.1 THIRD_PARTY_LICENSES update (#2746) * Update THIRD_PARTY_LICENSES * Support async invocations using optional synthetic SimplyTimed behavior (#2745) * Add support for async invocations for optional inferred SimplyTimed behavior on JAX-RS endpoints Signed-off-by: tim.quinn@oracle.com <tim.quinn@oracle.com> * Do not attempt to access the request context in Fallback callback. If used together with Retry, it is possible for the fallback to be called in a fresh thread for which there is no current request scope. Instead just use the original value obtained in this class' constructor. Updated functional test (with some class renaming) to cover this use case. (#2748) Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fix for native image. (#2753) Signed-off-by: Tomas Langer <tomas.langer@oracle.com> * Fixed checkstyle issues. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> Co-authored-by: Tomas Langer <tomas.langer@gmail.com> Co-authored-by: Daniel Kec <daniel.kec@oracle.com> Co-authored-by: Joe DiPol <joe.dipol@oracle.com> Co-authored-by: Tomáš Kraus <tomas.kraus@oracle.com> Co-authored-by: Romain Grecourt <romain.grecourt@oracle.com> Co-authored-by: Jonathan Knight <jk@thegridman.com> Co-authored-by: Laird Nelson <laird.nelson@oracle.com> Co-authored-by: David Král <david.k.kral@oracle.com> Co-authored-by: Tim Quinn <tim.quinn@oracle.com> * Fixed problems in RetryImpl after merge. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed problems with metrics after merge. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Updated version in suite file. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed problem retrieving registry for metrics. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed more problems after merge. All tests are passing now. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed checkstyle errors. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Fixed TODO. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Enabled TCK's by default and removed generated file. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * One more checkstyle violation. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> * Removed duplicate test after merge. Signed-off-by: Santiago Pericasgeertsen <santiago.pericasgeertsen@oracle.com> Co-authored-by: Dmitry Aleksandrov <mitiaalexandrov@gmail.com> Co-authored-by: Tomas Langer <tomas.langer@gmail.com> Co-authored-by: Daniel Kec <daniel.kec@oracle.com> Co-authored-by: Joe DiPol <joe.dipol@oracle.com> Co-authored-by: Tomáš Kraus <tomas.kraus@oracle.com> Co-authored-by: Romain Grecourt <romain.grecourt@oracle.com> Co-authored-by: Jonathan Knight <jk@thegridman.com> Co-authored-by: Laird Nelson <laird.nelson@oracle.com> Co-authored-by: David Král <david.k.kral@oracle.com> Co-authored-by: Tim Quinn <tim.quinn@oracle.com>
JMH test results: