Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IGNITE-22478 Use single argument in Compute API #3926

Merged
merged 110 commits into from
Jun 28, 2024

Conversation

PakhomovAlexander
Copy link
Contributor

@PakhomovAlexander PakhomovAlexander commented Jun 14, 2024

The Compute API is changed:

  • Only single argument can be accepted or null by any compute job.
  • Marshaler API is introduced as a part of public API and can be used in Compute API.
  • Default marshaller is implemented but not used explicitly. By default we support only
    primitives. If the one want to use custom marshaler they should declare it explicitly.

Marshalers allow us to define the default way to serialize objects: java serialization.
At the same time client can implement theirs own marshalers and use them in any
public API that supports it. This is the first implementation and it is supposed to be
improved in the future. The only thing that will not be changed -- Marshaler and
Compute interfaces (their public parts).

…-22478

# Conflicts:
#	modules/api/src/main/java/org/apache/ignite/compute/IgniteCompute.java
#	modules/api/src/main/java/org/apache/ignite/compute/task/ComputeJobRunner.java
#	modules/client/src/main/java/org/apache/ignite/internal/client/compute/ClientCompute.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeBaseTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeErrorsBaseTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeTestEmbedded.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeTestStandalone.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItExecutionsCleanerTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItFailoverCandidateNotFoundTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItWorkerShutdownTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/threading/ItComputeApiThreadingTest.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/AntiHijackIgniteCompute.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/IgniteComputeImpl.java
#	modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/PlatformTestNodeRunner.java
…-22478

# Conflicts:
#	modules/api/src/main/java/org/apache/ignite/compute/ComputeJob.java
#	modules/api/src/main/java/org/apache/ignite/compute/task/MapReduceTask.java
#	modules/client-handler/src/main/java/org/apache/ignite/client/handler/requests/table/ClientStreamerWithReceiverBatchSendRequest.java
#	modules/client/src/test/java/org/apache/ignite/client/fakes/FakeCompute.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeTestEmbedded.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/threading/ItComputeApiThreadingTest.java
#	modules/compute/src/jobs/java/org/apache/ignite/internal/compute/FailingJob.java
#	modules/compute/src/jobs/java/org/apache/ignite/internal/compute/GetNodeNameJob.java
#	modules/compute/src/jobs/java/org/apache/ignite/internal/compute/MapReduce.java
#	modules/compute/src/jobs/java/org/apache/ignite/internal/compute/NonEmptyConstructorJob.java
#	modules/compute/src/jobs/java/org/apache/ignite/internal/compute/SleepJob.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/executor/ComputeExecutorImpl.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/task/TaskExecutionInternal.java
#	modules/compute/src/test/java/org/apache/ignite/internal/compute/loader/JobClassLoaderFactoryTest.java
#	modules/compute/src/unit1/java/org/apache/ignite/internal/compute/UnitJob.java
#	modules/compute/src/unit2/java/org/apache/ignite/internal/compute/UnitJob.java
#	modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/PlatformTestNodeRunner.java
…-22478

# Conflicts:
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItMapReduceTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/utils/InteractiveTasks.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/executor/ComputeExecutorImpl.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/task/TaskExecutionInternal.java
…-22478

# Conflicts:
#	modules/api/src/main/java/org/apache/ignite/compute/task/MapReduceTask.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeTestEmbedded.java
#	modules/compute/src/jobs/java/org/apache/ignite/internal/compute/MapReduce.java
…-22478

# Conflicts:
#	modules/api/src/main/java/org/apache/ignite/compute/IgniteCompute.java
#	modules/api/src/main/java/org/apache/ignite/table/DataStreamerTarget.java
#	modules/client-handler/src/main/java/org/apache/ignite/client/handler/requests/compute/ClientComputeExecuteColocatedRequest.java
#	modules/client-handler/src/main/java/org/apache/ignite/client/handler/requests/compute/ClientComputeExecuteMapReduceRequest.java
#	modules/client-handler/src/main/java/org/apache/ignite/client/handler/requests/compute/ClientComputeExecuteRequest.java
#	modules/client/src/main/java/org/apache/ignite/internal/client/compute/ClientCompute.java
#	modules/client/src/main/java/org/apache/ignite/internal/client/compute/ClientJobExecution.java
#	modules/client/src/main/java/org/apache/ignite/internal/client/table/ClientKeyValueBinaryView.java
#	modules/client/src/main/java/org/apache/ignite/internal/client/table/ClientKeyValueView.java
#	modules/client/src/main/java/org/apache/ignite/internal/client/table/ClientRecordBinaryView.java
#	modules/client/src/main/java/org/apache/ignite/internal/client/table/ClientRecordView.java
#	modules/client/src/test/java/org/apache/ignite/client/ClientComputeTest.java
#	modules/client/src/test/java/org/apache/ignite/client/PartitionAwarenessTest.java
#	modules/client/src/test/java/org/apache/ignite/client/fakes/FakeCompute.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeBaseTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeErrorsBaseTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeTestEmbedded.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItComputeTestStandalone.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItExecutionsCleanerTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItFailoverCandidateNotFoundTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItWorkerShutdownTest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/threading/ItComputeApiThreadingTest.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/AntiHijackIgniteCompute.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/IgniteComputeImpl.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/executor/JobExecutionInternal.java
#	modules/compute/src/test/java/org/apache/ignite/internal/compute/ComputeComponentImplTest.java
#	modules/rest/src/integrationTest/java/org/apache/ignite/internal/rest/compute/ItComputeControllerTest.java
#	modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/client/ItThinClientComputeTest.java
#	modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/client/ItThinClientPartitionAwarenessTest.java
#	modules/table/src/main/java/org/apache/ignite/internal/table/KeyValueBinaryViewImpl.java
#	modules/table/src/main/java/org/apache/ignite/internal/table/KeyValueViewImpl.java
#	modules/table/src/main/java/org/apache/ignite/internal/table/PublicApiThreadingViewBase.java
#	modules/table/src/main/java/org/apache/ignite/internal/table/RecordBinaryViewImpl.java
#	modules/table/src/main/java/org/apache/ignite/internal/table/RecordViewImpl.java
@PakhomovAlexander PakhomovAlexander changed the title IGNITE-22478 IGNITE-22478 Change Compute API Jun 24, 2024
…-22478

# Conflicts:
#	modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/PlatformTestNodeRunner.java
@ptupitsyn ptupitsyn changed the title IGNITE-22478 Change Compute API IGNITE-22478 Use single argument in Compute API Jun 25, 2024
PakhomovAlexander and others added 16 commits June 28, 2024 11:18
…ute.java

Co-authored-by: Vadim Pakhnushev <8614891+valepakh@users.noreply.github.com>
…ute.java

Co-authored-by: Vadim Pakhnushev <8614891+valepakh@users.noreply.github.com>
…ute.java

Co-authored-by: Vadim Pakhnushev <8614891+valepakh@users.noreply.github.com>
…ute.java

Co-authored-by: Vadim Pakhnushev <8614891+valepakh@users.noreply.github.com>
…ute.java

Co-authored-by: Vadim Pakhnushev <8614891+valepakh@users.noreply.github.com>
…ute.java

Co-authored-by: Vadim Pakhnushev <8614891+valepakh@users.noreply.github.com>
PakhomovAlexander and others added 9 commits June 28, 2024 15:47
…-22478

# Conflicts:
#	modules/api/src/main/java/org/apache/ignite/compute/JobDescriptor.java
#	modules/api/src/main/java/org/apache/ignite/table/ReceiverDescriptor.java
…-22478

# Conflicts:
#	modules/client-handler/src/main/java/org/apache/ignite/client/handler/requests/compute/ClientComputeExecuteMapReduceRequest.java
#	modules/compute/src/integrationTest/java/org/apache/ignite/internal/compute/ItMapReduceTest.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/ComputeComponentImpl.java
#	modules/compute/src/main/java/org/apache/ignite/internal/compute/task/TaskExecutionInternal.java
#	modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/client/ItThinClientComputeTest.java
@PakhomovAlexander PakhomovAlexander merged commit 692ecde into apache:main Jun 28, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants