[Bug] [oracle] maximum number of processes exceeded #1902
Comments
|
I submitted a pr #1903 |
|
Hello, are you sure you're into this method? I added some log printing to this method, but the program did not print any logs when it ran, and I also changed the original log printing, and the program ran not as I changed the log, but as the original log..... |
|
mark! |
I'm sure it's in this method. Maybe you can add StartupMode.INITIAL when starting the job. |
|
The database connection pool will not be closed. |
There are two problems. One is that the connection pool is not used. The second is that after the connection pool is used, the connection is not be closed , which causes other threads to be unable to obtain the connection |
|
Last Week，The production server crashed directly. @Cheers0606 This PR can Solve the problem? |
You can try. I have tested and successfully synchronized a table with 100 million data |
|
Caused by: java.sql.SQLTransientConnectionException: connection-pool-192.168.1.192:1521 - Connection is not available, request timed out after 30000ms. |
2023-02-20 18:11:39,159 WARN org.apache.flink.runtime.taskmanager.Task [] - Source: no_primary_1_source[1] -> ConstraintEnforcer[2] (1/1)#8 (8e29a0661c3ee0b7fdbd27166cffd3c7_cbc357ccb763df2852fee8c4fc7d55f2_0_8) switched from RUNNING to FAILED with failure cause: java.lang.RuntimeException: One or more fetchers have encountered exception
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:225)
at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:169)
at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:130)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385)
at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:150)
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
... 1 more
Caused by: org.apache.flink.util.FlinkRuntimeException: Read split SnapshotSplit{tableId=pdb.C##FLINKUSER.NO_PRIMARY_1, splitId='pdb.C##FLINKUSER.NO_PRIMARY_1:1', splitKeyType=[`ROWID` STRING], splitStart=[AAAR2iAAOAAAACgAB6], splitEnd=[AAAR2iAAOAAAAC3AD7], highWatermark=null} error due to java.lang.NullPointerException.
at com.ververica.cdc.connectors.base.source.reader.external.IncrementalSourceScanFetcher.checkReadException(IncrementalSourceScanFetcher.java:181)
at com.ververica.cdc.connectors.base.source.reader.external.IncrementalSourceScanFetcher.pollSplitRecords(IncrementalSourceScanFetcher.java:128)
at com.ververica.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.fetch(IncrementalSourceSplitReader.java:73)
at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58)
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:142)
... 6 more
Caused by: io.debezium.DebeziumException: java.lang.NullPointerException
at com.ververica.cdc.connectors.oracle.source.reader.fetch.OracleScanFetchTask$OracleSnapshotSplitReadTask.execute(OracleScanFetchTask.java:251)
at com.ververica.cdc.connectors.oracle.source.reader.fetch.OracleScanFetchTask.execute(OracleScanFetchTask.java:110)
at com.ververica.cdc.connectors.base.source.reader.external.IncrementalSourceScanFetcher.lambda$submitTask$0(IncrementalSourceScanFetcher.java:94)
... 5 more
Caused by: java.lang.NullPointerException
at com.ververica.cdc.connectors.oracle.source.reader.fetch.OracleScanFetchTask$OracleSnapshotSplitReadTask.createDataEventsForTable(OracleScanFetchTask.java:332)
at com.ververica.cdc.connectors.oracle.source.reader.fetch.OracleScanFetchTask$OracleSnapshotSplitReadTask.createDataEvents(OracleScanFetchTask.java:316)
at com.ververica.cdc.connectors.oracle.source.reader.fetch.OracleScanFetchTask$OracleSnapshotSplitReadTask.doExecute(OracleScanFetchTask.java:276)
at com.ververica.cdc.connectors.oracle.source.reader.fetch.OracleScanFetchTask$OracleSnapshotSplitReadTask.execute(OracleScanFetchTask.java:246)
... 7 more |
|
hello, oracle 19c support? @Cheers0606 |
I just tested with oracle 11g, flink 1.15. Let me try later. |
There is no problem with the test on my side. Can you describe your represent steps? |
IN FLINK SESSION |
|
Have fixed in #2254. |
Search before asking
Flink version
1.15.2
Flink CDC version
2.3
Database and its version
oracle 11.2.0.4.0
Minimal reproduce step
What did you expect to see?
The task is executed normally and the data can be synchronized in full
What did you see instead?
An error occurred after execution for a period of time: oracle maximum number of processes exceeded
Anything else?
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: