Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flink error: More buffers requested available than totally available. #29

Open
ziXet opened this issue Mar 4, 2021 · 1 comment
Open

Comments

@ziXet
Copy link

ziXet commented Mar 4, 2021

I get the following error when I want to build the Texas state tiles.

Command:
java -jar sharedstreets-builder-0.3.1.jar --input osm/texas-latest.osm.pbf --output ./texas_tiles

Error:

03/04/2021 21:16:20 CHAIN Join(Join at buildWays(OSMDataStream.java:179)) -> Map (Map at buildWays(OSMDataStream.java:179))(6/16) switched to FAILED
java.lang.IllegalArgumentException: More buffers requested available than totally available.
at org.apache.flink.runtime.operators.hash.MutableHashTable.ensureNumBuffersReturned(MutableHashTable.java:1333)
at org.apache.flink.runtime.operators.hash.MutableHashTable.initTable(MutableHashTable.java:1156)
at org.apache.flink.runtime.operators.hash.MutableHashTable.buildTableFromSpilledPartition(MutableHashTable.java:921)
at org.apache.flink.runtime.operators.hash.MutableHashTable.prepareNextPartition(MutableHashTable.java:631)
at org.apache.flink.runtime.operators.hash.MutableHashTable.nextRecord(MutableHashTable.java:666)
at org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashJoinIterator.callWithNextKey(NonReusingBuildFirstHashJoinIterator.java:116)
at org.apache.flink.runtime.operators.JoinDriver.run(JoinDriver.java:222)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:490)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:355)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:748)

There is not much explanation about this error on the Internet.

I already tried to increase the Flink taskmanager heap size.
taskmanager.memory.size
taskmanager.heap.size

Environment:
java -version
openjdk version "1.8.0_282" OpenJDK Runtime Environment (build 1.8.0_282-8u282-b08-0ubuntu1~16.04-b08) OpenJDK 64-Bit Server VM (build 25.282-b08, mixed mode)

CPU: 16 - Memory 60GB

Any help would be appreciated.

@ziXet
Copy link
Author

ziXet commented Mar 5, 2021

Solved it by increasing JVM heap size with -Xmx flag!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant