ignoring connection above current capacity: NoSlabSpace(<hidden seed>)

[roderik]

I'm running:

• Parity version: Parity/v1.7.0-beta-5f2cabd-20170727/x86_64-linux-gnu/rustc1.18.0
• Operating system: Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-89-generic x86_64)
• And installed: bash <(curl https://get.parity.io -kL)

This node is one in three of a PoA private setup. On top of the RPC of this node we are running a very complex, and active multi-tenant API layer using web3 to interface with the node. There are quite some watches for events going on.

It has been working splendidly for a while, and then all of a sudden, the api loses connection to the node which displays a streaming list of the same message:

ignoring connection above current capacity: NoSlabSpace(<hidden seed>)

• Searched the code to see where and why this message was thrown, but could not find it
• Looked for a max connection parameter for the RPC, could not find one (only between nodes)
• Increased the server size from 1GB/1CPU to 4GB/2CPU, it still happens

I'm looking for

• a cause, i assume the RPC connections, but i'm not sure.
• what is the limit, and how to increase it
• other solutions i have not thought about (and preferably not "set less watches" since i need them)

For reference, all relevant configs below

toml

[parity]
chain = "spec.json"
base_path = "/root/parity0"
[network]
port = 30300

[rpc]
interface = "xx.xx.xx.xx"
hosts = ["all"]
port = 8545
apis = ["web3", "eth", "net", "personal", "parity", "parity_set", "traces", "rpc", "parity_accounts"]

[ui]
interface = "xx.xx.xx.xx"
port = 8180

[websockets]
interface = "xx.xx.xx.xx"

[dapps]
interface = "xx.xx.xx.xx"
hosts = ["all"]
port = 8080

[account]
password = ["node.pwds"]

[mining]
engine_signer = "0x00..."
reseal_on_txs = "none"
gas_floor_target = "47000000"
gas_cap = "62831840"
tx_gas_limit = "62831840"

Note: I'm aware of the depreciation notices on the dapps etc parameters

spec.json

{
    "name": "MyNet",
    "engine": {
        "authorityRound": {
            "params": {
                "gasLimitBoundDivisor": "0x400",
                "stepDuration": "1",
                "blockReward": "0x4563918244F40000",
                "validators" : {
                    "list": [
                        "0x00...",
                        "0x00...",
                        "0x00..."
                    ]
                }
            }
        }
    },
    "params": {
        "maximumExtraDataSize": "0x20",
        "minGasLimit": "0x1388",
        "networkID" : "0x2323"
    },
    "genesis": {
        "seal": {
            "authorityRound": {
                "step": "0x0",
                "signature": "0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
            }
        },
        "difficulty": "0x20000",
        "gasLimit": "0x5B8D80"
    },
    "accounts": {
        "0x0000000000000000000000000000000000000001": { "balance": "1", "builtin": { "name": "ecrecover", "pricing": { "linear": { "base": 3000, "word": 0 } } } },
        "0x0000000000000000000000000000000000000002": { "balance": "1", "builtin": { "name": "sha256", "pricing": { "linear": { "base": 60, "word": 12 } } } },
        "0x0000000000000000000000000000000000000003": { "balance": "1", "builtin": { "name": "ripemd160", "pricing": { "linear": { "base": 600, "word": 120 } } } },
        "0x0000000000000000000000000000000000000004": { "balance": "1", "builtin": { "name": "identity", "pricing": { "linear": { "base": 15, "word": 3 } } } }
    }
}

[roderik]

ps. i'm not unsetting the watches before shutting down to do a new deploy. Maybe they stay active in the node?

[tomusdrw]

Seems that the node is reaching maximum of 4096 open sockets. It's currently not configurable but will be addressed for 1.8 release (switching to tokio-based hyper).
You may try switching to fast http server implementation that shouldn't have this limitation: --jsonrpc-server-threads 1, when you run more than one thread you can hit #6096, but if you don't send many parallel transactions from the same account it works fine.

[roderik]

Perfect, trying it with '--jsonrpc-server-threads 1' for a while!

[roderik]

Error has dissapeared, but with --jsonrpc-server-threads 1 activated the dapps server is deactivated and hitting the UI gets me the 'generate a new token' modal i cannot get past

[5chdn]

The dapps server was removed / merged with rpc server in 1.7.0.

What happens if you enter a new token in the wallet?

[tomusdrw]

Seems that the UI is not handling missing dapps server correctly (but it should). Will have a look at that today.

[5chdn]

Did #6296 address this?

[tomusdrw]

After #6296 the UI works without dapps server, the issue with NoSlabSpace will be resolved when we update to jsonrpc-http-server (no issue for that, feel free to create one and assign me).

[stone212]

I am also having this issue on a node that was working great for some time. I am using Parity 1.6.10 (there is a reason for the older version but I forget why at this moment). The reason I think it is worth mentioning this here is because I also have the symptom that "hitting the UI gets me the 'generate a new token' modal i cannot get past" as I have referenced here:

#6586

Because there are now two connections between these two seemingly unconnected issues I hope it is helpful to add my issue to this thread.

Incidentally, stopping and restarting Parity makes the NoSlapSpace error go away.

[AyushyaChitransh]

I am also facing the same issue and restarting parity makes the error go away.

[tomusdrw]

@AyushyaChitransh Can you check if the error is still present in 1.8?

[stone212]

@tomusdrw

I have not had the problem since Parity 1.7.6. I didn't think to update the thread until I got pinged with your and AC's replies.