Cannot subscribe more than 5-6 topics [ipfs-http-client]

[TheDiscordian]

• Version:

ipfs-core
{
  version: '0.8.0',
  repo: '10',
  commit: '81f944163f5a78ef15fdc23b2c488ba7f44af0be',
  'interface-ipfs-core': '^0.147.0'
}
ipfs-http-client
{
  version: '0.8.0',
  commit: 'ce693d7e8',
  repo: '11',
  system: 'amd64/linux',
  golang: 'go1.16.2'
}

• Platform:
  Linux TheDesktop 5.10.42-1-MANJARO #1 SMP PREEMPT Thu Jun 3 14:37:11 UTC 2021 x86_64 GNU/Linux

• Subsystem:
  ipfs-http-client

Severity:

High

Description:

Issue found at: https://stackoverflow.com/questions/68257274/cannot-subscribe-to-more-than-6-ipfs-pubsub-channels-with-ipfs-http-client

1. Subscribe to 20 topics
2. Publish to all 20 topics
3. Only first 5-6 subs will get messages

Steps to reproduce the error:

First subscribe to the topics:

const { create } = require('ipfs-http-client');

async function echo(msg) {
	console.log(`TopicID: ${msg.topicIDs[0]}, Msg: ${new TextDecoder().decode(msg.data)}`);
}

async function run() {
	// connect to the default API address http://localhost:5001
	const client = create();
	console.log(await client.version());
	
	for (let i = 0; i < 20; i++) {
		await client.pubsub.subscribe(parseInt(i), echo);
	}
}

run();

Then publish:

#!/bin/bash
for i in {1..20}
do
   ipfs pubsub pub $i $i
done

Output is only:

{
  version: '0.8.0',
  commit: 'ce693d7e8',
  repo: '11',
  system: 'amd64/linux',
  golang: 'go1.16.2'
}
TopicID: 1, Msg: 1
TopicID: 2, Msg: 2
TopicID: 3, Msg: 3
TopicID: 4, Msg: 4
TopicID: 5, Msg: 5

[vasco-santos]

Given it is not the same behaviour all the time, I would say the previous subscriptions are still being propagated in the network while the publish happens.

When a pubsub subscription is done, it will take a bit before nodes exchange their subscription messages around. So, I suspect this is the problem. If so, we should improve our docs to make the expectations clear.

@TheDiscordian could you wrap your bash script with an extra loop that sends the messages for the 20 topics multiple times to confirm these assumptions?

[TheDiscordian]

@TheDiscordian could you wrap your bash script with an extra loop that sends the messages for the 20 topics multiple times to confirm these assumptions?

Sure thing! I'll make it loop 20 times, with a delay between each attempt.

[TheDiscordian]

@vasco-santos no change, still just first 5 subs using:

 #!/bin/bash
for ii in {1..20}
do
	echo "BATCH "$ii
	for i in {1..20}
	do
		ipfs pubsub pub $i $i
	done
	sleep 10
done

FWIW I've never personally seen 6, I'm reporting 5-6 because the SO user reported 6 (here). It's possible they were mistaken.

Edit: Log: https://ipfs.io/ipfs/QmdFNKLtp2RUheirioNFURfzRF9BwCbjM9Pf5T5c9Q3qgA

[achingbrain]

The problem here is that browsers limit the number of simultaneously open connections to a given hostname to six.

In node you can increase this limit by configuring an agent for the http client to use but browsers are stuck with max six per hostname.

Each subscription uses one connection so that's why you hit the limit.

js-IPFS has an experimental gRPC-over-websockets server though, and it doesn't have this limitation.

It only supports a few methods, the rest are all patched in by ipfs-http-client (so you still have the full IPFS core-api available).

I've opened #3813 which implements ipfs.pubsub.subscribe and ipfs.pubsub.unsubscribe over gRPC - once it's merged & released, just switch out ipfs-http-client for ipfs-client, ensure the Addresses.RPC config key is configured in your ~/.jsipfs/config file and you should be good to go.

[vasco-santos]

The problem here is that browsers limit the number of simultaneously open connections to a given hostname to six.

Is this in the browser? Per the information, in the issue and stackoverflow this does not seem to be browser environment?

[achingbrain]

In node the default agent used by the http client limits connections to 6 so it's consistent with the browser. You can configure your own agent to change this:

const { create } = require('ipfs-http-client');
const http = require('http')

async function echo(msg) {
	console.log(`TopicID: ${msg.topicIDs[0]}, Msg: ${new TextDecoder().decode(msg.data)}`);
}

async function run() {
	// connect to the default API address http://localhost:5001
	const client = create({
		agent: http.Agent({ keepAlive: true, maxSockets: Infinity })
	});
	console.log(await client.version());

	for (let i = 0; i < 20; i++) {
		await client.pubsub.subscribe(parseInt(i), echo);
	}
}

run();

{
  version: '0.8.0',
  commit: '',
  repo: '11',
  system: 'amd64/darwin',
  golang: 'go1.15.8'
}
TopicID: 1, Msg: 1
TopicID: 2, Msg: 2
TopicID: 3, Msg: 3
TopicID: 4, Msg: 4
TopicID: 5, Msg: 5
TopicID: 6, Msg: 6
TopicID: 7, Msg: 7
TopicID: 8, Msg: 8
TopicID: 9, Msg: 9
TopicID: 10, Msg: 10
TopicID: 11, Msg: 11
TopicID: 12, Msg: 12
TopicID: 13, Msg: 13
TopicID: 14, Msg: 14
TopicID: 15, Msg: 15
TopicID: 16, Msg: 16
TopicID: 17, Msg: 17
TopicID: 18, Msg: 18
TopicID: 19, Msg: 19

[achingbrain]

This looks like it's been resolved, please re-open or create a new issue if you're still having problems.