huge memory and high CPU

[suutaku]

js-ipfs v 0.38.1
Mac OS X 10.14.6
node 12.10.0

Launch js-ipfs bundle and do nothing.
About 1 hours, memory and cpu usage was going to very hight.
CPU over 130%
memory over 3.5G
process sample:
sample.txt

please take a look. thanks

[suutaku]

another cpu profile

CPU-20191116T231957.cpuprofile.zip

it's really a big problem...

[suutaku]

any update???

[achingbrain]

Can you share a bit more about your configuration? Some sample code to run that replicates the problem would be best.

[suutaku]

Can you share a bit more about your configuration? Some sample code to run that replicates the problem would be best.

here is configuration

{
        silent:true,
        repo: home,
        config: {
            Addresses: {
              Swarm: [
                "/ip4/0.0.0.0/tcp/4001",
                "/ip4/127.0.0.1/tcp/4003/ws",
                "/dns4/xxx/tcp/9090/ws/p2p-webrtc-star"
              ]
            }
          },
        libp2p:  {
            switch: {
              blacklistTTL: 2 * 60 * 1e3, // 2 minute base
              blackListAttempts: 5, // back off 5 times
              maxParallelDials: 100,
              maxColdCalls: 100,
              dialTimeout: 10e3 // Be strict with dial time
            },
            modules: {
              transport: [
                TCP,
                WebSockets,
                webrtcStar,
                wss
                
              ],
              streamMuxer: [
                Multiplex
                //SPDY
              ],
              connEncryption: [
                SECIO
              ],
              connProtector: new Protector(swarmKeyBuffer),
              peerDiscovery: [
                MulticastDNS,
                Bootstrap,
                webrtcStar.discovery,
                wss.discovery
               
              ],
              dht: KadDHT
            },
            config: {
              peerDiscovery: {
                autoDial: true,
                mdns: {
                  enabled: true
                },
                bootstrap: {
                  enabled: true,
                  list: bootstrapers
                },
                websocketStar: {
                  enabled: true
                }
              },
              dht: {
                kBucketSize: 20,
                enabled: true,
                randomWalk: {
                  enabled: true
                }
              },
              relay: { 
                enabled: true, 
                hop: { 
                  enabled: true,
                   active: true 
                } 
              }
            }
        },
        connectionManager: {
          minPeers: 1,
          maxPeers: 50
        }
    }

and I only set up 3 to 4 peers in my private network.

[alanshaw]

cc @jacobheun @vasco-santos

[jacobheun]

@suutaku you have 3 or 4 nodes in a private network, are they all sharing the same config you provided above? Are these all running on Node.js or do you also have browser/go nodes?

There are existing performance issues we are aware of with the DHT which is why we don't currently recommend running it with ipfs at the moment, but 4 nodes really shouldn't be a problem.

[suutaku]

Yes, nodes sharing same config.
3 go version nodes running with cluster-server-ctl, without /wssupported.
1 node running js-ipfs bundle.
After did same reaches. I removed package libp2p-kad-dht@0.16.0 and install libp2p-kad-dht@0.16.1, memory keeping on 100M for now. I will do more testing to make sure if 0.16.1 is ok.
from @alanshaw 's relay at #2311 , I know you are doing a big refactor now. it will be awesome if the performance problem fixed.