Daemon v0.4.0-dev Kills Netgear C3700

[nginnever]

A new time warner cable plan and a "new" (new in quotes since time warner was kind enough to inform me that my mac address was already registered to an account and they would need to talk to a supervisor to replace it, so maybe it is not out of the box with factory settings as it was sold to me) netgear c3700. Plugged it in with all (maybe) factory settings. Running go-ipfs daemon for more than 5 minutes will consistently knock all active connections to the modem/router offline. All wifi connected devices and any ethernet connect devices. Modem reboot restores the network.

I haven't spent much time troubleshooting what could cause this and will probably replace the modem tomorrow out of laziness or lack of time. If that is not the case and anyone has suggestions on things we can try to figure this out I will of course try.

I did dump some logs from the router during a 5 minute interval of running the daemon before it crashed the network.

[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:04:56 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 8   Sat Jan 23 05:04:48 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:04:36 2016    0.0.0.0:0   192.168.0.12:0
[LAN access from remote] from 12.201.242.227:60876 to 192.168.0.12:4001 1   Sat Jan 23 05:04:33 2016    192.168.0.12:4001   12.201.242.227:60876
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 8   Sat Jan 23 05:04:32 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:04:16 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 16  Sat Jan 23 05:04:16 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:03:56 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 8   Sat Jan 23 05:03:44 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:03:35 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 8   Sat Jan 23 05:03:29 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:03:15 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 16  Sat Jan 23 05:03:14 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:02:54 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 8   Sat Jan 23 05:02:43 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:02:34 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 8   Sat Jan 23 05:02:28 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:02:14 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 3   Sat Jan 23 05:02:11 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:01:53 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 1   Sat Jan 23 05:01:33 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:01:33 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetExternalIPAddress] from source 192.168.0.12 1   Sat Jan 23 05:01:13 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: AddPortMapping] from source 192.168.0.12   1   Sat Jan 23 05:01:13 2016    0.0.0.0:0   192.168.0.12:0
[UPnP set event: GetNATRSIPStatus] from source 192.168.0.12 1   Sat Jan 23 05:01:13 2016    0.0.0.0:0   192.168.0.12:0
[admin login] from source 192.168.0.12  1   Sat Jan 23 04:58:06 2016    0.0.0.0:0   192.168.0.12:0

Cheers!

[whyrusleeping]

Thanks! I love seeing how fragile ISP's infrastructure is. We really should have an option to disable upnp or something along those lines. Maybe its just that its happening too often? This is kinda strange.

[nginnever]

Np! I wish I could have recorded some of the conversations I had with them while trying to register an already registered mac address. Also this is my first all powerful fry's electronics buying xp (midwest lacks fry's). -.-

googling "UPnP set event kills c3700" brings this issue to first search result :)

googling "UPnP set event c3700" returns from an "esteemed contributer" on the netgear forums
" if you have an application that is using UPnP it may well then show connectivity issues"

Only thing I can think to try is disabling UPnP at the router. Which I have no idea what that would do to a p2p app like ipfs... edit... disabling UPnP results in active TCP connections still. Maybe the router will stay alive?

[nginnever]

Disabling UPnP seems to have fixed the logs getting dumped, however the problem persists this morning. Seems to kick me off more randomly now and was staying online with ipfs longer than normal last night.

[jbenet]

@nginnever it could be a few other things, too:

• tcp allocations in some symmetric nat table. try using 0.4.0 with utp listener only. (will still dial out tcp, but wont get any inbound conns that way)
• if the router is getting overloaded and rebooting... get a new router. cheaper than the hours you will put in dealing with this.
• If the router is dropping the connection, but the hardware (and OS) is fine, i suspect conscious attacks by the ISP configuration itself.

Try using cjdns and pipe all your connections through some host outside your crap router / isp. i've had bad experiences with ISPs including throttling torrent downloads of linux ISOs, and of blizzard game updates.

In the US, Comcast for example would trickle your speed down dramatically, introduce fake latency, and intermittent disconnections (forged TCP RST packets AND dropping all traffic for some time). the FCC got on their case about it: https://www.eff.org/deeplinks/2008/08/fcc-rules-against-comcast-bit-torrent-blocking -- But then! this happened: https://en.wikipedia.org/wiki/Comcast_Corp._v._FCC so it's not clear what the state is today.

[palkeo]

Hi !

I have the same problem. With a technicolor modem from my ISP : http://wwwch.upc-cablecom.ch/fr/tc7200.u_user_manual_eng_v17.pdf

It never crashed before, but each time I start ipfs it will hang (the wifi AP disappear, ethernet links too) after a while (between 5 and 30 minutes).
If I stop IPFS the router come back up by itself after ~5 minutes.

If you want, I can do further analysis of that issue (even if I'm not sure where to start). Any advice is welcome !

[Kubuxu]

I also had issues with similar Technicolor modem, also from ISP. Similar symptoms but it isn't 100% reproducible.

[whyrusleeping]

@palkeo @Kubuxu help debugging this would be great. I'm not really sure where to start yet other than looking at what upnp requests cause it to fail. and maybe even trying to break the modems outside of ipfs? (using some sort of upnp tool by hand maybe)

[whyrusleeping]

you might have some luck using this to send upnp requests to your modem manually

[palkeo]

I have a wireshark capture of the moment when a cut happens.

There is an unusual burst of IPv6 communication from/to port 4001.
Then, a MDNS query (which is not so unusual, it's every 10 seconds, from what I have seen) :
MDNS 83 Standard query 0x7aa0 PTR discovery.ipfs.io.local, "QM" question

And less than 3 seconds later the modem is dead. Interestingly, it is back after ~5 minutes, by itself.

I'll try to disable IPv6 and see if it changes something.

[ghost]

MDNS is only used for local discovery, I'd be surprised if that's an issue, but you can still try disabling it in the config: Discovery.MDNS.Enabled = false

[palkeo]

I have disabled IPv6 and still encountered the issue. I disabled MDNS like you said, and my router seems to be no longer crashing. Strange…
I will try to send a MDNS packet every second and see if that makes it crash more often.

[palkeo]

It is indeed the case, when I do one MDNS query per second, my router is slower and crash after a few minutes. I don't really know what to do now.

[whyrusleeping]

hrm... i'm guessing your router has buggy support for mdns then... We should start a list for modem/router issues like this

[ghost]

@palkeo in the meantime you could disable MDNS, or give OpenWrt a try: https://wiki.openwrt.org/toh/start

The wiki doesn't list it as supported yet though.

[epremuz]

@palkeo Can't thank you enough!

I can confirm another Technicolor router has this issue. My model is DPC3848VE. Disabling MDNS seems to do the trick.

EDIT: Alas, I spoke too soon. It certainly improved the time it took before crashing but that's all.
EDIT2: I bought a separate router and configured my cablemodem in bridged mode (no routing). IPFS still makes my cablemodem crash.