Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rails server spams log due to getsockopt(2) TCP_INFO ENOPROTOPT #1982

Closed
ArsalanDotMe opened this issue Apr 22, 2017 · 34 comments
Closed

Rails server spams log due to getsockopt(2) TCP_INFO ENOPROTOPT #1982

ArsalanDotMe opened this issue Apr 22, 2017 · 34 comments

Comments

@ArsalanDotMe
Copy link

ArsalanDotMe commented Apr 22, 2017

Microsoft Windows [Version 10.0.15063]

Clean installed rails 5.0.2 on Ruby 2.4.0 through rebenv 1.1.0-2-g4f8925a.
Went into a directory owned by Windows. Created a new project and started the server with rails s. The command was extremely slow to start and once it showed the following output, I made a simple GET request to localhost:3000.

/home/arsalan/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/railties-5.0.2/lib/rails/app_loader.rb:40: warning: Insecure world writable dir /home/arsalan/.rbenv/versions in PATH, mode 040777
=> Booting Puma
=> Rails 5.0.2 application starting in development on http://0.0.0.0:3000
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.8.1 (ruby 2.4.0-p0), codename: Sassy Salamander
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://0.0.0.0:3000
Use Ctrl-C to stop

On making the GET request, the following error showed up.

2017-04-22 11:40:02 +0500: Read error: #<Errno::ENOPROTOOPT: Protocol not available - getsockopt(2)>
/home/arsalan/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/puma-3.8.1/lib/puma/server.rb:124:in `getsockopt'
/home/arsalan/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/puma-3.8.1/lib/puma/server.rb:124:in `closed_socket?'
/home/arsalan/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/puma-3.8.1/lib/puma/server.rb:563:in `handle_request'
/home/arsalan/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/puma-3.8.1/lib/puma/server.rb:425:in `process_client'
/home/arsalan/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/puma-3.8.1/lib/puma/server.rb:289:in `block in run'
/home/arsalan/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/puma-3.8.1/lib/puma/thread_pool.rb:120:in `block in spawn_thread'

I don't know if strace of this command will be useful as it keeps continuously spitting out lines at the speed of light. Never exits.

The relevant error-throwing line from puma seems to be

socket.getsockopt(Socket::SOL_TCP, Socket::TCP_INFO)
@ArsalanDotMe
Copy link
Author

It looks like the TCP_INFO option on the linux getsockopt(2) is not supported at this point.

Please see: TCP Options

TCP_INFO (since Linux 2.4)
Used to collect information about this socket. The kernel
returns a struct tcp_info as defined in the file
/usr/include/linux/tcp.h. This option should not be used in
code intended to be portable.

@sunilmut
Copy link
Member

@ArsalanDotMe - Thanks for reporting the issue and the detailed information. Very helpful. Yes, we don't need the whole trace. You have already provided the relevant pieces. I can confirm that WSL currently does not implement the TCP_INFO socket option.

I have opened a task at our end to track this. Thanks again for trying WSL and providing feedback.

@guenard
Copy link

guenard commented Jun 6, 2017

Just wanted to add that I got a similar log from PHP-FPM 5.6. ~ one line per second is written in /var/log/php5.6-fpm.log (TCP_INFO for socket: Protocol not available). Same as #2090.

@adamluzsi
Copy link

+1

1 similar comment
@laughloudlylovelife
Copy link

+1

@sunilmut
Copy link
Member

Can someone share clear (and if possible targeted) repro steps for this? I am looking into it right now and need that for validation.

@ArsalanDotMe
Copy link
Author

I'm happy to provide more detailed reproduction steps, but looking at the first message in the thread, what do you think is missing in detail to reproduce the error? I tried to provide exact version numbers and steps for all software involved.

@sunilmut
Copy link
Member

@ArsalanDotMe - Thanks. Yes, the first thread does provide the exact versions, but I was hoping to get exact instructions since I am not familiar with rails. That way, I can copy paste the instructions for the repro and focus on fixing the issue.

@ArsalanDotMe
Copy link
Author

The guide I followed is this one: https://gorails.com/setup/windows/10

You don't need to setup mysql or postgres, otherwise the guide is the same.

@adamluzsi
Copy link

Hy @sunilmut ,

Here is a example repo where all you need is to install and run (or if you have both ruby and bundler all you need is to start the app).
I tried to create an install script but I couldn't test it because I'm not in wsl/ubuntu now.

I met with this too and the problem is about a yet not implemented socket protocol.
I can't confirm that the bug stills on, but if you clone the repo and run it and try call it with some http requests, than you can reproduce the problem.

@Marthyn
Copy link

Marthyn commented Jul 26, 2017

👍
Ruby 2.2.4
Rails 4.2.7.1
Running bundle exec unicorn -p 5001

Results in
Protocol not available- setsockopt(2) (Errno::ENOPROTOOPT)

@bitcrazed
Copy link
Contributor

@ArsalanDotMe Could you try repro'ing on a recent Insiders build? I am running build 17364, and cannot repro your scenario - all looks good to me!

I installed rvm using DigitalOcean's instructions, and then installed rails, and created a new rails templated site using rails new ./rubytest. I then ran rails s to build & serve the site, and navigated to its URL:

image

Please let us know if your issue persists.

@therealkenc
Copy link
Collaborator

I think this got picked up with #393 #68. AFAIK, Rails has worked for a while (there would be more noise if it didn't). Best @sunilmut weigh in on the status of TCP_INFO. Caveat being, stuff is being worked around upstream, and it is hard to tell whether the surface was fixed or just dealt with. There is no mention of TCP_INFO for some time. Nothing in the release notes (natch).

@bitcrazed
Copy link
Contributor

Yep - difficult to know if this is upstream or WSL now supporting TCP_INFO - what say you @sunilmut? 😁

@Yay295
Copy link

Yay295 commented May 3, 2018

This is still not supported in Redstone 4. A quick test with Python

import socket
socket.socket().getsockopt(socket.SOL_TCP, socket.TCP_INFO)

shows

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 92] Protocol not available

@brucek2
Copy link

brucek2 commented Jun 18, 2018

Since #2090 was duped to here and closed, I'll note that for me php7.1-fpm is still generating a once per second log entry:
ERROR: failed to retrieve TCP_INFO for socket: Protocol not available (92)

FPM is otherwise working but it'd be nice if there was a fix for this nuisance. Anyone have a good workaround?

@therealkenc
Copy link
Collaborator

therealkenc commented Jun 18, 2018

Thanks @brucek2 and @Yay295 for confirming. So TCP_INFO is just not implemented. The work-around would be to recompile php without config HAVE_LQ_TCP_INFO. Or for that matter, just comment out the zlog() line. It isn't fatal, but I can imagine the over-aggressive logging once per second is pretty problematic.

@sunilmut - You don't from any practical standpoint have to wait on the NT tcp stack people to implement this. The feature wasn't forthcoming from them in April of 2017, and it probably isn't forthcoming in 2018 either. Just let getsockopt() succeed and fill in the struct tcp_info with zeros. As long as .tcpi_sacked is zero then php will just move on (and, mercifully, does so without spamming the log).

There is a decent backgrounder on TCP_INFO here.

@therealkenc therealkenc changed the title Rails server doesn't boot up due to getsockopt(2) error Rails server spams log due to getsockopt(2) TCP_INFO ENOPROTOPT Jun 18, 2018
@jbrooksuk
Copy link

I'm seeing:

ERROR: failed to retrieve TCP_INFO for socket: Protocol not available (92)

Every second (although not since changing log_level = alert) but nginx does not respond with anything, it's just a blank screen - although this may be fault of valet-wsl?

@SamSaffron
Copy link

The ruby unicorn web server is definitely not working right per:

https://bogomips.org/unicorn-public/CAAtdryOtTO8HGTeKLy_JbeRhWLC7JZpCABVpbpEK+z67JHx=ew@mail.gmail.com/T/#u

@tara-raj
Copy link

We’re investigating approaches to address this issue and will report back when we have something concrete to share. Will update this thread at that time with details!

@SamSaffron
Copy link

Awesome @tara-raj , if you need any help testing I am happy to flick whatever switches I need to run bleeding edge wsl.

@haacked
Copy link

haacked commented Sep 20, 2018

I'm happy to test as well. Ran into this with nginx and a ruby sinatra app.

@maatinito
Copy link

maatinito commented Oct 20, 2018

So to resume:
Unicorn doesn't work on WSL and there are two problems:

  • getsockopt(socket.SOL_TCP, socket.TCP_INFO) ==> ENOPROTOPT
  • getsockopt(TCP_CORK/TCP_NOPUSH) ==> ENOPROTOPT

Waiting for more support, as Unicorn is just a way to laucnh multiple rails server for performance, for the dev, I removed the dependency on Unicorn in file Gemfile, uninstall unicorn using sudo gem uninstall unicorn and everything works fine :-)

WSL is very promising, very exciting so good luck guys and thanks :-)

@lustremedia
Copy link

So to resume:
Unicorn doesn't work on WSL and there are two problems:

* getsockopt(socket.SOL_TCP, socket.TCP_INFO) ==> ENOPROTOPT

* getsockopt(TCP_CORK/TCP_NOPUSH) ==> ENOPROTOPT

Waiting for more support, as Unicorn is just a way to laucnh multiple rails server for performance, for the dev, I removed the dependency on Unicorn in file Gemfile, uninstall unicorn using sudo gem uninstall unicorn and everything works fine :-)

WSL is very promising, very exciting so good luck guys and thanks :-)

👍
Ruby 2.2.4
Rails 4.2.7.1
Running bundle exec unicorn -p 5001

Results in
Protocol not available- setsockopt(2) (Errno::ENOPROTOOPT)

Unicorn is still not working for me, ruby 2.4.0 rails 4.2.8

@jobs-git
Copy link

jobs-git commented Dec 30, 2019

Which windows version has this been solved?

I got a way to skip the error for PHP working for 1803. Use unix socket instead of tcp https://www.nginx.com/resources/wiki/start/topics/examples/phpfcgi/ & fastcgi_buffering off

However, a proper fix to tcp should be done since other applications relies on TCP_INFO.

@Morgy93
Copy link

Morgy93 commented Feb 17, 2020

Any update on this one?
Seems to be the same case for symfony:

$ symfony serve

Feb 17 18:44:17 |DEBUG| PHP    Reloading PHP versions 
Feb 17 18:44:18 |DEBUG| PHP    Using PHP version 7.4.2 (from default version in $PATH) 
Feb 17 18:44:18 |INFO | PHP    listening path="/usr/sbin/php-fpm7.4" php="7.4.2" port=51199
Feb 17 18:44:18 |DEBUG| PHP    started 
Feb 17 18:44:19 |INFO | PHP    'user' directive is ignored when FPM is not running as root 
Feb 17 18:44:19 |INFO | PHP    'group' directive is ignored when FPM is not running as root 
Feb 17 18:44:19 |ERROR| PHP    failed to retrieve TCP_INFO for socket: Protocol not available (92) 
Feb 17 18:44:19 |INFO | PHP    fpm is running, pid 26731 
Feb 17 18:44:19 |INFO | PHP    ready to handle connections 
Feb 17 18:44:19 |INFO | PHP    systemd monitor interval set to 10000ms 
                                                                                                                        
 [OK] Web server listening on http://127.0.0.1:8001 (PHP FPM 7.4.2)                                                     
                                                                                                                        

WARNING unable to find the application log
Feb 17 18:44:20 |ERROR| PHP    failed to retrieve TCP_INFO for socket: Protocol not available (92) 
Feb 17 18:44:21 |ERROR| PHP    failed to retrieve TCP_INFO for socket: Protocol not available (92) 
Feb 17 18:44:22 |ERROR| PHP    failed to retrieve TCP_INFO for socket: Protocol not available (92) 
Feb 17 18:44:23 |ERROR| PHP    failed to retrieve TCP_INFO for socket: Protocol not available (92) 
Feb 17 18:44:24 |ERROR| PHP    failed to retrieve TCP_INFO for socket: Protocol not available (92)
... and so on ... every second ...

@SamSaffron
Copy link

strongly recommend just closing this issue as "is fixed with WSL2, will not be fixed with WSL1"

No point carrying it around forever.

@Yay295
Copy link

Yay295 commented Feb 18, 2020

It would be nice if it were fixed in WSL1 though.

@sarim
Copy link

sarim commented Jun 5, 2020

strongly recommend just closing this issue as "is fixed with WSL2, will not be fixed with WSL1"

No point carrying it around forever.

@SamSaffron why? Did Microsoft announced WSL1 as abandoned, deprecated product already? As far as I remember when announcing WSL2, they said both wsl1 and wsl2 would be maintained. I'd like to see msft continuing to make wsl1 as complete as possible, which means implementing TCP_INFO somewhere in the future.

@Morgy93
Copy link

Morgy93 commented Jun 6, 2020

strongly recommend just closing this issue as "is fixed with WSL2, will not be fixed with WSL1"
No point carrying it around forever.

@SamSaffron why? Did Microsoft announced WSL1 as abandoned, deprecated product already? As far as I remember when announcing WSL2, they said both wsl1 and wsl2 would be maintained. I'd like to see msft continuing to make wsl1 as complete as possible, which means implementing TCP_INFO somewhere in the future.

You're right. In fact there are even official exceptions for using WSL 1 rather than WSL 2

But to answer the question:

What will happen to WSL 1? Will it be abandoned?
We currently have no plans to deprecate WSL 1. You can run WSL 1 and WSL 2 distros side by side, and can upgrade and downgrade any distro at any time. Adding WSL 2 as a new architecture presents a better platform for the WSL team to deliver features that make WSL an amazing way to run a Linux environment in Windows.
Source: https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq#what-will-happen-to-wsl-1-will-it-be-abandoned

@sarim
Copy link

sarim commented Jun 6, 2020

well if existing valid issue about wsl1 keep getting closed with fixed-in-ws2 tag, I don't see much hope for msft fixing wsl1 issues. Sad.

@jobs-git
Copy link

My hardware has driver problem with recent windows 10 releases so I am using wsl1. I hope they get it fixed there.

@privatehosting
Copy link

privatehosting commented Aug 24, 2020

@jobs-git Install Update KB4566116 for Windows 10 1909 and set wsl to version 2. Then you can continue your work.

@abdulrehman3222
Copy link

I was having a similar issue with

WSL: 1, Ruby: 2.5.0, Rails: 4.2.1, and Puma: 3.8.0

my rails s was working I was getting the following error whenever I send a Get request:

    Read error: #<Errno::ENOPROTOOPT: Protocol not available - getsockopt(2)>
    /home/abr/.rvm/gems/ruby-2.5.0@helpy/gems/puma-3.8.1/lib/puma/server.rb:124:in `getsockopt'
    /home/abr/.rvm/gems/ruby-2.5.0@helpy/gems/puma-3.8.1/lib/puma/server.rb:124:in `closed_socket?'
    /home/abr/.rvm/gems/ruby-2.5.0@helpy/gems/puma-3.8.1/lib/puma/server.rb:563:in `handle_request'
    /home/abr/.rvm/gems/ruby-2.5.0@helpy/gems/puma-3.8.1/lib/puma/server.rb:425:in `process_client'
    /home/abr/.rvm/gems/ruby-2.5.0@helpy/gems/puma-3.8.1/lib/puma/server.rb:289:in `block in run'
    /home/abr/.rvm/gems/ruby-2.5.0@helpy/gems/puma-3.8.1/lib/puma/thread_pool.rb:120:in `block in spawn_thread'

I struggle a lot for fixing that issue even I try the above solution but that didn't work either.

at last, I found a simple solution that I need to update my Pum from 3.8 to 3.12+ and it works perfectly for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests