Assertion in state_read_server_response_header (v7.1.0) #1629

scw00 · 2017-04-03T13:53:32Z

In my opinion, the EPOLLERR always with read.enabled or write.enabled(otherwise, there is no sm could handle this event, we just ignore). So when EPOLLERR happens, we could push the vc into read_ready_list if the read.enabled == 1, otherwise push it into write_ready_list.

zwoop · 2017-04-03T15:13:09Z

[approve ci]

atsci · 2017-04-03T15:14:37Z

Linux build failed! https://ci.trafficserver.apache.org/job/linux-github/1712/

atsci · 2017-04-03T15:15:39Z

clang format failed! https://ci.trafficserver.apache.org/job/clang-format-github/125/

atsci · 2017-04-03T15:15:59Z

RAT check failed! https://ci.trafficserver.apache.org/job/RAT-github/139/

atsci · 2017-04-03T15:16:18Z

FreeBSD11 build failed! https://ci.trafficserver.apache.org/job/freebsd-github/1821/

atsci · 2017-04-03T15:16:35Z

clang-analyzer build failed! https://ci.trafficserver.apache.org/job/clang-analyzer-github/383/

atsci · 2017-04-03T15:16:42Z

Intel CC build failed! https://ci.trafficserver.apache.org/job/icc-github/251/

atsci · 2017-04-03T15:18:37Z

AU check failed! https://ci.trafficserver.apache.org/job/autest-github/122/

oknet · 2017-04-05T04:27:26Z

@scw00 Can you have a try with only these 2 lines change?

diff --git a/iocore/net/UnixNet.cc b/iocore/net/UnixNet.cc
index c7f1798..2f98269 100644
--- a/iocore/net/UnixNet.cc
+++ b/iocore/net/UnixNet.cc
@@ -450,7 +450,7 @@ NetHandler::mainNetEvent(int event, Event *e)
       if (cop_list.in(vc)) {
         cop_list.remove(vc);
       }
-      if (get_ev_events(pd, x) & EVENTIO_READ) {
+      if (get_ev_events(pd, x) & (EVENTIO_READ | EVENTIO_ERROR)) {
         vc->read.triggered = 1;
         if (get_ev_events(pd, x) & EVENTIO_ERROR) {
           vc->read.error = 1;
@@ -466,7 +466,7 @@ NetHandler::mainNetEvent(int event, Event *e)
         }
       }
       vc = epd->data.vc;
-      if (get_ev_events(pd, x) & EVENTIO_WRITE) {
+      if (get_ev_events(pd, x) & (EVENTIO_WRITE | EVENTIO_ERROR)) {
         vc->write.triggered = 1;
         if (get_ev_events(pd, x) & EVENTIO_ERROR) {
           vc->write.error = 1;

To resolve the issue we just revert these 2 lines to the state #947 .

oknet · 2017-04-05T05:49:03Z

iocore/net/UnixNet.cc

-                get_ev_events(pd, x), vc->write.enabled, vc->closed, write_ready_list.in(vc));
+          if (!write_ready_list.in(vc)) {
+            write_ready_list.enqueue(vc);
+          }


You just ignore the error if enabled==0. This is equal to revert #947 .
I think this is not a good fix to the issue.

scw00 · 2017-04-05T05:53:43Z

Based on @oknet , Here is a new logic. We just handle write error when read error was handled, or there is only a write error.

oknet · 2017-04-05T09:56:39Z

[approve ci]

oknet · 2017-04-05T10:03:19Z

Oops ... I don't have permission to execute CI. @zwoop Can I have "approve ci" ? :-D

oknet · 2017-04-08T05:28:55Z

@zwoop Can you have a try on docs ?

zwoop · 2017-04-12T17:41:53Z

[approve ci]

atsci · 2017-04-12T17:45:33Z

AU check failed! https://ci.trafficserver.apache.org/job/autest-github/165/

atsci · 2017-04-12T17:45:46Z

Linux build failed! https://ci.trafficserver.apache.org/job/linux-github/1755/

atsci · 2017-04-12T17:46:19Z

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/184/

atsci · 2017-04-12T17:46:26Z

clang format successful! https://ci.trafficserver.apache.org/job/clang-format-github/169/

atsci · 2017-04-12T17:47:36Z

Intel CC build failed! https://ci.trafficserver.apache.org/job/icc-github/295/

atsci · 2017-04-12T17:47:39Z

FreeBSD11 build failed! https://ci.trafficserver.apache.org/job/freebsd-github/1865/

atsci · 2017-04-12T17:52:05Z

clang-analyzer build successful! https://ci.trafficserver.apache.org/job/clang-analyzer-github/428/

zwoop · 2017-04-12T17:52:16Z

@oknet I've added you as admin of all the builds.

zwoop · 2017-04-12T17:52:59Z

Looks like there's a build failure here:

UnixNetVConnection.cc: In function 'void write_to_net_io(NetHandler*, UnixNetVConnection*, EThread*)':
UnixNetVConnection.cc:472:39: error: 'struct NetState' has no member named '_cont'
       if (!vc->read.error || vc->read._cont == nullptr) {
                                       ^

oknet · 2017-04-13T03:45:40Z

[approve ci]

atsci · 2017-04-13T03:51:14Z

clang format successful! https://ci.trafficserver.apache.org/job/clang-format-github/175/

atsci · 2017-04-13T03:51:17Z

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/190/

atsci · 2017-04-13T03:55:41Z

AU check successful! https://ci.trafficserver.apache.org/job/autest-github/171/

atsci · 2017-04-13T03:56:30Z

Intel CC build successful! https://ci.trafficserver.apache.org/job/icc-github/301/

atsci · 2017-04-13T04:01:49Z

Linux build successful! https://ci.trafficserver.apache.org/job/linux-github/1761/

atsci · 2017-04-13T04:01:52Z

FreeBSD11 build successful! https://ci.trafficserver.apache.org/job/freebsd-github/1871/

atsci · 2017-04-13T04:11:32Z

clang-analyzer build successful! https://ci.trafficserver.apache.org/job/clang-analyzer-github/434/

oknet · 2017-04-13T06:23:58Z

@zwoop Please test this PR on docs. thanks. @scw00 have tested it by a python script.

oknet · 2017-06-05T04:20:45Z

[approve ci] @scw00 Can you write a autest script ?

bryancall · 2017-06-08T18:24:42Z

This looks like it is related to #947

zwoop · 2017-06-08T18:47:14Z

Since we're reverting #947, I'm closing this (please reopen if that is not right).

This reverts PRs apache#1559, apache#1522 and apache#947 PR apache#947 made the HTTP state machine unstable and lead to crashes in production like apache#1930 apache#1559 apache#1522 apache#1531 apache#1629 This reverts commit c1ac5f8.

This reverts PRs #1559, #1522 and #947 PR #947 made the HTTP state machine unstable and lead to crashes in production like #1930 #1559 #1522 #1531 #1629 This reverts commit c1ac5f8.

scw00 mentioned this pull request Apr 3, 2017

Assertion in state_read_server_response_header (v7.1.0) #1531

Closed

zwoop added the Network label Apr 3, 2017

zwoop added this to the 7.2.0 milestone Apr 3, 2017

zwoop requested a review from oknet April 3, 2017 15:13

zwoop assigned scw00 Apr 3, 2017

oknet requested changes Apr 5, 2017

View reviewed changes

oknet approved these changes Apr 5, 2017

View reviewed changes

Assertion in state_read_server_response_header (v7.1.0)

811017d

oknet approved these changes Apr 13, 2017

View reviewed changes

zwoop modified the milestones: 7.2.0, 8.0.0 Apr 25, 2017

scw00 mentioned this pull request Jun 1, 2017

WRITE_COMPLETE event in http server open causes traffic server to abort #1930

Closed

vmamidi mentioned this pull request Jun 8, 2017

Revert PR #947 #2093

Merged

zwoop closed this Jun 8, 2017

vmamidi mentioned this pull request Jun 8, 2017

Revert PR #947 - merge to 7.1.x #2103

Merged

zwoop pushed a commit that referenced this pull request Jun 8, 2017

Revert PR #947 - merge to 7.1.x

65a001b

This reverts PRs #1559, #1522 and #947 PR #947 made the HTTP state machine unstable and lead to crashes in production like #1930 #1559 #1522 #1531 #1629 This reverts commit c1ac5f8.

zwoop modified the milestone: 8.0.0 Jul 7, 2017

Assertion in state_read_server_response_header (v7.1.0) #1629

Assertion in state_read_server_response_header (v7.1.0) #1629

Uh oh!

Conversation

scw00 commented Apr 3, 2017

Uh oh!

zwoop commented Apr 3, 2017

Uh oh!

atsci commented Apr 3, 2017

Uh oh!

atsci commented Apr 3, 2017

Uh oh!

atsci commented Apr 3, 2017

Uh oh!

atsci commented Apr 3, 2017

Uh oh!

atsci commented Apr 3, 2017

Uh oh!

atsci commented Apr 3, 2017

Uh oh!

atsci commented Apr 3, 2017

Uh oh!

oknet commented Apr 5, 2017

Uh oh!

oknet Apr 5, 2017

Choose a reason for hiding this comment

Uh oh!

scw00 commented Apr 5, 2017

Uh oh!

oknet commented Apr 5, 2017

Uh oh!

oknet commented Apr 5, 2017

Uh oh!

oknet commented Apr 8, 2017

Uh oh!

zwoop commented Apr 12, 2017

Uh oh!

atsci commented Apr 12, 2017

Uh oh!

atsci commented Apr 12, 2017

Uh oh!

atsci commented Apr 12, 2017

Uh oh!

atsci commented Apr 12, 2017

Uh oh!

atsci commented Apr 12, 2017

Uh oh!

atsci commented Apr 12, 2017

Uh oh!

atsci commented Apr 12, 2017

Uh oh!

zwoop commented Apr 12, 2017

Uh oh!

zwoop commented Apr 12, 2017

Uh oh!

oknet commented Apr 13, 2017

Uh oh!

atsci commented Apr 13, 2017

Uh oh!

atsci commented Apr 13, 2017

Uh oh!

atsci commented Apr 13, 2017

Uh oh!

atsci commented Apr 13, 2017

Uh oh!

atsci commented Apr 13, 2017

Uh oh!

atsci commented Apr 13, 2017

Uh oh!

atsci commented Apr 13, 2017

Uh oh!

oknet commented Apr 13, 2017

Uh oh!

oknet commented Jun 5, 2017

Uh oh!

bryancall commented Jun 8, 2017

Uh oh!

zwoop commented Jun 8, 2017

Uh oh!

Reviewers