upload module 2.2.0 fails to build against nginx 1.3.9

[chrislea]

When compiling nginx 1.3.9 and using upload module 2.2.0 as an addon module, the compilation fails (at least for me on Linux using gcc). The error is as follows:

gcc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g   -I src/core -I src/event -I src/event/modules -I src/os/unix -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I src/mail \
        -o objs/addon/nginx_upload_module-2.2.0/ngx_http_upload_module.o \
        ../nginx_upload_module-2.2.0/ngx_http_upload_module.c
../nginx_upload_module-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_read_upload_client_request_body’:
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2628: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2687: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_do_read_upload_client_request_body’:
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2769: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2785: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2877: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
make[1]: *** [objs/addon/nginx_upload_module-2.2.0/ngx_http_upload_module.o] Error 1

Thanks for your work on this software!

[7Two1]

To clarify, this module is ONLY supported up to nginx versions 1.3.8 (dev). up to release-1.2.6 branch is unaffected.

and this applies to upload module version 2.2 (https://github.com/vkholodkov/nginx-upload-module/tree/2.2)

Background....
nginx revision 4931 (released in v1.3.9) introduces "Request body: chunked transfer encoding support."
from nginx changelog: "Feature: support for chunked transfer encoding while reading client request body."

Unfortunately, in ngx_http_request.h -> the to_write pointer was removed from the ngx_http_request_body_t structure.

Therefore, someone needs to update this code in order to meet up with the new structure and enhancements.

[perestoronin]

When planned will be corrected module upload for the nginx 1.3.9 ?

[davromaniak]

Hello.

I'm also confronted to this issue when testing the upgrade to nginx 1.3.9

As maintainer of the Debian packages for nginx, we might consider dropping you module from the packages if it stays unbuildable for long time.

In parallel, I will try to propose a patch for this issue.

Thanks.

[perestoronin]

In parallel, I will try to propose a patch for this issue.

It will be nice.

[davromaniak]

Apparently, correcting this issue won't be simple as clapping hands.

With the 1.3.9 release, the way data are transfered is changed, this lead to changes in the structure ngx_http_request_body_t, used in this module.

So this would need to perform some big changes in the modules.

I will try to do this, but it will require heavy testing, as I'm not a C developer, and I'm still learning it.

Thanks.

[davromaniak]

Hello again.

I just finished patching this module. For the moment, the only thing I was able to test is the building... and it works.

Here is the patch : http://paste.davromaniak.eu/index.php?show=110

I need some people to heavily test it as I'm not an expert in C programming, and it's more a kludge than a clean fix.

Thanks.

[perestoronin]

Thanks. Currently I switched with your help my test server on nginx 1.3.9 from 1.3.8.

[benjaminbarbe]

@davromaniak Thanks for the effort but I have an issue with this patch : Nginx: 413 Request Entity Too Large Error
client_max_body_size is correctly configured, the same config works on old version of nginx.

@vkholodkov Do you have any plan to update your code?

Thanks

[davromaniak]

@benjaminbarbe Thanks for testing, actually, I think my action didn't help advancing on this subject.

@vkholodkov Valery, are you still maintaining this module ?

Thanks

[benjaminbarbe]

@davromaniak https://github.com/agentzh/lua-resty-upload

[pgaertig]

@benjaminbarbe thanks for the link, lua-resty-upload looks promising as shows power of Lua in Nginx, but for now it is just an upload stream framework and misses many functions like resumable uploads, crc32, passing control to backend or error handling (correct me if I lost something). You have to implement that logic yourself in Lua. I am sticking with stable ngx 1.2.x now but I see only two ways for future: fork upload-module or make something with that lua stuff (looks easier) - opensource world guys!

[benjaminbarbe]

@pgaertig You are right, but look at this example https://gist.github.com/benjaminbarbe/5206051
It's a good starting point. You want to pass control to backend? Easily made with ngx.location.capture("/api/persist.json", { method = ngx.HTTP_POST, body = json });

Scripting Nginx with Lua, useful links: https://twitter.com/benjaminbarbe/status/311827277493182464
Thanks

[mikhailov]

as I understand nginx-upload-module doesn't work properly with Nginx 1.3.9+?
@vkholodkov any plan to support such great module?

[mikhailov]

Nginx has built-in functionality to provide basic async non-blocking upload without upstream:

    location /upload {
      limit_except POST              { deny all; } 

      client_body_temp_path          /tmp/;
      client_body_in_file_only       on;
      client_body_buffer_size        128K;
      client_max_body_size           50M;
      proxy_pass_request_headers     on;
      proxy_set_body                 $request_body_file;
      proxy_pass                     http://upstream; # will receive file_name only
      proxy_redirect                 off;
    }

[pgaertig]

And after a few evenings here is my attempt: https://github.com/pgaertig/nginx-big-upload . BTW thanks go to @benjaminbarbe for convincing me that Lua is easy.

[pgaertig]

@mikhailov you ask for reason to continue support of external modules, and in the same time you say nginx-upload-module is great. Why is it great then if you have client_body_in_file_only?

My production services process 1GB+ file uploads everyday, resumable is must have in the mobile and WiFi dominated world. For some clients there are non-resumable Flash-based uploads peaking 1GB in one multi-part request. Your solution looks very good but the problem is that multi-part body has to be extracted in 2nd pass by the upstream, which means lag for bigger files.

It looks like to me that resumability can be implemented easily using client_body_in_file_only and back-end logic assembling chunks. However the problem is that each chunk is saved as separate file which then has to be merged into target file. Depending on chunk size that operation may introduce some response lag. On the other side specialized upload modules append to an existing file and calculate CRC32 and other stuff progressively.

Anyway client_body_in_file_only looks interesting to me I am going to experiment with that as well.

[vkholodkov]

Hi! I feel that I need to comment on this issue.

For historical reasons upload module is based on nginx interfaces that are not quite documented. To date there is no incentive from Nginx Inc. to standardize them. Supporting every little change in the core from my side means diving into support hell that doesn't guarantee permanent compatibility either. Doing this means investing a lot of time an effort with questionable outcome.

I feel that I did a great job creating all these features and it's time for me to go and find new challenges. I'm pretty sure you will be easily able to fix all compatibility issues.

[benjaminbarbe]

@pgaertig You're welcome, thanks to you for nginx-big-upload :-)

[Dexus]

@vkholodkov I dont think the same. Your code is used by so many admins and somany peoples, i cant understand why you brake out to finde new challanges.

It was a horror to read it here... now i cant update NGINX because my website will die if i update... thanks for that. :(

[vkholodkov]

@Dexus why don't you take over the maintenance then? It's not difficult and many people will be grateful to you, including yourself!

I'll be always ready to consult you if you need my help...

[Dexus]

@vkholodkov my skills are not so good that i can make it. :(

But it looks like i have to learn more about c programming.

Am 14.06.2013 um 17:02 schrieb Valery Kholodkov notifications@github.com:

@Dexus why don't you take over the maintenance then? It's not difficult and many people will be grateful to you, including yourself!

I'll be always ready to consult you if you need my help...

—
Reply to this email directly or view it on GitHub.

[vkholodkov]

Good. You'll get a challenge as well. As I said, I'll always be available for questions.

[adamchal]

This is sad. I'm on the fence about using LUA or another approach for handling uploads. This module was really good and reliable at handling resumable uploads, solid incremental checksums, etc. I'd be interested in sponsoring someone to port this module to work with the nginx development branch (1.5). This upload module saved me many times, especially when having to rely on less-than-ideal app servers (PHP) for uploads.

[pgaertig]

@mikhailov - Source? To be more precise, Lua module doesn't break any internals of SPDY module. They both work right in separate requests (production tested), however I had some issues when they are both used in one request. SPDY support is new stuff natively introduced with Nginx 1.4.0 so we have teething time. Production SPDY is still rare outside big G. However if somebody really needs to use both then the workaround is to keep them separated in different server blocks thus domains.

[pgaertig]

@mikhailov some of them old, outdated or without evidence. One is even mine, but I assure you currently the Lua module is usable in production if you don't use SPDY. That is OT really.

[vkholodkov]

I'm afraid you need to complain to Nginx Inc in order to get cooperation on this issue.

[ltpao]

Here is the patch : http://paste.davromaniak.eu/index.php?show=110
line 601:

•            rb->to_write = rb->bufs->next ? rb->bufs->next : rb->bufs;
  

•            rb->bufs = rb->bufs->next ? rb->bufs->next : rb->bufs; 
  

----- Original Message -----
From: "Kieran Pilkington" notifications@github.com
To: "vkholodkov/nginx-upload-module" nginx-upload-module@noreply.github.com
Cc: "ltpao" paul0@139.com
Sent: Wednesday, September 04, 2013 6:40 AM
Subject: Re: [nginx-upload-module] upload module 2.2.0 fails to build against nginx 1.3.9 (#41)

Any progress with this? I understand the original author doesn't want to maintain it anymore, is someone else able to fork it and make the changes required to run on Nginx 1.4.2+?

---

Reply to this email directly or view it on GitHub:
https://github.com/vkholodkov/nginx-upload-module/issues/41#issuecomment-23753012

[adamchal]

@ltpao, can you post a complete patch with your bugfix applied? I'm not following your what you're saying.

[adamchal]

Nevermind, I've created a Gist for ngx_http_upload_module.c that has the patch and bugfix applied. I've tested with Nginx v1.5.1 and it seems to be working properly. I'll keep testing and identify any problems on this thread. Much thanks to @davromaniak for creating this patch, @ltpao for finding the bug, and of course @vkholodkov for creating this add-on.

[chrislea]

I've made Ubuntu packages for supported releases from Lucid and going newer
that have this module included with the new patch applied. I can't attest
to anything other than that it compiles, but if anybody wants to try it
out, install the "nginx-extras" package to get a binary with the upload
module built in: http://bit.ly/e6De6N

On Thu, Sep 5, 2013 at 3:28 PM, Adam Chalemian notifications@github.comwrote:

Nevermind, I've created a Gist for ngx_http_upload_module.chttps://gist.github.com/adamchal/6457039that has the patch and bugfix applied. I've tested with Nginx v1.5.1 and it
seems to be working properly. I'll keep testing and identify any problems
on this thread. Much thanks to @davromaniakhttps://github.com/davromaniakfor creating this patch,
@ltpao https://github.com/ltpao for finding the bug, and of course
@vkholodkov https://github.com/vkholodkov for creating this add-on.

—
Reply to this email directly or view it on GitHubhttps://github.com/vkholodkov/nginx-upload-module/issues/41#issuecomment-23906545
.

__
https://chrislea.com
http://about.me/chrislea
310-709-4021

[chrislea]

Sorry, I should add that my packages are builds of 1.5.4 which is the current newest release as of this writing. http://bit.ly/e6De6N

[pservit]

@adamchal, thank you for you gist

In rare cases nginx terminated with signal 11

nginx/1.4.2 on FreeBSD 9.2-RC3 amd64

here is backtrace:

Core was generated by `nginx'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000000000 in ?? ()

(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00000000004f578d in ngx_http_do_read_client_request_body (r=0x4410b050) at /usr/ports/www/nginx/work/nginx_upload_module-2.2.0/ngx_http_upload_module.c:3601
#2  0x00000000004f45af in ngx_http_read_client_request_body_handler (r=0x4410b050) at /usr/ports/www/nginx/work/nginx_upload_module-2.2.0/ngx_http_upload_module.c:3618
#3  0x000000000046bc1b in ngx_http_request_handler (ev=0x47801960) at src/http/ngx_http_request.c:2133
#4  0x0000000000437770 in ngx_event_process_posted (cycle=0x43cfd050, posted=0x739c68 <ngx_posted_events>) at src/event/ngx_event_posted.c:40
#5  0x0000000000435ecf in ngx_process_events_and_timers (cycle=0x43cfd050) at src/event/ngx_event.c:276
#6  0x0000000000445a31 in ngx_worker_process_cycle (cycle=0x43cfd050, data=0x1) at src/os/unix/ngx_process_cycle.c:807
#7  0x000000000044150e in ngx_spawn_process (cycle=0x43cfd050, proc=0x445840 <ngx_worker_process_cycle>, data=0x1, name=0x50755e "worker process", respawn=1)
    at src/os/unix/ngx_process.c:198
#8  0x00000000004439ab in ngx_reap_children (cycle=0x43cfd050) at src/os/unix/ngx_process_cycle.c:619
#9  0x0000000000442d97 in ngx_master_process_cycle (cycle=0x43cfd050) at src/os/unix/ngx_process_cycle.c:180
#10 0x0000000000407f8f in main (argc=1, argv=0x7fffffffda80) at src/core/nginx.c:412

[pservit]

full backtrace:

(gdb) bt full
#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00000000004f578d in ngx_http_do_read_client_request_body (r=0x4410b050) at /usr/ports/www/nginx/work/nginx_upload_module-2.2.0/ngx_http_upload_module.c:3601
        rest = 290
        size = 290
        n = 290
        rc = 0
        b = 0x7fffffffda80
        cl = 0x43cfd068
        out = {buf = 0x43c70100, next = 0x0}
        c = 0x45002d50
        rb = 0x4410cab0
        clcf = 0x0
#2  0x00000000004f45af in ngx_http_read_client_request_body_handler (r=0x4410b050) at /usr/ports/www/nginx/work/nginx_upload_module-2.2.0/ngx_http_upload_module.c:3618
        rc = 4294967295
#3  0x000000000046bc1b in ngx_http_request_handler (ev=0x47801960) at src/http/ngx_http_request.c:2133
        c = 0x45002d50
        r = 0x4410b050
        ctx = 0x43c700e8
#4  0x0000000000437770 in ngx_event_process_posted (cycle=0x43cfd050, posted=0x739c68 <ngx_posted_events>) at src/event/ngx_event_posted.c:40
        ev = 0x47801960
#5  0x0000000000435ecf in ngx_process_events_and_timers (cycle=0x43cfd050) at src/event/ngx_event.c:276
        flags = 3
        timer = 29456
        delta = 1
#6  0x0000000000445a31 in ngx_worker_process_cycle (cycle=0x43cfd050, data=0x1) at src/os/unix/ngx_process_cycle.c:807
        worker = 1
        i = 4631248425700960387
        c = 0x738df8 <cached_http_log_iso8601+312>
#7  0x000000000044150e in ngx_spawn_process (cycle=0x43cfd050, proc=0x445840 <ngx_worker_process_cycle>, data=0x1, name=0x50755e "worker process", respawn=1)
    at src/os/unix/ngx_process.c:198
        on = 1
        pid = 0
        s = 1
#8  0x00000000004439ab in ngx_reap_children (cycle=0x43cfd050) at src/os/unix/ngx_process_cycle.c:619
        i = 1
        n = 2
---Type <return> to continue, or q <return> to quit---
        live = 1
        ch = {command = 2, pid = 57268, slot = 1, fd = -1}
        ccf = 0x7dd00000009
#9  0x0000000000442d97 in ngx_master_process_cycle (cycle=0x43cfd050) at src/os/unix/ngx_process_cycle.c:180
        title = 0x443a018c "master process /usr/local/sbin/nginx"
        p = 0x443a01b0 ""
        size = 37
        i = 1
        n = 0
        sigio = 0
        set = {__bits = {0, 0, 0, 0}}
        itv = {it_interval = {tv_sec = 0, tv_usec = 0}, it_value = {tv_sec = 0, tv_usec = 6}}
        live = 1
        delay = 0
        ls = 0x0
        ccf = 0x43cfe070
#10 0x0000000000407f8f in main (argc=1, argv=0x7fffffffda80) at src/core/nginx.c:412
        i = 59
        log = 0x737308 <ngx_log>
        cycle = 0x43cfd050
        init_cycle = {conf_ctx = 0x0, pool = 0x43c07800, log = 0x737308 <ngx_log>, new_log = {log_level = 0, file = 0x0, connection = 0, handler = 0x0, data = 0x0,
            action = 0x0}, files = 0x0, free_connections = 0x0, free_connection_n = 0, reusable_connections_queue = {prev = 0x0, next = 0x0}, listening = {elts = 0x0, nelts = 0,
            size = 0, nalloc = 0, pool = 0x0}, paths = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = {elts = 0x0, nelts = 0,
              next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0},
          connection_n = 0, files_n = 0, connections = 0x0, read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 31,
            data = 0x503da9 "/usr/local/etc/nginx/nginx.conf"}, conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 21, data = 0x503da9 "/usr/local/etc/nginx/nginx.conf"},
          prefix = {len = 21, data = 0x503d93 "/usr/local/etc/nginx/"}, lock_file = {len = 0, data = 0x0}, hostname = {len = 0, data = 0x0}}
        ccf = 0x43cfe070

[robink]

@chrislea > your package seems to not include the upload module :(

[chrislea]

I'm using it right now so I'm pretty sure it does. Did you install the
nginx-extras package?

On Thu, Nov 7, 2013 at 3:43 PM, Robin Komiwes notifications@github.comwrote:

@chrislea https://github.com/chrislea > your package seems to not
include the upload module :(

—
Reply to this email directly or view it on GitHubhttps://github.com/vkholodkov/nginx-upload-module/issues/41#issuecomment-28018564
.

__
https://chrislea.com
http://about.me/chrislea
310-709-4021

[robink]

Yep, here is what I'm getting: https://gist.github.com/robink/016c0a9033a09a5212da

[chrislea]

You're using Ubuntu Oneiric, which is no longer supported so the newer
builds that have the upload module included aren't available. If you switch
to a supported distro it'll update to Nginx 1.5.6 (currently) which has it
included.

On Thu, Nov 7, 2013 at 4:13 PM, Robin Komiwes notifications@github.comwrote:

Yep, here is what I'm getting:
https://gist.github.com/robink/016c0a9033a09a5212da

—
Reply to this email directly or view it on GitHubhttps://github.com/vkholodkov/nginx-upload-module/issues/41#issuecomment-28020183
.

__
https://chrislea.com
http://about.me/chrislea
310-709-4021

[wtn]

The author of nginx-upload-module added updates to the 2.2 branch on 2013-10-03.

[goodforever]

@mikhailov my system is nginx 1.4.3 +passenger ,I set config according you said:
location /upload {
limit_except POST { deny all; }

  client_body_temp_path          /tmp/;
  client_body_in_file_only       on;
  client_body_buffer_size        128K;
  client_max_body_size           50M;
  proxy_pass_request_headers     on;
  proxy_set_body                 $request_body_file;
  proxy_pass                     http://upstream; # will receive file_name only
  proxy_redirect                 off;
}

I still cannot upload file, Could you help me ?

[peterpeng001]

I used the code on branch 2.2 and succeed in the installation for Nginx 1.4.4. Uploading file also works well.

[untidy-hair]

Branch 2.2 was successfully installed (and is used) with Nginx 1.5.10. I believe there were great efforts.

[tchatzi]

I had issues with the gist in https://gist.github.com/adamchal/6457039 on nginx-1.6.2 due to .state files being incorrectly created. I have modified the code in https://gist.github.com/tchatzi/02dfdf41fafee878cdd5 and it seems to have resolved the issue I was having.
Only lightly tested, so YMMV

[7Two1]

@tchatzi

Given that the original module author (vkholodkov) fixed outstanding issues (apparently) on branch 2.2, then doesn't his work supersede the gist by adam and therefore your fix should be applied to this 2.2 (vkholodkov) branch and not to adamchal as you have done? Doing a code review, your gist is currently missing changes that affect the variable "u" included in branch 2.2.

Thanks in advance.

[tchatzi]

vkholodkov's master branch still doesn't build against nginx-1.6.2, while mine (basically adamchal's) does and works properly (barring SPDY issues that every module that plays with downstream connections directly seems to have, see openresty/lua-nginx-module#252 and that are a different set of patches anyhow)

[vkholodkov]

Yeah, would be great to get the patch against 2.2 and not master.

[tchatzi]

You're right about version 2.2.0. Barring the known SPDY issues, it seems to work just as well (no thorough code-reading and only light testing done on my part).
It's quite smaller than adamchal's version and that can't possibly be bad though.
Still had an issue with the .state files, available at https://gist.github.com/tchatzi/a5920cce439dfed03944

[tchatzi]

sorry, my bad, I should have explained in more detail:
(1) Yes, https://gist.github.com/tchatzi/a5920cce439dfed03944 is based on 2.2.0
It is a rather simple patch, to avoid issues like this one that I naively thought everyone was facing:

2014/10/15 11:55:09 [error] 18036#0: *1 failed to create or open state file "/opt/nginx/proxy_temp/e/at/4546948080.state" (2: No such file or directory), client: 192.168.100.120, server: nginx-dev2.arx.net, request: "POST /uploadsmobile HTTP/1.1", host: "nginx-dev2.arx.net:8443", referrer: "https://nginx-dev2.arx.net:8443/"

or worse in adamchal's gist

2014/10/15 12:04:10 [error] 20770#0: *1 failed to create or open state file "/opt/nginx/proxy_temp/^@/te/3072265362.state^@" (21: Is a directory), client: 192.168.100.120, server: nginx-dev2.arx.net, request: "POST /uploadsmobile HTTP/1.1", host: "nginx-dev2.arx.net:8443", referrer: "https://nginx-dev2.arx.net:8443/"

I couldn't decipher what kind of directories the hashing wanted to pre-exist (in the first case, the second is clearly wrong use of ngx_memcpy()), so I settled for the state file being named <full_path_to_upload_file>.state, which might be a little simplistic, but it guarantees that the directory exists and nginx can write inside it. Besides, I think that's the way it worked before 1.3.x that broke the original upload-module.

(2) So no, after my simple 3 line patch the .state file problem does not manifest anymore.
Again here it is, for completeness (against 2.2.0):

--- ngx_http_upload_module.c.orig   2014-10-14 21:35:34.024008009 +0300
+++ ngx_http_upload_module.c    2014-10-15 12:14:15.436008728 +0300
@@ -1203,7 +1203,6 @@

     ngx_file_t  *file = &u->output_file;
     ngx_path_t  *path = u->store_path;
-    ngx_path_t  *state_path = u->state_store_path;
     uint32_t    n;
     ngx_uint_t  i;
     ngx_int_t   rc;
@@ -1251,17 +1250,14 @@
                         return NGX_UPLOAD_NOMEM;
                 }

-                state_file->name.len = state_path->name.len + 1 + state_path->len + u->session_id.len + sizeof(".state")-1;
+                state_file->name.len = file->name.len + 1 + sizeof(".state");
                 state_file->name.data = ngx_palloc(u->request->pool, state_file->name.len + 1);

                 if(state_file->name.data == NULL)
                     return NGX_UPLOAD_NOMEM;

-                ngx_memcpy(state_file->name.data, state_path->name.data, state_path->name.len);
-                (void) ngx_sprintf(state_file->name.data + state_path->name.len + 1 + state_path->len,
-                        "%V.state%Z", &u->session_id);
-
-                ngx_create_hashed_filename(state_path, state_file->name.data, state_file->name.len);
+                ngx_memcpy(state_file->name.data, file->name.data, file->name.len);
+                ngx_memcpy(state_file->name.data + file->name.len, ".state", sizeof(".state") + 1);

                 ngx_log_debug1(NGX_LOG_DEBUG_CORE, file->log, 0,
                                "hashed path of state file: %s", state_file->name.data);

(3) In order to replicate the issue, one need only do a chunked http upload, something that requires a valid .state file

(4) I did, as I said before, there seems to be a general issue with nginx SPDY and every module that plays with downstream connections directly like openresty/lua-nginx-module#252 quoted above.
The issue writes stuff like this in the error log:

2014/10/09 19:53:40 [debug] 23688#0: *1 event timer add: 3: 60000:1412873680218
2014/10/09 19:53:40 [debug] 23688#0: *1 epoll add event: fd:3 op:1 ev:80002001
2014/10/09 19:53:40 [alert] 23688#0: *1 epoll_ctl(1, 3) failed (17: File exists), client: 192.168.100.120, server: nginx-dev2.arx.net, request: "POST /uploadsmobile HTTP/1.1", host: "nginx-dev2.arx.net:8443", referrer: "https://nginx-dev2.arx.net:8443/"
2014/10/09 19:53:40 [debug] 23688#0: *1 http finalize request: 500, "/uploadsmobile" a:1, c:2

and returns an nginx error 500

(5) I'd recommend we fix this, but I'm not (yet ?) knowledgeable enough of nginx internals to figure it out myself. If you can live without SPDY, upload-module works quite as expected. If not, I suppose you'll have to make do with client_body_in_file_only on; that works with SPDY (or so I heard, didn't have time to test it myself)

[7Two1]

@tchatzi thank you very much, this is far better for everyone :)

With any luck and if your code is correct, then @vkholodkov may merge.

I have reposted the questions that I asked and you answered as follows....

@tchatzi quote "Still had an issue with the .state files"

First, thanks for your replies.

I am trying to help all readers get to a place of complete understanding... therefore...

just to be clear, I have only ever asked about branch 2.2.0 and not master...

1) Is the gist link (a592 ... 3944) your previous fixes applied to the 2.2.0 branch code?
2) If 1 == yes, are you saying that even with your changes, that there is still an issue with .state files?
3) If 2 == yes, then can you describe how to replicate the issue
4) Did you test SPDY against 2.2.0?
5) If 4 == yes and SPDY issues are still valid, then would you recommend to disable SPDY?

[schue]

I built and ran the 2.2 version of the upload module against NGINX 1.9.4 without any problems. This issue should really be closed or at least reworded. I took it at face value initially and discarded upload-module as a possible solution when, in actuality, it seems to work.

[verges-io]

On a brand new Ubuntu 14.04 server I'm running exactly into this issue when building against nginx 1.9.5:

cc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g   -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules -I /u1/software/nginx/nginx-upload-module-master-2.2.0 -I src/mail -I src/stream \
        -o objs/addon/nginx-upload-module-master-2.2.0/ngx_http_upload_module.o \
        /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_upload_merge_loc_conf’:
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1104:31: error: ‘ngx_garbage_collector_temp_handler’ undeclared (first use in this function)
                               ngx_garbage_collector_temp_handler, cf);
                               ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1104:31: note: each undeclared identifier is reported only once for each function it appears in
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1104:31: error: passing argument 1 of ‘ngx_conf_merge_path_value’ from incompatible pointer type [-Werror]
In file included from src/core/ngx_core.h:66:0,
                 from /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:7:
src/core/ngx_file.h:151:7: note: expected ‘struct ngx_conf_t *’ but argument is of type ‘struct ngx_path_t *’
 char *ngx_conf_merge_path_value(ngx_conf_t *cf, ngx_path_t **path,
       ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1104:31: error: passing argument 2 of ‘ngx_conf_merge_path_value’ from incompatible pointer type [-Werror]
                               ngx_garbage_collector_temp_handler, cf);
                               ^
In file included from src/core/ngx_core.h:66:0,
                 from /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:7:
src/core/ngx_file.h:151:7: note: expected ‘struct ngx_path_t **’ but argument is of type ‘struct ngx_path_t *’
 char *ngx_conf_merge_path_value(ngx_conf_t *cf, ngx_path_t **path,
       ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1104:31: error: passing argument 3 of ‘ngx_conf_merge_path_value’ from incompatible pointer type [-Werror]
                               ngx_garbage_collector_temp_handler, cf);
                               ^
In file included from src/core/ngx_core.h:66:0,
                 from /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:7:
src/core/ngx_file.h:151:7: note: expected ‘struct ngx_path_t *’ but argument is of type ‘char *’
 char *ngx_conf_merge_path_value(ngx_conf_t *cf, ngx_path_t **path,
       ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1104:31: error: passing argument 4 of ‘ngx_conf_merge_path_value’ makes pointer from integer without a cast [-Werror]
                               ngx_garbage_collector_temp_handler, cf);
                               ^
In file included from src/core/ngx_core.h:66:0,
                 from /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:7:
src/core/ngx_file.h:151:7: note: expected ‘struct ngx_path_init_t *’ but argument is of type ‘int’
 char *ngx_conf_merge_path_value(ngx_conf_t *cf, ngx_path_t **path,
       ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1104:31: error: too many arguments to function ‘ngx_conf_merge_path_value’
                               ngx_garbage_collector_temp_handler, cf);
                               ^
In file included from src/core/ngx_core.h:66:0,
                 from /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:7:
src/core/ngx_file.h:151:7: note: declared here
 char *ngx_conf_merge_path_value(ngx_conf_t *cf, ngx_path_t **path,
       ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_upload_pass_form_field’:
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1490:5: error: passing argument 1 of ‘ngx_regex_compile’ from incompatible pointer type [-Werror]
     f->regex = ngx_regex_compile(&value[1], 0, cf->pool, &err);
     ^
In file included from src/core/ngx_core.h:71:0,
                 from /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:7:
src/core/ngx_regex.h:50:11: note: expected ‘struct ngx_regex_compile_t *’ but argument is of type ‘struct ngx_str_t *’
 ngx_int_t ngx_regex_compile(ngx_regex_compile_t *rc);
           ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1490:5: error: too many arguments to function ‘ngx_regex_compile’
     f->regex = ngx_regex_compile(&value[1], 0, cf->pool, &err);
     ^
In file included from src/core/ngx_core.h:71:0,
                 from /u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:7:
src/core/ngx_regex.h:50:11: note: declared here
 ngx_int_t ngx_regex_compile(ngx_regex_compile_t *rc);
           ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1490:14: error: assignment makes pointer from integer without a cast [-Werror]
     f->regex = ngx_regex_compile(&value[1], 0, cf->pool, &err);
              ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1497:5: error: implicit declaration of function ‘ngx_regex_capture_count’ [-Werror=implicit-function-declaration]
     n = ngx_regex_capture_count(f->regex);
     ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1501:28: error: ‘ngx_regex_capture_count_n’ undeclared (first use in this function)
                            ngx_regex_capture_count_n " failed for "
                            ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1501:54: error: expected ‘)’ before string constant
                            ngx_regex_capture_count_n " failed for "
                                                      ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_read_upload_client_request_body’:
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1922:7: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
     rb->to_write = rb->bufs;
       ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_do_read_upload_client_request_body’:
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1991:56: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
                     ngx_http_process_request_body(r, rb->to_write));
                                                        ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:1997:19: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
                 rb->to_write = rb->bufs->next ? rb->bufs->next : rb->bufs;
                   ^
/u1/software/nginx/nginx-upload-module-master-2.2.0/ngx_http_upload_module.c:2063:44: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
         ngx_http_process_request_body(r, rb->to_write));
                                            ^
cc1: all warnings being treated as errors

I'll try to patch this with the solution @tchatzi pointed out. But closing it seems to be a bit too early. :)

[wyzssw]

@mikhailov
i tried

location /upload {
      limit_except POST              { deny all; } 

      client_body_temp_path          /tmp/;
      client_body_in_file_only       on;
      client_body_buffer_size        128K;
      client_max_body_size           50M;
      proxy_pass_request_headers     on;
      proxy_set_body                 $request_body_file;
      proxy_pass                     http://upstream; # will receive file_name only
      proxy_redirect                 off;
    }

still not work,can you help me,and i have no back-end,how can i config

[mikhailov]

@goodforever @wyzssw I got few questions from you and have an answer here

[khavishbhundoo]

@fdintino You can close this issue