Lua module for nginx/openresty to stream zip archives assembled on the fly.
The only dependendencies of autozip are either pure lua modules themselves and
small enough to be included in the project, the lfs
(lua-filesystem
) module
which seems to be widely available in package managers around the globe, and
the modules provided by the target environment, namely the ngx
module
provided by openresty and possibly in the future dependencies on the modules
included in LuaJIT as it is the interpreter used by openresty.
There is an additional optional dependency on zlib, which is installed on most
base linux distributions, to dramatically improve performances of the CRC-32
computations compared to the pure lua implementation. The zlib library is loaded
through LuaJIT's ffi
module, which makes ffi
only an optional dependency for
now.
Autozip can be used when mod_zip is not available out of the box or via your operating system's package manager (and you don't want to recompile nginx yourself), but nginx' lua module is.
It can be used to complement your autoindex (ideally with a custom stylesheet) when you want to provide visitors the convenience of both a directory to explore and a zip of said directory to download, but don't want to waste twice the space for it.
Another reason to use autozip is when you could use mod_zip but you don't have an upstream server telling it which files to serve, or don't want to set up one to serve local files (ideally mod_zip should be able to figure out for itself and with minimal configuration which files can be zipped locally). Or if you think that any upstream sending the right magic header triggering mod_zip is a bit too much, and you would prefer an opt-in mechanism instead.
However if you still want to use mod_zip (and there are reasons you might want
to, the least of them not being performance), autozip has a function to output
the list of files required by mod_zip: check out autozip.mod_zip_filelist
.
It is painfully slow without the support of the native zlib library, and still somewhat slow with it. And it uses a lot of CPU. Please open tickets or contribute if you know how to improve the situation.
It is not well tested. It is not super complex, but a few tests here and there wouldn't hurt. Dear reader, you are welcome to help with that.
It is not tested on a variety of distributions or operating systems, and might break on yours. Feedbacks are appreciated.
It doesn't have the zip feature that you need. The selected zip feature set is pretty much on point for the target usage: zipping on the fly potentially large directories and files, no need for compression as most file formats would handle that better, minimal external dependencies. Unless there is a clear benefit at a relatively low cost, it probably won't land in this project.
There are probably many other reasons. If you feel like it, you can add some to this list.
Copy the lua files somewhere on nginx' lua package path.
If you copy them somewhere else, say under /home/nginx/lua/
, you must adapt
the package path using the following directive in the http
context of your
site or nginx' configuration:
lua_package_path '/home/nginx/lua/?.lua;;';
You can then require and use autozip
to fit your situation.
If you have an interesting use-case that does not follow the examples below, please share!
Let's assume that an autoindex is enabled on /public/
and that we want to zip
directories on the fly by appending '.zip' to the directory name in the URL:
requesting /public/banana/
would list the content of the 'banana' directory,
while requesting /public/banana.zip
would download 'banana.zip' generated on
the fly with the content of /public/banana/
.
To achieve this, you can use the following location
configuration:
location /public/ {
location ~ \.zip$ {
try_files $uri $uri/ @try_zip;
}
# some configuration
...
}
This will intercept requests for a zip file under /public/
matching on the
extension, and if no actual file has the requested name, nor directory (which
can also have their name end with '.zip'), the fallback named location
@try_zip
will be used:
location @try_zip {
content_by_lua_block {
local autozip = require 'autozip'
autozip.try_zip('=404')
}
}
Much like try_files
, you provide a fallback location to autozip.try_zip
(here a redirection to an HTTP 404 error page, handled by nginx' error handler).
Autozip will try to figure out whether it can serve the directory
/public/banana/
as a zip, and if not redirect to the fallback location.
Let's assume that an autoindex is enabled on /public/
and that we want to zip
directories on the fly when the query parameter download
is present.
Matching query parameters is a bit more tricky, as nginx does not let you directly match locations on them.
One way to handle this would be to configure a custom error handler and trigger it when the query parameters match:
location / {
error_page 418 @try_zip;
recursive_error_pages on;
if ($args ~ (^|&)download(&|$)) {
return 418;
}
# some configuration
...
}
The @try_zip
named location (see previous example) would generate the zip.
Note that with the current autozip.lua
code this scenario would require a
slight adaptation to make it work, as the primary focus was the "Virtual files"
use-case. Future versions might improve the situation, and so can you by
contributing!