Adding and enabling third party modules.

[srivathsan-srinivasan]

Hi everyone.

Goal :

I am trying to include a thirdparty module a Http_more_headers to provide a custom name to the server in the headers.

I am able to build a custom controller image with the module installed with a slightly tweaked DockerFile.

Issue:

To enable this third party module I need to add the load_module directive in the nginx.conf I am however confused on how the ingress controller interprets the nginx.conf file. If I add the load_module in the server-snippet annotation, will it work? or do I have to modify the .tmpl file to enable the third party module? or should I just modify the nginx.conf and use a COPY during the image build itself? which would be the best way to achieve the goal?

[Rulox]

Hi @chronograph3r

The load_module directive can be only used in the main block. Because of that, server-snippets won't work for you. You need to use main-snippets (more info about snippets here).

The easiest way for you would be to add the following Configmap key:
main-snippets: load_module modules/ngx_http_opentracing_module.so;

Let me know if it works and/or you have any other question.

[srivathsan-srinivasan]

Hi @Rulox Thanks for the information. It works as you have mentioned and I was also able to have the nginx modules loaded by modifying the nginx.tmpl file.

Could you help me out in setting up a configuration for a custom 404/500 html that is present on the /usr/share/nginx/html inside the container? Like you have mentioned, should I use Server-snippet to add this configuration?

nginx.org/server-snippet: 
error_page 404 /404.html;
        location = /404.html {
                root /usr/share/nginx/html;
                internal;
        }

        error_page 500 502 503 504 /custom_50x.html;
        location = /custom_50x.html {
                root /usr/share/nginx/html;
                internal;
}

 

or is there any other way or a best practice that you would suggest to do?
Your help is much appreciated. Thanks.

[Rulox]

Hey @chronograph3r glad you did this.

About your approach, yeah server-snippets is a good solution for your problem (this is better than modifying the nginx.tmpl file because you don't have to modify the file again when you update the Ingress Controller version). Additionally, and if you are using our custom NGINX Ingress Resources (VirtualServer and VirtualServerRoutes), you can also use the errorPages feature.

By the way, if you want to modify the template anyways, you can do it using a ConfigMap key as well (so you can change it without having to re-build the Ingress Controller image), check this. This way you don't need to change the tmpl file.

Let me know if this makes sense, thanks!

[srivathsan-srinivasan]

Hey @Rulox That makes sense and thank you for notifying me of the useful features.
Here's an interesting thing, I was able to get to the custom error page when I try to hit the loadbalancer IP, i.e, I was able to get the /usr/share/nginx/html/custom404.html with the configuration I modified with nginx.tmpl. But I was not able to achieve the custom error page via the ingress. even after adding the same in the server-snippets of ingress or on the config map. Is there a specific syntax that I have to provide on the template section?

I wonder this behavior is because of how the grpc error codes mentioned in the nginx.ingress.tmpl redirects to a plain and simple default nginx 404 page.

I am yet to try out the errorPages feature with VirtualServer. I will let you know how that goes as well.

[Rulox]

Hi @chronograph3r

Can you show me your Ingress Resources/Config map keys and the changes you did in the template? That would help me to understand what's happening in your config and I'll be able to assist you better. Thanks!

PS: I don't think grpc error pages is the issue here.

[srivathsan-srinivasan]

Thank you so much @Rulox for helping me understand the usage of snippets and template, Yes you're correct, grpc wasn't the issue. i was having a duplicate location block. I was able to achieve a custom 404 error page via the ingress as well. I just had to add the proxy_intercept_errors on; for the ingress resources. and the error_page directive worked.

I was also able to add custom cors configuration as well with the help of @pleshakov on this comment

I was able to modify the desired configurations with this

By the way, if you want to modify the template anyways, you can do it using a ConfigMap key as well (so you can change it without having to re-build the Ingress Controller image), check this. This way you don't need to change the tmpl file.

This guided me in the right path to enabling the third party modules compiled on a custom image.

The load_module directive can be only used in the main block. Because of that, server-snippets won't work for you. You need to use main-snippets (more info about snippets here).

The easiest way for you would be to add the following Configmap key:
main-snippets: load_module modules/ngx_http_opentracing_module.so;

[Rulox]

That's great news @chronograph3r

Closing this issue then, as it seems you're happy with the result.

[Kishore-kalastri]

@chronograph3r Can you please share Dockerfile to create nginx-ingress image with more_header module. I need to create nginx-ingress with all modules.

[Kishore-kalastri]

@chronograph3r can you please share Dockerfile to create nginx-ingress image with more_header module. I need to create nginx-ingress with all modules.

[srivathsan-srinivasan]

@Kishore-kalastri

You can find the Dockerfile here, I wrote an article about it. I don't know if it is relevant now, please let me know if it's useful

Creating a Custom Nginx INgress controller