HTTP proxy mode for egress-router #13586

danwinship · 2017-03-30T13:20:45Z

This adds an http/https proxy mode to egress-router. If you don't specify an EGRESS_DESTINATION, then instead of acting as an iptables-based reflector, it will run squid and act as an HTTP proxy (to any destination).

I wanted to use something like tinyproxy, but that's not currently shipped as part of RHEL. It looked like squid and apache were the only options; if anyone knows of a smaller http proxy option that is in RHEL, let me know. (haproxy is not an option; it only does reverse proxying.)

It doesn't have any configurability at the moment, but some could be added. (Eg, limiting destination addresses?) We'd want to do that by having generic environment variables in the pod spec, not by letting the user provide their own squid.conf, because we don't want to be tied to squid necessarily.

Also, maybe this should be explicitly configured somehow (EGRESS_ROUTER_MODE=http-proxy?) rather than being implicit if you don't specify EGRESS_DESTINATION. Or maybe it should be a separate image entirely?

@openshift/networking PTAL
Trello Card: https://trello.com/c/9RHw27K7

danwinship

FYI:

danwinship · 2017-03-30T13:21:31Z

images/router/egress/squid.conf

+# This is a mostly-transparent HTTP/HTTPS proxy; it only exists to
+# redirect requests, not to authenticate them in any way.
+
+http_port 8080


8080 seemed like the most "generic HTTP proxy" port to me? I didn't want to use 3128 because that's explicitly squid's port

8080 is fine, but 3128 wouldn't be bad too. FoxyProxy and the Android proxy config both default to 3128 for the port.

danwinship · 2017-03-30T13:21:44Z

images/router/egress/squid.conf

+# redirect requests, not to authenticate them in any way.
+
+http_port 8080
+http_access allow all


"Allow all HTTP requests from all sources"

danwinship · 2017-03-30T13:22:04Z

images/router/egress/squid.conf

+
+http_port 8080
+http_access allow all
+cache deny all


"Don't cache any requests"

danwinship · 2017-03-30T13:25:08Z

images/router/egress/squid.conf

+http_access allow all
+cache deny all
+access_log none all
+debug_options ALL,0


"Don't log every request/response"
"Print only critical errors (level 0) in all log domains"

The container ends up logging

WARNING: Cannot write log file: /var/log/squid/cache.log /var/log/squid/cache.log: No such file or directory messages will be sent to 'stderr'.

twice at startup. (It shows up in oc logs for the pod.) I couldn't find any way to get it to just log to stderr without complaining about it...

It's not the end of the world. But you could try /proc/self/fd/2 if the message really annoyed you. I'd probably just live with that error though.

Yeah, that doesn't work either. (Same error)

knobunc

LGTM, thanks Dan!

danwinship · 2017-03-30T14:10:18Z

(pushed an update with the addition of shutdown_lifetime 0 to cause squid to exit immediately when you delete the pod, rather than lingering 30 seconds)

danwinship · 2017-03-30T16:04:26Z

we should implement this as two separate containers, with only the "setup" container being privileged

stevekuznetsov · 2017-03-31T14:33:18Z

images/router/egress/egress-router.sh

+    # bash to react to SIGTERM if we explicitly set a trap for it, except
+    # that bash doesn't process signal traps while it is waiting for a
+    # process to finish, and we have to be waiting for a process to finish
+    # because there's no way to sleep forever within bash.


I have a slight feeling that this leaves the tail around as a zombie ... why not

trap "exit" TERM while true; do sleep 60 done

It doesn't leave a zombie (note that this is existing code, it's just getting re-indented). But yeah, a loop-and-sleep would be more obvious anyway...

I'm also thinking about making the setup be done in an initContainer, and then run origin-pod as the "main" container. (Or is that "cheating" because in the future origin-pod might do something other than just sit and wait for a SIGTERM?)

Using an init container doesn't sound like a bad idea to me but I'm not sure I can be authoritative on that

imcsk8 · 2017-04-03T18:38:34Z

You could touch /var/log/squid/cache.log in the Dockerfile in order to avoid the warning message

danwinship · 2017-04-03T19:05:15Z

You could touch /var/log/squid/cache.log in the Dockerfile in order to avoid the warning message

No, then it would actually log to that file; I want it to log to stderr, so that any errors will show up in "oc logs"

danwinship · 2017-04-26T16:50:25Z

Rebased on top of #13837. Now it uses two containers (a privileged setup container and an unprivileged squid container), is much more configurable, and has a unit test.

If you wanted to test it:

    apiVersion: v1
    kind: Pod
    metadata:
      name: egress-http
      labels:
        name: egress-http
      annotations:
        pod.network.openshift.io/assign-macvlan: "true"
        pod.beta.kubernetes.io/init-containers: '[
            {
                "name": "egress-init-container",
                "image": "openshift/origin-egress-router",
                "imagePullPolicy": "IfNotPresent",
                "env": [
                    { "name": "EGRESS_SOURCE", "value": "172.17.0.99" },
                    { "name": "EGRESS_GATEWAY", "value": "172.17.0.1" },
                    { "name": "EGRESS_ROUTER_MODE", "value": "http-proxy" }
                ],
                "securityContext": { "privileged": true }
            }
        ]'
    spec:
      containers:
      - name: egress-proxy
        image: openshift/origin-egress-http-proxy
        imagePullPolicy: IfNotPresent
        env:
        - name: EGRESS_HTTP_PROXY_DESTINATION
          value: |
            !*.example.com
            *

(that means "allow to anywhere except example.com and its subdomains". See the unit test for more syntax examples. I'll submit a docs bug once this is finalized. Note that once the kube 1.6 rebase completes you'll be able to specify the init container "normally" rather than as an annotation.)

danwinship · 2017-05-08T15:47:54Z

Just to clarify, this is reviewable, it's just do-not-merge because it's built on top of #13837 which isn't merged yet. (Only the last commit is new in this PR.)

dcbw · 2017-05-27T15:25:47Z

LGTM

knobunc · 2017-05-30T19:42:28Z

[merge]

knobunc · 2017-05-31T14:11:11Z

This is not a test failure... it's me forgetting the merge state. I'll re-merge this on Monday.

openshift-bot · 2017-06-05T14:05:27Z

[Test]ing while waiting on the merge queue

Update the base egress-router image and add a new egress-http-proxy image that runs squid to proxy HTTP connections to some or all destinations.

danwinship · 2017-06-05T19:48:23Z

flake #14229, [test]

openshift-bot · 2017-06-05T20:05:38Z

Evaluated for origin test up to 351b252

openshift-bot · 2017-06-05T21:47:47Z

continuous-integration/openshift-jenkins/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin/1943/) (Base Commit: 09f08bd)

openshift-bot · 2017-06-09T11:13:45Z

Evaluated for origin merge up to 351b252

openshift-bot · 2017-06-09T11:13:54Z

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_origin/936/) (Base Commit: 2458531) (Image: devenv-rhel7_6333)

danwinship commented Mar 30, 2017

View reviewed changes

knobunc approved these changes Mar 30, 2017

View reviewed changes

danwinship force-pushed the egress-router-proxy branch from 64db4c4 to 438d3e6 Compare March 30, 2017 14:09

danwinship changed the title ~~HTTP proxy mode for egress-router~~ [DO NOT MERGE] HTTP proxy mode for egress-router Mar 30, 2017

stevekuznetsov reviewed Mar 31, 2017

View reviewed changes

danwinship mentioned this pull request Apr 20, 2017

Allow multiple destinations in egress-router #13837

Merged

danwinship force-pushed the egress-router-proxy branch from 438d3e6 to a6e3db6 Compare April 26, 2017 16:46

danwinship force-pushed the egress-router-proxy branch from a6e3db6 to fda905b Compare April 26, 2017 18:16

danwinship mentioned this pull request Apr 27, 2017

Document new egress-router features openshift/openshift-docs#4277

Merged

danwinship force-pushed the egress-router-proxy branch from fda905b to bf0cb21 Compare May 17, 2017 12:12

danwinship changed the title ~~[DO NOT MERGE] HTTP proxy mode for egress-router~~ HTTP proxy mode for egress-router May 17, 2017

openshift-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 27, 2017

knobunc added the component/routing label May 31, 2017

danwinship force-pushed the egress-router-proxy branch from bf0cb21 to e7c465c Compare June 5, 2017 14:02

openshift-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 5, 2017

Add an HTTP proxy mode to egress router

351b252

Update the base egress-router image and add a new egress-http-proxy image that runs squid to proxy HTTP connections to some or all destinations.

danwinship force-pushed the egress-router-proxy branch from e7c465c to 351b252 Compare June 5, 2017 16:38

openshift-bot merged commit cc2ed8f into openshift:master Jun 9, 2017

danwinship deleted the egress-router-proxy branch June 10, 2017 14:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HTTP proxy mode for egress-router #13586

HTTP proxy mode for egress-router #13586

danwinship commented Mar 30, 2017 •

edited by knobunc

Loading

danwinship left a comment

danwinship Mar 30, 2017

knobunc Mar 30, 2017

danwinship Mar 30, 2017

danwinship Mar 30, 2017

danwinship Mar 30, 2017

knobunc Mar 30, 2017

danwinship Mar 30, 2017

knobunc left a comment

danwinship commented Mar 30, 2017

danwinship commented Mar 30, 2017

stevekuznetsov Mar 31, 2017

danwinship Mar 31, 2017

stevekuznetsov Apr 3, 2017

imcsk8 commented Apr 3, 2017

danwinship commented Apr 3, 2017

danwinship commented Apr 26, 2017 •

edited by stevekuznetsov

Loading

danwinship commented May 8, 2017

dcbw commented May 27, 2017

knobunc commented May 30, 2017

knobunc commented May 31, 2017

openshift-bot commented Jun 5, 2017

danwinship commented Jun 5, 2017

openshift-bot commented Jun 5, 2017

openshift-bot commented Jun 5, 2017

openshift-bot commented Jun 9, 2017

openshift-bot commented Jun 9, 2017 •

edited

Loading

HTTP proxy mode for egress-router #13586

HTTP proxy mode for egress-router #13586

Conversation

danwinship commented Mar 30, 2017 • edited by knobunc Loading

danwinship left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

knobunc left a comment

Choose a reason for hiding this comment

danwinship commented Mar 30, 2017

danwinship commented Mar 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

imcsk8 commented Apr 3, 2017

danwinship commented Apr 3, 2017

danwinship commented Apr 26, 2017 • edited by stevekuznetsov Loading

danwinship commented May 8, 2017

dcbw commented May 27, 2017

knobunc commented May 30, 2017

knobunc commented May 31, 2017

openshift-bot commented Jun 5, 2017

danwinship commented Jun 5, 2017

openshift-bot commented Jun 5, 2017

openshift-bot commented Jun 5, 2017

openshift-bot commented Jun 9, 2017

openshift-bot commented Jun 9, 2017 • edited Loading

danwinship commented Mar 30, 2017 •

edited by knobunc

Loading

danwinship commented Apr 26, 2017 •

edited by stevekuznetsov

Loading

openshift-bot commented Jun 9, 2017 •

edited

Loading