diff --git a/CHANGELOG.md b/CHANGELOG.md index ecc9afc4..f6ad9156 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,22 @@ +# v0.16.0 - 22 September 2020 + +## New Feature + +- (#152, #155) Add In-App WAF protection to Echo's request parameter parser: + [`Context`](https://pkg.go.dev/github.com/labstack/echo/v4#Context)'s method + `Bind()` is now protected by the In-App WAF. The Go value it parses from the + HTTP request is made available to the In-App WAF rules via the + `GET/POST parameters` field. + When blocked, `Bind()` returns a non-nil [`SqreenError` value](https://godoc.org/github.com/sqreen/go-agent/sdk/types#SqreenError) + and its caller should immediately return. + Read more about the blocking behavior of Sqreen for Go at . + +## Fix + +- (#153) RASP shellshock: properly handle environment variables containing + variable definitions (eg. `TERMCAP`). + + # v0.15.0 - 9 September 2020 ## New Feature diff --git a/README.md b/README.md index cac850b2..da8142df 100644 --- a/README.md +++ b/README.md @@ -4,11 +4,6 @@ # [Sqreen](https://www.sqreen.com/)'s Application Security Management for Go -[![Release](https://img.shields.io/github/release/sqreen/go-agent.svg)](https://github.com/sqreen/go-agent/releases) -[![GoDoc](https://godoc.org/github.com/sqreen/go-agent?status.svg)](https://godoc.org/github.com/sqreen/go-agent) -[![Go Report Card](https://goreportcard.com/badge/github.com/sqreen/go-agent)](https://goreportcard.com/report/github.com/sqreen/go-agent) -[![Build Status](https://dev.azure.com/sqreenci/Go%20Agent/_apis/build/status/sqreen.go-agent?branchName=master)](https://dev.azure.com/sqreenci/Go%20Agent/_build/latest?definitionId=8&branchName=master) - After performance monitoring (APM), error and log monitoring it’s time to add a security component into your app. Sqreen’s microagent automatically monitors sensitive app’s routines, blocks attacks and reports actionable infos to your @@ -34,41 +29,29 @@ For more details, visit [sqreen.com](https://www.sqreen.com/) # Quick start 1. Use the middleware function for the Go web framework you use: - - [sqhttp](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqhttp) for the standard `net/http` package. - - [Gin](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqgin) for `github.com/gin-gonic/gin`. - - [Echo](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqecho) for `github.com/labstack/echo`. + - [net/http](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqhttp) + - [Gin](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqgin) + - [Echo](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqecho/v4) If your framework is not listed, it is usually possible to use instead the standard `net/http` middleware. If not, please, let us know by [creating an issue](http://github.com/sqreen/go-agent/issues/new). -1. Without Go modules: Download the new dependencies - - `go get` will automatically download the new dependencies of the SDK, including - Sqreen's agent for Go: - - ```consol - $ go get -d -v ./... - ``` - 1. Compile your program with Sqreen Sqreen's dynamic configuration of your protection is made possible thanks to Go instrumentation. It is safely performed at compilation time by the following instrumentation tool. - Install the following instrumentation tool and compile your program using it in - order to enable Sqreen. + Install the following instrumentation tool and compile your program using it + in order to enable Sqreen. - 1. Use `go install` to compile the instrumentation tool: + 1. Use `go build` to download and compile the instrumentation tool: ```console - $ go install github.com/sqreen/go-agent/sdk/sqreen-instrumentation + $ go build github.com/sqreen/go-agent/sdk/sqreen-instrumentation ``` - By default, the resulting `sqreen-instrumentation` tool is installed in the - `bin` directory of the `GOPATH`. You can find it using `go env GOPATH`. - 1. Configure the Go toolchain to use it: Use the instrumentation tool using the go options @@ -79,15 +62,15 @@ For more details, visit [sqreen.com](https://www.sqreen.com/) For example, the following two commands are equivalent: ```console - $ go build -a -toolexec $(go env GOPATH)/bin/sqreen-instrumentation my-project - $ env GOFLAGS="-a -toolexec $(go env GOPATH)/bin/sqreen-instrumentation" go build my-project + $ go build -a -toolexec $PWD/sqreen-instrumentation-tool my-project + $ env GOFLAGS="-a -toolexec $PWD/sqreen-instrumentation-tool" go build my-project ``` 1. [Signup to Sqreen](https://my.sqreen.io/signup) to get a token for your app, and store it in the agent's configuration file `sqreen.yaml`: ```sh - app_name: Your Go service name + app_name: Your Go app name token: your token ``` @@ -96,13 +79,15 @@ For more details, visit [sqreen.com](https://www.sqreen.com/) path by defining the configuration file location into the environment variable `SQREEN_CONFIG_FILE`. -1. You are done! - Just recompile your Go program and the go toolchain will download the latests - agent version. +Congratulations, your Go web application is now protected by Sqreen! + +

+Sqreen for Go +

+ -1. Optionally, use the [SDK](https://godoc.org/github.com/sqreen/go-agent/sdk) - to perform [user monitoring](https://godoc.org/github.com/sqreen/go-agent/sdk#HTTPRequestRecord.ForUser) - (eg. signing-in) or [custom security events](https://godoc.org/github.com/sqreen/go-agent/sdk#HTTPRequestRecord.TrackEvent) - you would like to track (eg. password changes). +# Advanced integration -Find out more about the agent setup at https://docs.sqreen.com/go/installation/ +Optionally, use the SDK to perform [user monitoring](https://docs.sqreen.com/go/user-monitoring/) +or [custom security events](https://docs.sqreen.com/go/custom-events/) you would +like to track and possibly block. \ No newline at end of file diff --git a/go.mod b/go.mod index 5f9e6412..cc1f2a26 100644 --- a/go.mod +++ b/go.mod @@ -16,10 +16,8 @@ require ( github.com/json-iterator/go v1.1.9 // indirect github.com/kentik/patricia v0.0.0-20190405133149-20eb46c597b3 github.com/labstack/echo v3.3.10+incompatible - github.com/labstack/gommon v0.2.9 // indirect + github.com/labstack/echo/v4 v4.1.17 github.com/magiconair/properties v1.8.1 // indirect - github.com/mattn/go-colorable v0.1.4 // indirect - github.com/mattn/go-isatty v0.0.11 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.1 // indirect github.com/onsi/ginkgo v1.7.0 @@ -30,11 +28,10 @@ require ( github.com/spf13/viper v1.3.2 github.com/sqreen/go-libsqreen v0.7.1 github.com/sqreen/go-sdk/signal v1.1.0 + github.com/stretchr/objx v0.2.0 // indirect github.com/stretchr/testify v1.6.1 - golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 // indirect - golang.org/x/net v0.0.0-20200513185701-a91f0712d120 - golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect - golang.org/x/text v0.3.2 // indirect + golang.org/x/net v0.0.0-20200904194848-62affa334b73 + golang.org/x/sys v0.0.0-20200918174421-af09f7315aff // indirect golang.org/x/tools v0.0.0-20200117065230-39095c1d176c // indirect golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898 gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect diff --git a/go.sum b/go.sum index f2cfdd31..6c43183f 100644 --- a/go.sum +++ b/go.sum @@ -13,6 +13,7 @@ github.com/dave/rebecca v0.9.1/go.mod h1:N6XYdMD/OKw3lkF3ywh8Z6wPGuwNFDNtWYEMFWE github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= github.com/dlclark/regexp2 v1.2.0 h1:8sAhBGEM0dRWogWqWyQeIJnxjWO6oIjl8FKqREDsGfk= github.com/dlclark/regexp2 v1.2.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc= github.com/dop251/goja v0.0.0-20200526165454-f1752421c432 h1:EIY1hqp9O08saJ41t7aQy0o1hhq3ByOy61AACthST5M= @@ -54,18 +55,21 @@ github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/labstack/echo v3.3.10+incompatible h1:pGRcYk231ExFAyoAjAfD85kQzRJCRI8bbnE7CX5OEgg= github.com/labstack/echo v3.3.10+incompatible/go.mod h1:0INS7j/VjnFxD4E2wkz67b8cVwCLbBmJyDaka6Cmk1s= -github.com/labstack/gommon v0.2.9 h1:heVeuAYtevIQVYkGj6A41dtfT91LrvFG220lavpWhrU= -github.com/labstack/gommon v0.2.9/go.mod h1:E8ZTmW9vw5az5/ZyHWCp0Lw4OH2ecsaBP1C/NKavGG4= +github.com/labstack/echo/v4 v4.1.17 h1:PQIBaRplyRy3OjwILGkPg89JRtH2x5bssi59G2EL3fo= +github.com/labstack/echo/v4 v4.1.17/go.mod h1:Tn2yRQL/UclUalpb5rPdXDevbkJ+lp/2svdyFBg6CHQ= +github.com/labstack/gommon v0.3.0 h1:JEeO0bvc78PKdyHxloTKiF8BD5iGrH8T6MSeGvSgob0= +github.com/labstack/gommon v0.3.0/go.mod h1:MULnywXg0yavhxWKc+lOruYdAhDwPK9wf0OL7NoOu+k= github.com/magiconair/properties v1.8.0 h1:LLgXmsheXeRoUOBOjtwPQCWIYqM/LU1ayDtDePerRcY= github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/magiconair/properties v1.8.1 h1:ZC2Vc7/ZFkGmsVC9KvOjumD+G5lXy2RtTKyzRKO2BQ4= github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= -github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA= -github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= +github.com/mattn/go-colorable v0.1.7 h1:bQGKb3vps/j0E9GfJQ03JyhRuxsvdAanXlT9BTw3mdw= +github.com/mattn/go-colorable v0.1.7/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= -github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM= -github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE= +github.com/mattn/go-isatty v0.0.9/go.mod h1:YNRxwqDuOph6SZLI9vUUz6OYw3QyUt7WiY2yME+cCiQ= +github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY= +github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE= github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -121,6 +125,8 @@ github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6Kllzaw github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc= github.com/valyala/fasttemplate v1.0.1 h1:tY9CJiPnMXf1ERmG2EyK7gNUd+c6RKGD0IfU8WdUSz8= github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8= +github.com/valyala/fasttemplate v1.2.1 h1:TVEnxayobAdVkhQfrfes2IzOB6o+z4roRkPF52WA1u4= +github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ= github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q= golang.org/x/arch v0.0.0-20180920145803-b19384d3c130 h1:Vsc61gop4hfHdzQNolo6Fi/sw7TnJ2yl3ZR4i7bYirs= golang.org/x/arch v0.0.0-20180920145803-b19384d3c130/go.mod h1:cYlCBUl1MsqxdiKgmc4uh7TxZfWSFLOGSRR090WDxt8= @@ -131,15 +137,18 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2 h1:VklqNMn3ovrHsnt90Pveol golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550 h1:ObdrDkeb4kJdCP557AjRjq69pTHfNouLtWZG7j9rPN8= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 h1:cg5LA/zNPRzIXIWSCxQW10Rvpy94aQh3LT/ShoCpkHw= -golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a h1:vclmkQCjlDX5OydZ9wv8rBCcS0QyQY66Mpf/7BZbInM= +golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190620200207-3b0461eec859 h1:R/3boaszxrf1GEUWTVDzSKVwLmSJpwZ1yqXm8j0v2QI= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200513185701-a91f0712d120 h1:EZ3cVSzKOlJxAd8e8YAJ7no8nNypTxexh/YE/xW3ZEY= -golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200822124328-c89045814202 h1:VvcQYSHwXgi7W+TpUR6A9g6Up98WAHf3f/ulnJ62IyA= +golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20200904194848-62affa334b73 h1:MXfv8rhZWmFeqX3GNZRsd6vOLoaCHjYEX3qkRo3YBUA= +golang.org/x/net v0.0.0-20200904194848-62affa334b73/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f h1:wMNYb4v58l5UBM7MYRLPG6ZhfOqbKu7X5eyFl8ZhKvA= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU= @@ -152,16 +161,18 @@ golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a h1:1BGLXjeY4akVXGgbC9HugT3Jv golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190602015325-4c4f7f33c9ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4= -golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 h1:YTzHMGlqJu67/uEo1lBv0n3wBXhXNeUbB1XfN2vmTm0= -golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200826173525-f9321e4c35a6 h1:DvY3Zkh7KabQE/kfzMvYvKirSiguP9Q/veMtkYyf0o8= +golang.org/x/sys v0.0.0-20200826173525-f9321e4c35a6/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200918174421-af09f7315aff h1:1CPUrky56AcgSpxz/KfgzQWzfG09u5YOL8MvPYBlrL8= +golang.org/x/sys v0.0.0-20200918174421-af09f7315aff/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs= -golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= +golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20181127232545-e782529d0ddd/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20200117065230-39095c1d176c h1:FodBYPZKH5tAN2O60HlglMwXGAeV/4k+NKbli79M/2c= diff --git a/internal/rule/callback/shellshock.go b/internal/rule/callback/shellshock.go index 941fb9f8..bab33a24 100644 --- a/internal/rule/callback/shellshock.go +++ b/internal/rule/callback/shellshock.go @@ -79,11 +79,15 @@ func newShellshockPrologCallback(rule RuleFace, blockingMode bool, regexps []*re } for _, env := range env { - v := strings.Split(env, `=`) - if len(v) != 2 { + v := strings.SplitN(env, `=`, 2) + if l := len(v); l <= 0 || l > 2 { ctx.Logger().Error(sqerrors.Errorf("unexpected number of elements split by `=` in `%s`", env)) return nil, nil + } else if l == 1 { + // Skip empty values + continue } + name, value := v[0], v[1] for _, re := range regexps { if re.MatchString(value) { diff --git a/internal/version/version.go b/internal/version/version.go index e7e0361c..be419b56 100644 --- a/internal/version/version.go +++ b/internal/version/version.go @@ -4,6 +4,6 @@ package version -const version = "0.15.0" +const version = "0.16.0" func Version() string { return version } diff --git a/sdk/middleware/sqecho/v4/echo.go b/sdk/middleware/sqecho/v4/echo.go new file mode 100644 index 00000000..d8a00432 --- /dev/null +++ b/sdk/middleware/sqecho/v4/echo.go @@ -0,0 +1,295 @@ +// Copyright (c) 2016 - 2019 Sqreen. All Rights Reserved. +// Please refer to our terms for more information: +// https://www.sqreen.io/terms.html + +package sqecho + +import ( + "bytes" + "io" + "net" + "net/http" + "net/textproto" + "net/url" + "strconv" + + "github.com/labstack/echo/v4" + "github.com/sqreen/go-agent/internal" + protectioncontext "github.com/sqreen/go-agent/internal/protection/context" + http_protection "github.com/sqreen/go-agent/internal/protection/http" + "github.com/sqreen/go-agent/internal/protection/http/types" + "github.com/sqreen/go-agent/sdk" +) + +// FromContext allows to access the HTTPRequestRecord from Echo request handlers +// if present, and nil otherwise. The value is stored in handler contexts by the +// middleware function, and is of type *HTTPRequestRecord. +// +// Note that Echo's context does not implement the `context.Context` interface, +// so `sdk.FromContext()` cannot be used with it, hence another `FromContext()` +// in this package to access the SDK context value from Echo's context. +// `sdk.FromContext()` can still be used on the request context. +func FromContext(c echo.Context) *sdk.Context { + return sdk.FromContext(c.Request().Context()) +} + +// Middleware is Sqreen's middleware function for echo to monitor and protect the +// requests echo receives. +// +// SDK methods can be called from request handlers by using the request context. +// It can be retrieved from the request context using `sdk.FromContext()` or +// on a echo's context. +// +// Usage example: +// +// e := echo.New() +// e.Use(sqecho.Middleware()) +// +// e.GET("/", func(c echo.Context) error { +// // Accessing the SDK through Echo's context +// sqecho.FromContext(c).TrackEvent("my.event.one") +// foo(c.Request()) +// return nil +// } +// +// func foo(req *http.Request) { +// // Accessing the SDK through the request context +// sdk.FromContext(req.Context()).TrackEvent("my.event.two") +// } +// +// e.GET("/", func(c echo.Context) { +// // Globally identifying a user and checking if the request should be +// // aborted. +// uid := sdk.EventUserIdentifiersMap{"uid": "my-uid"} +// sqUser := sqecho.FromContext(c).ForUser(uid) +// // Globally associate this user to the current request and check if it got +// // blocked. +// if err := sqUser.Identify(); err != nil { +// // Return to stop further handling the request +// return err +// } +// // ... not blocked ... +// return nil +// } +// +func Middleware() echo.MiddlewareFunc { + internal.Start() + return func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + return middlewareHandler(internal.Agent(), next, c) + } + } +} + +func middlewareHandler(agent protectioncontext.AgentFace, next echo.HandlerFunc, c echo.Context) error { + if agent == nil { + return next(c) + } + + requestReader := &requestReaderImpl{c: c} + responseWriter := &responseWriterImpl{c: c} + + req := c.Request() + + ctx, reqCtx, cancelHandlerContext := http_protection.NewRequestContext(req.Context(), agent, responseWriter, requestReader) + if ctx == nil { + return next(c) + } + + defer func() { + cancelHandlerContext() + _ = ctx.Close(responseWriter.closeResponseWriter()) + }() + + req = ctx.WrapRequest(reqCtx, req) + c.SetRequest(req) + c.Set(protectioncontext.ContextKey.String, ctx) + + if err := ctx.Before(); err != nil { + return err + } + if err := next(c); err != nil { + // Handler-based protection such as user security responses or RASP + // protection may lead to aborted requests bubbling up the error that + // was returned. + return err + } + if err := ctx.After(); err != nil { + return err + } + return nil +} + +type requestReaderImpl struct { + c echo.Context + bodyReadBuffer bytes.Buffer + queryForm url.Values +} + +func (r *requestReaderImpl) Body() []byte { + return nil +} + +func (r *requestReaderImpl) UserAgent() string { + return r.c.Request().UserAgent() +} + +func (r *requestReaderImpl) Referer() string { + return r.c.Request().Referer() +} + +func (r *requestReaderImpl) ClientIP() net.IP { + return nil // Delegated to the middleware according the agent configuration +} + +func (r *requestReaderImpl) Method() string { + return r.c.Request().Method +} + +func (r *requestReaderImpl) URL() *url.URL { + return r.c.Request().URL +} + +func (r *requestReaderImpl) RequestURI() string { + return r.c.Request().RequestURI +} + +func (r *requestReaderImpl) Host() string { + return r.c.Request().Host +} + +func (r *requestReaderImpl) IsTLS() bool { + return r.c.IsTLS() +} + +func (r *requestReaderImpl) Params() types.RequestParamMap { + params := r.c.ParamNames() + l := len(params) + if l == 0 { + return nil + } + + m := make(types.RequestParamMap, l) + for _, key := range params { + m.Add(key, r.c.Param(key)) + } + return m +} + +func (r *requestReaderImpl) QueryForm() url.Values { + if r.queryForm == nil { + r.queryForm = r.c.Request().URL.Query() + } + return r.queryForm +} + +func (r *requestReaderImpl) PostForm() url.Values { + return r.c.Request().PostForm +} + +func (r *requestReaderImpl) Headers() http.Header { + return r.c.Request().Header +} + +func (r *requestReaderImpl) Header(h string) *string { + headers := r.c.Request().Header + if headers == nil { + return nil + } + v := headers[textproto.CanonicalMIMEHeaderKey(h)] + if len(v) == 0 { + return nil + } + return &v[0] +} + +func (r *requestReaderImpl) RemoteAddr() string { + return r.c.Request().RemoteAddr +} + +type responseWriterImpl struct { + c echo.Context + closed bool +} + +func (w *responseWriterImpl) closeResponseWriter() types.ResponseFace { + if !w.closed { + w.closed = true + } + return newObservedResponse(w) +} + +func (w *responseWriterImpl) Header() http.Header { + return w.c.Response().Header() +} + +func (w *responseWriterImpl) Write(b []byte) (int, error) { + if w.closed { + return 0, types.WriteAfterCloseError{} + } + return w.c.Response().Write(b) +} + +func (w *responseWriterImpl) WriteString(s string) (int, error) { + if w.closed { + return 0, types.WriteAfterCloseError{} + } + return io.WriteString(w.c.Response(), s) +} + +// Static assert that the io.StringWriter is implemented +var _ io.StringWriter = (*responseWriterImpl)(nil) + +func (w *responseWriterImpl) WriteHeader(statusCode int) { + if w.closed { + return + } + w.c.Response().WriteHeader(statusCode) +} + +// response observed by the response writer +type observedResponse struct { + contentType string + contentLength int64 + status int +} + +func newObservedResponse(r *responseWriterImpl) *observedResponse { + // Content-Type will be not empty only when explicitly set. + // It could be guessed as net/http does. Not implemented for now. + ct := r.Header().Get("Content-Type") + + response := r.c.Response() + + // Content-Length is either explicitly set or the amount of written data. + cl := response.Size + if contentLength := r.Header().Get("Content-Length"); contentLength != "" { + if l, err := strconv.ParseInt(contentLength, 10, 0); err == nil { + cl = l + } + } + + status := response.Status + + return &observedResponse{ + contentType: ct, + contentLength: cl, + status: status, + } +} + +func (r *observedResponse) Status() int { + if status := r.status; status != 0 { + return status + } + // Default net/http status is 200 + return http.StatusOK +} + +func (r *observedResponse) ContentType() string { + return r.contentType +} + +func (r *observedResponse) ContentLength() int64 { + return r.contentLength +} diff --git a/sdk/middleware/sqecho/v4/echo_test.go b/sdk/middleware/sqecho/v4/echo_test.go new file mode 100644 index 00000000..749e812e --- /dev/null +++ b/sdk/middleware/sqecho/v4/echo_test.go @@ -0,0 +1,416 @@ +// Copyright (c) 2016 - 2019 Sqreen. All Rights Reserved. +// Please refer to our terms for more information: +// https://www.sqreen.io/terms.html + +package sqecho + +import ( + "errors" + "io/ioutil" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" + + "github.com/labstack/echo/v4" + protectioncontext "github.com/sqreen/go-agent/internal/protection/context" + "github.com/sqreen/go-agent/internal/protection/http/types" + "github.com/sqreen/go-agent/sdk" + "github.com/sqreen/go-agent/sdk/middleware/_testlib/mockups" + "github.com/sqreen/go-agent/tools/testlib" + "github.com/stretchr/testify/mock" + "github.com/stretchr/testify/require" +) + +func TestMiddleware(t *testing.T) { + t.Run("without middleware", func(t *testing.T) { + body := testlib.RandUTF8String(4096) + req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body)) + + // Create an Echo context + e := echo.New() + rec := httptest.NewRecorder() + c := e.NewContext(req, rec) + // Define a Echo handler + h := func(c echo.Context) error { + { + // using echo's context interface + sq := FromContext(c) + require.NotNil(t, sq) + sq.TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()).WithUserIdentifiers(sdk.EventUserIdentifiersMap{"my": "id"}) + sq.ForUser(sdk.EventUserIdentifiersMap{"my": "id"}).TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()) + } + { + // using echo's context interface + sq := sdk.FromContext(c.Request().Context()) + require.NotNil(t, sq) + sq.TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()).WithUserIdentifiers(sdk.EventUserIdentifiersMap{"my": "id"}) + sq.ForUser(sdk.EventUserIdentifiersMap{"my": "id"}).TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()) + } + body, err := ioutil.ReadAll(c.Request().Body) + if err != nil { + return err + } + return c.String(http.StatusOK, string(body)) + } + // Perform the request and record the output + err := h(c) + // Check the request was performed as expected + require.NoError(t, err) + require.Equal(t, http.StatusOK, rec.Code) + require.Equal(t, body, rec.Body.String()) + }) + + t.Run("without middleware", func(t *testing.T) { + body := testlib.RandUTF8String(4096) + req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body)) + + // Create an Echo context + e := echo.New() + rec := httptest.NewRecorder() + c := e.NewContext(req, rec) + // Define a Echo handler + h := func(c echo.Context) error { + { + // using echo's context interface + sq := FromContext(c) + require.NotNil(t, sq) + sq.TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()).WithUserIdentifiers(sdk.EventUserIdentifiersMap{"my": "id"}) + sq.ForUser(sdk.EventUserIdentifiersMap{"my": "id"}).TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()) + } + { + // using echo's context interface + sq := sdk.FromContext(c.Request().Context()) + require.NotNil(t, sq) + sq.TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()).WithUserIdentifiers(sdk.EventUserIdentifiersMap{"my": "id"}) + sq.ForUser(sdk.EventUserIdentifiersMap{"my": "id"}).TrackEvent("my event").WithProperties(sdk.EventPropertyMap{"my": "prop"}).WithTimestamp(time.Now()) + } + body, err := ioutil.ReadAll(c.Request().Body) + if err != nil { + return err + } + return c.String(http.StatusOK, string(body)) + } + // Perform the request and record the output + err := Middleware()(h)(c) + // Check the request was performed as expected + require.NoError(t, err) + require.Equal(t, http.StatusOK, rec.Code) + require.Equal(t, body, rec.Body.String()) + }) + + t.Run("control flow", func(t *testing.T) { + middlewareResponseBody := testlib.RandUTF8String(4096) + middlewareResponseStatus := 433 + handlerResponseBody := testlib.RandUTF8String(4096) + handlerResponseStatus := 533 + agent := &mockups.AgentMockup{} + agent.ExpectConfig().Return(&mockups.AgentConfigMockup{}) + agent.ExpectIsIPAllowed(mock.Anything).Return(false) + agent.ExpectIsPathAllowed(mock.Anything).Return(false) + agent.ExpectSendClosedRequestContext(mock.Anything).Return(nil) + defer agent.AssertExpectations(t) // inaccurate but worth it + + for _, tc := range []struct { + name string + agent protectioncontext.AgentFace + }{ + { + name: "agent disabled", + agent: nil, + }, + { + name: "agent enabled", + agent: agent, + }, + } { + tc := tc + t.Run(tc.name, func(t *testing.T) { + for _, tc := range []struct { + name string + middlewares []echo.MiddlewareFunc + handler func(echo.Context) error + test func(t *testing.T, rec *httptest.ResponseRecorder, err error) + }{ + { + name: "sqreen first/the middleware aborts before the handler", + middlewares: []echo.MiddlewareFunc{ + middleware(tc.agent), + func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + c.String(middlewareResponseStatus, middlewareResponseBody) + return errors.New("middleware abort") + } + }, + }, + handler: func(echo.Context) error { + panic("unexpected control flow") + return nil + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.Error(t, err) + require.Equal(t, "middleware abort", err.Error()) + require.Equal(t, middlewareResponseStatus, rec.Code) + require.Equal(t, middlewareResponseBody, rec.Body.String()) + }, + }, + + { + name: "sqreen first/the handler aborts", + middlewares: []echo.MiddlewareFunc{ + middleware(tc.agent), + func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + err := next(c) + require.Error(t, err) + return err + } + }, + }, + handler: func(c echo.Context) error { + c.String(handlerResponseStatus, handlerResponseBody) + return errors.New("handler abort") + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.Error(t, err) + require.Equal(t, "handler abort", err.Error()) + require.Equal(t, handlerResponseStatus, rec.Code) + require.Equal(t, handlerResponseBody, rec.Body.String()) + }, + }, + + { + name: "sqreen first/no one aborts", + middlewares: []echo.MiddlewareFunc{ + middleware(tc.agent), + func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + err := c.String(middlewareResponseStatus, middlewareResponseBody) + require.NoError(t, err) + err = next(c) + require.NoError(t, err) + err = c.String(middlewareResponseStatus, middlewareResponseBody) + require.NoError(t, err) + return err + } + }, + }, + handler: func(c echo.Context) error { + return c.String(handlerResponseStatus, handlerResponseBody) + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.NoError(t, err) + require.Equal(t, middlewareResponseStatus, rec.Code) + require.Equal(t, middlewareResponseBody+handlerResponseBody+middlewareResponseBody, rec.Body.String()) + }, + }, + + { + name: "sqreen first/the middleware aborts after the handler", + middlewares: []echo.MiddlewareFunc{ + middleware(tc.agent), + func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + err := next(c) + require.NoError(t, err) + err = c.String(middlewareResponseStatus, middlewareResponseBody) + require.NoError(t, err) + return errors.New("middleware abort") + } + }, + }, + handler: func(c echo.Context) error { + return nil + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.Error(t, err) + require.Equal(t, "middleware abort", err.Error()) + require.Equal(t, middlewareResponseStatus, rec.Code) + require.Equal(t, middlewareResponseBody, rec.Body.String()) + }, + }, + + { + name: "sqreen after another middleware/issue AGO-29: the middleware aborts after the next handler", + middlewares: []echo.MiddlewareFunc{ + func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + err := next(c) + require.NoError(t, err) + err = c.String(middlewareResponseStatus, middlewareResponseBody) + require.NoError(t, err) + return errors.New("middleware abort") + } + }, + middleware(tc.agent), + }, + handler: func(c echo.Context) error { + // Do nothing so that the calling middleware can handle the response. + return nil + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.Error(t, err) + require.Equal(t, "middleware abort", err.Error()) + require.Equal(t, middlewareResponseStatus, rec.Code) + require.Equal(t, middlewareResponseBody, rec.Body.String()) + }, + }, + + { + name: "sqreen after another middleware/the middleware aborts before the next handler", + middlewares: []echo.MiddlewareFunc{ + func(echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + err := c.String(middlewareResponseStatus, middlewareResponseBody) + require.NoError(t, err) + return errors.New("middleware abort") + } + }, + func(echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + // Make sure echo doesn't call the next middleware when the + // previous one returns an error. + panic("unexpected control flow") + } + }, + middleware(tc.agent), + }, + handler: func(echo.Context) error { + // Make sure echo doesn't call the handler when one of the + // previous middlewares return an error. + panic("unexpected control flow") + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.Error(t, err) + require.Equal(t, "middleware abort", err.Error()) + require.Equal(t, middlewareResponseStatus, rec.Code) + require.Equal(t, middlewareResponseBody, rec.Body.String()) + }, + }, + + { + name: "sqreen after another middleware/the handler aborts", + middlewares: []echo.MiddlewareFunc{ + func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + err := next(c) + require.Error(t, err) + return err + } + }, + middleware(tc.agent), + }, + handler: func(c echo.Context) error { + err := c.String(handlerResponseStatus, handlerResponseBody) + require.NoError(t, err) + return errors.New("handler abort") + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.Error(t, err) + require.Equal(t, "handler abort", err.Error()) + require.Equal(t, handlerResponseStatus, rec.Code) + require.Equal(t, handlerResponseBody, rec.Body.String()) + }, + }, + + { + name: "sqreen after another middleware/no one aborts", + middlewares: []echo.MiddlewareFunc{ + func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + err := c.String(middlewareResponseStatus, middlewareResponseBody) + require.NoError(t, err) + err = next(c) + require.NoError(t, err) + err = c.String(middlewareResponseStatus, middlewareResponseBody) + require.NoError(t, err) + return nil + } + }, + middleware(tc.agent), + }, + handler: func(c echo.Context) error { + err := c.String(handlerResponseStatus, handlerResponseBody) + require.NoError(t, err) + return nil + }, + test: func(t *testing.T, rec *httptest.ResponseRecorder, err error) { + require.NoError(t, err) + require.Equal(t, middlewareResponseStatus, rec.Code) + require.Equal(t, middlewareResponseBody+handlerResponseBody+middlewareResponseBody, rec.Body.String()) + }, + }, + } { + tc := tc + t.Run(tc.name, func(t *testing.T) { + // Create a echo router + router := echo.New() + // Setup the middleware + router.Use(tc.middlewares...) + // Add the endpoint + router.GET("/", tc.handler) + + // Perform the request and record the output + rec := httptest.NewRecorder() + req, _ := http.NewRequest("GET", "/", nil) + var err error + router.HTTPErrorHandler = func(e error, _ echo.Context) { + err = e + } + router.ServeHTTP(rec, req) + + // Check the request was performed as expected + tc.test(t, rec, err) + }) + } + }) + } + }) + + t.Run("response observation", func(t *testing.T) { + expectedStatusCode := 433 + + agent := &mockups.AgentMockup{} + agent.ExpectConfig().Return(&mockups.AgentConfigMockup{}).Once() + agent.ExpectIsIPAllowed(mock.Anything).Return(false).Once() + agent.ExpectIsPathAllowed(mock.Anything).Return(false).Once() + var responseStatusCode int + agent.ExpectSendClosedRequestContext(mock.MatchedBy(func(recorded types.ClosedRequestContextFace) bool { + resp := recorded.Response() + responseStatusCode = resp.Status() + return true + })).Return(nil) + defer agent.AssertExpectations(t) + + // Create a route + router := echo.New() + router.Use(middleware(agent)) + router.GET("/", func(c echo.Context) error { + return c.NoContent(expectedStatusCode) + }) + + // Perform the request and record the output + rec := httptest.NewRecorder() + req, _ := http.NewRequest("GET", "/", nil) + var err error + router.HTTPErrorHandler = func(e error, _ echo.Context) { + err = e + } + router.ServeHTTP(rec, req) + + // Check the result + require.NoError(t, err) + require.Equal(t, expectedStatusCode, responseStatusCode) + require.Equal(t, expectedStatusCode, rec.Code) + }) + +} + +func middleware(agent protectioncontext.AgentFace) echo.MiddlewareFunc { + return func(next echo.HandlerFunc) echo.HandlerFunc { + return func(c echo.Context) error { + return middlewareHandler(agent, next, c) + } + } +} diff --git a/sdk/sqreen-instrumentation-tool/instrumentation.go b/sdk/sqreen-instrumentation-tool/instrumentation.go index e4cb6f60..48e51082 100644 --- a/sdk/sqreen-instrumentation-tool/instrumentation.go +++ b/sdk/sqreen-instrumentation-tool/instrumentation.go @@ -202,6 +202,8 @@ var ( "os", "net/http", "github.com/gin-gonic/gin", + "github.com/labstack/echo", + "github.com/labstack/echo/v4", "go.mongodb.org/mongo-driver/mongo", } @@ -227,6 +229,14 @@ var ( // Limited for performance reasons to: "mongo.go", // mongo.go contains the bson transformation function }, + "github.com/labstack/echo": { + // Same comment as net/http + "context.go", // context.go contains the body parsers + }, + "github.com/labstack/echo/v4": { + // Same comment as net/http + "context.go", // context.go contains the body parsers + }, } )