Skip to content
This repository has been archived by the owner on Nov 2, 2023. It is now read-only.

Commit

Permalink
New Feature:
Browse files Browse the repository at this point in the history
- (#137) RASP: add noSQL Injection protection support for the Go MongoDB driver
  `go.mongodb.org/mongo-driver/mongo`. This protection can be configured at
  <https://my.sqreen.com/application/goto/modules/rasp/details/nosql_injection>.

Internal Changes:

- (#138) Health-check the HTTPS connectivity to the new backend API
  `ingestion.sqreen.com` before using it. Fallback to the usual
  `back.sqreen.com` in case of a connection issue. Therefore, the agent can take
  up to 30 seconds to connect to Sqreen if the health-check timeouts. Please
  make sure to add this new  firewall and proxy configurations.

- (#136) Add support to attach multiple security protections per hook point.

Fixes:

- (#140) Fix the In-App WAF metadata PII scrubbing to also match substrings.
  • Loading branch information
Julio Guerra committed Jul 24, 2020
2 parents 4a71cf1 + 53e2d05 commit b3b89ec
Show file tree
Hide file tree
Showing 33 changed files with 1,532 additions and 679 deletions.
57 changes: 40 additions & 17 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,27 @@
# v0.12.1
# v0.13.0 - 24 July 2020

## New Feature

- (#137) RASP: add noSQL Injection protection support for the Go MongoDB driver
`go.mongodb.org/mongo-driver/mongo`. This protection can be configured at
<https://my.sqreen.com/application/goto/modules/rasp/details/nosql_injection>.

## Internal Changes

- (#138) Health-check the HTTPS connectivity to the new backend API
`ingestion.sqreen.com` before using it. Fallback to the usual
`back.sqreen.com` in case of a connection issue. Therefore, the agent can take
up to 30 seconds to connect to Sqreen if the health-check timeouts. Please
make sure to add this new firewall and proxy configurations.

- (#136) Add support to attach multiple security protections per hook point.

## Fixes

- (#140) Fix the In-App WAF metadata PII scrubbing to also match substrings.


# v0.12.1 - 13 July 2020

## Fixes

Expand All @@ -19,7 +42,7 @@
- (eeb1dca) Avoid copying the metadata returned by the In-App WAF.


# v0.12.0
# v0.12.0 - 6 July 2020

## New Features

Expand Down Expand Up @@ -53,7 +76,7 @@
- (794d6e2) Allow port numbers in the `X-Forwarded-For` header.


# v0.11.0
# v0.11.0 - 19 June 2020

## New Features

Expand Down Expand Up @@ -90,14 +113,14 @@
- (#114) Add Goroutine Local Storage (GLS) support through static instrumentation of the Go runtime.


# v0.10.1
# v0.10.1 - 5 June 2020

## Fix

- (#116) Fix the instrumentation tool ignoring vendored packages, leading to
missing hook points in the agent.

# v0.10.0
# v0.10.0 - 20 May 2020

## New Features

Expand Down Expand Up @@ -136,7 +159,7 @@

- Document PII scrubbing configuration at <https://docs.sqreen.com/go/configuration/#personally-identifiable-information-scrubbing>.

# v0.9.1
# v0.9.1 - 31 March 2020

## Fixes

Expand All @@ -150,7 +173,7 @@
- (#101) Prevent starting the agent when the instrumentation tool and agent
versions are not the same.

# v0.9.0
# v0.9.0 - 19 February 2020

This new major version says farewell to the `beta` and adds SQL-injection
run time protection thanks the first building blocks of [RASP][RASP-Wikipedia]
Expand Down Expand Up @@ -233,7 +256,7 @@ Because we now want a stable public API, find below the breaking changes:
compiled as a Go module. This is also shown by the dashboard when the list
of dependencies is empty.

# v0.1.0-beta.10
# v0.1.0-beta.10 - 24 January 2020

## Breaking Change

Expand Down Expand Up @@ -264,7 +287,7 @@ Because we now want a stable public API, find below the breaking changes:
- (#92) Vendoring using `go mod vendor` could lead to compilation errors due to
missing files.

# v0.1.0-beta.9
# v0.1.0-beta.9 - 19 December 2019

## New Features

Expand All @@ -283,7 +306,7 @@ Because we now want a stable public API, find below the breaking changes:
- The In-App WAF has been intensively optimized so that large requests can no longer impact
its execution time. (#83)

# v0.1.0-beta.8
# v0.1.0-beta.8 - 15 October 2019

## Internal Changes

Expand All @@ -292,7 +315,7 @@ Because we now want a stable public API, find below the breaking changes:
- Ignore WAF timeout errors and add more context when reporting an error (#80).
- Update the libsqreen to v0.4.0 to add support for the `@pm` operator.

# v0.1.0-beta.7
# v0.1.0-beta.7 - 26 September 2019

## Breaking Changes

Expand All @@ -319,7 +342,7 @@ Because we now want a stable public API, find below the breaking changes:
- Fix a compilation error on 32-bit target architectures.


# v0.1.0-beta.6
# v0.1.0-beta.6 - 25 July 2019

## New Features

Expand Down Expand Up @@ -354,7 +377,7 @@ Because we now want a stable public API, find below the breaking changes:
log-level.


# v0.1.0-beta.5
# v0.1.0-beta.5 - 23 May 2019

## New Features

Expand All @@ -380,7 +403,7 @@ Because we now want a stable public API, find below the breaking changes:
processing loop.


# v0.1.0-beta.4
# v0.1.0-beta.4 - 16 April 2019

This release adds the ability to block IP addresses or users into your Go web
services by adding support for [Security Automation] according to your
Expand Down Expand Up @@ -440,7 +463,7 @@ Note that redirecting users or IP addresses is not supported yet.
- Avoid performing multiple times commands within the same command batch. (51)


# v0.1.0-beta.3
# v0.1.0-beta.3 - 22 March 2019

## New Features

Expand Down Expand Up @@ -477,15 +500,15 @@ Note that redirecting users or IP addresses is not supported yet.
self-managing the initializations. (#28)


# v0.1.0-beta.2
# v0.1.0-beta.2 - 14 February 2019

## New feature

- Add a new `Identify()` method allowing to explicitly associate a user to the
current request. As soon as we add the support for the security reponses, it
will allow to block users (#26).

# v0.1.0-beta.1
# v0.1.0-beta.1 - 7 February 2019

This version is a new major version towards the v0.1.0 as it proposes a new and
stable SDK API, that now will only be updated upon user feedback. So please,
Expand Down
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
[![GoDoc](https://godoc.org/github.com/sqreen/go-agent?status.svg)](https://godoc.org/github.com/sqreen/go-agent)
[![Go Report Card](https://goreportcard.com/badge/github.com/sqreen/go-agent)](https://goreportcard.com/report/github.com/sqreen/go-agent)
[![Build Status](https://dev.azure.com/sqreenci/Go%20Agent/_apis/build/status/sqreen.go-agent?branchName=master)](https://dev.azure.com/sqreenci/Go%20Agent/_build/latest?definitionId=8&branchName=master)
[![Sourcegraph](https://sourcegraph.com/github.com/sqreen/go-agent/-/badge.svg)](https://sourcegraph.com/github.com/sqreen/go-agent?badge)

After performance monitoring (APM), error and log monitoring it’s time to add a
security component into your app. Sqreen’s microagent automatically monitors
Expand Down
5 changes: 3 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ require (
github.com/spf13/pflag v1.0.5 // indirect
github.com/spf13/viper v1.3.2
github.com/sqreen/go-libsqreen v0.7.0
github.com/sqreen/go-sdk/signal v1.0.0
github.com/stretchr/testify v1.5.1
github.com/sqreen/go-sdk/signal v1.1.0
github.com/stretchr/testify v1.6.1
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 // indirect
golang.org/x/net v0.0.0-20200513185701-a91f0712d120
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect
Expand All @@ -40,4 +40,5 @@ require (
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/go-playground/assert.v1 v1.2.1
gopkg.in/go-playground/validator.v8 v8.18.2 // indirect
gopkg.in/yaml.v2 v2.3.0 // indirect
)
13 changes: 7 additions & 6 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -101,12 +101,10 @@ github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.3.2 h1:VUFqw5KcqRf7i70GOzW7N+Q7+gxVBkSSqiXB12+JQ4M=
github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
github.com/sqreen/go-libsqreen v0.6.1 h1:+SHH3h8qHhINEzgRVqTZ40YxqwDjSVxU5r4isUeg+C8=
github.com/sqreen/go-libsqreen v0.6.1/go.mod h1:D324eoKlZGfW+TF3WGg+2fUtpdrI+cEK5UYwpxfaeUc=
github.com/sqreen/go-libsqreen v0.7.0 h1:MRX/KB5lX3O6ucvmTUap6iSDt27bM+76MQpuDNjL+1o=
github.com/sqreen/go-libsqreen v0.7.0/go.mod h1:D324eoKlZGfW+TF3WGg+2fUtpdrI+cEK5UYwpxfaeUc=
github.com/sqreen/go-sdk/signal v1.0.0 h1:WNjufvcjKYOgSZHPCwqG0Od5eVAD8wxwmiIe6ZCqoNE=
github.com/sqreen/go-sdk/signal v1.0.0/go.mod h1:UksuO4mxxDMFw3el+R9mW9tmCgdc94WiDcGuCXU/pwU=
github.com/sqreen/go-sdk/signal v1.1.0 h1:l22lqlUNDlEaqsNjpgVelGteBCwGodZqUDPUMBOLzhE=
github.com/sqreen/go-sdk/signal v1.1.0/go.mod h1:XWJV0TzuoN6PotzRn4YSe6fhTxyw67yRpVYr9NJTzto=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.2.0 h1:Hbg2NidpLE8veEBkEZTL3CvlkUIVzuU9jDplZO54c48=
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
Expand All @@ -115,8 +113,8 @@ github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8 h1:3SVOIvH7Ae1KRYyQWRjXWJEA9sS/c/pjvH++55Gr648=
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
Expand Down Expand Up @@ -190,3 +188,6 @@ gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 h1:tQIYjPdBoyREyB9XMu+nnTclpTYkz2zFM+lzLJFO4gQ=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
9 changes: 9 additions & 0 deletions internal/adapter.go
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,15 @@ func newMetricsAPIAdapter(logger plog.ErrorLogger, expiredMetrics map[string]*me
return metricsArray
}

type variousInfoAPIAdapter struct {
*appInfoAPIAdapter
sqreenDomains api.SqreenDomainStatusMap
}

func (v variousInfoAPIAdapter) GetSqreenDomains() api.SqreenDomainStatusMap {
return v.sqreenDomains
}

type appInfoAPIAdapter app.Info

func (a *appInfoAPIAdapter) unwrap() *app.Info { return (*app.Info)(a) }
Expand Down
23 changes: 14 additions & 9 deletions internal/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -228,9 +228,9 @@ func New(cfg *config.Config) *AgentType {
logger.Info(message)
return nil
}
// TODO: agent.Health() + waf.Health()

if waf.Version() == nil {
message := fmt.Sprintf("in-app waf disabled: cgo was disabled during the program compilation while required by the in-app waf")
message := "in-app waf disabled: cgo was disabled during the program compilation while required by the in-app waf"
backend.SendAgentMessage(logger, cfg, message)
logger.Info("agent: ", message)
}
Expand All @@ -240,9 +240,11 @@ func New(cfg *config.Config) *AgentType {
sdkMetricsPeriod := time.Duration(cfg.SDKMetricsPeriod()) * time.Second
logger.Debugf("agent: using sdk metrics store time period of %s", sdkMetricsPeriod)

piiScrubber, err := sqsanitize.NewScrubber(cfg.StripSensitiveKeyRegexp(), cfg.StripSensitiveValueRegexp(), config.ScrubberRedactedString)
piiScrubber := sqsanitize.NewScrubber(cfg.StripSensitiveKeyRegexp(), cfg.StripSensitiveValueRegexp(), config.ScrubberRedactedString)

client, err := backend.NewClient(cfg.BackendHTTPAPIBaseURL(), cfg.BackendHTTPAPIProxy(), logger)
if err != nil {
logger.Error(sqerrors.Wrap(err, "ecdsa public key"))
logger.Error(sqerrors.Wrap(err, "agent: could not create the backend client"))
return nil
}

Expand All @@ -264,7 +266,7 @@ func New(cfg *config.Config) *AgentType {
cancel: cancel,
config: cfg,
appInfo: app.NewInfo(logger),
client: backend.NewClient(cfg.BackendHTTPAPIBaseURL(), cfg.BackendHTTPAPIProxy(), logger),
client: client,
actors: actor.NewStore(logger),
rules: rulesEngine,
piiScrubber: piiScrubber,
Expand Down Expand Up @@ -341,7 +343,7 @@ func (a *AgentType) Serve() error {

token := a.config.BackendHTTPAPIToken()
appName := a.config.AppName()
appLoginRes, err := appLogin(a.ctx, a.logger, a.client, token, appName, a.appInfo, a.config.UseSignalBackend())
appLoginRes, err := appLogin(a.ctx, a.logger, a.client, token, appName, a.appInfo, a.config.DisableSignalBackend())
if err != nil {
if xerrors.Is(err, context.Canceled) {
a.logger.Debug(err)
Expand Down Expand Up @@ -586,11 +588,14 @@ func stopTimer(t *time.Timer) {

func (m *eventManager) Loop(ctx context.Context, client *backend.Client) {
var (
stalenessTimer = time.NewTimer(m.maxStaleness)
// We can't create a stopped timer so we initializae it with a large value
// of 24 hours and stop it immediately. Calls to Reset() will correctly
// set the configured timer value.
stalenessTimer = time.NewTimer(24 * time.Hour)
stalenessChan <-chan time.Time
)
defer stopTimer(stalenessTimer)
stopTimer(stalenessTimer)
defer stopTimer(stalenessTimer)

batch := make([]Event, 0, m.count)
for {
Expand Down Expand Up @@ -647,7 +652,7 @@ func (m *eventManager) sendBatch(ctx context.Context, client *backend.Client, ba
if _, err := m.agent.piiScrubber.Scrub(event, nil); err != nil {
// Only log this unexpected error and keep the event that may have been
// partially scrubbed.
m.agent.logger.Error(errors.Wrap(err, "could not send the event batch"))
m.agent.logger.Error(errors.Wrap(err, "could not scrub the event"))
}
req.Batch = append(req.Batch, *api.NewBatchRequest_EventFromFace(event))
}
Expand Down
Loading

0 comments on commit b3b89ec

Please sign in to comment.