Panic using default docker-compose configuration with docker log driver

[mrlake]

Describe the bug
Panic using default docker-compose configuration with docker log driver
Loki Runs for a while, logs are visible via grafana, then the loki container exits with Code:2
To Reproduce
Steps to reproduce the behavior:

1. Enabled logging on influxdb via compose config:

2. Started Loki (docker-compose up) version=master-b652f0a, branch=master, revision=b652f0a3)

Expected behavior
Loki to continue running

Environment:

• Infrastructure: AWS EC2 Ubuntu 18.04
• Deployment tool: Docker Compose

Loki config file vanilla from here.

https://github.com/grafana/loki/blob/master/cmd/loki/loki-docker-config.yaml

Pasted below for reference:

auth_enabled: false

server:
http_listen_port: 3100

ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s
max_transfer_retries: 0

schema_config:
configs:
- from: 2018-04-15
store: boltdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 168h

storage_config:
boltdb:
directory: /loki/index

filesystem:
directory: /loki/chunks

limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h

chunk_store_config:
max_look_back_period: 0s

table_manager:
retention_deletes_enabled: false
retention_period: 0s

Docker Compose Config below:

version: "3.3"

services:

loki:
image: grafana/loki:latest
volumes:
- "loki-config1:/etc/loki"
command:
- --config.file=/etc/loki/loki-docker-config.yaml
- --log.level=debug
ports:
- "3100:3100"

influxdb:
image: influxdb:1.7.10
restart: always
ports:
- 8086:8086
volumes:
- influxdb:/var/lib/influxdb
logging:
driver: loki
options:
loki-url: "http://localhost:3100/loki/api/v1/push"

volumes:
loki-config1:

Screenshots, Promtail config, or terminal output
If applicable, add any output to help explain your problem.

Attached a copy of the debug output from the container log.
loki-errorlog.LOG

[cyriltovena]

Thanks for catching this.

Sharing the error as plain text for others.

level=debug ts=2020-04-27T03:00:40.05956351Z caller=flush.go:183 msg="Ingester.flushLoop() exited"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x111aa6f]

goroutine 288 [running]:
github.com/grafana/loki/pkg/chunkenc.(*Facade).Size(0xc00075b500, 0x3c2c640)
	<autogenerated>:1 +0x2f
github.com/cortexproject/cortex/pkg/chunk/storage.metricsChunkClient.PutChunks(0x3c2cac0, 0xc0000af600, 0xc00000ed80, 0xc00000ed88, 0xc00000ed90, 0xc00000ed98, 0x3c2c640, 0xc0007bacf0, 0xc0005d8780, 0x1, ...)
	/src/loki/vendor/github.com/cortexproject/cortex/pkg/chunk/storage/metrics.go:71 +0x205
github.com/cortexproject/cortex/pkg/chunk.(*seriesStore).PutOne(0xc0001b4000, 0x3c2c640, 0xc0007bacf0, 0x171b98f66fe, 0x171b98f6c6c, 0xb297a3bc0ecadaa0, 0x34edc0e, 0x4, 0x171b98f66fe, 0x171b98f6c6c, ...)
	/src/loki/vendor/github.com/cortexproject/cortex/pkg/chunk/series_store.go:438 +0x672
github.com/cortexproject/cortex/pkg/chunk.compositeStore.Put.func1(0x171b98f66fe, 0x171b98f6c6c, 0x3c76aa0, 0xc0001b4000, 0x44615c, 0x50f8bb)
	/src/loki/vendor/github.com/cortexproject/cortex/pkg/chunk/composite_store.go:95 +0xd3
github.com/cortexproject/cortex/pkg/chunk.compositeStore.forStores(0xc0000af620, 0x1, 0x1, 0x171b98f66fe, 0x171b98f6c6c, 0xc000ca9838, 0x196, 0xac8)
	/src/loki/vendor/github.com/cortexproject/cortex/pkg/chunk/composite_store.go:231 +0x1e0
github.com/cortexproject/cortex/pkg/chunk.compositeStore.Put(0xc0000af620, 0x1, 0x1, 0x3c2c600, 0xc0003905a0, 0xc0007ad900, 0x1, 0x1, 0x4a91d0, 0xc00018ae10)
	/src/loki/vendor/github.com/cortexproject/cortex/pkg/chunk/composite_store.go:94 +0x14b
github.com/grafana/loki/pkg/ingester.(*Ingester).flushChunks(0xc0001ebc00, 0x3c2c600, 0xc0003905a0, 0xb297a3bc0ecadaa0, 0xc000cb0c00, 0x6, 0x6, 0xc00000eda8, 0x1, 0x1, ...)
	/src/loki/pkg/ingester/flush.go:337 +0x82d
github.com/grafana/loki/pkg/ingester.(*Ingester).flushUserSeries(0xc0001ebc00, 0x34edc0e, 0x4, 0xb297a3bc0ecadaa0, 0x0, 0x0, 0x0)
	/src/loki/pkg/ingester/flush.go:224 +0x274
github.com/grafana/loki/pkg/ingester.(*Ingester).flushLoop(0xc0001ebc00, 0x0)
	/src/loki/pkg/ingester/flush.go:196 +0x304
created by github.com/grafana/loki/pkg/ingester.(*Ingester).starting
	/src/loki/pkg/ingester/ingester.go:168 +0x68

Please use 1.4.1 for now, This seems to be a master issue.

/cc @slim-bean @owen-d

[cyriltovena]

Fixed by #1987