Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Crashes on boot #281

Closed
ViRb3 opened this issue Jun 5, 2022 · 37 comments
Closed

[BUG] Crashes on boot #281

ViRb3 opened this issue Jun 5, 2022 · 37 comments
Labels
bug Something isn't working

Comments

@ViRb3
Copy link

ViRb3 commented Jun 5, 2022

Describe the bug
After updating to 0.4.9-omnibus, scrutiny can no longer boot - it always crashes as it starts up.

System

  • Ubuntu 22.04 arm64
  • Docker Compose
  scrutiny:
    container_name: scrutiny
    image: ghcr.io/analogj/scrutiny:v0.4.9-omnibus
    privileged: true
    volumes:
      - /run/udev:/run/udev:ro
      - /dev:/dev
      - scrutiny-config:/opt/scrutiny/config
      - scrutiny-db:/opt/scrutiny/influxdb
    networks:
      - scrutiny-nginx

Log Files

 ___   ___  ____  __  __  ____  ____  _  _  _  _
/ __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
\__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
(___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
github.com/AnalogJ/scrutiny                             dev-0.4.9

Start the scrutiny server
time="2022-06-05T18:02:42Z" level=info msg="Successfully connected to scrutiny sqlite db: /opt/scrutiny/config/scrutiny.db\n"
panic: failed to check influxdb setup status - Get "http://localhost:8086/api/v2/setup": dial tcp: lookup localhost: device or resource busy

goroutine 1 [running]:
github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x103a6c0, 0x4000010ce8}, {0x1043620, 0x4000430070})
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:14 +0xd4
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0x400042a630, {0x1043620, 0x4000430070})
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0x90
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0x400042a630)
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x530
main.main.func2(0x40003ffcc0)
	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x288
github.com/urfave/cli/v2.(*Command).Run(0x400042e120, 0x40003ffb40)
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x648
github.com/urfave/cli/v2.(*App).RunContext(0x40002fc480, {0x1026870, 0x400003a028}, {0x4000032060, 0x2, 0x2})
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0x840
github.com/urfave/cli/v2.(*App).Run(...)
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215
main.main()
	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x73c
@ViRb3 ViRb3 added the bug Something isn't working label Jun 5, 2022
@AnalogJ
Copy link
Owner

AnalogJ commented Jun 6, 2022

can you try running without --privileged and -v /dev:/dev?
Instead just start the container with --device flags?

@ViRb3
Copy link
Author

ViRb3 commented Jun 6, 2022

Just tried, exactly same error:

  scrutiny:
    container_name: scrutiny
    image: ghcr.io/analogj/scrutiny:v0.4.9-omnibus
    volumes:
      - /run/udev:/run/udev:ro
      - scrutiny-config:/opt/scrutiny/config
      - scrutiny-db:/opt/scrutiny/influxdb
    networks:
      - scrutiny-nginx
    devices:
      - "/dev/sda"
      - "/dev/sdb"
      - "/dev/sdc"
      - "/dev/sdd"
      - "/dev/sde"
      - "/dev/sdf"

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 6, 2022

can you paste the contents of your scrutiny config file(s)?

@ViRb3
Copy link
Author

ViRb3 commented Jun 6, 2022

I don't have a config, so it should be using all defaults.

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 6, 2022

I'm unable to replicate this issue locally, and it seems like this isn't a problem for most of my users. I wonder if this is an ARM issue. Investigating.

Could be similar to #282

@ViRb3
Copy link
Author

ViRb3 commented Jun 6, 2022

#282 is a dupe of #278 I think, but I stopped getting that error and now i get this one.

I can test scrutiny on another arm64 system tonight.

@agneevX
Copy link

agneevX commented Jun 6, 2022

Repro on arm64 here.

@feroxy
Copy link

feroxy commented Jun 6, 2022

Also experiencing this with 0.4.9 omnibus. No config set, all defaults, x86 here.
scrutiny_0.4.9.log

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 8, 2022

Sorry everyone, this is incredibly difficult to debug without a way to reproduce.

So there's 2 things that I'm currently investigating as possible root-causes:

  • the go binaries are compiled in a golang:1.17.10-buster image while the runtime container is based on ubuntu:latest. While the binaries are supposed to be statically compiled, IIRC they still use the system glibc, which will be different in different OS versions. The weird thing is that an issue with incompatible glibc versions should cause issues for everyone, not just some users.
  • It could be related to static compilation + CGO + DNS: https://engineering.grab.com/dns-resolution-in-go-and-cgo

While I investigate further, can those of you running into this issue run the following command and paste the output as a comment: docker info

Thanks for your patience!

@rshxyz
Copy link

rshxyz commented Jun 8, 2022

@AnalogJ

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.5.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 3
 Server Version: 20.10.16
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc version: v1.1.1-0-g52de29d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.17.11-200.fc35.x86_64
 Operating System: Fedora Linux 35 (Workstation Edition)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 31.33GiB
 Name: fedora
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 8, 2022

I updated the beta branch to use debian:bullseye consistently during compilation and runtime. (Potential solution 1)

Can you try using ghcr.io/analogj/scrutiny:beta-omnibus in your docker-compose/docker run command, and report back?

@rshxyz
Copy link

rshxyz commented Jun 8, 2022

That worked, here is the log, didn't include all the info. I can also reach the webapp GUI and the influx db GUI asks for a username and password.

scrutiny-omnibus    | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
scrutiny-omnibus    | [s6-init] ensuring user provided files have correct perms...exited 0.
scrutiny-omnibus    | [fix-attrs.d] applying ownership & permissions fixes...
scrutiny-omnibus    | [fix-attrs.d] done.
scrutiny-omnibus    | [cont-init.d] executing container initialization scripts...
scrutiny-omnibus    | [cont-init.d] 01-timezone: executing... 
scrutiny-omnibus    | [cont-init.d] 01-timezone: exited 0.
scrutiny-omnibus    | [cont-init.d] 50-cron-config: executing... 
scrutiny-omnibus    | [cont-init.d] 50-cron-config: exited 0.
scrutiny-omnibus    | [cont-init.d] done.
scrutiny-omnibus    | [services.d] starting services
scrutiny-omnibus    | waiting for influxdb
scrutiny-omnibus    | starting cron
scrutiny-omnibus    | waiting for scrutiny service to start
scrutiny-omnibus    | [services.d] done.
scrutiny-omnibus    | starting influxdb
scrutiny-omnibus    | influxdb not ready
scrutiny-omnibus    | scrutiny api not ready
scrutiny-omnibus    | ts=2022-06-08T05:53:53.313870Z lvl=info msg="Welcome to InfluxDB" log_id=0axTs6dG000 version=v2.2.0 commit=a2f8538837 build_date=2022-04-06T17:36:40Z
scrutiny-omnibus    | ts=2022-06-08T05:53:53.316487Z lvl=info msg="Resources opened" log_id=0axTs6dG000 service=bolt path=/opt/scrutiny/influxdb/influxd.bolt
scrutiny-omnibus    | ts=2022-06-08T05:53:53.316528Z lvl=info msg="Resources opened" log_id=0axTs6dG000 service=sqlite path=/opt/scrutiny/influxdb/influxd.sqlite
scrutiny-omnibus    | ts=2022-06-08T05:53:53.317131Z lvl=info msg="Bringing up metadata migrations" log_id=0axTs6dG000 service="KV migrations" migration_count=19
scrutiny-omnibus    | ts=2022-06-08T05:53:53.370563Z lvl=info msg="Bringing up metadata migrations" log_id=0axTs6dG000 service="SQL migrations" migration_count=5
scrutiny-omnibus    | ts=2022-06-08T05:53:53.380089Z lvl=info msg="Using data dir" log_id=0axTs6dG000 service=storage-engine service=store path=/opt/scrutiny/influxdb/engine/data
scrutiny-omnibus    | ts=2022-06-08T05:53:53.380174Z lvl=info msg="Compaction settings" log_id=0axTs6dG000 service=storage-engine service=store max_concurrent_compactions=6 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
scrutiny-omnibus    | ts=2022-06-08T05:53:53.380184Z lvl=info msg="Open store (start)" log_id=0axTs6dG000 service=storage-engine service=store op_name=tsdb_open op_event=start
scrutiny-omnibus    | ts=2022-06-08T05:53:53.380221Z lvl=info msg="Open store (end)" log_id=0axTs6dG000 service=storage-engine service=store op_name=tsdb_open op_event=end op_elapsed=0.038ms
scrutiny-omnibus    | ts=2022-06-08T05:53:53.380241Z lvl=info msg="Starting retention policy enforcement service" log_id=0axTs6dG000 service=retention check_interval=30m
scrutiny-omnibus    | ts=2022-06-08T05:53:53.380255Z lvl=info msg="Starting precreation service" log_id=0axTs6dG000 service=shard-precreation check_interval=10m advance_period=30m
scrutiny-omnibus    | ts=2022-06-08T05:53:53.380786Z lvl=info msg="Starting query controller" log_id=0axTs6dG000 service=storage-reads concurrency_quota=1024 initial_memory_bytes_quota_per_query=9223372036854775807 memory_bytes_quota_per_query=9223372036854775807 max_memory_bytes=0 queue_size=1024
scrutiny-omnibus    | ts=2022-06-08T05:53:53.381866Z lvl=info msg="Configuring InfluxQL statement executor (zeros indicate unlimited)." log_id=0axTs6dG000 max_select_point=0 max_select_series=0 max_select_buckets=0
scrutiny-omnibus    | ts=2022-06-08T05:53:53.384861Z lvl=info msg=Listening log_id=0axTs6dG000 service=tcp-listener transport=http addr=:8086 port=8086
scrutiny-omnibus    | scrutiny api not ready
scrutiny-omnibus    | starting scrutiny
scrutiny-omnibus    | 2022/06/08 05:53:58 No configuration file found at /opt/scrutiny/config/scrutiny.yaml. Using Defaults.
scrutiny-omnibus    | 
scrutiny-omnibus    |  ___   ___  ____  __  __  ____  ____  _  _  _  _
scrutiny-omnibus    | / __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
scrutiny-omnibus    | \__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
scrutiny-omnibus    | (___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
scrutiny-omnibus    | github.com/AnalogJ/scrutiny                             dev-0.4.9
scrutiny-omnibus    | 
scrutiny-omnibus    | Start the scrutiny server
scrutiny-omnibus    | [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
scrutiny-omnibus    |  - using env:	export GIN_MODE=release
scrutiny-omnibus    |  - using code:	gin.SetMode(gin.ReleaseMode)
scrutiny-omnibus    | 
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=info msg="Trying to connect to scrutiny sqlite db: /opt/scrutiny/config/scrutiny.db\n"
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=info msg="Successfully connected to scrutiny sqlite db: /opt/scrutiny/config/scrutiny.db\n"
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=debug msg="InfluxDB url: http://localhost:8086"
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=debug msg="Determine Influxdb setup status..."
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=debug msg="Influxdb un-initialized, running first-time setup..."
scrutiny-omnibus    | fatal error: unexpected signal during runtime execution
scrutiny-omnibus    | [signal SIGSEGV: segmentation violation code=0x1 addr=0x63 pc=0x7f89941d8298]
scrutiny-omnibus    | 
scrutiny-omnibus    | runtime stack:
scrutiny-omnibus    | runtime.throw({0xf8c76b, 0x7f8970000d53})
scrutiny-omnibus    | 	/usr/local/go/src/runtime/panic.go:1198 +0x71
scrutiny-omnibus    | runtime.sigpanic()
scrutiny-omnibus    | 	/usr/local/go/src/runtime/signal_unix.go:719 +0x396
scrutiny-omnibus    | 
scrutiny-omnibus    | goroutine 36 [syscall]:
scrutiny-omnibus    | runtime.cgocall(0xc0bb10, 0xc000060590)
scrutiny-omnibus    | 	/usr/local/go/src/runtime/cgocall.go:156 +0x5c fp=0xc000060568 sp=0xc000060530 pc=0x4056dc
scrutiny-omnibus    | net._C2func_getaddrinfo(0xc00011a4e0, 0x0, 0xc000110f00, 0xc000118120)
scrutiny-omnibus    | 	_cgo_gotypes.go:91 +0x56 fp=0xc000060590 sp=0xc000060568 pc=0x58ce36
scrutiny-omnibus    | net.cgoLookupIPCNAME.func1({0xc00011a4e0, 0x0, 0x0}, 0xc00011a4c0, 0xc000060650)
scrutiny-omnibus    | 	/usr/local/go/src/net/cgo_unix.go:163 +0x9f fp=0xc0000605e8 sp=0xc000060590 pc=0x58e6ff
scrutiny-omnibus    | net.cgoLookupIPCNAME({0xf6a57b, 0x3}, {0xc00011a4c0, 0x4})
scrutiny-omnibus    | 	/usr/local/go/src/net/cgo_unix.go:163 +0x16d fp=0xc000060738 sp=0xc0000605e8 pc=0x58df4d
scrutiny-omnibus    | net.cgoIPLookup(0x16d0f30, {0xf6a57b, 0xc00011a4d0}, {0xc00011a4c0, 0xc00015e000})
scrutiny-omnibus    | 	/usr/local/go/src/net/cgo_unix.go:220 +0x3b fp=0xc0000607a8 sp=0xc000060738 pc=0x58e7bb
scrutiny-omnibus    | net.cgoLookupIP·dwrap·25()
scrutiny-omnibus    | 	/usr/local/go/src/net/cgo_unix.go:230 +0x36 fp=0xc0000607e0 sp=0xc0000607a8 pc=0x58ec36
scrutiny-omnibus    | runtime.goexit()
scrutiny-omnibus    | 	/usr/local/go/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc0000607e8 sp=0xc0000607e0 pc=0x465f41
scrutiny-omnibus    | created by net.cgoLookupIP
scrutiny-omnibus    | 	/usr/local/go/src/net/cgo_unix.go:230 +0x125
scrutiny-omnibus    | 
scrutiny-omnibus    | goroutine 1 [select]:
scrutiny-omnibus    | net/http.(*Transport).getConn(0xc000422f00, 0xc00011c180, {{}, 0x0, {0xc0001121e0, 0x4}, {0xc00011a4c0, 0xe}, 0x0})
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1372 +0x5d2
scrutiny-omnibus    | net/http.(*Transport).roundTrip(0xc000422f00, 0xc00015c200)
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:581 +0x774
scrutiny-omnibus    | net/http.(*Transport).RoundTrip(0xc00015c200, 0x10a4c20)
scrutiny-omnibus    | 	/usr/local/go/src/net/http/roundtrip.go:18 +0x19
scrutiny-omnibus    | net/http.send(0xc00015c100, {0x10a4c20, 0xc000422f00}, {0xf3f3a0, 0x4c3101, 0x16d1f40})
scrutiny-omnibus    | 	/usr/local/go/src/net/http/client.go:252 +0x5d8
scrutiny-omnibus    | net/http.(*Client).send(0xc0004928d0, 0xc00015c100, {0xc000068800, 0xc000110cf0, 0x16d1f40})
scrutiny-omnibus    | 	/usr/local/go/src/net/http/client.go:176 +0x9b
scrutiny-omnibus    | net/http.(*Client).do(0xc0004928d0, 0xc00015c100)
scrutiny-omnibus    | 	/usr/local/go/src/net/http/client.go:725 +0x908
scrutiny-omnibus    | net/http.(*Client).Do(0xee3f00, 0xc000110cc0)
scrutiny-omnibus    | 	/usr/local/go/src/net/http/client.go:593 +0x19
scrutiny-omnibus    | github.com/influxdata/influxdb-client-go/v2/api/http.(*service).DoHTTPRequestWithResponse(0xc000490400, 0xc00015c100, 0x0)
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/http/service.go:136 +0x343
scrutiny-omnibus    | github.com/influxdata/influxdb-client-go/v2/domain.(*Client).PostSetup(0xc000405c60, {0x10b84b0, 0xc000036080}, 0xe0bd01, {{0xf6e249, 0x7}, {0xf6f6b2, 0x8}, 0xc000114210, 0xc00011a2c0, ...})
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:2731 +0x275
scrutiny-omnibus    | github.com/influxdata/influxdb-client-go/v2/domain.(*ClientWithResponses).PostSetupWithResponse(0xe0bda0, {0x10b84b0, 0xc000036080}, 0xe66340, {{0xf6e249, 0x7}, {0xf6f6b2, 0x8}, 0xc000114210, 0xc00011a2c0, ...})
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:23859 +0x58
scrutiny-omnibus    | github.com/influxdata/influxdb-client-go/v2.(*clientImpl).SetupWithToken(0xc0004500c0, {0x10b84b0, 0xc000036080}, {0xf6b6a9, 0x5}, {0xf73411, 0xd}, {0xf6f6b2, 0x8}, {0xf6e249, ...}, ...)
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/client.go:191 +0x371
scrutiny-omnibus    | github.com/analogj/scrutiny/webapp/backend/pkg/database.NewScrutinyRepository({0x10ce1e0, 0xc000406088}, {0x10d78b0, 0xc000472460})
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/database/scrutiny_repository.go:102 +0x751
scrutiny-omnibus    | github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x10ce1e0, 0xc000406088}, {0x10d78b0, 0xc000472460})
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:12 +0x31
scrutiny-omnibus    | github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0xc000405210, {0x10d78b0, 0xc000472460})
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0xb4
scrutiny-omnibus    | github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0xc000405210)
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x46b
scrutiny-omnibus    | main.main.func2(0xc000413280)
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x1f7
scrutiny-omnibus    | github.com/urfave/cli/v2.(*Command).Run(0xc000478240, 0xc000413100)
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x64a
scrutiny-omnibus    | github.com/urfave/cli/v2.(*App).RunContext(0xc00047a000, {0x10b84b0, 0xc000036080}, {0xc000032040, 0x2, 0x2})
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0x926
scrutiny-omnibus    | github.com/urfave/cli/v2.(*App).Run(...)
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215
scrutiny-omnibus    | main.main()
scrutiny-omnibus    | 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x679
scrutiny-omnibus    | 
scrutiny-omnibus    | goroutine 18 [select]:
scrutiny-omnibus    | database/sql.(*DB).connectionOpener(0xc0004169c0, {0x10b8478, 0xc0004901c0})
scrutiny-omnibus    | 	/usr/local/go/src/database/sql/sql.go:1196 +0x93
scrutiny-omnibus    | created by database/sql.OpenDB
scrutiny-omnibus    | 	/usr/local/go/src/database/sql/sql.go:794 +0x188
scrutiny-omnibus    | 
scrutiny-omnibus    | goroutine 23 [IO wait]:
scrutiny-omnibus    | internal/poll.runtime_pollWait(0x7f89941eff08, 0x72)
scrutiny-omnibus    | 	/usr/local/go/src/runtime/netpoll.go:303 +0x85
scrutiny-omnibus    | internal/poll.(*pollDesc).wait(0xc00040d180, 0xc00047d000, 0x0)
scrutiny-omnibus    | 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32
scrutiny-omnibus    | internal/poll.(*pollDesc).waitRead(...)
scrutiny-omnibus    | 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
scrutiny-omnibus    | internal/poll.(*FD).Read(0xc00040d180, {0xc00047d000, 0x1000, 0x1000})
scrutiny-omnibus    | 	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a
scrutiny-omnibus    | net.(*netFD).Read(0xc00040d180, {0xc00047d000, 0x438827, 0xc000077c30})
scrutiny-omnibus    | 	/usr/local/go/src/net/fd_posix.go:56 +0x29
scrutiny-omnibus    | net.(*conn).Read(0xc000406418, {0xc00047d000, 0xe, 0xc0000001a0})
scrutiny-omnibus    | 	/usr/local/go/src/net/net.go:183 +0x45
scrutiny-omnibus    | net/http.(*persistConn).Read(0xc000478c60, {0xc00047d000, 0xc00003e5a0, 0xc000077d30})
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1926 +0x4e
scrutiny-omnibus    | bufio.(*Reader).fill(0xc000400fc0)
scrutiny-omnibus    | 	/usr/local/go/src/bufio/bufio.go:101 +0x103
scrutiny-omnibus    | bufio.(*Reader).Peek(0xc000400fc0, 0x1)
scrutiny-omnibus    | 	/usr/local/go/src/bufio/bufio.go:139 +0x5d
scrutiny-omnibus    | net/http.(*persistConn).readLoop(0xc000478c60)
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:2087 +0x1ac
scrutiny-omnibus    | created by net/http.(*Transport).dialConn
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1747 +0x1e05
scrutiny-omnibus    | 
scrutiny-omnibus    | goroutine 24 [select]:
scrutiny-omnibus    | net/http.(*persistConn).writeLoop(0xc000478c60)
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:2386 +0xfb
scrutiny-omnibus    | created by net/http.(*Transport).dialConn
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1748 +0x1e65
scrutiny-omnibus    | 
scrutiny-omnibus    | goroutine 34 [select]:
scrutiny-omnibus    | net.(*Resolver).lookupIPAddr(0x16d0f20, {0x10b84e8, 0xc00010e8a0}, {0xf6a57b, 0xc000138140}, {0xc00011a4c0, 0x9})
scrutiny-omnibus    | 	/usr/local/go/src/net/lookup.go:302 +0x5c7
scrutiny-omnibus    | net.(*Resolver).internetAddrList(0x10b84e8, {0x10b84e8, 0xc00010e8a0}, {0xf6a57b, 0x3}, {0xc00011a4c0, 0xe})
scrutiny-omnibus    | 	/usr/local/go/src/net/ipsock.go:288 +0x67a
scrutiny-omnibus    | net.(*Resolver).resolveAddrList(0x170a0c0, {0x10b84e8, 0xc00010e8a0}, {0xf6ac29, 0x4}, {0xf6a57b, 0x0}, {0xc00011a4c0, 0xe}, {0x0, ...})
scrutiny-omnibus    | 	/usr/local/go/src/net/dial.go:221 +0x41b
scrutiny-omnibus    | net.(*Dialer).DialContext(0xc000400a80, {0x10b84e8, 0xc00010e6c0}, {0xf6a57b, 0x7f89bbf0ae20}, {0xc00011a4c0, 0x118})
scrutiny-omnibus    | 	/usr/local/go/src/net/dial.go:406 +0x448
scrutiny-omnibus    | net/http.(*Transport).dial(0xc00010e6c0, {0x10b84e8, 0xc00010e6c0}, {0xf6a57b, 0x0}, {0xc00011a4c0, 0x0})
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1166 +0xda
scrutiny-omnibus    | net/http.(*Transport).dialConn(0xc000422f00, {0x10b84e8, 0xc00010e6c0}, {{}, 0x0, {0xc0001121e0, 0x4}, {0xc00011a4c0, 0xe}, 0x0})
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1604 +0x845
scrutiny-omnibus    | net/http.(*Transport).dialConnFor(0x712fc6, 0xc00015e000)
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1446 +0xb0
scrutiny-omnibus    | created by net/http.(*Transport).queueForDial
scrutiny-omnibus    | 	/usr/local/go/src/net/http/transport.go:1415 +0x3d7
scrutiny-omnibus    | 
scrutiny-omnibus    | goroutine 35 [select]:
scrutiny-omnibus    | net.cgoLookupIP({0x10b8478, 0xc00011c1c0}, {0xf6a57b, 0x9}, {0xc00011a4c0, 0xe})
scrutiny-omnibus    | 	/usr/local/go/src/net/cgo_unix.go:231 +0x1b7
scrutiny-omnibus    | net.(*Resolver).lookupIP(0x16d0f20, {0x10b8478, 0xc00011c1c0}, {0xf6a57b, 0x3}, {0xc00011a4c0, 0x9})
scrutiny-omnibus    | 	/usr/local/go/src/net/lookup_unix.go:97 +0x128
scrutiny-omnibus    | net.glob..func1({0x10b8478, 0xc00011c1c0}, 0x0, {0xf6a57b, 0xc0000607b8}, {0xc00011a4c0, 0x7100f0})
scrutiny-omnibus    | 	/usr/local/go/src/net/hook.go:23 +0x3d
scrutiny-omnibus    | net.(*Resolver).lookupIPAddr.func1()
scrutiny-omnibus    | 	/usr/local/go/src/net/lookup.go:296 +0x9f
scrutiny-omnibus    | internal/singleflight.(*Group).doCall(0x16d0f30, 0xc000138190, {0xc00011a4d0, 0xd}, 0xc00015e000)
scrutiny-omnibus    | 	/usr/local/go/src/internal/singleflight/singleflight.go:95 +0x3b
scrutiny-omnibus    | created by internal/singleflight.(*Group).DoChan
scrutiny-omnibus    | 	/usr/local/go/src/internal/singleflight/singleflight.go:88 +0x2f1
scrutiny-omnibus    | waiting for influxdb
scrutiny-omnibus    | starting scrutiny
scrutiny-omnibus    | 2022/06/08 05:53:58 No configuration file found at /opt/scrutiny/config/scrutiny.yaml. Using Defaults.
scrutiny-omnibus    | 
scrutiny-omnibus    |  ___   ___  ____  __  __  ____  ____  _  _  _  _
scrutiny-omnibus    | / __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
scrutiny-omnibus    | \__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
scrutiny-omnibus    | (___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
scrutiny-omnibus    | github.com/AnalogJ/scrutiny                             dev-0.4.9
scrutiny-omnibus    | 
scrutiny-omnibus    | Start the scrutiny server
scrutiny-omnibus    | [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
scrutiny-omnibus    |  - using env:	export GIN_MODE=release
scrutiny-omnibus    |  - using code:	gin.SetMode(gin.ReleaseMode)
scrutiny-omnibus    | 
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=info msg="Trying to connect to scrutiny sqlite db: /opt/scrutiny/config/scrutiny.db\n"
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=info msg="Successfully connected to scrutiny sqlite db: /opt/scrutiny/config/scrutiny.db\n"
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=debug msg="InfluxDB url: http://localhost:8086"
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=debug msg="Determine Influxdb setup status..."
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=debug msg="Influxdb un-initialized, running first-time setup..."
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=info msg="Database migration starting. Please wait, this process may take a long time...."
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=info msg="Database migration completed successfully"
scrutiny-omnibus    | time="2022-06-08T05:53:58Z" level=debug msg="basepath: "
scrutiny-omnibus    | [GIN-debug] GET    /api/health               --> github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup.func1 (5 handlers)
scrutiny-omnibus    | [GIN-debug] POST   /api/health/notify        --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.SendTestNotification (5 handlers)
scrutiny-omnibus    | [GIN-debug] POST   /api/devices/register     --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.RegisterDevices (5 handlers)
scrutiny-omnibus    | [GIN-debug] GET    /api/summary              --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.GetDevicesSummary (5 handlers)
scrutiny-omnibus    | [GIN-debug] GET    /api/summary/temp         --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.GetDevicesSummaryTempHistory (5 handlers)
scrutiny-omnibus    | [GIN-debug] POST   /api/device/:wwn/smart    --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.UploadDeviceMetrics (5 handlers)
scrutiny-omnibus    | [GIN-debug] POST   /api/device/:wwn/selftest --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.UploadDeviceSelfTests (5 handlers)
scrutiny-omnibus    | [GIN-debug] GET    /api/device/:wwn/details  --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.GetDeviceDetails (5 handlers)
scrutiny-omnibus    | [GIN-debug] DELETE /api/device/:wwn          --> github.com/analogj/scrutiny/webapp/backend/pkg/web/handler.DeleteDevice (5 handlers)
scrutiny-omnibus    | [GIN-debug] GET    /web/*filepath            --> github.com/gin-gonic/gin.(*RouterGroup).createStaticHandler.func1 (5 handlers)
scrutiny-omnibus    | [GIN-debug] HEAD   /web/*filepath            --> github.com/gin-gonic/gin.(*RouterGroup).createStaticHandler.func1 (5 handlers)
scrutiny-omnibus    | [GIN-debug] GET    /                         --> github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup.func2 (5 handlers)
scrutiny-omnibus    | [GIN-debug] Listening and serving HTTP on 0.0.0.0:8080
scrutiny-omnibus    | time="2022-06-08T05:54:03Z" level=info msg="127.0.0.1 - scrutiny-omnibus [08/Jun/2022:05:54:03 +0000] \"HEAD /api/health\" 200 0 \"\" \"curl/7.74.0\" (6ms)" clientIP=127.0.0.1 hostname=scrutiny-omnibus latency=6 method=HEAD path=/api/health referer= respLength=0 statusCode=200 userAgent=curl/7.74.0
scrutiny-omnibus    | time="2022-06-08T05:54:03Z" level=debug bodyType=response clientIP=127.0.0.1 hostname=scrutiny-omnibus latency=6 method=HEAD path=/api/health referer= respLength=0 statusCode=200 userAgent=curl/7.74.0
scrutiny-omnibus    | starting scrutiny collector (run-once mode. subsequent calls will be triggered via cron service)
scrutiny-omnibus    | 2022/06/08 05:54:03 No configuration file found at /opt/scrutiny/config/collector.yaml. Using Defaults.
scrutiny-omnibus    | 
scrutiny-omnibus    |  ___   ___  ____  __  __  ____  ____  _  _  _  _
scrutiny-omnibus    | / __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
scrutiny-omnibus    | \__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
scrutiny-omnibus    | (___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
scrutiny-omnibus    | AnalogJ/scrutiny/metrics                                dev-0.4.9
scrutiny-omnibus    | 
scrutiny-omnibus    | time="2022-06-08T05:54:03Z" level=info msg="Verifying required tools" type=metrics
scrutiny-omnibus    | time="2022-06-08T05:54:03Z" level=info msg="Executing command: smartctl --scan --json" type=metrics

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 8, 2022

Great! 🎉 It seems like this may have all been caused by the mismatch between the compilation image and the runtime image (different glibc versions).
I'm going to wait for a couple of other users to try out the beta image and report back, but after that I can merge the changes into master 👍

Thanks for your help!

@ViRb3
Copy link
Author

ViRb3 commented Jun 8, 2022

Confirming fixed with beta, but #278 returned now. Same as two comments above.

@feroxy
Copy link

feroxy commented Jun 8, 2022

I updated the beta branch to use debian:bullseye consistently during compilation and runtime. (Potential solution 1)

Can you try using ghcr.io/analogj/scrutiny:beta-omnibus in your docker-compose/docker run command, and report back?

This restores things to a working state for me

@agneevX
Copy link

agneevX commented Jun 8, 2022

This does not work for me.

Expand for logs

ghcr.io/analogj/scrutiny:beta-omnibus

Container log

2022-06-08T16:00:25.442646269Z [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
2022-06-08T16:00:25.994034155Z [s6-init] ensuring user provided files have correct perms...exited 0.
2022-06-08T16:00:26.010814942Z [fix-attrs.d] applying ownership & permissions fixes...
2022-06-08T16:00:26.021550517Z [fix-attrs.d] done.
2022-06-08T16:00:26.033484738Z [cont-init.d] executing container initialization scripts...
2022-06-08T16:00:26.044495013Z [cont-init.d] 01-timezone: executing... 
2022-06-08T16:00:26.062240616Z [cont-init.d] 01-timezone: exited 0.
2022-06-08T16:00:26.067823076Z [cont-init.d] 50-cron-config: executing... 
2022-06-08T16:00:26.095929317Z [cont-init.d] 50-cron-config: exited 0.
2022-06-08T16:00:26.099444053Z [cont-init.d] done.
2022-06-08T16:00:26.111921006Z [services.d] starting services
2022-06-08T16:00:26.157791461Z waiting for influxdb
2022-06-08T16:00:26.159618356Z waiting for scrutiny service to start
2022-06-08T16:00:26.170442893Z starting cron
2022-06-08T16:00:26.184983477Z influxdb config file already exists. skipping.
2022-06-08T16:00:26.185055735Z starting influxdb
2022-06-08T16:00:26.223608704Z influxdb not ready
2022-06-08T16:00:26.232649901Z scrutiny api not ready
2022-06-08T16:00:26.246227058Z [services.d] done.
2022-06-08T16:00:27.080953021Z ts=2022-06-08T16:00:27.079794Z lvl=info msg="Welcome to InfluxDB" log_id=0ay0_Pml000 version=v2.2.0 commit=a2f8538837 build_date=2022-04-06T17:36:38Z
2022-06-08T16:00:27.083807343Z ts=2022-06-08T16:00:27.083496Z lvl=info msg="Resources opened" log_id=0ay0_Pml000 service=bolt path=/opt/scrutiny/influxdb/influxd.bolt
2022-06-08T16:00:27.084148670Z ts=2022-06-08T16:00:27.083902Z lvl=info msg="Resources opened" log_id=0ay0_Pml000 service=sqlite path=/opt/scrutiny/influxdb/influxd.sqlite
2022-06-08T16:00:27.097214076Z ts=2022-06-08T16:00:27.096407Z lvl=info msg="Checking InfluxDB metadata for prior version." log_id=0ay0_Pml000 bolt_path=/opt/scrutiny/influxdb/influxd.bolt
2022-06-08T16:00:27.097295408Z ts=2022-06-08T16:00:27.096779Z lvl=info msg="Using data dir" log_id=0ay0_Pml000 service=storage-engine service=store path=/opt/scrutiny/influxdb/engine/data
2022-06-08T16:00:27.097316926Z ts=2022-06-08T16:00:27.096915Z lvl=info msg="Compaction settings" log_id=0ay0_Pml000 service=storage-engine service=store max_concurrent_compactions=2 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
2022-06-08T16:00:27.097337352Z ts=2022-06-08T16:00:27.096959Z lvl=info msg="Open store (start)" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open op_event=start
2022-06-08T16:00:27.166460039Z ts=2022-06-08T16:00:27.166174Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.191171171Z ts=2022-06-08T16:00:27.189008Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/1d7b14b7fd72697a/autogen/26/000000001-000000001.tsm id=0 duration=2.252ms
2022-06-08T16:00:27.218268207Z ts=2022-06-08T16:00:27.217884Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.244109431Z ts=2022-06-08T16:00:27.242312Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/1d7b14b7fd72697a/autogen/2/000000001-000000001.tsm id=0 duration=2.485ms
2022-06-08T16:00:27.318946983Z ts=2022-06-08T16:00:27.318539Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.339621630Z ts=2022-06-08T16:00:27.339204Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/13/000000001-000000001.tsm id=0 duration=13.814ms
2022-06-08T16:00:27.349349778Z ts=2022-06-08T16:00:27.348952Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.359883968Z ts=2022-06-08T16:00:27.359220Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/1/000000001-000000001.tsm id=0 duration=2.169ms
2022-06-08T16:00:27.483321280Z ts=2022-06-08T16:00:27.482575Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/1 duration=225.314ms
2022-06-08T16:00:27.483431353Z ts=2022-06-08T16:00:27.482710Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/1d7b14b7fd72697a/autogen/2 duration=338.430ms
2022-06-08T16:00:27.483454111Z ts=2022-06-08T16:00:27.482575Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/1d7b14b7fd72697a/autogen/26 duration=337.667ms
2022-06-08T16:00:27.483473204Z ts=2022-06-08T16:00:27.482641Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/13 duration=225.377ms
2022-06-08T16:00:27.555598154Z ts=2022-06-08T16:00:27.555203Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.561650254Z ts=2022-06-08T16:00:27.561250Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.574078467Z ts=2022-06-08T16:00:27.572328Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/23/000000001-000000001.tsm id=0 duration=5.025ms
2022-06-08T16:00:27.574192058Z ts=2022-06-08T16:00:27.572433Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/23/000000002-000000001.tsm id=1 duration=5.035ms
2022-06-08T16:00:27.574215354Z ts=2022-06-08T16:00:27.573444Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/23 duration=90.572ms
2022-06-08T16:00:27.575433870Z ts=2022-06-08T16:00:27.574601Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/14/000000001-000000001.tsm id=0 duration=6.049ms
2022-06-08T16:00:27.577091916Z ts=2022-06-08T16:00:27.576698Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/14 duration=93.924ms
2022-06-08T16:00:27.577171951Z ts=2022-06-08T16:00:27.576858Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.600352536Z ts=2022-06-08T16:00:27.599845Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/3/000000001-000000001.tsm id=0 duration=15.513ms
2022-06-08T16:00:27.600936193Z ts=2022-06-08T16:00:27.600612Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/3 duration=117.873ms
2022-06-08T16:00:27.601273631Z ts=2022-06-08T16:00:27.600969Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.614904287Z ts=2022-06-08T16:00:27.614321Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/16/000000001-000000001.tsm id=0 duration=6.025ms
2022-06-08T16:00:27.616343559Z ts=2022-06-08T16:00:27.615612Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/16 duration=132.787ms
2022-06-08T16:00:27.640005783Z ts=2022-06-08T16:00:27.639609Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.655031267Z ts=2022-06-08T16:00:27.654552Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.665035910Z ts=2022-06-08T16:00:27.664360Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/27/000000001-000000001.tsm id=0 duration=1.924ms
2022-06-08T16:00:27.666499070Z ts=2022-06-08T16:00:27.665520Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/27 duration=88.668ms
2022-06-08T16:00:27.667629384Z ts=2022-06-08T16:00:27.666795Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/25/000000001-000000001.tsm id=0 duration=3.927ms
2022-06-08T16:00:27.679231926Z ts=2022-06-08T16:00:27.678831Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/25 duration=105.249ms
2022-06-08T16:00:27.687513877Z ts=2022-06-08T16:00:27.686966Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.690186165Z ts=2022-06-08T16:00:27.689720Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.699998478Z ts=2022-06-08T16:00:27.699205Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/6/000000001-000000001.tsm id=0 duration=2.103ms
2022-06-08T16:00:27.701407973Z ts=2022-06-08T16:00:27.700789Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/6 duration=100.033ms
2022-06-08T16:00:27.705744787Z ts=2022-06-08T16:00:27.704942Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/4/000000001-000000001.tsm id=0 duration=2.811ms
2022-06-08T16:00:27.708605590Z ts=2022-06-08T16:00:27.708084Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/4 duration=91.410ms
2022-06-08T16:00:27.734461684Z ts=2022-06-08T16:00:27.733962Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.744953653Z ts=2022-06-08T16:00:27.744647Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.753564635Z ts=2022-06-08T16:00:27.753134Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/7/000000001-000000001.tsm id=0 duration=2.877ms
2022-06-08T16:00:27.754097422Z ts=2022-06-08T16:00:27.753803Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/7 duration=74.177ms
2022-06-08T16:00:27.756724562Z ts=2022-06-08T16:00:27.756360Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/5/000000001-000000001.tsm id=0 duration=2.287ms
2022-06-08T16:00:27.762018990Z ts=2022-06-08T16:00:27.761617Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.763988975Z ts=2022-06-08T16:00:27.763401Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/5 duration=97.085ms
2022-06-08T16:00:27.779474340Z ts=2022-06-08T16:00:27.778718Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.780320325Z ts=2022-06-08T16:00:27.779820Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/10/000000001-000000001.tsm id=0 duration=3.131ms
2022-06-08T16:00:27.781322290Z ts=2022-06-08T16:00:27.780820Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/10 duration=79.070ms
2022-06-08T16:00:27.787042840Z ts=2022-06-08T16:00:27.786276Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/11/000000002-000000001.tsm id=1 duration=1.225ms
2022-06-08T16:00:27.789877810Z ts=2022-06-08T16:00:27.789356Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/11/000000001-000000001.tsm id=0 duration=1.696ms
2022-06-08T16:00:27.790255507Z ts=2022-06-08T16:00:27.789821Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.793073033Z ts=2022-06-08T16:00:27.792781Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.794216476Z ts=2022-06-08T16:00:27.793745Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/11 duration=85.535ms
2022-06-08T16:00:27.799859546Z ts=2022-06-08T16:00:27.799342Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/8/000000001-000000001.tsm id=0 duration=2.600ms
2022-06-08T16:00:27.803864848Z ts=2022-06-08T16:00:27.803544Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/8 duration=49.474ms
2022-06-08T16:00:27.805037198Z ts=2022-06-08T16:00:27.803034Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/9/000000001-000000001.tsm id=0 duration=2.063ms
2022-06-08T16:00:27.806864649Z ts=2022-06-08T16:00:27.806490Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/9 duration=42.833ms
2022-06-08T16:00:27.810394366Z ts=2022-06-08T16:00:27.809919Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.828178784Z ts=2022-06-08T16:00:27.827615Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/12/000000001-000000001.tsm id=0 duration=6.751ms
2022-06-08T16:00:27.834424029Z ts=2022-06-08T16:00:27.834020Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/45d3adcb018f45aa/autogen/12 duration=52.330ms
2022-06-08T16:00:27.846706985Z ts=2022-06-08T16:00:27.846225Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.853483276Z ts=2022-06-08T16:00:27.853163Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.857057937Z ts=2022-06-08T16:00:27.856744Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.864993376Z ts=2022-06-08T16:00:27.864570Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/15/000000001-000000001.tsm id=0 duration=1.885ms
2022-06-08T16:00:27.866460739Z ts=2022-06-08T16:00:27.866138Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/15 duration=59.525ms
2022-06-08T16:00:27.878787973Z ts=2022-06-08T16:00:27.878436Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.879910324Z ts=2022-06-08T16:00:27.879547Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/952187b03a0c36f1/autogen/21/000000001-000000001.tsm id=0 duration=9.249ms
2022-06-08T16:00:27.900029220Z ts=2022-06-08T16:00:27.899359Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/952187b03a0c36f1/autogen/20/000000002-000000001.tsm id=1 duration=25.734ms
2022-06-08T16:00:27.901260847Z ts=2022-06-08T16:00:27.899322Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/952187b03a0c36f1/autogen/20/000000001-000000001.tsm id=0 duration=24.555ms
2022-06-08T16:00:27.902700804Z ts=2022-06-08T16:00:27.900759Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/952187b03a0c36f1/autogen/21/000000002-000000001.tsm id=1 duration=29.717ms
2022-06-08T16:00:27.905499367Z ts=2022-06-08T16:00:27.904860Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/952187b03a0c36f1/autogen/21 duration=110.982ms
2022-06-08T16:00:27.906446332Z ts=2022-06-08T16:00:27.905943Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/952187b03a0c36f1/autogen/20 duration=100.574ms
2022-06-08T16:00:27.915374217Z ts=2022-06-08T16:00:27.914127Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/19/000000002-000000001.tsm id=1 duration=1.226ms
2022-06-08T16:00:27.919024321Z ts=2022-06-08T16:00:27.918528Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/19/000000001-000000001.tsm id=0 duration=1.444ms
2022-06-08T16:00:27.919338038Z ts=2022-06-08T16:00:27.919084Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/19 duration=84.914ms
2022-06-08T16:00:27.920446167Z ts=2022-06-08T16:00:27.920070Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.929818710Z ts=2022-06-08T16:00:27.929292Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/17/000000002-000000001.tsm id=1 duration=1.174ms
2022-06-08T16:00:27.932100486Z ts=2022-06-08T16:00:27.931524Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/17/000000001-000000001.tsm id=0 duration=1.628ms
2022-06-08T16:00:27.933428130Z ts=2022-06-08T16:00:27.932731Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/17 duration=66.456ms
2022-06-08T16:00:27.935472002Z ts=2022-06-08T16:00:27.934862Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.939037886Z ts=2022-06-08T16:00:27.938580Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.944624864Z ts=2022-06-08T16:00:27.944036Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/18/000000001-000000001.tsm id=0 duration=2.104ms
2022-06-08T16:00:27.944906841Z ts=2022-06-08T16:00:27.944590Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/18 duration=39.439ms
2022-06-08T16:00:27.945952601Z ts=2022-06-08T16:00:27.945578Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/22/000000002-000000001.tsm id=1 duration=1.355ms
2022-06-08T16:00:27.946833252Z ts=2022-06-08T16:00:27.946479Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/22/000000001-000000001.tsm id=0 duration=1.648ms
2022-06-08T16:00:27.947004879Z ts=2022-06-08T16:00:27.946606Z lvl=info msg="index opened with 8 partitions" log_id=0ay0_Pml000 service=storage-engine index=tsi
2022-06-08T16:00:27.947409705Z ts=2022-06-08T16:00:27.947177Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/22 duration=39.705ms
2022-06-08T16:00:27.951622300Z ts=2022-06-08T16:00:27.951297Z lvl=info msg="Opened file" log_id=0ay0_Pml000 service=storage-engine engine=tsm1 service=filestore path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/24/000000001-000000001.tsm id=0 duration=1.552ms
2022-06-08T16:00:27.951985553Z ts=2022-06-08T16:00:27.951790Z lvl=info msg="Opened shard" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/opt/scrutiny/influxdb/engine/data/b5e446e7f9aa37cb/autogen/24 duration=32.611ms
2022-06-08T16:00:27.953703116Z ts=2022-06-08T16:00:27.953460Z lvl=info msg="Open store (end)" log_id=0ay0_Pml000 service=storage-engine service=store op_name=tsdb_open op_event=end op_elapsed=856.503ms
2022-06-08T16:00:27.953775263Z ts=2022-06-08T16:00:27.953542Z lvl=info msg="Starting retention policy enforcement service" log_id=0ay0_Pml000 service=retention check_interval=30m
2022-06-08T16:00:27.953793966Z ts=2022-06-08T16:00:27.953574Z lvl=info msg="Starting precreation service" log_id=0ay0_Pml000 service=shard-precreation check_interval=10m advance_period=30m
2022-06-08T16:00:27.956327534Z ts=2022-06-08T16:00:27.956086Z lvl=info msg="Starting query controller" log_id=0ay0_Pml000 service=storage-reads concurrency_quota=1024 initial_memory_bytes_quota_per_query=9223372036854775807 memory_bytes_quota_per_query=9223372036854775807 max_memory_bytes=0 queue_size=1024
2022-06-08T16:00:28.002476596Z ts=2022-06-08T16:00:28.001331Z lvl=info msg="Configuring InfluxQL statement executor (zeros indicate unlimited)." log_id=0ay0_Pml000 max_select_point=0 max_select_series=0 max_select_buckets=0
2022-06-08T16:00:28.021729951Z ts=2022-06-08T16:00:28.021378Z lvl=info msg=Listening log_id=0ay0_Pml000 service=tcp-listener transport=http addr=:8086 port=8086
2022-06-08T16:00:31.266455185Z starting scrutiny
2022-06-08T16:00:31.267135711Z scrutiny api not ready
2022-06-08T16:00:31.294996012Z 2022/06/08 16:00:31 Loading configuration file: /opt/scrutiny/config/scrutiny.yaml
2022-06-08T16:00:31.296501357Z 
2022-06-08T16:00:31.296592004Z  ___   ___  ____  __  __  ____  ____  _  _  _  _
2022-06-08T16:00:31.296625262Z / __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
2022-06-08T16:00:31.296652873Z \__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
2022-06-08T16:00:31.296674817Z (___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
2022-06-08T16:00:31.296694150Z github.com/AnalogJ/scrutiny                             dev-0.4.9
2022-06-08T16:00:31.296713613Z 
2022-06-08T16:00:31.296732409Z Start the scrutiny server
2022-06-08T16:00:31.306435595Z fatal error: unexpected signal during runtime execution
2022-06-08T16:00:31.308612687Z [signal SIGSEGV: segmentation violation code=0x1 addr=0x62 pc=0xffff7d073c44]
2022-06-08T16:00:31.308710111Z 
2022-06-08T16:00:31.308735685Z runtime stack:
2022-06-08T16:00:31.308757036Z runtime.throw({0xf0f0f1, 0x2a})
2022-06-08T16:00:31.308798962Z 	/usr/local/go/src/runtime/panic.go:1198 +0x54
2022-06-08T16:00:31.308822758Z runtime.sigpanic()
2022-06-08T16:00:31.308843535Z 	/usr/local/go/src/runtime/signal_unix.go:719 +0x230
2022-06-08T16:00:31.308864479Z 
2022-06-08T16:00:31.308884997Z goroutine 36 [syscall]:
2022-06-08T16:00:31.308905978Z runtime.cgocall(0xbdb960, 0x40004b85a8)
2022-06-08T16:00:31.308928663Z 	/usr/local/go/src/runtime/cgocall.go:156 +0x50 fp=0x40004b8570 sp=0x40004b8530 pc=0x4039b0
2022-06-08T16:00:31.308950533Z net._C2func_getaddrinfo(0x4000480350, 0x0, 0x40004848a0, 0x400048a090)
2022-06-08T16:00:31.308972996Z 	_cgo_gotypes.go:91 +0x48 fp=0x40004b85a0 sp=0x40004b8570 pc=0x58eb68
2022-06-08T16:00:31.309084772Z net.cgoLookupIPCNAME.func1({0x4000480350, 0xa, 0xa}, 0x40004848a0, 0x400048a090)
2022-06-08T16:00:31.309141734Z 	/usr/local/go/src/net/cgo_unix.go:163 +0xa8 fp=0x40004b85f0 sp=0x40004b85a0 pc=0x590148
2022-06-08T16:00:31.309240991Z net.cgoLookupIPCNAME({0xeecfd2, 0x3}, {0x4000480330, 0x9})
2022-06-08T16:00:31.309287453Z 	/usr/local/go/src/net/cgo_unix.go:163 +0x150 fp=0x40004b8710 sp=0x40004b85f0 pc=0x58fa80
2022-06-08T16:00:31.309379026Z net.cgoIPLookup(0x4000494720, {0xeecfd2, 0x3}, {0x4000480330, 0x9})
2022-06-08T16:00:31.309446914Z 	/usr/local/go/src/net/cgo_unix.go:220 +0x40 fp=0x40004b87b0 sp=0x40004b8710 pc=0x5901d0
2022-06-08T16:00:31.309493487Z runtime.goexit()
2022-06-08T16:00:31.309548486Z 	/usr/local/go/src/runtime/asm_arm64.s:1133 +0x4 fp=0x40004b87b0 sp=0x40004b87b0 pc=0x468ef4
2022-06-08T16:00:31.309571541Z created by net.cgoLookupIP
2022-06-08T16:00:31.309621596Z 	/usr/local/go/src/net/cgo_unix.go:230 +0x98
2022-06-08T16:00:31.309644373Z 
2022-06-08T16:00:31.309665095Z goroutine 1 [select]:
2022-06-08T16:00:31.309787074Z net/http.(*Transport).getConn(0x400033aa00, 0x40004a60c0, {{}, 0x0, {0x4000496240, 0x4}, {0x4000480330, 0xe}, 0x0})
2022-06-08T16:00:31.309832944Z 	/usr/local/go/src/net/http/transport.go:1372 +0x4e8
2022-06-08T16:00:31.309886925Z net/http.(*Transport).roundTrip(0x400033aa00, 0x40004aa200)
2022-06-08T16:00:31.309946646Z 	/usr/local/go/src/net/http/transport.go:581 +0x7d4
2022-06-08T16:00:31.309969516Z net/http.(*Transport).RoundTrip(0x400033aa00, 0x40004aa200)
2022-06-08T16:00:31.310027385Z 	/usr/local/go/src/net/http/roundtrip.go:18 +0x30
2022-06-08T16:00:31.310634282Z net/http.send(0x40004aa100, {0x10282e0, 0x400033aa00}, {0xc0a04fccd234fc06, 0x4a9f83b3a, 0x1634cc0})
2022-06-08T16:00:31.310686985Z 	/usr/local/go/src/net/http/client.go:252 +0x5ac
2022-06-08T16:00:31.310713003Z net/http.(*Client).send(0x4000366510, 0x40004aa100, {0xc0a04fccd234fc06, 0x4a9f83b3a, 0x1634cc0})
2022-06-08T16:00:31.310736077Z 	/usr/local/go/src/net/http/client.go:176 +0xac
2022-06-08T16:00:31.310765872Z net/http.(*Client).do(0x4000366510, 0x40004aa100)
2022-06-08T16:00:31.310795946Z 	/usr/local/go/src/net/http/client.go:725 +0x888
2022-06-08T16:00:31.310827371Z net/http.(*Client).Do(0x4000366510, 0x40004aa100)
2022-06-08T16:00:31.310851649Z 	/usr/local/go/src/net/http/client.go:593 +0x30
2022-06-08T16:00:31.310872945Z github.com/influxdata/influxdb-client-go/v2/api/http.(*service).DoHTTPRequestWithResponse(0x4000362d80, 0x40004aa100, 0x0)
2022-06-08T16:00:31.310949073Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/http/service.go:136 +0x38c
2022-06-08T16:00:31.310982461Z github.com/influxdata/influxdb-client-go/v2/domain.(*Client).GetOrgs(0x40003578d0, {0x103b030, 0x40000c2000}, 0x40004a6040)
2022-06-08T16:00:31.311006831Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:2056 +0x288
2022-06-08T16:00:31.311352881Z github.com/influxdata/influxdb-client-go/v2/domain.(*ClientWithResponses).GetOrgsWithResponse(0x40003578e0, {0x103b030, 0x40000c2000}, 0x40004a6040)
2022-06-08T16:00:31.311453416Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:23204 +0x4c
2022-06-08T16:00:31.311481045Z github.com/influxdata/influxdb-client-go/v2/api.(*organizationsAPI).getOrganizations(0x400048a050, {0x103b030, 0x40000c2000}, 0x40004a6040, {0x0, 0x0, 0x0})
2022-06-08T16:00:31.311504471Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/organizations.go:84 +0x138
2022-06-08T16:00:31.311526841Z github.com/influxdata/influxdb-client-go/v2/api.(*organizationsAPI).FindOrganizationByName(0x400048a050, {0x103b030, 0x40000c2000}, {0xef2125, 0x8})
2022-06-08T16:00:31.311549470Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/organizations.go:100 +0xd8
2022-06-08T16:00:31.315449681Z github.com/analogj/scrutiny/webapp/backend/pkg/database.NewScrutinyRepository({0x104ee80, 0x40000becf0}, {0x1057de0, 0x4000361730})
2022-06-08T16:00:31.315596864Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/database/scrutiny_repository.go:139 +0xa94
2022-06-08T16:00:31.315649122Z github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x104ee80, 0x40000becf0}, {0x1057de0, 0x4000361730})
2022-06-08T16:00:31.315675548Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:12 +0x40
2022-06-08T16:00:31.315699455Z github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0x4000356f40, {0x1057de0, 0x4000361730})
2022-06-08T16:00:31.315722103Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0x90
2022-06-08T16:00:31.315743973Z github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0x4000356f40)
2022-06-08T16:00:31.315766009Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x530
2022-06-08T16:00:31.315799842Z main.main.func2(0x4000335cc0)
2022-06-08T16:00:31.315824916Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x288
2022-06-08T16:00:31.315854860Z github.com/urfave/cli/v2.(*Command).Run(0x400035d9e0, 0x4000335b40)
2022-06-08T16:00:31.315880804Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x648
2022-06-08T16:00:31.315904896Z github.com/urfave/cli/v2.(*App).RunContext(0x4000226480, {0x103b030, 0x40000c2000}, {0x40000c0000, 0x2, 0x2})
2022-06-08T16:00:31.315929173Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0x840
2022-06-08T16:00:31.315953284Z github.com/urfave/cli/v2.(*App).Run(...)
2022-06-08T16:00:31.315976969Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215
2022-06-08T16:00:31.316000950Z main.main()
2022-06-08T16:00:31.316037227Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x73c
2022-06-08T16:00:31.316064578Z 
2022-06-08T16:00:31.316087133Z goroutine 22 [select]:
2022-06-08T16:00:31.316110263Z database/sql.(*DB).connectionOpener(0x4000333450, {0x103aff8, 0x4000362bc0})
2022-06-08T16:00:31.316191595Z 	/usr/local/go/src/database/sql/sql.go:1196 +0x94
2022-06-08T16:00:31.316218279Z created by database/sql.OpenDB
2022-06-08T16:00:31.316240427Z 	/usr/local/go/src/database/sql/sql.go:794 +0x158
2022-06-08T16:00:31.316262390Z 
2022-06-08T16:00:31.316283797Z goroutine 27 [IO wait]:
2022-06-08T16:00:31.316306407Z internal/poll.runtime_pollWait(0xffff7e155718, 0x72)
2022-06-08T16:00:31.316330092Z 	/usr/local/go/src/runtime/netpoll.go:303 +0xb4
2022-06-08T16:00:31.316354314Z internal/poll.(*pollDesc).wait(0x40002ef418, 0x72, 0x0)
2022-06-08T16:00:31.316378369Z 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x38
2022-06-08T16:00:31.316400646Z internal/poll.(*pollDesc).waitRead(...)
2022-06-08T16:00:31.316429924Z 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
2022-06-08T16:00:31.316452664Z internal/poll.(*FD).Read(0x40002ef400, {0x400037a000, 0x1000, 0x1000})
2022-06-08T16:00:31.316474664Z 	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x1e4
2022-06-08T16:00:31.316495812Z net.(*netFD).Read(0x40002ef400, {0x400037a000, 0x1000, 0x1000})
2022-06-08T16:00:31.316516941Z 	/usr/local/go/src/net/fd_posix.go:56 +0x44
2022-06-08T16:00:31.316538052Z net.(*conn).Read(0x40000bf100, {0x400037a000, 0x1000, 0x1000})
2022-06-08T16:00:31.316559773Z 	/usr/local/go/src/net/net.go:183 +0x4c
2022-06-08T16:00:31.316581588Z net/http.(*persistConn).Read(0x4000368240, {0x400037a000, 0x1000, 0x1000})
2022-06-08T16:00:31.316604162Z 	/usr/local/go/src/net/http/transport.go:1926 +0x64
2022-06-08T16:00:31.316625643Z bufio.(*Reader).fill(0x4000093c20)
2022-06-08T16:00:31.316647864Z 	/usr/local/go/src/bufio/bufio.go:101 +0x10c
2022-06-08T16:00:31.316668883Z bufio.(*Reader).Peek(0x4000093c20, 0x1)
2022-06-08T16:00:31.316690567Z 	/usr/local/go/src/bufio/bufio.go:139 +0x74
2022-06-08T16:00:31.316711826Z net/http.(*persistConn).readLoop(0x4000368240)
2022-06-08T16:00:31.316733104Z 	/usr/local/go/src/net/http/transport.go:2087 +0x170
2022-06-08T16:00:31.316754177Z created by net/http.(*Transport).dialConn
2022-06-08T16:00:31.316775603Z 	/usr/local/go/src/net/http/transport.go:1747 +0x18b0
2022-06-08T16:00:31.316796973Z 
2022-06-08T16:00:31.316818547Z goroutine 28 [select]:
2022-06-08T16:00:31.316839991Z net/http.(*persistConn).writeLoop(0x4000368240)
2022-06-08T16:00:31.316861657Z 	/usr/local/go/src/net/http/transport.go:2386 +0xa8
2022-06-08T16:00:31.316971877Z created by net/http.(*Transport).dialConn
2022-06-08T16:00:31.317002136Z 	/usr/local/go/src/net/http/transport.go:1748 +0x18d0
2022-06-08T16:00:31.317040524Z 
2022-06-08T16:00:31.317063876Z goroutine 34 [select]:
2022-06-08T16:00:31.317239947Z net.(*Resolver).lookupIPAddr(0x1633ca0, {0x103b068, 0x40004945a0}, {0xeecfd2, 0x3}, {0x4000480330, 0x9})
2022-06-08T16:00:31.317286335Z 	/usr/local/go/src/net/lookup.go:302 +0x5c4
2022-06-08T16:00:31.317310020Z net.(*Resolver).internetAddrList(0x1633ca0, {0x103b068, 0x40004945a0}, {0xeecfd2, 0x3}, {0x4000480330, 0xe})
2022-06-08T16:00:31.317332779Z 	/usr/local/go/src/net/ipsock.go:288 +0x68c
2022-06-08T16:00:31.317355149Z net.(*Resolver).resolveAddrList(0x1633ca0, {0x103b068, 0x40004945a0}, {0xeed674, 0x4}, {0xeecfd2, 0x3}, {0x4000480330, 0xe}, {0x0, ...})
2022-06-08T16:00:31.317377871Z 	/usr/local/go/src/net/dial.go:221 +0x484
2022-06-08T16:00:31.317413833Z net.(*Dialer).DialContext(0x40000936e0, {0x103b068, 0x4000494300}, {0xeecfd2, 0x3}, {0x4000480330, 0xe})
2022-06-08T16:00:31.317439221Z 	/usr/local/go/src/net/dial.go:406 +0x3b0
2022-06-08T16:00:31.317461517Z net/http.(*Transport).dial(0x400033aa00, {0x103b068, 0x4000494300}, {0xeecfd2, 0x3}, {0x4000480330, 0xe})
2022-06-08T16:00:31.317484461Z 	/usr/local/go/src/net/http/transport.go:1166 +0x178
2022-06-08T16:00:31.317506572Z net/http.(*Transport).dialConn(0x400033aa00, {0x103b068, 0x4000494300}, {{}, 0x0, {0x4000496240, 0x4}, {0x4000480330, 0xe}, 0x0})
2022-06-08T16:00:31.317534794Z 	/usr/local/go/src/net/http/transport.go:1604 +0x66c
2022-06-08T16:00:31.317557053Z net/http.(*Transport).dialConnFor(0x400033aa00, 0x40004ae000)
2022-06-08T16:00:31.317578367Z 	/usr/local/go/src/net/http/transport.go:1446 +0x90
2022-06-08T16:00:31.317600089Z created by net/http.(*Transport).queueForDial
2022-06-08T16:00:31.317621866Z 	/usr/local/go/src/net/http/transport.go:1415 +0x398
2022-06-08T16:00:31.317642940Z 
2022-06-08T16:00:31.317664384Z goroutine 35 [select]:
2022-06-08T16:00:31.317685513Z net.cgoLookupIP({0x103aff8, 0x40004a6100}, {0xeecfd2, 0x3}, {0x4000480330, 0x9})
2022-06-08T16:00:31.317707328Z 	/usr/local/go/src/net/cgo_unix.go:231 +0x104
2022-06-08T16:00:31.317740975Z net.(*Resolver).lookupIP(0x1633ca0, {0x103aff8, 0x40004a6100}, {0xeecfd2, 0x3}, {0x4000480330, 0x9})
2022-06-08T16:00:31.317765345Z 	/usr/local/go/src/net/lookup_unix.go:97 +0x13c
2022-06-08T16:00:31.317787030Z net.glob..func1({0x103aff8, 0x40004a6100}, 0x400048c2a0, {0xeecfd2, 0x3}, {0x4000480330, 0x9})
2022-06-08T16:00:31.317809493Z 	/usr/local/go/src/net/hook.go:23 +0x58
2022-06-08T16:00:31.317830863Z net.(*Resolver).lookupIPAddr.func1()
2022-06-08T16:00:31.317852011Z 	/usr/local/go/src/net/lookup.go:296 +0x9c
2022-06-08T16:00:31.317886640Z internal/singleflight.(*Group).doCall(0x1633cb0, 0x40004ac0a0, {0x4000480340, 0xd}, 0x40004a6140)
2022-06-08T16:00:31.317908954Z 	/usr/local/go/src/internal/singleflight/singleflight.go:95 +0x28
2022-06-08T16:00:31.317930954Z created by internal/singleflight.(*Group).DoChan
2022-06-08T16:00:31.317952120Z 	/usr/local/go/src/internal/singleflight/singleflight.go:88 +0x3d4
2022-06-08T16:00:31.338342642Z waiting for influxdb
2022-06-08T16:00:31.369620607Z starting scrutiny
2022-06-08T16:00:31.401338732Z 2022/06/08 16:00:31 Loading configuration file: /opt/scrutiny/config/scrutiny.yaml
2022-06-08T16:00:31.402796429Z 
2022-06-08T16:00:31.402931852Z  ___   ___  ____  __  __  ____  ____  _  _  _  _
2022-06-08T16:00:31.402968926Z / __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
2022-06-08T16:00:31.402991314Z \__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
2022-06-08T16:00:31.403018203Z (___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
2022-06-08T16:00:31.403037851Z github.com/AnalogJ/scrutiny                             dev-0.4.9
2022-06-08T16:00:31.403057813Z 
2022-06-08T16:00:31.405053279Z Start the scrutiny server
2022-06-08T16:00:31.413400599Z fatal error: unexpected signal during runtime execution
2022-06-08T16:00:31.416996482Z [signal SIGSEGV: segmentation violation code=0x1 addr=0x62 pc=0xffff72465c44]
2022-06-08T16:00:31.417093055Z 
2022-06-08T16:00:31.417121351Z runtime stack:
2022-06-08T16:00:31.417141591Z runtime.throw({0xf0f0f1, 0x2a})
2022-06-08T16:00:31.417170331Z 	/usr/local/go/src/runtime/panic.go:1198 +0x54
2022-06-08T16:00:31.417191461Z runtime.sigpanic()
2022-06-08T16:00:31.417211868Z 	/usr/local/go/src/runtime/signal_unix.go:719 +0x230
2022-06-08T16:00:31.417231015Z 
2022-06-08T16:00:31.417249219Z goroutine 27 [syscall]:
2022-06-08T16:00:31.417267756Z runtime.cgocall(0xbdb960, 0x400004ada8)
2022-06-08T16:00:31.417286144Z 	/usr/local/go/src/runtime/cgocall.go:156 +0x50 fp=0x400004ad70 sp=0x400004ad30 pc=0x4039b0
2022-06-08T16:00:31.417305329Z net._C2func_getaddrinfo(0x400031b600, 0x0, 0x4000458300, 0x4000314520)
2022-06-08T16:00:31.417324218Z 	_cgo_gotypes.go:91 +0x48 fp=0x400004ada0 sp=0x400004ad70 pc=0x58eb68
2022-06-08T16:00:31.417343180Z net.cgoLookupIPCNAME.func1({0x400031b600, 0xa, 0xa}, 0x4000458300, 0x4000314520)
2022-06-08T16:00:31.417362235Z 	/usr/local/go/src/net/cgo_unix.go:163 +0xa8 fp=0x400004adf0 sp=0x400004ada0 pc=0x590148
2022-06-08T16:00:31.417390568Z net.cgoLookupIPCNAME({0xeecfd2, 0x3}, {0x400031b5e0, 0x9})
2022-06-08T16:00:31.417411049Z 	/usr/local/go/src/net/cgo_unix.go:163 +0x150 fp=0x400004af10 sp=0x400004adf0 pc=0x58fa80
2022-06-08T16:00:31.417430271Z net.cgoIPLookup(0x400030fda0, {0xeecfd2, 0x3}, {0x400031b5e0, 0x9})
2022-06-08T16:00:31.417448938Z 	/usr/local/go/src/net/cgo_unix.go:220 +0x40 fp=0x400004afb0 sp=0x400004af10 pc=0x5901d0
2022-06-08T16:00:31.417467678Z runtime.goexit()
2022-06-08T16:00:31.417486270Z 	/usr/local/go/src/runtime/asm_arm64.s:1133 +0x4 fp=0x400004afb0 sp=0x400004afb0 pc=0x468ef4
2022-06-08T16:00:31.417505566Z created by net.cgoLookupIP
2022-06-08T16:00:31.417564325Z 	/usr/local/go/src/net/cgo_unix.go:230 +0x98
2022-06-08T16:00:31.417585232Z 
2022-06-08T16:00:31.417603805Z goroutine 1 [select]:
2022-06-08T16:00:31.417622120Z net/http.(*Transport).getConn(0x40003b4f00, 0x400042c7c0, {{}, 0x0, {0x40003eac00, 0x4}, {0x400031b5e0, 0xe}, 0x0})
2022-06-08T16:00:31.417641119Z 	/usr/local/go/src/net/http/transport.go:1372 +0x4e8
2022-06-08T16:00:31.417659841Z net/http.(*Transport).roundTrip(0x40003b4f00, 0x40003d2700)
2022-06-08T16:00:31.417678674Z 	/usr/local/go/src/net/http/transport.go:581 +0x7d4
2022-06-08T16:00:31.426150548Z net/http.(*Transport).RoundTrip(0x40003b4f00, 0x40003d2700)
2022-06-08T16:00:31.426272231Z 	/usr/local/go/src/net/http/roundtrip.go:18 +0x30
2022-06-08T16:00:31.426303027Z net/http.send(0x40003d2600, {0x10282e0, 0x40003b4f00}, {0xc0a04fccd8979f1e, 0x4aa0673ca, 0x1634cc0})
2022-06-08T16:00:31.426326212Z 	/usr/local/go/src/net/http/client.go:252 +0x5ac
2022-06-08T16:00:31.426347360Z net/http.(*Client).send(0x4000419260, 0x40003d2600, {0xc0a04fccd8979f1e, 0x4aa0673ca, 0x1634cc0})
2022-06-08T16:00:31.426367007Z 	/usr/local/go/src/net/http/client.go:176 +0xac
2022-06-08T16:00:31.426386600Z net/http.(*Client).do(0x4000419260, 0x40003d2600)
2022-06-08T16:00:31.426405747Z 	/usr/local/go/src/net/http/client.go:725 +0x888
2022-06-08T16:00:31.426425043Z net/http.(*Client).Do(0x4000419260, 0x40003d2600)
2022-06-08T16:00:31.426444376Z 	/usr/local/go/src/net/http/client.go:593 +0x30
2022-06-08T16:00:31.426463691Z github.com/influxdata/influxdb-client-go/v2/api/http.(*service).DoHTTPRequestWithResponse(0x400042c440, 0x40003d2600, 0x0)
2022-06-08T16:00:31.426483135Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/http/service.go:136 +0x38c
2022-06-08T16:00:31.426502375Z github.com/influxdata/influxdb-client-go/v2/domain.(*Client).GetOrgs(0x400042a4b0, {0x103b030, 0x400003a028}, 0x400042c740)
2022-06-08T16:00:31.426521745Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:2056 +0x288
2022-06-08T16:00:31.426541356Z github.com/influxdata/influxdb-client-go/v2/domain.(*ClientWithResponses).GetOrgsWithResponse(0x400042a4c0, {0x103b030, 0x400003a028}, 0x400042c740)
2022-06-08T16:00:31.426560986Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:23204 +0x4c
2022-06-08T16:00:31.426580300Z github.com/influxdata/influxdb-client-go/v2/api.(*organizationsAPI).getOrganizations(0x40003144e8, {0x103b030, 0x400003a028}, 0x400042c740, {0x0, 0x0, 0x0})
2022-06-08T16:00:31.426600448Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/organizations.go:84 +0x138
2022-06-08T16:00:31.426619429Z github.com/influxdata/influxdb-client-go/v2/api.(*organizationsAPI).FindOrganizationByName(0x40003144e8, {0x103b030, 0x400003a028}, {0xef2125, 0x8})
2022-06-08T16:00:31.426638558Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/organizations.go:100 +0xd8
2022-06-08T16:00:31.426701205Z github.com/analogj/scrutiny/webapp/backend/pkg/database.NewScrutinyRepository({0x104ee80, 0x4000314078}, {0x1057de0, 0x4000403c70})
2022-06-08T16:00:31.426731983Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/database/scrutiny_repository.go:139 +0xa94
2022-06-08T16:00:31.426752908Z github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x104ee80, 0x4000314078}, {0x1057de0, 0x4000403c70})
2022-06-08T16:00:31.426772482Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:12 +0x40
2022-06-08T16:00:31.426791963Z github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0x4000313b20, {0x1057de0, 0x4000403c70})
2022-06-08T16:00:31.426811555Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0x90
2022-06-08T16:00:31.426831018Z github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0x4000313b20)
2022-06-08T16:00:31.426850258Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x530
2022-06-08T16:00:31.426869462Z main.main.func2(0x400031d380)
2022-06-08T16:00:31.426938257Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x288
2022-06-08T16:00:31.426966034Z github.com/urfave/cli/v2.(*Command).Run(0x4000409c20, 0x400031d200)
2022-06-08T16:00:31.426985386Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x648
2022-06-08T16:00:31.427004867Z github.com/urfave/cli/v2.(*App).RunContext(0x4000416000, {0x103b030, 0x400003a028}, {0x4000032060, 0x2, 0x2})
2022-06-08T16:00:31.427024515Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0x840
2022-06-08T16:00:31.427044570Z github.com/urfave/cli/v2.(*App).Run(...)
2022-06-08T16:00:31.427063792Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215
2022-06-08T16:00:31.427082662Z main.main()
2022-06-08T16:00:31.427101087Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x73c
2022-06-08T16:00:31.427119957Z 
2022-06-08T16:00:31.427138365Z goroutine 18 [select]:
2022-06-08T16:00:31.427157272Z database/sql.(*DB).connectionOpener(0x40003a6820, {0x103aff8, 0x400042c280})
2022-06-08T16:00:31.427176642Z 	/usr/local/go/src/database/sql/sql.go:1196 +0x94
2022-06-08T16:00:31.427195604Z created by database/sql.OpenDB
2022-06-08T16:00:31.427221826Z 	/usr/local/go/src/database/sql/sql.go:794 +0x158
2022-06-08T16:00:31.427241344Z 
2022-06-08T16:00:31.427259844Z goroutine 25 [select]:
2022-06-08T16:00:31.427278807Z net.(*Resolver).lookupIPAddr(0x1633ca0, {0x103b068, 0x400030fce0}, {0xeecfd2, 0x3}, {0x400031b5e0, 0x9})
2022-06-08T16:00:31.427298195Z 	/usr/local/go/src/net/lookup.go:302 +0x5c4
2022-06-08T16:00:31.427359953Z net.(*Resolver).internetAddrList(0x1633ca0, {0x103b068, 0x400030fce0}, {0xeecfd2, 0x3}, {0x400031b5e0, 0xe})
2022-06-08T16:00:31.427382620Z 	/usr/local/go/src/net/ipsock.go:288 +0x68c
2022-06-08T16:00:31.427401804Z net.(*Resolver).resolveAddrList(0x1633ca0, {0x103b068, 0x400030fce0}, {0xeed674, 0x4}, {0xeecfd2, 0x3}, {0x400031b5e0, 0xe}, {0x0, ...})
2022-06-08T16:00:31.427426545Z 	/usr/local/go/src/net/dial.go:221 +0x484
2022-06-08T16:00:31.427446100Z net.(*Dialer).DialContext(0x400030f1a0, {0x103b068, 0x400030fb00}, {0xeecfd2, 0x3}, {0x400031b5e0, 0xe})
2022-06-08T16:00:31.427465785Z 	/usr/local/go/src/net/dial.go:406 +0x3b0
2022-06-08T16:00:31.427485081Z net/http.(*Transport).dial(0x40003b4f00, {0x103b068, 0x400030fb00}, {0xeecfd2, 0x3}, {0x400031b5e0, 0xe})
2022-06-08T16:00:31.427504173Z 	/usr/local/go/src/net/http/transport.go:1166 +0x178
2022-06-08T16:00:31.427523358Z net/http.(*Transport).dialConn(0x40003b4f00, {0x103b068, 0x400030fb00}, {{}, 0x0, {0x40003eac00, 0x4}, {0x400031b5e0, 0xe}, 0x0})
2022-06-08T16:00:31.427543247Z 	/usr/local/go/src/net/http/transport.go:1604 +0x66c
2022-06-08T16:00:31.427562931Z net/http.(*Transport).dialConnFor(0x40003b4f00, 0x40003b8370)
2022-06-08T16:00:31.427582024Z 	/usr/local/go/src/net/http/transport.go:1446 +0x90
2022-06-08T16:00:31.427601023Z created by net/http.(*Transport).queueForDial
2022-06-08T16:00:31.427619930Z 	/usr/local/go/src/net/http/transport.go:1415 +0x398
2022-06-08T16:00:31.427639208Z 
2022-06-08T16:00:31.427657708Z goroutine 23 [IO wait]:
2022-06-08T16:00:31.427676763Z internal/poll.runtime_pollWait(0xffff7248cf08, 0x72)
2022-06-08T16:00:31.427696022Z 	/usr/local/go/src/runtime/netpoll.go:303 +0xb4
2022-06-08T16:00:31.427715058Z internal/poll.(*pollDesc).wait(0x40003a3198, 0x72, 0x0)
2022-06-08T16:00:31.427734317Z 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x38
2022-06-08T16:00:31.427859519Z internal/poll.(*pollDesc).waitRead(...)
2022-06-08T16:00:31.427880981Z 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
2022-06-08T16:00:31.427901129Z internal/poll.(*FD).Read(0x40003a3180, {0x4000456000, 0x1000, 0x1000})
2022-06-08T16:00:31.427921018Z 	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x1e4
2022-06-08T16:00:31.427940425Z net.(*netFD).Read(0x40003a3180, {0x4000456000, 0x1000, 0x1000})
2022-06-08T16:00:31.427959999Z 	/usr/local/go/src/net/fd_posix.go:56 +0x44
2022-06-08T16:00:31.427980832Z net.(*conn).Read(0x4000314488, {0x4000456000, 0x1000, 0x1000})
2022-06-08T16:00:31.428000924Z 	/usr/local/go/src/net/net.go:183 +0x4c
2022-06-08T16:00:31.428019924Z net/http.(*persistConn).Read(0x40004325a0, {0x4000456000, 0x1000, 0x1000})
2022-06-08T16:00:31.428039608Z 	/usr/local/go/src/net/http/transport.go:1926 +0x64
2022-06-08T16:00:31.428058978Z bufio.(*Reader).fill(0x400030f740)
2022-06-08T16:00:31.428124403Z 	/usr/local/go/src/bufio/bufio.go:101 +0x10c
2022-06-08T16:00:31.428146903Z bufio.(*Reader).Peek(0x400030f740, 0x1)
2022-06-08T16:00:31.428166106Z 	/usr/local/go/src/bufio/bufio.go:139 +0x74
2022-06-08T16:00:31.428185569Z net/http.(*persistConn).readLoop(0x40004325a0)
2022-06-08T16:00:31.428204698Z 	/usr/local/go/src/net/http/transport.go:2087 +0x170
2022-06-08T16:00:31.428225087Z created by net/http.(*Transport).dialConn
2022-06-08T16:00:31.428243679Z 	/usr/local/go/src/net/http/transport.go:1747 +0x18b0
2022-06-08T16:00:31.428262605Z 
2022-06-08T16:00:31.428281197Z goroutine 24 [select]:
2022-06-08T16:00:31.428300122Z net/http.(*persistConn).writeLoop(0x40004325a0)
2022-06-08T16:00:31.428319548Z 	/usr/local/go/src/net/http/transport.go:2386 +0xa8
2022-06-08T16:00:31.428338992Z created by net/http.(*Transport).dialConn
2022-06-08T16:00:31.428364270Z 	/usr/local/go/src/net/http/transport.go:1748 +0x18d0
2022-06-08T16:00:31.428384417Z 
2022-06-08T16:00:31.428403102Z goroutine 26 [select]:
2022-06-08T16:00:31.428422417Z net.cgoLookupIP({0x103aff8, 0x400042c800}, {0xeecfd2, 0x3}, {0x400031b5e0, 0x9})
2022-06-08T16:00:31.428441842Z 	/usr/local/go/src/net/cgo_unix.go:231 +0x104
2022-06-08T16:00:31.428461083Z net.(*Resolver).lookupIP(0x1633ca0, {0x103aff8, 0x400042c800}, {0xeecfd2, 0x3}, {0x400031b5e0, 0x9})
2022-06-08T16:00:31.428480101Z 	/usr/local/go/src/net/lookup_unix.go:97 +0x13c
2022-06-08T16:00:31.428499915Z net.glob..func1({0x103aff8, 0x400042c800}, 0x400042a8d0, {0xeecfd2, 0x3}, {0x400031b5e0, 0x9})
2022-06-08T16:00:31.428519526Z 	/usr/local/go/src/net/hook.go:23 +0x58
2022-06-08T16:00:31.428540026Z net.(*Resolver).lookupIPAddr.func1()
2022-06-08T16:00:31.428559285Z 	/usr/local/go/src/net/lookup.go:296 +0x9c
2022-06-08T16:00:31.428579303Z internal/singleflight.(*Group).doCall(0x1633cb0, 0x40003a93b0, {0x400031b5f0, 0xd}, 0x400042c840)
2022-06-08T16:00:31.428598858Z 	/usr/local/go/src/internal/singleflight/singleflight.go:95 +0x28
2022-06-08T16:00:31.428618080Z created by internal/singleflight.(*Group).DoChan
2022-06-08T16:00:31.428637246Z 	/usr/local/go/src/internal/singleflight/singleflight.go:88 +0x3d4
2022-06-08T16:00:32.342866731Z waiting for influxdb
2022-06-08T16:00:32.371347633Z starting scrutiny
2022-06-08T16:00:32.398483077Z 2022/06/08 16:00:32 Loading configuration file: /opt/scrutiny/config/scrutiny.yaml
2022-06-08T16:00:32.399816072Z 
2022-06-08T16:00:32.400038383Z  ___   ___  ____  __  __  ____  ____  _  _  _  _
2022-06-08T16:00:32.400090346Z / __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
2022-06-08T16:00:32.400114901Z \__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
2022-06-08T16:00:32.400136808Z (___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
2022-06-08T16:00:32.400191566Z github.com/AnalogJ/scrutiny                             dev-0.4.9
2022-06-08T16:00:32.400213455Z 
2022-06-08T16:00:32.400270713Z Start the scrutiny server
2022-06-08T16:00:32.407949693Z fatal error: unexpected signal during runtime execution
2022-06-08T16:00:32.411048621Z [signal SIGSEGV: segmentation violation code=0x1 addr=0x62 pc=0xffff8c2a9c44]
2022-06-08T16:00:32.411151860Z 
2022-06-08T16:00:32.411176397Z runtime stack:
2022-06-08T16:00:32.411196915Z runtime.throw({0xf0f0f1, 0x2a})
2022-06-08T16:00:32.411217396Z 	/usr/local/go/src/runtime/panic.go:1198 +0x54
2022-06-08T16:00:32.411237192Z runtime.sigpanic()
2022-06-08T16:00:32.411255914Z 	/usr/local/go/src/runtime/signal_unix.go:719 +0x230
2022-06-08T16:00:32.411275469Z 
2022-06-08T16:00:32.411294395Z goroutine 9 [syscall]:
2022-06-08T16:00:32.411312672Z runtime.cgocall(0xbdb960, 0x4000050da8)
2022-06-08T16:00:32.411331616Z 	/usr/local/go/src/runtime/cgocall.go:156 +0x50 fp=0x4000050d70 sp=0x4000050d30 pc=0x4039b0
2022-06-08T16:00:32.411351690Z net._C2func_getaddrinfo(0x40002f8dc0, 0x0, 0x400028bc20, 0x4000010d18)
2022-06-08T16:00:32.411370838Z 	_cgo_gotypes.go:91 +0x48 fp=0x4000050da0 sp=0x4000050d70 pc=0x58eb68
2022-06-08T16:00:32.411389764Z net.cgoLookupIPCNAME.func1({0x40002f8dc0, 0xa, 0xa}, 0x400028bc20, 0x4000010d18)
2022-06-08T16:00:32.411408541Z 	/usr/local/go/src/net/cgo_unix.go:163 +0xa8 fp=0x4000050df0 sp=0x4000050da0 pc=0x590148
2022-06-08T16:00:32.411428541Z net.cgoLookupIPCNAME({0xeecfd2, 0x3}, {0x40002f8da0, 0x9})
2022-06-08T16:00:32.411447763Z 	/usr/local/go/src/net/cgo_unix.go:163 +0x150 fp=0x4000050f10 sp=0x4000050df0 pc=0x58fa80
2022-06-08T16:00:32.411467096Z net.cgoIPLookup(0x4000082d80, {0xeecfd2, 0x3}, {0x40002f8da0, 0x9})
2022-06-08T16:00:32.411486262Z 	/usr/local/go/src/net/cgo_unix.go:220 +0x40 fp=0x4000050fb0 sp=0x4000050f10 pc=0x5901d0
2022-06-08T16:00:32.411506002Z runtime.goexit()
2022-06-08T16:00:32.414689578Z 	/usr/local/go/src/runtime/asm_arm64.s:1133 +0x4 fp=0x4000050fb0 sp=0x4000050fb0 pc=0x468ef4
2022-06-08T16:00:32.414786835Z created by net.cgoLookupIP
2022-06-08T16:00:32.414813557Z 	/usr/local/go/src/net/cgo_unix.go:230 +0x98
2022-06-08T16:00:32.414834779Z 
2022-06-08T16:00:32.414854019Z goroutine 1 [select]:
2022-06-08T16:00:32.414873056Z net/http.(*Transport).getConn(0x4000352f00, 0x40000faa80, {{}, 0x0, {0x400003e720, 0x4}, {0x40002f8da0, 0xe}, 0x0})
2022-06-08T16:00:32.414948999Z 	/usr/local/go/src/net/http/transport.go:1372 +0x4e8
2022-06-08T16:00:32.414973517Z net/http.(*Transport).roundTrip(0x4000352f00, 0x40000ad100)
2022-06-08T16:00:32.414992998Z 	/usr/local/go/src/net/http/transport.go:581 +0x7d4
2022-06-08T16:00:32.415062645Z net/http.(*Transport).RoundTrip(0x4000352f00, 0x40000ad100)
2022-06-08T16:00:32.415084830Z 	/usr/local/go/src/net/http/roundtrip.go:18 +0x30
2022-06-08T16:00:32.415104626Z net/http.send(0x40000ad000, {0x10282e0, 0x4000352f00}, {0xc0a04fcd18440524, 0x4a9c6ecd4, 0x1634cc0})
2022-06-08T16:00:32.415123755Z 	/usr/local/go/src/net/http/client.go:252 +0x5ac
2022-06-08T16:00:32.415142607Z net/http.(*Client).send(0x4000435290, 0x40000ad000, {0xc0a04fcd18440524, 0x4a9c6ecd4, 0x1634cc0})
2022-06-08T16:00:32.415161736Z 	/usr/local/go/src/net/http/client.go:176 +0xac
2022-06-08T16:00:32.415180791Z net/http.(*Client).do(0x4000435290, 0x40000ad000)
2022-06-08T16:00:32.415199587Z 	/usr/local/go/src/net/http/client.go:725 +0x888
2022-06-08T16:00:32.415218680Z net/http.(*Client).Do(0x4000435290, 0x40000ad000)
2022-06-08T16:00:32.415238179Z 	/usr/local/go/src/net/http/client.go:593 +0x30
2022-06-08T16:00:32.415257438Z github.com/influxdata/influxdb-client-go/v2/api/http.(*service).DoHTTPRequestWithResponse(0x4000446440, 0x40000ad000, 0x0)
2022-06-08T16:00:32.415277290Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/http/service.go:136 +0x38c
2022-06-08T16:00:32.415297493Z github.com/influxdata/influxdb-client-go/v2/domain.(*Client).GetOrgs(0x40004444c0, {0x103b030, 0x400003a028}, 0x40000faa00)
2022-06-08T16:00:32.415317178Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:2056 +0x288
2022-06-08T16:00:32.415337067Z github.com/influxdata/influxdb-client-go/v2/domain.(*ClientWithResponses).GetOrgsWithResponse(0x40004444d0, {0x103b030, 0x400003a028}, 0x40000faa00)
2022-06-08T16:00:32.415357048Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/domain/client.gen.go:23204 +0x4c
2022-06-08T16:00:32.415376640Z github.com/influxdata/influxdb-client-go/v2/api.(*organizationsAPI).getOrganizations(0x4000010cd8, {0x103b030, 0x400003a028}, 0x40000faa00, {0x0, 0x0, 0x0})
2022-06-08T16:00:32.415396714Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/organizations.go:84 +0x138
2022-06-08T16:00:32.415421102Z github.com/influxdata/influxdb-client-go/v2/api.(*organizationsAPI).FindOrganizationByName(0x4000010cd8, {0x103b030, 0x400003a028}, {0xef2125, 0x8})
2022-06-08T16:00:32.415441898Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/influxdata/influxdb-client-go/v2/api/organizations.go:100 +0xd8
2022-06-08T16:00:32.415461805Z github.com/analogj/scrutiny/webapp/backend/pkg/database.NewScrutinyRepository({0x104ee80, 0x4000338070}, {0x1057de0, 0x400041fdc0})
2022-06-08T16:00:32.415481805Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/database/scrutiny_repository.go:139 +0xa94
2022-06-08T16:00:32.415501027Z github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x104ee80, 0x4000338070}, {0x1057de0, 0x400041fdc0})
2022-06-08T16:00:32.415520304Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:12 +0x40
2022-06-08T16:00:32.415581359Z github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0x4000335b30, {0x1057de0, 0x400041fdc0})
2022-06-08T16:00:32.415604951Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0x90
2022-06-08T16:00:32.415624969Z github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0x4000335b30)
2022-06-08T16:00:32.418735842Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x530
2022-06-08T16:00:32.418836914Z main.main.func2(0x400033f380)
2022-06-08T16:00:32.418863766Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x288
2022-06-08T16:00:32.418923876Z github.com/urfave/cli/v2.(*Command).Run(0x4000425c20, 0x400033f200)
2022-06-08T16:00:32.419016467Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x648
2022-06-08T16:00:32.419085058Z github.com/urfave/cli/v2.(*App).RunContext(0x4000432000, {0x103b030, 0x400003a028}, {0x4000032060, 0x2, 0x2})
2022-06-08T16:00:32.419111909Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0x840
2022-06-08T16:00:32.419131872Z github.com/urfave/cli/v2.(*App).Run(...)
2022-06-08T16:00:32.419931099Z 	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215
2022-06-08T16:00:32.420009709Z main.main()
2022-06-08T16:00:32.420167299Z 	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x73c
2022-06-08T16:00:32.420327278Z 
2022-06-08T16:00:32.422015415Z goroutine 18 [select]:
2022-06-08T16:00:32.422109284Z database/sql.(*DB).connectionOpener(0x4000344820, {0x103aff8, 0x4000446280})
2022-06-08T16:00:32.422136784Z 	/usr/local/go/src/database/sql/sql.go:1196 +0x94
2022-06-08T16:00:32.422157098Z created by database/sql.OpenDB
2022-06-08T16:00:32.422176487Z 	/usr/local/go/src/database/sql/sql.go:794 +0x158
2022-06-08T16:00:32.422195838Z 
2022-06-08T16:00:32.422214912Z goroutine 23 [IO wait]:
2022-06-08T16:00:32.422233689Z internal/poll.runtime_pollWait(0xffff8c2ccf08, 0x72)
2022-06-08T16:00:32.422252948Z 	/usr/local/go/src/runtime/netpoll.go:303 +0xb4
2022-06-08T16:00:32.422272022Z internal/poll.(*pollDesc).wait(0x4000371118, 0x72, 0x0)
2022-06-08T16:00:32.422291096Z 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x38
2022-06-08T16:00:32.422310244Z internal/poll.(*pollDesc).waitRead(...)
2022-06-08T16:00:32.422328780Z 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
2022-06-08T16:00:32.422347539Z internal/poll.(*FD).Read(0x4000371100, {0x4000466000, 0x1000, 0x1000})
2022-06-08T16:00:32.422366743Z 	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x1e4
2022-06-08T16:00:32.422386316Z net.(*netFD).Read(0x4000371100, {0x4000466000, 0x1000, 0x1000})
2022-06-08T16:00:32.422405020Z 	/usr/local/go/src/net/fd_posix.go:56 +0x44
2022-06-08T16:00:32.422470352Z net.(*conn).Read(0x4000338480, {0x4000466000, 0x1000, 0x1000})
2022-06-08T16:00:32.422492981Z 	/usr/local/go/src/net/net.go:183 +0x4c
2022-06-08T16:00:32.422512407Z net/http.(*persistConn).Read(0x400044c5a0, {0x4000466000, 0x1000, 0x1000})
2022-06-08T16:00:32.422540555Z 	/usr/local/go/src/net/http/transport.go:1926 +0x64
2022-06-08T16:00:32.422561795Z bufio.(*Reader).fill(0x400032f740)
2022-06-08T16:00:32.422580887Z 	/usr/local/go/src/bufio/bufio.go:101 +0x10c
2022-06-08T16:00:32.423637221Z bufio.(*Reader).Peek(0x400032f740, 0x1)
2022-06-08T16:00:32.423943197Z 	/usr/local/go/src/bufio/bufio.go:139 +0x74
2022-06-08T16:00:32.424347524Z net/http.(*persistConn).readLoop(0x400044c5a0)
2022-06-08T16:00:32.424576612Z 	/usr/local/go/src/net/http/transport.go:2087 +0x170
2022-06-08T16:00:32.425017512Z created by net/http.(*Transport).dialConn
2022-06-08T16:00:32.426336082Z 	/usr/local/go/src/net/http/transport.go:1747 +0x18b0
2022-06-08T16:00:32.428542785Z 
2022-06-08T16:00:32.429671303Z goroutine 24 [select]:
2022-06-08T16:00:32.429755153Z net/http.(*persistConn).writeLoop(0x400044c5a0)
2022-06-08T16:00:32.429778653Z 	/usr/local/go/src/net/http/transport.go:2386 +0xa8
2022-06-08T16:00:32.429805467Z created by net/http.(*Transport).dialConn
2022-06-08T16:00:32.429833226Z 	/usr/local/go/src/net/http/transport.go:1748 +0x18d0
2022-06-08T16:00:32.429854800Z 
2022-06-08T16:00:32.429874633Z goroutine 7 [select]:
2022-06-08T16:00:32.432984469Z net.(*Resolver).lookupIPAddr(0x1633ca0, {0x103b068, 0x4000082c00}, {0xeecfd2, 0x3}, {0x40002f8da0, 0x9})
2022-06-08T16:00:32.433081819Z 	/usr/local/go/src/net/lookup.go:302 +0x5c4
2022-06-08T16:00:32.433106615Z net.(*Resolver).internetAddrList(0x1633ca0, {0x103b068, 0x4000082c00}, {0xeecfd2, 0x3}, {0x40002f8da0, 0xe})
2022-06-08T16:00:32.433127892Z 	/usr/local/go/src/net/ipsock.go:288 +0x68c
2022-06-08T16:00:32.433149318Z net.(*Resolver).resolveAddrList(0x1633ca0, {0x103b068, 0x4000082c00}, {0xeed674, 0x4}, {0xeecfd2, 0x3}, {0x40002f8da0, 0xe}, {0x0, ...})
2022-06-08T16:00:32.433169651Z 	/usr/local/go/src/net/dial.go:221 +0x484
2022-06-08T16:00:32.433188502Z net.(*Dialer).DialContext(0x400032f1a0, {0x103b068, 0x4000082960}, {0xeecfd2, 0x3}, {0x40002f8da0, 0xe})
2022-06-08T16:00:32.433207669Z 	/usr/local/go/src/net/dial.go:406 +0x3b0
2022-06-08T16:00:32.433227002Z net/http.(*Transport).dial(0x4000352f00, {0x103b068, 0x4000082960}, {0xeecfd2, 0x3}, {0x40002f8da0, 0xe})
2022-06-08T16:00:32.433245835Z 	/usr/local/go/src/net/http/transport.go:1166 +0x178
2022-06-08T16:00:32.433264723Z net/http.(*Transport).dialConn(0x4000352f00, {0x103b068, 0x4000082960}, {{}, 0x0, {0x400003e720, 0x4}, {0x40002f8da0, 0xe}, 0x0})
2022-06-08T16:00:32.433330203Z 	/usr/local/go/src/net/http/transport.go:1604 +0x66c
2022-06-08T16:00:32.433351370Z net/http.(*Transport).dialConnFor(0x4000352f00, 0x40000c2580)
2022-06-08T16:00:32.433370832Z 	/usr/local/go/src/net/http/transport.go:1446 +0x90
2022-06-08T16:00:32.433389832Z created by net/http.(*Transport).queueForDial
2022-06-08T16:00:32.433408647Z 	/usr/local/go/src/net/http/transport.go:1415 +0x398
2022-06-08T16:00:32.433427387Z 
2022-06-08T16:00:32.433445813Z goroutine 8 [select]:
2022-06-08T16:00:32.433506460Z net.cgoLookupIP({0x103aff8, 0x40000faac0}, {0xeecfd2, 0x3}, {0x40002f8da0, 0x9})
2022-06-08T16:00:32.433530145Z 	/usr/local/go/src/net/cgo_unix.go:231 +0x104
2022-06-08T16:00:32.433549385Z net.(*Resolver).lookupIP(0x1633ca0, {0x103aff8, 0x40000faac0}, {0xeecfd2, 0x3}, {0x40002f8da0, 0x9})
2022-06-08T16:00:32.433639124Z 	/usr/local/go/src/net/lookup_unix.go:97 +0x13c
2022-06-08T16:00:32.433662494Z net.glob..func1({0x103aff8, 0x40000faac0}, 0x40002e7720, {0xeecfd2, 0x3}, {0x40002f8da0, 0x9})
2022-06-08T16:00:32.433940952Z 	/usr/local/go/src/net/hook.go:23 +0x58
2022-06-08T16:00:32.434185430Z net.(*Resolver).lookupIPAddr.func1()
2022-06-08T16:00:32.434219559Z 	/usr/local/go/src/net/lookup.go:296 +0x9c
2022-06-08T16:00:32.434239762Z internal/singleflight.(*Group).doCall(0x1633cb0, 0x40001bd590, {0x40002f8db0, 0xd}, 0x40000fab00)
2022-06-08T16:00:32.436185599Z 	/usr/local/go/src/internal/singleflight/singleflight.go:95 +0x28
2022-06-08T16:00:32.436255357Z created by internal/singleflight.(*Group).DoChan
2022-06-08T16:00:32.436280116Z 	/usr/local/go/src/internal/singleflight/singleflight.go:88 +0x3d4
2022-06-08T16:00:33.345547821Z waiting for influxdb
2022-06-08T16:00:33.392342225Z starting scrutiny
2022-06-08T16:00:33.423794503Z 2022/06/08 16:00:33 Loading configuration file: /opt/scrutiny/config/scrutiny.yaml
2022-06-08T16:00:33.425342921Z 
2022-06-08T16:00:33.425598453Z  ___   ___  ____  __  __  ____  ____  _  _  _  _
2022-06-08T16:00:33.425755432Z / __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
2022-06-08T16:00:33.425843820Z \__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
2022-06-08T16:00:33.425877875Z (___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
2022-06-08T16:00:33.425899207Z github.com/AnalogJ/scrutiny                             dev-0.4.9
2022-06-08T16:00:33.425919448Z 
2022-06-08T16:00:33.425945114Z Start the scrutiny server
2022-06-08T16:00:36.303953032Z scrutiny api not ready
2022-06-08T16:00:41.343342599Z scrutiny api not ready
2022-06-08T16:00:46.378004665Z scrutiny api not ready
2022-06-08T16:00:51.414459932Z scrutiny api not ready
2022-06-08T16:00:56.457782369Z scrutiny api not ready
2022-06-08T16:01:01.494237965Z scrutiny api not ready

@PhilipWhiteside
Copy link

This has worked for me, using the beta-omnibus tag. Using beta-web with separated containers did not. I am happy to use omnibus.

@benamajin
Copy link

ghcr.io/analogj/scrutiny:beta-omnibus does not work for me either

running linux aarch64 Debian GNU/Linux 11 (bullseye)

Log:

 ___   ___  ____  __  __  ____  ____  _  _  _  _
/ __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
\__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
(___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
github.com/AnalogJ/scrutiny                             dev-0.4.9

Start the scrutiny server
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

time="2022-06-08T18:22:39-04:00" level=info msg="Trying to connect to scrutiny sqlite db: \n"
time="2022-06-08T18:22:39-04:00" level=info msg="Successfully connected to scrutiny sqlite db: \n"
time="2022-06-08T18:22:39-04:00" level=debug msg="InfluxDB url: ://:"
time="2022-06-08T18:22:39-04:00" level=debug msg="Determine Influxdb setup status..."
panic: failed to check influxdb setup status - parse "://:": missing protocol scheme

goroutine 1 [running]:
github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x104ee80, 0x400009c080}, {0x1057de0, 0x40000fe5b0})
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:14 +0xd4
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0x4000099250, {0x1057de0, 0x40000fe5b0})
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0x90
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0x4000099250)
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x530
main.main.func2(0x4000091280)
	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x288
github.com/urfave/cli/v2.(*Command).Run(0x4000404360, 0x4000091100)
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x648
github.com/urfave/cli/v2.(*App).RunContext(0x4000408000, {0x103b030, 0x4000036050}, {0x4000032060, 0x2, 0x2})
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0x840
github.com/urfave/cli/v2.(*App).Run(...)
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215
main.main()
	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x73c
waiting for influxdb
starting scrutiny
2022/06/08 18:22:40 No configuration file found at /opt/scrutiny/config/scrutiny.yaml. Using Defaults.

 ___   ___  ____  __  __  ____  ____  _  _  _  _
/ __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
\__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
(___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
github.com/AnalogJ/scrutiny                             dev-0.4.9

Start the scrutiny server
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

time="2022-06-08T18:22:40-04:00" level=info msg="Trying to connect to scrutiny sqlite db: \n"
time="2022-06-08T18:22:40-04:00" level=info msg="Successfully connected to scrutiny sqlite db: \n"
time="2022-06-08T18:22:40-04:00" level=debug msg="InfluxDB url: ://:"
time="2022-06-08T18:22:40-04:00" level=debug msg="Determine Influxdb setup status..."
panic: failed to check influxdb setup status - parse "://:": missing protocol scheme

goroutine 1 [running]:
github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x104ee80, 0x40003a8088}, {0x1057de0, 0x40004125b0})
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:14 +0xd4
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0x40003a7240, {0x1057de0, 0x40004125b0})
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0x90
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0x40003a7240)
	/go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x530
main.main.func2(0x40003b1280)
	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x288
github.com/urfave/cli/v2.(*Command).Run(0x4000418240, 0x40003b1100)
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x648
github.com/urfave/cli/v2.(*App).RunContext(0x400041c000, {0x103b030, 0x4000036050}, {0x4000032060, 0x2, 0x2})
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0x840
github.com/urfave/cli/v2.(*App).Run(...)
	/go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215
main.main()
	/go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x73c

@robflate
Copy link

robflate commented Jun 8, 2022

Confirming beta fixes this issue for me. Thanks.

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 9, 2022

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 9, 2022

@agneevX Ugh, I was really hoping that we had solved this issue, but you and @ViRb3 are both still seeing segfaults, so maybe I'll need to implement the fix for CGO as well.

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 9, 2022

Also, thanks @robflate @PhilipWhiteside @feroxy @ViRb3 @agneevX @rshxyz and everyone who tested the beta version and provided feedback.
I'm currently merging the changes from the beta version into master, and will release them as v0.4.10

@captainnapalm
Copy link

captainnapalm commented Jun 9, 2022

Although this already seems to be headed to master, I can also confirm that beta-omnibus resolved this issue for me.

@agneevX
Copy link

agneevX commented Jun 9, 2022

Is the image at latest supposed to resolve this? I still get segv faults with that image (built 10hrs ago).

EDIT: :beta-omnibus seems to exhibit similar behavior.

@benamajin
Copy link

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 10, 2022

I made some compilation changes to the beta-omnibus docker image. Can those of you still seeing segmentation faults pull that image and try it out? If your username isn't in the table below, please include the output of docker info when responding.

@agneevX @benamajin @ViRb3

Just for tracking purposes:

user arch segfaults?
@agneevX arm64 ?
@benamajin arm64 ?
@ViRb3 arm64 ?

@ViRb3
Copy link
Author

ViRb3 commented Jun 10, 2022

Still segfaults for me. Latest beta-omnibus update was 2 days ago, though. You sure it's pushed?

@causalityloop
Copy link

Still segfaults for me. Latest beta-omnibus update was 2 days ago, though. You sure it's pushed?

try latest...it works for me and had the same issue. looks like latest is today - from docker inspect

docker inspect ghcr.io/analogj/scrutiny:master-omnibu

"Labels": {
                "org.opencontainers.image.created": "2022-06-10T13:23:15.554Z",
                "org.opencontainers.image.description": "Hard Drive S.M.A.R.T Monitoring, Historical Trends & Real World Failure Thresholds",
                "org.opencontainers.image.licenses": "MIT",
                "org.opencontainers.image.revision": "7ba7b6efda1ed86f92214a434cf1b25e1d4f6fc3",
                "org.opencontainers.image.source": "https://github.com/AnalogJ/scrutiny",
                "org.opencontainers.image.title": "scrutiny",
                "org.opencontainers.image.url": "https://github.com/AnalogJ/scrutiny",
                "org.opencontainers.image.version": "master-omnibus"
            }

@ViRb3
Copy link
Author

ViRb3 commented Jun 10, 2022

Yes, but latest is master-omnibus, that's the stable version, not the beta.

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 10, 2022

Hey @causalityloop I think there's actually 2 issues being conflated here. Both cause segfaults which is why its even more confusing 😭

@ViRb3 apologies, looks like the CI failed for beta-omnibus I just retriggered it.

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 10, 2022

@ViRb3 can you try pulling now

@ViRb3
Copy link
Author

ViRb3 commented Jun 10, 2022

@AnalogJ Fixed!

@agneevX
Copy link

agneevX commented Jun 10, 2022

I can confirm that the segv faults are no longer present in the :beta-omninus image.

However, I cannot access the web UI (scrutiny api not ready in logs).

@AnalogJ
Copy link
Owner

AnalogJ commented Jun 11, 2022

Closing this issue as fixed. It'll be released in v0.4.11.
If anyone runs into the same issue, please feel free to comment or reopen this issue.

@agneevX can you open a new issue to track your scrutiny api not ready problem? I'll need your logs as well.

@paulmorabito
Copy link

I'm getting the same error using Unraid 6.10.3. I'm not sure how long it's been happening as I just noticed an alert my docker logs were filling fast.

I was previously using the linuxserver/scrutiny image but have swapped to the above recommended image and tags. Error is below:

2022/07/18 08:53:34 Loading configuration file: /opt/scrutiny/config/scrutiny.yaml
time="2022-07-18T08:53:34+02:00" level=info msg="Trying to connect to scrutiny sqlite db: \n"
time="2022-07-18T08:53:34+02:00" level=info msg="Successfully connected to scrutiny sqlite db: \n"
panic: failed to check influxdb setup status - parse "://:": missing protocol scheme

goroutine 1 [running]:
github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware.RepositoryMiddleware({0x119d3a0?, 0xc000118058?}, {0x119f210?, 0xc0004938f0?})
        /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:14 +0xa5
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Setup(0xc0001159b0, {0x119f210, 0xc0004938f0})
waiting for influxdb    /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:27 +0xb4
github.com/analogj/scrutiny/webapp/backend/pkg/web.(*AppEngine).Start(0xc0001159b0)

        /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/server.go:105 +0x6e5
main.main.func2(0xc000121200)
        /go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:112 +0x1f7
github.com/urfave/cli/v2.(*Command).Run(0xc000481e60, 0xc000121080)
        /go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/command.go:164 +0x5bb
github.com/urfave/cli/v2.(*App).RunContext(0xc00049e000, {0x1198308?, 0xc000036058}, {0xc000032040, 0x2, 0x2})
        /go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:306 +0xbc5
github.com/urfave/cli/v2.(*App).Run(...)
starting scrutiny       /go/src/github.com/analogj/scrutiny/vendor/github.com/urfave/cli/v2/app.go:215

main.main()
        /go/src/github.com/analogj/scrutiny/webapp/backend/cmd/scrutiny/scrutiny.go:137 +0x659

Any ideas on how to resolve this?

@AnalogJ
Copy link
Owner

AnalogJ commented Jul 18, 2022

@paulmorabito
Copy link

Thanks. I made some progress but the WebUI is not accessible (404 error) and the API is not ready. Logs are below:

2022/07/19 09:11:26 Loading configuration file: /opt/scrutiny/config/scrutiny.yaml

 ___   ___  ____  __  __  ____  ____  _  _  _  _
/ __) / __)(  _ \(  )(  )(_  _)(_  _)( \( )( \/ )
\__ \( (__  )   / )(__)(   )(   _)(_  )  (  \  /
(___/ \___)(_)\_)(______) (__) (____)(_)\_) (__)
github.com/AnalogJ/scrutiny                            dev-0.4.16

Start the scrutiny server
time="2022-07-19T09:11:26+02:00" level=info msg="Trying to connect to scrutiny sqlite db: /opt/scrutiny/config/scrutiny.db\n"
time="2022-07-19T09:11:26+02:00" level=info msg="Successfully connected to scrutiny sqlite db: /opt/scrutiny/config/scrutiny.db\n"
time="2022-07-19T09:11:26+02:00" level=info msg="Database migration starting. Please wait, this process may take a long time...."
time="2022-07-19T09:11:26+02:00" level=info msg="Database migration completed successfully"
time="2022-07-19T09:11:31+02:00" level=warning msg="127.0.0.1 - d1db5d7a8818 [19/Jul/2022:09:11:31 +0200] \"HEAD /api/health\" 404 19 \"\" \"curl/7.74.0\" (1ms)" clientIP=127.0.0.1 hostname=d1db5d7a8818 latency=1 method=HEAD path=/api/health referer= respLength=19 statusCode=404 userAgent=curl/7.74.0
scrutiny api not ready
time="2022-07-19T09:11:36+02:00" level=warning msg="127.0.0.1 - d1db5d7a8818 [19/Jul/2022:09:11:36 +0200] \"HEAD /api/health\" 404 19 \"\" \"curl/7.74.0\" (1ms)" clientIP=127.0.0.1 hostname=d1db5d7a8818 latency=1 method=HEAD path=/api/health referer= respLength=19 statusCode=404 userAgent=curl/7.74.0
scrutiny api not ready
time="2022-07-19T09:11:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests