latest error: error querying Consul checks: time: invalid duration

[SergeyOvsienko]

Hi all.
I have nomad cluster:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:        18.04
Codename:       bionic

nomad version
Nomad v0.8.7 (21a2d93eecf018ad2209a5eab6aae6c359267933+CHANGES)

consul version
Consul v1.4.1
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

I am create service for test:

job "nginx" {
  datacenters = ["dc1"]
  type = "service"

  group "nginx" {
    count = 3

    task "nginx" {
      driver = "docker"

      config {
        image = "nginx"
        port_map {
          http = 80
        }
        volumes = [
          "custom/default.conf:/etc/nginx-test/conf.d/default.conf"
        ]
      }

      template {
        data = <<EOH
          server {
            location / {
              root /local/data/;
            }
          }
        EOH

        destination = "custom/default.conf"
      }

      template {
        data = <<EOH
          Good morning, huy.
        EOH

        destination = "local/data/index.html"
      }

      resources {
        cpu    = 100 # 100 MHz
        memory = 128 # 128 MB
        network {
          mbits = 10
          port "http" {}
        }
      }

      service {
        name = "nginx"
        tags = ["nginx"]
        port = "http"
        check {
          type     = "http"
          port     = "http"
          path     = "/"
          interval = "10s"
          timeout  = "2s"
        }
      }
    }
  }
}

Service is created success:

nomad job status nginx
ID            = nginx
Name          = nginx
Submit Date   = 2019-02-06T16:07:36Z
Type          = service
Priority      = 50
Datacenters   = dc1
Status        = running
Periodic      = false
Parameterized = false

Summary
Task Group  Queued  Starting  Running  Failed  Complete  Lost
nginx       0       0         3        0       0         0

Allocations
ID        Node ID   Task Group  Version  Desired  Status   Created     Modified
3215a919  13587718  nginx       0        run      running  18h55m ago  18h50m ago
7d4b7d8d  fe634a83  nginx       0        run      running  18h55m ago  18h50m ago
a7e16307  54482f07  nginx       0        run      running  18h55m ago  18h50m ago

But this service does not register in consul. In nomad logs:

Feb 07 10:48:59 ip-172-31-45-174 nomad[12808]:     2019/02/07 10:48:59.861949 [ERR] consul.sync: still unable to update services in Consul after 2420 failures; latest error: error querying Consul checks: time: invalid duration
Feb 07 10:50:02 ip-172-31-45-174 nomad[12771]:     2019/02/07 10:50:02.616703 [ERR] consul.sync: still unable to update services in Consul after 2380 failures; latest error: error querying Consul checks: time: invalid duration
Feb 07 10:53:59 ip-172-31-45-174 nomad[12808]:     2019/02/07 10:53:59.876478 [ERR] consul.sync: still unable to update services in Consul after 2430 failures; latest error: error querying Consul checks: time: invalid duration
Feb 07 10:55:02 ip-172-31-45-174 nomad[12771]:     2019/02/07 10:55:02.631797 [ERR] consul.sync: still unable to update services in Consul after 2390 failures; latest error: error querying Consul checks: time: invalid duration

What i am doing wrong?

[SergeyOvsienko]

System type service is registered normal

job "fabio" {
  datacenters = ["dc1"]
  type = "system"

  group "fabio" {
    task "fabio" {
      driver = "raw_exec"

      artifact {
        source = "https://github.com/fabiolb/fabio/releases/download/v1.5.10/fabio-1.5.10-go1.11.1-linux_amd64"
      }

      config {
        command = "fabio-1.5.10-go1.11.1-linux_amd64"
      }

      resources {
        cpu = 512
        memory = 512
        network {
          port "http" {
            static = 9999
          }
          port "admin" {
            static = 9998
          }
        }
      }
    }
  }
}

[shantanugadgil]

Hi,
There was a bug in Consul 1.4.1 (fixed in v 1.4.2) ...
Ref: hashicorp/consul#5270

Would it be possible to upgrade Consul to 1.4.2 and see if the problem "goes away" 😁

Cheers,
Shantanu

[SergeyOvsienko]

Yes, i can update consul cluster. I am will inform.

[SergeyOvsienko]

@shantanugadgil Thx for help. Yes, consul update is helped

[shantanugadgil]

Glad I could help, Thanks! 😄

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.