Could not start job with variables after stopping (UI)

[urusha]

Nomad version

Nomad v1.6.2
BuildDate 2023-09-13T16:47:25Z
Revision 73e372a

Operating system and Environment details

debian 12 amd64

Issue

After stopping job with variables, it could not be started again in UI. Nomad 1.6.1 was OK.

Reproduction steps

Create job with variable (used in group/task definition):

variable "instances" {
  type = list(object({
    name  = string
    count = number
    tasks = list(object({
      name         = string
      memory       = number
      cpu          = number
      entrypoint   = list(string)
    }))
  }))
}

There are correct Job Spec,HCL Variable Values in Definition tab.
Stop the job in UI. Make no changes. Start the job in UI. Get error:

Could Not Start Job
Failed to parse job: <nil>: Unset variable "instances"; A used variable must be set or have a default value; see https://www.nomadproject.io/docs/job-specification/hcl2/variables for details.

This bash script starts the job successfully:

nomad job run -json -detach <(nomad job inspect -namespace "$n" "$j" | sed 's/"Stop": true,/"Stop": false,/')

Now we have only json data in Definition. It's ok. If the job is stopped again, on start (UI) we have another error:

Could Not Start Job
Job ID hasn't been provided

The same bash script still starts the job.

Expected Result

Job should start.

Actual Result

Error.

Job file (if appropriate)

Nomad Server logs (if appropriate)

Nomad Client logs (if appropriate)

[qk4l]

Is it possible to make hot fix for this?

[luhhujbb]

Hello,
In UI when trying to start a previously stopped job a call to https://nomad/v1/job/my-job/submission?version=1 return 404 error and thus fail to start the job as already mentioned in issue description.

[tgross]

Hi folks! This looked at first like duplicate of #17806 which was fixed in #18120 and shipped in Nomad 1.6.2. But it looks like this is specific to HCL variables so there might be something we missed there.

Is it possible to make hot fix for this?

I've marked the issue for triage. We don't comment on when things will be fixed.

In UI when trying to start a previously stopped job a call to https://nomad/v1/job/my-job/submission?version=1 return 404 error and thus fail to start the job as already mentioned in issue description.

If you don't have a problem specifically with HCL variables as in the issue description above (and you're already upgraded to 1.6.2), then please open a new issue describing the problem.

[philrenaud]

Hi @urusha — I've been able to test this out and reproduce locally, and it seems like it's less related to HCL variables and more related to the UI not handling failure cases well on stop/start. There's a PR to fix this at #18621 which I think should solve this issue as well. As such, I'm going mark it as resolving this issue as well.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.