Onefuzz reports a 'missing field' error for the optional field preserve_existing_outputs

[puhley]

Information

• Onefuzz version: 5.0.0
• OS: Initiated from a Mac CLI for an Ubuntu scaleset

Provide detailed reproduction steps (if any)

1. Attempt to launch a libfuzzer merge task with the following command line:
   onefuzz template libfuzzer merge my_project my_build v1.0 my_pool --existing_inputs azure-container-with-files --output_container azure-output-container --target_exe my_libfuzzer.exe --preserve_existing_outputs
   This will launch the job without any visible errors:

INFO:onefuzz:creating libfuzzer merge from template
INFO:onefuzz:creating job (runtime: 24 hours)
INFO:onefuzz:created job: 12345678-aaaa-eeee-dddd-386a493dea6f
INFO:onefuzz:using container: oft-setup-272c805....
INFO:onefuzz:using container: oft-unique-inputs-80534....
INFO:onefuzz:uploading target exe `my_libfuzzer.exe`
INFO:onefuzz:creating libfuzzer_merge task
INFO:onefuzz:done creating tasks
{
    "config": {
        "build": "v1.0",
        "duration": 24,
        "name": "my_build",
        "project": "my_project"
    },
    "job_id": "12345678-aaaa-eeee-dddd-386a493dea6f",
    "state": "init",
    "user_info": {
        "application_id": "43210765-aaaa-3333-aaaa-123679c3d834",
        "object_id": "87654321-ffff-1111-2222-7ebace27ca0e",
        "upn": "onefuzz@example.org.test"
    }
}

2. Use onefuzz jobs list to notice that the job immediately fails with a state of stopped. Use the job_id to retrieve the relevant tasks via: onefuzz jobs tasks list {job_id}.
3. Inspect the output for the task and notice the following:

...
            "task": {
                "check_asan_log": false,
                "check_debugger": true,
                "check_fuzzer_help": true,
                "duration": 24,
                "preserve_existing_outputs": true,
                "reboot_after_setup": false,
                "rename_output": false,
                "target_exe": "my_libfuzzer.exe",
                "target_options_merge": false,
                "type": "libfuzzer_merge"
            }
        },
        "end_time": "2022-02-24 06:48:46+00:00",
        "error": {
            "code": "TASK_FAILED",
            "errors": [
                "task failed. exit_status:code=1 signal=None success=False",
                "",
                "Error: missing field `preserve_existing_outputs`\n"
            ]
        },

It is worth noting that in the above output, that the task flags show that preserve_existing_outputs is set to true which is what is expected when the flag is set in the command line.

4. Performing the above steps again without the --preserve_existing_outputs flag will also produce the same error regarding the missing field. In the task section of the output, the preserve_existing_outputs flag is correctly marked as false since the flag was omitted.

...
            "task": {
                "check_asan_log": false,
                "check_debugger": true,
                "check_fuzzer_help": true,
                "duration": 24,
                "preserve_existing_outputs": false,
                "reboot_after_setup": false,
                "rename_output": false,
                "target_exe": "my_libfuzzer.exe",
                "target_options_merge": false,
                "type": "libfuzzer_merge"
            }
        },
        "end_time": "2022-02-24 06:57:16+00:00",
        "error": {
            "code": "TASK_FAILED",
            "errors": [
                "task failed. exit_status:code=1 signal=None success=False",
                "",
                "Error: missing field `preserve_existing_outputs`\n"
            ]
        },

Expected result

The preserve_existing_outputs field is a boolean. As such, it should always have a default value and setting the flag is only necessary when overriding the value. As onefuzz template libfuzzer merge --help points out:

  --preserve_existing_outputs
                        (Default: False. Sets value to True)

The libfuzzer merge task should execute appropriately and pass the value for preserve_existing_outputs through the entire workflow.

Actual result

The libfuzzer task is recognizing the flag from the command line since it is correctly showing up in the task output for the job. However, that value is not being passed all the way through the execution flow which results in a missing field error. This error is showing up regardless of whether the flag was passed on the command line.

Other notes

The missing field error appears to come from the handlers.py file for job_templates:

onefuzz/src/cli/onefuzz/job_templates/handlers.py

Line 135 in 1822acf

raise Exception("missing field: %s" % field.name)

In the tasks defs for the Azure functions app, preserve_existing_outputs does show up in the generic_merge definition:

onefuzz/src/api-service/__app__/onefuzzlib/tasks/defs.py

Line 335 in 338b541

TaskType.generic_merge: TaskDefinition(

However, that field is missing from the libfuzzer_merge definition in the same file:

onefuzz/src/api-service/__app__/onefuzzlib/tasks/defs.py

Line 216 in 338b541

TaskType.libfuzzer_merge: TaskDefinition(

The field does show up in both the Rust libfuzzer_merge code:

onefuzz/src/agent/onefuzz-agent/src/tasks/merge/libfuzzer_merge.rs

Line 34 in 94e2904

pub preserve_existing_outputs: bool,

and the Python templates code:

onefuzz/src/cli/onefuzz/templates/libfuzzer.py

Line 399 in b49d140

preserve_existing_outputs: bool = False,

AB#35875

[chkeita]

@puhley are you open to submit a PR for the fix? From your investigation it looks like adding preserve_existing_outputs to the libfuzzer_merge definition should fix the issue.

[puhley]

A quick update from my side: I was mistaken as to the source of the missing field error. It is actually coming from the Rust application's Serde library: https://github.com/serde-rs/serde/blob/c3ce2c934a9123b5a309256d65cca6b6d19466ec/serde/src/de/mod.rs#L285

Adding preserve_existing_outputs to the Task Definition did not fix the issue. The tasks output from the API service shows that it received the parameter. However, it is failing to be acquired by the Rust application. I am still researching where it is getting lost in the process.

[puhley]

I believe that the above patch will address the issue. I've tested locally but please give it a review. I copied the approach from the check_fuzzer_help flag.

[mgreisen]

The referenced PR needs to be converted to C# code now.