Qemu@SSH Connection failed while buliding diskImage using Packer for full system simu.

[haotianmichael]

Hello dear gem5's developers,
I've been troubled for days by this problem. dont' know what to do~
I using packer validate mm2.json, it is good; packer builid mm2.json. It's always stuck in ssh connection. I don't know exactly how qemu's details about creating VM, but I am wondering if this problem comes from my env. I use docker to run gem5, and my server in which docker is running is a Intel Xeon server, it disables kvm for some reason, so i have to change qemucpu to qemu64, and following is my stdout and .json of packer, And i start ssh on my docker-machine using sudo service ssh enable; sudo service ssh start.Hoping I can get some hints about solutions.

root@9b9497153f71:/gem5/gem5/mm2/gem5/diskImage# ./packer build mm2.json
qemu: output will be in this color.

==> qemu: Retrieving ISO
==> qemu: Trying http://old-releases.ubuntu.com/releases/18.04.2/ubuntu-18.04.2-server-amd64.iso
==> qemu: Trying http://old-releases.ubuntu.com/releases/18.04.2/ubuntu-18.04.2-server-amd64.iso?checksum=md5%3A34416ff83179728d54583bf3f18d42d2
==> qemu: http://old-releases.ubuntu.com/releases/18.04.2/ubuntu-18.04.2-server-amd64.iso?checksum=md5%3A34416ff83179728d54583bf3f18d42d2 => /root/.cache/packer/e94c06241a5e6ba0c4b9618ddcda39ba2ee27e9a.iso
==> qemu: Creating floppy disk...
    qemu: Copying files flatly from floppy_files
    qemu: Copying file: shared/preseed.cfg
    qemu: Done copying files from floppy_files
    qemu: Collecting paths from floppy_dirs
    qemu: Resulting paths from floppy_dirs : []
    qemu: Done copying paths from floppy_dirs
    qemu: Copying files from floppy_content
    qemu: Done copying files from floppy_content
==> qemu: Starting HTTP server on port 8809
==> qemu: Found port for communicator (SSH, WinRM, etc): 3100.
==> qemu: Looking for available port between 5900 and 6000 on 127.0.0.1
==> qemu: Starting VM, booting from CD-ROM
    qemu: The VM will be run headless, without a GUI. If you want to
    qemu: view the screen of the VM, connect via VNC without a password to
    qemu: vnc://127.0.0.1:5908
==> qemu: Overriding default Qemu arguments with qemuargs template option...
==> qemu: Waiting 10s for boot...
==> qemu: Connecting to VM via VNC (127.0.0.1:5908)
==> qemu: Typing the boot commands over VNC...
    qemu: Not using a NetBridge -- skipping StepWaitGuestAddress
==> qemu: Using SSH communicator to connect: 127.0.0.1
==> qemu: Waiting for SSH to become available...
==> qemu: Timeout waiting for SSH.
==> qemu: Deleting output directory...
Build 'qemu' errored after 1 hour 2 minutes: Timeout waiting for SSH.

==> Wait completed after 1 hour 2 minutes

==> Some builds didn't complete successfully and had errors:
--> qemu: Timeout waiting for SSH.

==> Builds finished but no artifacts were created.

this is my mm2.json

{
    "_author": "HaotianMichael@Ucas",
    "_license": "",
    "builders":
    [
        {
            "type": "qemu",
            "format": "raw",
            "boot_command":
            [
                "{{ user `boot_command_prefix` }}",
                "debian-installer={{ user `locale` }} auto locale={{ user `locale` }} kbd-chooser/method=us ",
                "file=/floppy/{{ user `preseed` }} ",
                "fb=false debconf/frontend=noninteractive ",
                "hostname={{ user `hostname` }} ",
                "/install/vmlinuz noapic ",
                "initrd=/install/initrd.gz ",
                "keyboard-configuration/modelcode=SKIP keyboard-configuration/layout=USA ",
                "keyboard-configuration/variant=USA console-setup/ask_detect=false ",
                "passwd/user-fullname={{ user `ssh_fullname` }} ",
                "passwd/user-password={{ user `ssh_password` }} ",
                "passwd/user-password-again={{ user `ssh_password` }} ",
                "passwd/username={{ user `ssh_username` }} ",
                "-- <enter>"
            ],
            "cpus": "{{ user `vm_cpus`}}",
            "disk_size": "{{ user `image_size` }}",
            "floppy_files":
            [
                "shared/{{ user `preseed` }}"
            ],
            "headless": "{{ user `headless` }}",
            "http_directory": "shared/",
            "iso_checksum": "{{ user `iso_checksum_type` }}:{{ user `iso_checksum` }}",
            "iso_urls": [ "{{ user `iso_url` }}" ],
            "memory": "{{ user `vm_memory`}}",
            "output_directory": "mm2/{{ user `image_name` }}-image",
            "qemuargs":
            [
                [ "-cpu", "host" ],
                [ "-display", "none" ]
            ],
            "qemu_binary":"/usr/bin/qemu-system-x86_64",
            "shutdown_command": "echo '{{ user `ssh_password` }}'|sudo -S shutdown -P now",
            "ssh_password": "{{ user `ssh_password` }}",
            "ssh_username": "{{ user `ssh_username` }}",
            "ssh_wait_timeout": "60m",
            "vm_name": "{{ user `image_name` }}"
        }
    ],
    "provisioners":
    [
        {
            "type": "file",
            "source": "/gem5/gem5/util/m5/build/x86/out/m5",
            "destination": "/home/gem5/"
        },
        {
            "type": "file",
            "source": "/gem5/gem5/mm2/gem5/archKernel/data/in4.txt",
            "destination": "/home/mm2/"
        },
        {
            "type": "file",
            "source": "/gem5/gem5/mm2/gem5/diskImage/runscript.sh",
            "destination": "/home/gem5/"
        },
        {
            "type": "file",
            "source": "/gem5/gem5/mm2/gem5/mm2",
            "destination": "/home/mm2/"
        },
        {
            "type": "shell",
            "execute_command": "echo '{{ user `ssh_password` }}' | {{.Vars}} sudo -E -S bash '{{.Path}}'",
            "scripts":
            [
                "post-installation.sh"
            ]
        }
    ],
    "variables":
    {
        "boot_command_prefix": "<enter><wait><f6><esc><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "desktop": "false",
        "image_size": "12000",
        "headless": "true",
        "iso_checksum": "34416ff83179728d54583bf3f18d42d2",
        "iso_checksum_type": "md5",
        "iso_name": "ubuntu-18.04.2-server-amd64.iso",
        "iso_url": "http://old-releases.ubuntu.com/releases/18.04.2/ubuntu-18.04.2-server-amd64.iso",
        "locale": "en_US",
        "preseed" : "preseed.cfg",
        "hostname": "gem5",
        "ssh_fullname": "gem5",
        "ssh_password": "12345",
        "ssh_username": "gem5",
        "vm_cpus": "4",
        "vm_memory": "8192",
        "image_name": "mm2"
  }

}

Thanks!

[Harshil2107]

I think that the reason packer is stuck on waiting for ssh could be due to either ubuntu not being installed correctly thus not booting properly or network not being initialized correctly. I would suggest adding a display as a qemu arg and using a vnc viewer to look at the install process.

In gem5 resources there we have disk images made via packer, I suggest taking a look at those as well: https://github.com/gem5/gem5-resources/tree/885e91ae8fa7003fddb9f78771049a7650ab6468/src/x86-ubuntu

[haotianmichael]

Thank you very much~The Problem has been solved.It turned out that ssh_wait_timeout is not long enough, I guess it's because qemu would spend more time withtcg accelerator than in kvm. So the ssh must wait for a long time? Because my server doesn't supoort kvm for some reason, so I set ssh_wait_timeout to 180m and update the scripts, now it works!

[haotianmichael]

And do you know there will be some simulation precision losing becase of the abscence of kvm? I'm trying to get accurate branch miss data of gem5's Minor core of which the workload is a binary mm2, I'm wondering if the data is not that accurate if I build a diskImage with kvm acc disabled. Thanks for your precious time again!

[Harshil2107]

There shouldnt be any difference in simulation precision as far as I know. The only difference between KVM and no KVM should be speed.

[haotianmichael]

OK, Thanks a lot!