fails to connect to ec2 if ip/host is not in known_hosts entry

[JigarJoshi]

it startsup ec2 instance when it attempts to ssh into it, it fails with

>>>>>> Converge failed on instance <default-centos-64>.
>>>>>> Please see .kitchen/logs/default-centos-64.log for more details 
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Authentication failed for user root@someip

before it attempts if I manually install entry in known_hosts list it goes further

[zerowolfgang]

@JigarJoshi Try running kitchen with the -l debug flags and if/when it fails again check the log file (as recommended) and search on SSH for hints. I've experienced this failure only when I either spec'd the wrong key in the .kitchen.yml config √, spec'd the AWS ssh key name, aws_ssh_key_id:, with it's suffix (not required) √, or did not have the (correct) key added to my ssh authentication agent √.

[JigarJoshi]

@zerowolfgang verified permission on key is correct, path to key is correct, I can ssh to instance using same key, also kitchen login works fine, it randomly fails with kitchen test

          [SSH] opening connection to root@SOME_IP_ADDRESS:22<{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :keys_only=>true, :port=>22, :keys=>["/path/to/some/key.pem"]}>
    D      Cleaning up local sandbox in /var/folders/h7/b7p0qb7n18ndmxcddxw_f9jmnr06m2/T/default-centos-64-sandbox-20140226-8307-wi91xx
    >>>>>> Converge failed on instance <default-centos-64>.
    >>>>>> Please see .kitchen/logs/default-centos-64.log for more details
    >>>>>> ------Exception-------
    >>>>>> Class: Kitchen::ActionFailed
    >>>>>> Message: Authentication failed for user root@SOME_IP
    >>>>>> ----------------------
    D      Converge failed on instance <default-centos-64>.
    D      ------Exception-------
    D      Class: Kitchen::InstanceFailure
    D      Message: Converge failed on instance <default-centos-64>.  Please see .kitchen/logs/default-centos-64.log for more details
    D      ---Nested Exception---
    D      Class: Kitchen::ActionFailed
    D      Message: Authentication failed for user root@SOME_IP
    D      ------Backtrace-------
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/driver/ssh_base.rb:112:in `rescue in run_remote'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/driver/ssh_base.rb:108:in `run_remote'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/driver/ssh_base.rb:44:in `block in converge'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/ssh.rb:47:in `initialize'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/driver/ssh_base.rb:43:in `new'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/driver/ssh_base.rb:43:in `converge'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/instance.rb:273:in `public_send'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/instance.rb:273:in `block in perform_action'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/instance.rb:308:in `call'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/instance.rb:308:in `synchronize_or_call'
    D      /Users/jigar.joshi/.rvm/gems/ruby-2.1.0/gems/test-kitchen-1.2.1/lib/kitchen/instance.rb:283:in `block in action'

[sethvargo]

@JigarJoshi by chance - are you using relative keys? Can you post your .kitchen.yml? There is a bug with using relative keys that might be the root cause of your issue.

[sethvargo]

And the output of kitchen diagnose is always helpful

[JigarJoshi]

@sethvargo Thanks, here is more detail

$ cat .kitchen.yml 
---
driver:
  name: ec2
  aws_access_key_id: SOME_KEY_ID
  aws_secret_access_key: SOME_KEY
  ssh_key: ~/.ssh/SOME_FILE.pem
  aws_ssh_key_id: SOME_ID
  region: us-west-2
  availability_zone: us-west-2c
  require_chef_omnibus: true
  subnet_id: subnet-SOME_ID
  security_group_ids: 'sg-SOME_GROUP_ID'
  flavor_id: 't1.micro'

provisioner:
  name: chef_solo

platforms:
  - name: centos-6.4

suites:
  - name: default
    run_list:
      - recipe[ci-testkitchen::default]
    attributes:

and

$ kitchen diagnose
---
timestamp: 2014-03-18 00:03:29.166126000 Z
kitchen_version: 1.2.1
instances:
  default-centos-64:
    state_file:
      hostname: SOME_IP
      last_action: verify
      server_id: SOME_ID
    driver:
      availability_zone: SOME_ZONE
      aws_access_key_id: SOME_KEY_ID
      aws_secret_access_key: SOME_KEY
      aws_session_token: 
      aws_ssh_key_id: SOME_KEY_ID_NAME
      ebs_optimized: false
      endpoint: https://ec2.us-west-2.amazonaws.com/
      flavor_id: t1.micro
      image_id: ami-SOME_ID
      interface: 
      kitchen_root: "/Users/jigar.joshi/dev-chef/cookbooks/ci-testkitchen"
      log_level: :info
      name: ec2
      port: 22
      region: us-west-2
      security_group_ids: sg-SOME_ID
      ssh_key: "~/.ssh/SOME_FILE_NAME.pem"
      subnet_id: subnet-SOME_ID
      sudo: true
      tags:
        created-by: test-kitchen
      test_base_path: "/Users/jigar.joshi/dev-chef/cookbooks/ci-testkitchen/test/integration"
      username: root
    provisioner:
      attributes: {}
      chef_omnibus_url: https://www.getchef.com/chef/install.sh
      clients_path: 
      cookbook_files_glob: README.*,metadata.{json,rb},attributes/**/*,definitions/**/*,files/**/*,libraries/**/*,providers/**/*,recipes/**/*,resources/**/*,templates/**/*
      data_bags_path: 
      data_path: 
      encrypted_data_bag_secret_key_path: 
      environments_path: 
      kitchen_root: "/Users/jigar.joshi/dev-chef/cookbooks/ci-testkitchen"
      log_level: :info
      name: chef_solo
      nodes_path: 
      require_chef_omnibus: true
      roles_path: 
      root_path: "/tmp/kitchen"
      run_list:
      - recipe[ci-testkitchen::default]
      solo_rb: {}
      sudo: true
      test_base_path: "/Users/jigar.joshi/dev-chef/cookbooks/ci-testkitchen/test/integration"
    busser:
      busser_bin: "/tmp/busser/bin/busser"
      kitchen_root: "/Users/jigar.joshi/dev-chef/cookbooks/ci-testkitchen"
      root_path: "/tmp/busser"
      ruby_bindir: "/opt/chef/embedded/bin"
      sudo: true
      suite_name: default
      test_base_path: "/Users/jigar.joshi/dev-chef/cookbooks/ci-testkitchen/test/integration"
      version: busser

[sethvargo]

Dupe of test-kitchen/test-kitchen#389

[jmahowald]

Not sure I agree that this is the same issue as supporting relative ssh keys. Mine has absolute in the rendered:

    ssh_key: <%= File.expand_path('~/.ssh/stage-voice.pem') %>

I still see this issue on the first run of converge. I tried setting

    no-host-key-verify:

and I still have the same issue.

[jmahowald]

I take back my previous statement. or more specifically having the value of

    no-host-key-verify: true

took care of the issues for me