communication failure when invoking qstat on remote host updated job state

[coleslaw481]

Panfish running on gordon moved a job to completed state when qstat came back with an error. The code should have just left the job in its current state.

Tue Nov 26 16:40:01 2013 (1385512801) INFO [main:243] Exit Code: 0
Tue Nov 26 16:41:01 2013 (1385512861) INFO [Panfish::QstatJobWatcher:188] State updated on 0 job(s) on gordon_shadow.q
Tue Nov 26 16:41:01 2013 (1385512861) INFO [main:243] Exit Code: 0
Unable to communicate with gordon-fe2.local(10.5.1.2)
Unable to communicate with gordon-fe2.local(10.5.1.2)
Communication failure.
qstat: cannot connect to server gordon-fe2.local (errno=15096) Unable to get connection to socket
Tue Nov 26 16:42:01 2013 (1385512921) ERROR [Panfish::PBSJobStateHashFactory:60] Unable to run /opt/torque/bin/qstat :
Tue Nov 26 16:42:01 2013 (1385512921) INFO [Panfish::QstatJobWatcher:188] State updated on 1 job(s) on gordon_shadow.q
Tue Nov 26 16:42:01 2013 (1385512921) INFO [main:243] Exit Code: 0
Tue Nov 26 16:43:01 2013 (1385512981) INFO [main:243] Exit Code: 0