Create functional testing suite

[peplin]

The currently lacks a good functional testing suite - i.e. something that runs on the actual VI hardware. We're really good as far as unit tests for the code that be compiled with a desktop C compiler (we're using LLVM's Clang these days) - 91% coverage! - but there are still some critical pieces of code that can only run on the target platform. One spot that's been a source of bugs in the past in the CAN acceptance filter configuration. This is some of my least favorite code in the entire project.

With that in mind, I'd like to take the few latest VI bugs that @zacnelson has reported (thanks, Zac) and use that to drive the creation of a functional test suite.

What Are We Trying to Test?

First up are a few tests of USB in isolation:

• Detect the USB device
• Query for version
• Query for device ID

Then, there are some integration tests that check both sending commands via USB and reading VI output at the same time they check that the VI's CAN controllers are configured properly. In all of these tests, each CAN controller is configured in loopback mode so a message sent on the bus is also received on that bus without requiring a physical loopback cable (e.g. to connect CAN1 to CAN2). If we want to also test the physical connection (e.g. when building a new VI) we could have another testing firmware build that requires a physical loopback.

For each controller (CAN1 and CAN2):

• Send CAN message and (after it loops back) receive that CAN message through USB
• Send a writable simple vehicle message via USB, receive that simple vehicle message via USB after it loops back
• Send a diagnostic request via USB, receive CAN message output after it loops back and verify
• Send a CAN message that contains a diagnostic response, receive diagnostic response output via USB after it loops back
• Send a CAN message that contains a diagnostic response, receive diagnostic response formatted with OBD-II output handler via USB after it loops back
• Add a recurring diagnostic request, confirm outgoing CAN messages
• Cancel recurring diagnostic request, confirm no outgoing CAN messages
• With CAN AF configured, send a message not in AF and confirm nothing is received.
  • Receipt of message matching AF is tested earlier
• With CAN AF in bypass, confirm both predefined and unrecognized CAN messages are received
• Dynamically turn bus 1 AF on, verify the AF can still be changed on the fly for diagnostic requests (right now because of global AF on LPC17xx, only really testing our software AF with the test firmware build)
• Test pre-defined OBD2 PIDs, received as simple vehicle messages
• Test sending and receiving extend CAN frames

TODO

• Add a control command to enable/disable CAN AF bypass mode #278
• Extend the canutil API to allow setting to loopback mode, not just AF on/off. #279
• Create a functional testing firmware build #280
• Create a script that uses openxc-python to send a receive the messages required for functional test suite #281
• Add a control command tochange CAN AF bypass on the fly #301
• Add a control command to change output format on the fly #302
• Add a control command to enable translating known OBD-II PIDs on the fly #303

Future Work

• A version of the testing firmware that requires a physical loopback cable (i.e. connecting CAN1 to CAN2) so we can make sure the physical connection is operational. This cable would require the termination resistors, I think. I've made one before for manufacturing testing for the Ford reference VI and it worked well.
• Implement as much of this as possible as a self-test mode in the VI so it doesn't require an external driver. It should be really, really fast, so it could theoretically run every time it boots up.

[peplin]

For the most part, done!

Split out into a few other issues (#297, #298, #300) and need to run the tests on the chipKIT and C5 to see if it can catch the reported issues with CAN2.