Add PSP unit tests

[skliper]

Currently, the JSC developed PSP unit tests are not included in the CFS release. They currently work with pc-linux and OrionSCP PSPs. Updates to so that they work with a VxWorks target PSP are in work (ut699-vxworks6.7 PSP)

These updates are being tracked in the JSC subversion repo and need to be pushed into a proper git branch and further work continued from there.

[skliper]

Imported from trac issue 10. Created by sduran on 2015-02-26T14:12:17, last modified: 2019-08-14T14:13:14

[skliper]

Trac comment by sduran on 2015-05-05 16:00:01:

This branch also contains updates to the grut699 PSP - they had to be done here to get for the unit tests to work. They involve issues discovered by the unit test or parts of the PSP that were incomplete(exception handling) that had to be completed to have something to tests the unit tests against. These will have to be merged into trac-11.

[skliper]

Trac comment by sduran on 2015-06-08 10:44:21:

Updates are no longer in JSC svn repo, but in bablefish branch trac-10-add-psp-unit-tests

[skliper]

Trac comment by sduran on 2015-06-08 10:47:51:

Not only are the updated PSP unit tests included, but VxWorks ut699 PSP updates are being made in order to pursue a Class A safety-critical classification of this software for use by Orion and potentially others. The CFS AES project performed static code analysis utilizing the Understand tool for all MISRA rules, ran static cppcheck tool for potential safety concerns, ran the existing unit tests on a vxworks/LEON3 target with WindRiver Coverage analysis enabled to determine unit test code coverage, and held a general code inspection process. Specific changes as a result will be as follows:

-Updating all ut699 PSP files to comply with "most" MISRA rules and cppcheck static analysis errors
-Updating all ut699 PSP files to remove compiler warnings
-Expanding existing unit tests to provide full API-level test coverage based on exposed uncovered code (black box testing)
-Adding a full ADDITIONAL unit test suite using UTAssert framework to provide full code and BRANCH level coverage of the ut699 PSP source code (white box testing)

Subsequent commits should address these issues.

[skliper]

Trac comment by gdecaruf on 2015-06-09 19:30:42:

Change to cfe_psp_start.c following Static Code analysis of cFE ES static code analysis.

• Change to CFE_ES_Main function !StartupFileName argument casting from uint8 to char to satisfy Misra rule 6.1, where strings must be char type.
• Corrections made to call in mac-osx, mcf5235-rtems, pc-cygwin, pc-linux

commit: [changeset:5a72a68]

[skliper]

Trac comment by sduran on 2015-06-10 15:15:06:

The change addressing MISRA rule 6.1 is in commit: 5a72a68

[skliper]

Trac comment by sduran on 2015-06-11 17:01:07:

Commit [changeset:6fd8801] addresses the MISRA rule violations.

[skliper]

Trac comment by dasp on 2015-08-01 15:43:48:

commit: [changeset:612f00f3] Initial white box tests for cfe_psp_memory.c

[skliper]

Trac comment by abrown4 on 2015-08-07 17:04:58:

commit: [changeset:c210e81] Trac #14 Added ut-assert for white box PSP tests.
(Needed to build and run [changeset:612f00f3].)

Yes, this is a duplicate ut-assert, but it is put here for expediency. Later, ut-assert should be pulled up out of every individual app, osal, and psp that it supports, but for now it is safer and more efficient to commit it here.

[skliper]

Trac comment by abrown4 on 2015-08-07 17:05:40:

commit: [changeset:4b9525c] Trac #14 added .gitignore for psp white box tests.

[skliper]

Trac comment by dasp on 2015-08-08 10:41:39:

commit: [changeset:ad4e7c5f] Initial white box unit tests for cfe_psp_start.c.
commit: [changeset:a7795ee1] Update to tests required to run on target

[skliper]

Trac comment by abrown4 on 2015-08-10 16:07:15:

commit: [changeset:a68d6f5] and [changeset:d91f334] cfe_psp_ram.c white box tests (linux and target).

[skliper]

Trac comment by abrown4 on 2015-08-10 17:34:22:

commit: [changeset:c3b9703] Trac #14 Added null pointer tests for psp ram white-box test. See #34

[skliper]

Trac comment by abrown4 on 2015-08-11 15:51:29:

commit: [changeset:a0a2a9b] & [changeset:05dfc40] Trac #14 Added white-box test for cfe_psp_memutils.c. See #35 #36

[skliper]

Trac comment by dasp on 2015-08-11 17:19:53:

Created ticket #37 for cfe_psp_memory.c white box test violations.

[skliper]

Trac comment by abrown4 on 2015-08-11 18:15:19:

commit: [changeset:ea19e38] Trac #14 Added white-box test for cfe_psp_memrange.c. See #38

[skliper]

Trac comment by abrown4 on 2015-08-11 20:47:27:

commit: [changeset:091e92e] and [changeset:6ba4fcb] Trac #14 Added white box test for cfe_psp_exception.c.

[skliper]

Trac comment by abrown4 on 2015-08-12 19:34:44:

comit: [changeset:2d93a56] Trac #14 Added white-box test for cfe_psp_support.c.

[skliper]

Trac comment by dasp on 2015-08-13 18:41:11:

commit: [changeset:998ebe4a] Initial white box unit tests for cfe_psp_watchdog.c

[skliper]

Trac comment by dasp on 2015-08-13 18:43:25:

commit: [changeset:eade7377] Initial white box unit tests for cfe_psp_timer.c

[skliper]

Trac comment by dasp on 2015-08-13 18:46:20:

Created ticket #39 for issues identified during white box testing of cfe_psp_watchdog.c

[skliper]

Trac comment by dasp on 2015-08-13 19:14:03:

Created ticket #40 for the issue identified during white box testing of cfe_psp_timer.c

[skliper]

Trac comment by dasp on 2015-08-13 19:21:15:

Created ticket #41 for the issue identified during white box testing of cfe_psp_start.c

[skliper]

Trac comment by abrown4 on 2015-08-17 15:42:24:

commit: [changeset:2234ae6] merged fixes for #34 and #36 into #14 branch.

[skliper]

Trac comment by abrown4 on 2015-08-17 15:51:48:

commit: [changeset:1868407] Trac #38 Updated CFE_PSP_MemRangeSet() comments.

[skliper]

Trac comment by abrown4 on 2015-08-17 17:29:26:

commit: [changeset:5563878] Trac #39 Added range-limited calaculation to CFE_PS_WatchdogSet().

[skliper]

Trac comment by abrown4 on 2015-08-18 12:23:31:

commit: [changeset:1931394] Trac #41 Updated CFE_PSP_GetRestartType() to ignore null pointer. Updated white-box unit test for testing & added "()" for compiler warning.

[skliper]

Trac comment by abrown4 on 2015-08-18 15:15:47:

commit: [changeset:319cce3] Merged changes from #35 and #37 into #14 branch.

[skliper]

Trac comment by jphickey on 2015-12-01 10:16:52:

I understand the genesis of this, but ultimately at the end of the day the CCB goal here is to release a source code package that can be distributed via SourceForge (or anywhere) that contains everything that a project would need to build, run, and test CFE on their target of choice.

If we don't include makefiles for the tests, then this is a problem.

In the "classic" GSFC build, the tarball/source repo has sample makefiles that are typically cloned and owned by the mission. While I don't advocate this approach (the clone/own part), at least it has all the necessary pieces of information within the source code repository such that someone can get it and build it.

We've gone over this time and time again - JSC uses their own custom makefiles, which is fine if you choose to do that, but as a CCB we cannot test against these since they are not in the CCB repository nor will they be included in a tarball when we finally release the code. We cannot expect non-JSC users to write the makefile from scratch though - we need to include something with the code.

[skliper]

Trac comment by sduran on 2016-02-08 15:26:33:

A build of the unit tests, with makefiles included in the PSP, has been accomplished. However, it does require several environment variables to be set(there are dependencies on cfe and osal .h files). The build also has dependencies on mission_inc, osconfig.h, and cfe_platform_cfg.h for a given psp. While the build does work, further though on where these files should be located and brought into the build is needed.

[skliper]

Trac comment by sduran on 2016-02-08 15:52:25:

Two specific changes in PSP trac-10 should be reviewed and considered for acceptance into PSPs. As the change is implemented, it has no impact to those that do not want to use it and is very useful to those that do. The changes of interest are in the following changes sets in psp trac-10.

commit ID: [changeset:d423ce6]

• Adds VOLTAB macro to cfe_psp_voltab.c
• Rather than only being able to use the default /cf physical location or having each developer make local changes to the voltab.c file, this modification allows the location of the /cf directory to be specified at build time(it could be provided on the command line or as we use the option at JSC, set in the top level makefile).
• If the default location is acceptable, then developers do nothing new at build time.

commit ID: [changeset:44a5ba3]

• Add += to TARGET_DEFS in compiler-opts.mak so that -DVOLTAB is included in build options

[skliper]

Trac comment by sduran on 2016-03-08 12:16:54:

Current compiler errors(per Susie's request) when building grut699-vxworks6 against the psp development branch - . The linux build, builds/runs with out issue. Nothing too complicate here, just have not had the time to trace it all down.

sparc-wrs-vxworks-gcc -I/home/sduran/COP_CFS_Workspace/osal/src/os/inc -I/home/sduran/COP_CFS_Workspace/cfe/fsw/cfe-core/src/inc -I/home/sduran/COP_CFS_Workspace/psp/fsw/inc -I/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/inc -I/home/sduran/CFSTST_Workspace/CFS_TST/CFS_TST/build/mission_inc -I../../inc -Wa,-a=cfe_psp_memory.lis -D_EB -DENDIAN=EB -DSOFTWARE_BIG_BIT_ORDER -DVOLTAB_PATH='"/ram0/cf"' -D_VXWORKS_OS -D__SPARC__ -Dvxworks6 -D_EMBED_ -DTOOL_FAMILY=gnu -DTOOL=gnuv8 -D_WRS_KERNEL -DCPU=SPARC -fno-builtin -fvolatile -Wcast-align -m32 -mtune=ut699 -mv8 -ffloat-int-mode -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/wrn/coreip -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/drv -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/drv/os_ext -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/src/config -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/src/drv -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/config/comps/src -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/config/comps/src/dosfs2 -Wall -ansi -g -O0 -DOS_DEBUG_LEVEL=3 -c -o cfe_psp_memory.o /home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c
In file included from /gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/private/memPartLibP.h:55,
from /gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/memPartLib.h:17,
from /gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/moduleLib.h:63,
from /home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:32:
/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/vsbConfig.h:42:2: warning: #warning "VxWorks Source Build (VSB) project not specified; using default VxWorks UP configuration under $WIND_BASE/target/lib/h/config"
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:209: error: conflicting types for 'CFE_PSP_GetResetArea'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:287: error: previous declaration of 'CFE_PSP_GetResetArea' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: In function 'CFE_PSP_GetResetArea':
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:219: warning: cast increases required alignment of target type
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: At top level:
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:248: error: conflicting types for 'CFE_PSP_GetUserReservedArea'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:294: error: previous declaration of 'CFE_PSP_GetUserReservedArea' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: In function 'CFE_PSP_GetUserReservedArea':
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:258: warning: cast increases required alignment of target type
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: At top level:
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:287: error: conflicting types for 'CFE_PSP_GetVolatileDiskMem'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:300: error: previous declaration of 'CFE_PSP_GetVolatileDiskMem' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: In function 'CFE_PSP_GetVolatileDiskMem':
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:297: warning: cast increases required alignment of target type
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: At top level:
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:363: error: conflicting types for 'CFE_PSP_GetKernelTextSegmentInfo'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:306: error: previous declaration of 'CFE_PSP_GetKernelTextSegmentInfo' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:405: error: conflicting types for 'CFE_PSP_GetCFETextSegmentInfo'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:311: error: previous declaration of 'CFE_PSP_GetCFETextSegmentInfo' was here
make[2]: *** [cfe_psp_memory.o] Error 1
make[2]: Leaving directory /home/sduran/CFSTST_Workspace/CFS_TST/CFS_TST/build/ut699-vxworks6/cfe/psp' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory /home/sduran/CFSTST_Workspace/CFS_TST/CFS_TST/build/ut699-vxworks6/cfe'
gmake: *** [one_cfs_build] Error 2
[jsc-er-gfe09:~/CFSTST_Workspace/CFS_TST/CFS_TST/build/ut699-vxworks6]$

[skliper]

Trac comment by jphickey on 2016-03-08 13:08:41:

This looks like a fairly straightforward error. The prototypes in cfe_psp.h were made to be type correct, so it uses the cpuaddr type instead of void* for returning the address. For instance, the prototype signature now looks like (for example):

{{{
int32 CFE_PSP_GetResetArea (cpuaddr *PtrToResetArea, uint32 *SizeOfResetArea)
}}}

(The first parameter is no longer void pointer type)

[skliper]

Trac comment by glimes on 2016-03-08 14:30:07:

I'm working with a GRUT-700, very similar; I will grab Steve's error messages,
move them into a new ticket, and provide an update to the PSP to resolve these
issues for these PSP APIs where parameter types were changed.

[skliper]

Trac comment by glimes on 2016-03-08 14:38:22:

See also PSP ticket #56

[skliper]

Trac comment by jhageman on 2019-05-01 16:44:50:

Refocusing suggestion - use this ticket to integrate the PSP black box tests that would be useful for testing/certifying PSPs.

[skliper]

Trac comment by jphickey on 2019-05-01 16:50:56:

Just to make sure we are on the same page -- I thought the unit test being discussed was a "white box" coverage test in the form of the other GSFC unit tests.

Can we clarify what exactly a "black box" test would be? From the sounds of it, This would be a new methodology that is different than what is in CFE.

[skliper]

Trac comment by jhageman on 2019-05-01 17:10:48:

I think of unit tests as coverage tests, which are by necessity "white box" and tightly coupled to the written code, and I wouldn't expect unit tests for PSPs to be common.

By "black box" I'm thinking fully exercising the API to show a given PSP conforms with the (possibly as-of-yet not fully defined) specification. This I would expect to be part of a common certification framework package to show the PSP behaves per the required specs (including error responses, nominal functionality, etc). I thought JSC had developed something like this (or at least a starting point) for the class A testing which would be a huge benefit to the community.

Jumping into the middle of all the history in this ticket I could easily be way off base. If the intent of this ticket was really just integrate a custom BSP's set of unit tests (tightly coupled to code and not a general testing solution), I'd suggest it isn't all that useful to control via the CCB (just make a techdev branch and close the ticket as invalid).

[skliper]

Trac comment by jhageman on 2019-05-06 09:46:07:

Note JSC branch name updates for naming standards compliance:

trac-10-add-psp-unit-tests -> techdev-pc-linux-uei
ic-sgd-dev-trac-10-merge -> techdev-sp0-vxworks6.9 (also likely contains latest tests)
add-ut700-vxworks6.9_psp -> techdev-ut700-vxworks6.9
additional_ut699_changes_from_trac-10 -> techdev-ut699-vxworks

[skliper]

Closing as duplicate of #168, really the thread is OBE anyways.