"Error, Can't Open ES App Startup file: /cf/cfe_es_startup.scr" when running a sample cpu1 application

[stanislaw]

Is your feature request related to a problem? Please describe.

I apologize in advance if the following issue that I am reporting is caused by my misunderstanding of the cFS/cFE CMake build system.

The issue is that I am getting the following issue when 1) I run the make command from the CFS repository and also 2) if I run it from CLion with the changes described in #478.

1)

cp cfe/cmake/Makefile.sample Makefile
cp -r cfe/cmake/sample_defs sample_defs
make && ./build/cpu1/cpu1/core-cpu1

The log:

...
Current working dir: /home/stanislaw/workspace/code/cFS
open(./cf/cfe_es_startup.scr): No such file or directory
1980-012-14:03:20.25202 zES Startup: Error, Can't Open ES App Startup file: /cf/cfe_es_startup.scr EC = 0xFFFFFFFF
1980-012-14:03:20.25205 ES Startup: CFE_ES_Main entering APPS_INIT state
1980-012-14:03:20.25206 ES Startup: CFE_ES_Main entering OPERATIONAL state
EVS Port1 42/1/CFE_TIME 21: Stop FLYWHEEL

2

cp cfe/cmake/Makefile.sample Makefile
cp -r cfe/cmake/sample_defs sample_defs
# Building cpu1-all and then core-cpu1 in CLion

The log:

1980-012-14:03:20.25314 ES Startup: CFE_ES_Main entering CORE_READY state
Current working dir: /home/stanislaw/workspace/code/cFS/cmake-build-debug-cfs-cpu1/cfe/cpu1
open(./cf/cfe_es_startup.scr): No such file or directory
1980-012-14:05:45.46664 zES Startup: Error, Can't Open ES App Startup file: /cf/cfe_es_startup.scr EC = 0xFFFFFFFF
1980-012-14:05:45.46666 ES Startup: CFE_ES_Main entering APPS_INIT state
1980-012-14:05:45.46667 ES Startup: CFE_ES_Main entering OPERATIONAL state
EVS Port1 42/1/CFE_TIME 21: Stop FLYWHEEL```

Looking for a missing file indicates that there is no way a running CFS can find a file because it is neither copied to a predetermined location nor the relative file ./cf/cfe_es_startup.scr points to a meaningful location.

$ find . | grep cfe_es_startup.scr
./cfe/cmake/sample_defs/cpu1_cfe_es_startup.scr
./sample_defs/cpu1_cfe_es_startup.scr

Describe the solution you'd like

Our understanding is that the cfe_es_startup.scr is an important file and without loading its contents CFS does not work properly. We would expect some action to exist in the CMake script files that would copy this file and make it available to the running cFS.

Describe alternatives you've considered

We assume that there is no other way of having CFS to run properly. There should be a procedure or instructions on how to make the .scr file available to CFS.

Additional context

None.

Requester Info

Stanislav Pankevich (PTS, private German space company)

[jphickey]

Please run make install which stages the build artifacts to an executable area. By default this stages to build/exe but you can override this to wherever you want your target files to go.

[jphickey]

Also - the POSIX OSAL assumes your cwd is the base of the staged directory tree. To run CFE, you would do something like

cd build/exe/cpu1 && ./core-cpu1

(i.e. do not invoke cfe with a relative path such as ./build/exe/cpu1/core-cpu1)

[stanislaw]

I have spent some time looking into the expectations that CLion has about the structure of the build folder and I think now I understand the issue.

CMake and CLion, that follows it closely, distinguish between the build output tree and the install tree.

The build tree is the one which is CMAKE_BINARY_DIR and this is where CMake builds all artifacts. For example, for binary files, this is controlled via CMAKE_{RUNTIME,LIBRARY,ARCHIVE}_OUTPUT_DIRECTORY variables and can be configured by CMake globally or set for specific targets using target_set_properties (like described here: How do I make CMake output into a 'bin' dir?.

When running CLion, it sets the current working directory for running the core-cpu1 binary to the CMAKE_RUNTIME_OUTPUT_DIRECTORY of that target which is in my case: /sandbox/cFS/build.cpu1.dir/cfe/cpu1/core-cpu1.

The install tree is controlled by CMAKE_INSTALL_PREFIX and assumes that a developer has to install the target to some final destination in order to run it. The install behavior is not taken into account by CLion which always runs the binary from the CMAKE_RUNTIME_OUTPUT_DIRECTORY directory so it is obviously missing everything that CFS configures for the install procedure.

I have prepared a patch without opening a PR yet to get your thoughts on whether that change would be seen as a positive one: along with the existing install command I am adding the necessary copy commands that also copy the needed files: .scr file and the .so files to the binary output directory of the core-* targets like core-cpu1 and this makes CFS runtime to recognize these and load these files correctly when I run them from CLion.

This is the branch and if you think it makes sense I can open the actual Pull Request:
https://github.com/nasa/cFE/compare/master...stanislaw:copy-the-files?expand=1.

---

This is the log that I am having now on macOS:

/sandbox/cFS/build.cpu1.dir/cfe/cpu1/core-cpu1
CFE_PSP: Default Reset Type = PO
CFE_PSP: Default Reset SubType = 1
CFE_PSP: Default CPU ID = 1
CFE_PSP: Default Spacecraft ID = 42
CFE_PSP: Default CPU Name: cpu1
CFE_PSP: Starting the cFE with a POWER ON reset.
Current working dir: /sandbox/cFS/build.cpu1.dir/cfe/cpu1
CFE_PSP: Clearing out CFE CDS Shared memory segment.
CFE_PSP: Clearing out CFE Reset Shared memory segment.
CFE_PSP: Clearing out CFE User Reserved Shared memory segment.
2030-021-16:00:32.71158 POWER ON RESET due to Power Cycle (Power Cycle).
2030-021-16:00:32.71162 ES Startup: CFE_ES_Main in EARLY_INIT state
CFE_PSP: CFE_PSP_AttachExceptions Called
2030-021-16:00:32.71164 ES Startup: CFE_ES_Main entering CORE_STARTUP state
2030-021-16:00:32.71165 ES Startup: Starting Object Creation calls.
2030-021-16:00:32.71165 ES Startup: Calling CFE_ES_CDSEarlyInit
2030-021-16:00:32.71169 ES Startup: Calling CFE_EVS_EarlyInit
2030-021-16:00:32.71170 Event Log cleared following power-on reset
2030-021-16:00:32.71170 ES Startup: Calling CFE_SB_EarlyInit
2030-021-16:00:32.71172 SB internal message format: CCSDS Space Packet Protocol version 1
2030-021-16:00:32.71172 ES Startup: Calling CFE_TIME_EarlyInit
1980-012-14:03:20.00000 ES Startup: Calling CFE_TBL_EarlyInit
1980-012-14:03:20.00005 ES Startup: Calling CFE_FS_EarlyInit
1980-012-14:03:20.00013 ES Startup: Core App: CFE_EVS created. App ID: 0
EVS Port1 42/1/CFE_EVS 1: cFE EVS Initialized. cFE Version 6.7.4.0
EVS Port1 42/1/CFE_EVS 14: No subscribers for MsgId 0x808,sender CFE_EVS
1980-012-14:03:20.05025 ES Startup: Core App: CFE_SB created. App ID: 1
1980-012-14:03:20.05027 SB:Registered 4 events for filtering
EVS Port1 42/1/CFE_SB 1: cFE SB Initialized
EVS Port1 42/1/CFE_SB 14: No subscribers for MsgId 0x808,sender CFE_SB
1980-012-14:03:20.10133 ES Startup: Core App: CFE_ES created. App ID: 2
EVS Port1 42/1/CFE_ES 1: cFE ES Initialized
EVS Port1 42/1/CFE_SB 14: No subscribers for MsgId 0x808,sender CFE_ES
EVS Port1 42/1/CFE_ES 2: Versions:cFE 6.7.4.0, OSAL 5.0.5.0, PSP 1.4.1.0, chksm 65535
EVS Port1 42/1/CFE_SB 14: No subscribers for MsgId 0x808,sender CFE_ES
EVS Port1 42/1/CFE_ES 91: Mission 6.7.3-bv-11-gdf8bb89-dirty.sample, CFE: 6.7.3-bv-16-g15ffb15-dirty, OSAL: 5.0.3-bv-25-g92ff052
EVS Port1 42/1/CFE_SB 14: No subscribers for MsgId 0x808,sender CFE_ES
EVS Port1 42/1/CFE_ES 92: Build 202001221419 Stanislaw@home
1980-012-14:03:20.15241 ES Startup: Core App: CFE_TIME created. App ID: 3
EVS Port1 42/1/CFE_TIME 1: cFE TIME Initialized
1980-012-14:03:20.20343 ES Startup: Core App: CFE_TBL created. App ID: 4
EVS Port1 42/1/CFE_TBL 1: cFE TBL Initialized.  cFE Version 6.7.4.0
1980-012-14:03:20.25444 ES Startup: Finished ES CreateObject table entries.
1980-012-14:03:20.25445 ES Startup: CFE_ES_Main entering CORE_READY state
1980-012-14:03:20.25453 ES Startup: Opened ES App Startup file: /cf/cfe_es_startup.scr
1980-012-14:03:20.25485 ES Startup: Loading shared library: /cf/sample_lib.so
Current working dir: /sandbox/cFS/build.cpu1.dir/cfe/cpu1
1980-012-14:03:20.27761 ES Startup: Loading file: /cf/sample_app.so, APP: SAMPLE_APP
Current working dir: /sandbox/cFS/build.cpu1.dir/cfe/cpu1
1980-012-14:03:20.30207 ES Startup: SAMPLE_APP loaded and created
...

[stanislaw]

I would like to add that we use CLion as a default IDE for all developers in our company, this is why some of the issues that I have been opening recently are coming from my attempts to run CFS on CLion, as easy as possible. Additionally, we think that CLion is a very intelligent IDE when it comes to working with CMake projects so it would be great if we could get the first-class CLion support for CFS and from what I see it should be just a few minor PRs that should not affect the non-CLion usage by other developers. Hope this gives a bit more context.

[stanislaw]

One more observation about the build trees and install trees that complements my above comment: #482 (comment).

Relying on install trees only also brings a disadvantage of having the installed binaries built in the Release configuration by CMake. One outcome of this is that it is not possible to debug things in the IDE easily (and I guess from the command line too!).

These are the flags that CMake uses now for example:

# cfe/fsw/cfe-core/cmake-build-debug/CMakeCache.txt
//Flags used by the C compiler during RELEASE builds.
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

//Flags used by the C compiler during RELWITHDEBINFO builds.
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

Not only you get O3 but also the -DNDEBUG flag which disables asserts.

This altogether creates a rather limited debugging experience for me as a C/C++ CLion IDE user.

[jphickey]

The fundamental issue is that CFE isn't the simplest piece of software in this regard, at least in the default config. It isn't a single executable that you can just run and expect it to work. There is an associated script and a number of binaries (i.e. each individual loadable application/library) and they all need to be arranged in a proper directory tree so the CFE can find them at runtime. This does not match the directory structure that that CMake outputs, hence why the "install" step is absolutely needed in order to stage the binaries and scripts into the right arrangement for execution.

IMO this is an issue with your CLion config. For instance in Eclipse I just change the build target to be install rather than all and point the debugger at ${workspace}/build/exe/cpu1. After this simple change, in-IDE building and debugging works just fine. I assume a similar change must be possible in CLion.

As for the release vs. debug issue, this is also expected. When you build with optimization, debugging is limited. That's why the default build for an IDE, when you anticipate debugging, should probably be Debug, not Release or RelWithDebInfo. If you want to build both, just make separate build trees (i.e. build-debug and build-release) with different build types, and point your debugger at the build-debug one.

[jphickey]

Also note, if you really do want to run CFE directly from the CMake build tree (without install) this is actually possible if you statically link the application and the apps. See the STATIC_APPLIST (rather than APPLIST) in the targets.cmake file. This causes the apps to be statically linked rather than dynamically linked. You can also statically link your startup script via a PSP module containing an embedded file, and enable OS_STATIC_LOADER in OSAL and list out each of your app entry points in a symlist in the targets file The result is a single self-contained executable that can be executed and has no dependency on dynamic loading.

[stanislaw]

The fundamental issue is that CFE isn't the simplest piece of software in this regard, at least in the default config. It isn't a single executable that you can just run and expect it to work. There is an associated script and a number of binaries (i.e. each individual loadable application/library) and they all need to be arranged in a proper directory tree so the CFE can find them at runtime. This does not match the directory structure that that CMake outputs, hence why the "install" step is absolutely needed in order to stage the binaries and scripts into the right arrangement for execution.

I have spent some time with CFS already and I started to appreciate the complexity and the need in the two-tree CMake build process. I have only one comment about the CMake capabilities: having the proper runtime structure is easily possible with CMake and build tree as well. In my patch quoted above I seem to be doing everything just fine from the build tree without relying on the installed parts: https://github.com/nasa/cFE/compare/master...stanislaw:copy-the-files?expand=1. I could have missed certain files to be copied but they can be added in a very similar way.

IMO this is an issue with your CLion config. For instance in Eclipse I just change the build target to be install rather than all and point the debugger at ${workspace}/build/exe/cpu1. After this simple change, in-IDE building and debugging works just fine. I assume a similar change must be possible in CLion.

I will check if it is possible but it looks like running from the build tree is the default way when CLion just works without additional customization.

As for the release vs. debug issue, this is also expected. When you build with optimization, debugging is limited. That's why the default build for an IDE, when you anticipate debugging, should probably be Debug, not Release or RelWithDebInfo. If you want to build both, just make separate build trees (i.e. build-debug and build-release) with different build types, and point your debugger at the build-debug one.

I understand this very well but thanks for the hint: I need to double-check this again in my settings and maybe I missed something because it is too many moving parts while I am learning about CFS.

P.S. Thanks for the hint and instructions how to switch to static linking. It is one of my next things on my list of learning.

[skliper]

Can we close based on original topic being resolved? Feel free to open a new thread for other topics.