ARMv7 compatibility and weird issues

[sagatowski]

Hello,
I've been trying to use TcUnit in my CX8180 project. Unfortunately, I've run into some weird issues and started to wonder if TcUnit is compatible with ARMv7 targets at all. I'm a twincat newbie so I'm not really sure whether the problem is in my libraries, settings or elsewhere and I'm at a loss as to why referencing a library which uses TcUnit causes all kinds of issues - unit testing code shouldn't even execute in such case, right?

At first, I developed and tested 2 libraries on local x64 twincat runtime and everything seemed ok. The moment I reference my libraries in CX8180 project (even if nothing uses them!) I lose ability to set breakpoints, value forcing is unreliable and PLC seems to randomly skip instructions.

To troubleshoot the issue I've tried the SequentialTestSuitesAndTests example project which when compiled and activated on CX8180 causes the same issue - even though the tests execute properly it is impossible to set a breakpoint anywhere and flow control doesn't work.

What am I doing wrong?

Originally posted by @chiveicrook2 in #147

[sagatowski]

Waiting for ArmV7 PLC before being able to reproduce.

[sagatowski]

Moved this to release 1.2. Was planning to release 1.2 next week but I think it's important to straighten this one out, as more and more people seem to be using the ARM-based PLCs (and probably even more once CX7000 is released).

[sagatowski]

Moving this to 1.3.0.0 because of time constraints. If anyone else has time to look into the issue, it would be highly appreciated.

[FirmacChris]

Just to give some feedback, I currently have what appears to be a very similar issue.
TwinCAT V3.1.4024.20 (I'm using the XAE Shell, I never tried the issue in a VS install).
TcUnit 1.2.0.0

The controller is a CX9020 running TwinCAT CE7 (ARM-V7).

Debugging/build/breakpoints worked fine using the XAR enviroment on my laptop (windows 10) but when I try to put it into my PLC I wasn't able to set breakpoints. Everything else seemed to work.

Trying to set a breakpoint anywhere in the program just gave a generic "PLC Control breakpoint could not be set on target" error message.

Removing TcUnit, and all references in the program, and then restarting XAE allowed me to set breakpoints again.

[oswin02]

Could not reproduce any of these problems on my setup.
Can anyone provide me a project where these issues appear?
There is a problem with the remote manager in TC 4024.20, so I will not update to this version.

My Config:

Engineering PC:
Windows 10
XAE Shell 3.1.4024.17 (tested also with remote manager to 4024.7)

Test PLC:
CX9020
TC 3.1.4024.12

TcUnit 1.2.0.0

[chiveicrook2]

On my CX8180 the issue is present in even the most basic projects, for example in all projects from tcunit example repos (https://github.com/tcunit/ExampleProjects). So far tested with 4024.7 and 4024.12 (both on engineering PC and CX) and TcUnit 1.1. Unfortunately, lack of time stopped me from debugging this further and I simply remove unit-tests before deploying to PLC...

[oswin02]

I could reproduce the issue on my plc with the SimpleTcUnitExampleProject and found a workaround.

Adding 'UnitTest' as condition for conditional referencing of the TcUnit library and adding 'UnitTest' in the compiler defines in the compile settings solved the brakepoint issue for me.

https://infosys.beckhoff.com/content/1033/tc3_plc_intro/2533017227.html?id=8439395065138194985
https://infosys.beckhoff.com/content/1033/tc3_plc_intro/4215784459.html?id=1823229847829719237

[sagatowski]

@FirmacChris and @chiveicrook2 Can you confirm that this work-around is OK/working?
I don't have an ARM-compatible controller on hand so I can't verify this.
Would it be enough to add this to the FAQ of TcUnit?

[Barteling]

Working on a CX9020 today and I had the same issues as discussed above.
The work around mentioned by @oswin02 did not help.

Some thoughts and remarks:

• Problem is not always present, sometimes settings breakpoints works fine, but once the breakpoint error comes up it's hard to get rid of it again.
• Does TwinCAT have some cache etc i can clear? sometime after restarting it seems to work fine again but not always.
• It smells like a memory problem, overlapping memory etc. Memory out of bounds, array index outside the bounds of the array, null reference pointer..
• Its already a problem before I even put the PLC in run, so i guess it must be in the declaration part of the code.

[candacespencer]

I am trying to test on on a CX7000 (ARMT2) and any project that references TcUnit will cause the project to build but fail to load on the controller.

I get the following error when trying to activate the project:
'Port_851' (851): PLC: Error (hr=0x98410027) loading boot project.

[sagatowski]

Is it enough for the project to reference TcUnit for it to fail? (so not executing TcUnit)?
Can you check the AMS memory allocation of the device with and without TcUnit?

Also, can you test to play around with the parameters related to memory allocation:
https://tcunit.org/frequently-asked-questions/?Display_FAQ=957

[candacespencer]

I was able to get the application to load and run a few unit tests by adjusting the parameters. Thanks.

[sagatowski]

@candacespencer Great! Could you post a screenshot of the values you used to get it to run?

[sagatowski]

@Barteling Can you confirm if changing the parameters will fix the issue for your CX9020 as well?

[maxxie85]

I modified the parameters and got it working on a CX8190 with the following settings

For clarification the following parameters are reduced in size

• MaxNumberOfTestSuites, dropped to 500
• MaxNumberOfTestsForEachTestSuite, dropped to 50
• MaxNumberOfAssertsForEachtSuite, dropped to 500

Increasing any of these settings to the original value will bring the problem back.

[sagatowski]

Thanks @maxxie85! It seems this is simply a memory problem as per @Barteling suggestion, probably due to the limited memory in the simple WinCE-CXs compared to the bigger W7/W10 cousins. I will add the above information to the FAQ of the TcUnit website and close this issue afterwards.

[sagatowski]

I have updated the FAQ with an entry for this in case anyone else encounters the same problem.

[ZigaJavornik]

I know this is a closed issue, but how does one work with 3rd party library that uses TcUnit? The parameters are not changeable in that case, I had to exclude the library for now:

[sagatowski]

I know this is a closed issue, but how does one work with 3rd party library that uses TcUnit? The parameters are not changeable in that case, I had to exclude the library for now:

How do you mean? If you're using a library that is using TcUnit, why would you want to change the parameters of TcUnit?

[ZigaJavornik]

I know this is a closed issue, but how does one work with 3rd party library that uses TcUnit? The parameters are not changeable in that case, I had to exclude the library for now:

How do you mean? If you're using a library that is using TcUnit, why would you want to change the parameters of TcUnit?

Well, If I attempt to download to a PLC that will be getting the upper errors, I would need to be able to either lower the test parameters or remove the reference due to lack of memory, is that not correct? From what I read, just a reference in the library manager is enough?

I was getting the mentioned breakpoints errors, I could not set it at all, using my own libraries, they also used Roalds TcError, which I also assumed is the problem. In the midst of debugging, and finding this in the FAQ, I had to just disable the tests from build and remove references from all the projects for now.

I will have to inspect further next week, this week I am on a different machine. But I have to note, once I change the parameters in my library that uses TcUnit, on my project that is using the created library, if I open the library manager, parameters are default values as well, despite me changing the value to a lower one inside the library project. Am I doing something wrong perhaps? I build, cleaned, installed, etc... No matter what, when i look in the library repo, the values are still at max, in the end I ended up getting "not enough memory" when I attempted to download to the PLC (CX8110) and I had to just remove the project reference as a whole, for now:

[I-Campbell]

to change library parameters, the tcUnit library needs to be added to the root level of the library manager in the final application. A library cannot influence the library parameters of another library. A bit annoying really, but workable.

[ZigaJavornik]

to change library parameters, the tcUnit library needs to be added to the root level of the library manager in the final application. A library cannot influence the library parameters of another library. A bit annoying really, but workable.

How do you go about using libraries and TcUnit then? I am not sure I understand how to deal with this.

[sagatowski]

@ZigaJavornik You only need to care about changing the parameters of YOUR library (that executes the TcUnit tests).
If your library depends on another library that uses TcUnit tests, that library-project will have its own set of parameters that will be stored only for that library. This is however not something you need to care about because that librarys tests will not be executed in your project. Therefore you don't need to be able to change the parameters of another library (Which as @I-Campbell has correctly noted, is not possible).

[oswin02]

I suggest deleting all tests before releasing a lib. Unit tests are not needed in a productive system.

[ZigaJavornik]

I suggest deleting all tests before releasing a lib. Unit tests are not needed in a productive system.

The whole point of tests is to have them run before a release is done. Deleting them, or disabling them, seems very unproductive way of going about it.

@sagatowski Even if I use my own library, as a released library, having TcUnit being referenced there, that is already the problem, as far as I can see. Do people just remove TcUnit when they release a public version to avoid this? I should add that I do not with to run unit tests on an ARM device, they will be ran on a virtual machine, I do however want to use these released libraries in production, of course.

[sagatowski]

I would NOT recommend to delete all tests prior to the release. The tests are excellent documentation of the behavior of your code that is released together with the library. I always ship the tests together with my POUs (albeit in a separate folder, like /test or alike in parallel to /POUs).

I don't understand why it is a problem to have TcUnit referenced in your library. Your libraries tests will not be executed in the context of whoever is using that library (whether that is another library or a PLC-executable).
The parameterization of TcUnit will ONLY affect whatever tests are running in that library. It will NOT affect any other tests that might be in any other libraries as these will simply not be executed.

If it's simply a matter of that you don't want to show the world that you are using TcUnit, you can hide your reference if you want to.

[ZigaJavornik]

I would NOT recommend to delete all tests prior to the release. The tests are excellent documentation of the behavior of your code that is released together with the library. I always ship the tests together with my POUs (albeit in a separate folder, like /test or alike in parallel to /POUs).

I don't understand why it is a problem to have TcUnit referenced in your library. Your libraries tests will not be executed in the context of whoever is using that library (whether that is another library or a PLC-executable). The parameterization of TcUnit will ONLY affect whatever tests are running in that library. It will NOT affect any other tests that might be in any other libraries as these will simply not be executed.

If it's simply a matter of that you don't want to show the world that you are using TcUnit, you can hide your reference if you want to.

Perhaps I have moved away from what the problem was, sorry.

My only "problem" (if you could call it that) with TcUnit was that the PLC that referenced the libraries using it, could not work normally (breakpoints error and during download to the PLC there would be "not enough memory" message), probably due to the memory problems mentioned in this thread. My whole point of how to change parameters/removing TcUnit was my thinking, that if there is only a reference to the library, the memory issue already exists, since I was able to download and debug normally after i removed TcUnit reference from my libraries and re-deployed them and added new reference in the PLC program to these new library releases that no longer had TcUnit in the.

I would never want to hide the fact that there are unit tests, or that I use TcUnit, on the contrary, it is a bonus and I want to know that the code is tested and written with best practices in mind. And just like you mentioned one time, the best way to get to know functionality of a library is trough tests, as opposed to comments (which are also nice, but together, its a win win).

I will put a pause on this until next week, when I will have time to re-check this on the PLC it-self as I am on other projects for the entire week. I hope that I explained better now what my only problem was/is.

[ZigaJavornik]

I can confirm that as soon as my project references my own library, which uses TcUnit, I am unable to set breakpoints. I am using CX8110, and I add my own library to the project, that library also contains unit tests and includes TcUnit. @sagatowski what would your suggestion be? For now, I disable TESTS folder and remove TcUnit reference before doing a release.

[sagatowski]

@ZigaJavornik Did you try any of the suggestions in this thread?
Also check this FAQ-entry: https://tcunit.org/#/faq?id=_12-i-have-problems-running-tcunit-on-a-armv7-controller-why

[ZigaJavornik]

I have, but as mentioned, when I import a library (my own) that uses TcUnit, it has default parameters, no matter if I change the parameters in the original project that uses TcUnit, you can see that here:

I have read trough the post, but it does not seem to solve the issue. So, step by step:

• I opened my library project that has TcUnit in it
• I set the number of tests to 100/20/100 for the parameters outlined (same as mentioned here https://tcunit.org/#/faq?id=_6-when-i-run-more-than-100-tests-in-a-single-test-suite-i-get-the-wrong-results-why)
• I recompiled saved and installed the library
• Then i open my project that only needs the "logic" of my library, but it still references TcUnit
• I reload the library reference to be the latest version, hoping to have reduced number of tests
• Looking at the parameters, they are still set to 1000, despite me attempting to set them to a lower value

If I understand correctly people ran tests on their ARMv7 devices, hence they were able to change the parameters to be able to run them without issues? I don't want to do that, all i need to do is use my tested libraries, which by default have TcUnit reference in it, and as mentioned, this automatically causes the bugs ...

[sagatowski]

@ZigaJavornik Thanks for clarification.
I don't know why you can't change the parameters for a library that you're not referencing directly. In one way I think it makes sense as your application does not use that library directly (it has no references to it), so it should not affect your execution.

But your PLC-program is not running any tests, right? It's only your library that is running tests.
So why do you need to change the TcUnit-parameters from the PLC-program? I don't think that will make any difference with the breakpoints.

Did you try to hide TcUnit completely? So it's not even visible in the PLC-program.

https://tcunit.org/#/faq?id=_4-is-there-a-way-to-hide-tcunit-in-my-libraries

[ZigaJavornik]

Yes, that is correct. I do not run any tests on the PLC, as it is a machine. I just have the library with tests included in the release. I will try by hiding the reference and see if it solves the problem. Do you perhaps have any clue if this is a "me" problem, or have you had similar experience before perhaps? I will let you know if I find a solution.

[ZigaJavornik]

I went ahead and read a bit about properties for libraries and I think this one might be what I am looking for :

If I understand correctly, I will put in some condition that will be false when instantiating it in the project and it should be disabled in that case - my assumption is going this way.

As for hide reference, it seems it only hides the reference from UI, it even says that compiler errors won't be shown and project might not compile if the error exists in the library, which leads me to believe that the library will still "exist":

[sagatowski]

That's what @oswin02 was referring to and it solved his problem. Try it out.

Also try hiding the library.

[ZigaJavornik]

Well, I missed that comment. Perhaps adding it to the FAQ or even the readme would be really neat :) I will do both next week, thank you @sagatowski

[sagatowski]

Well, I missed that comment. Perhaps adding it to the FAQ or even the readme would be really neat :) I will do both next week, thank you @sagatowski

You're more than welcome to make a PR and make an addition to the FAQ for this. If this is updated the website is automatically updated.

[ZigaJavornik]

Once I confirm it fixes the the issue I will do that, cheers.

[ZigaJavornik]

@sagatowski An update, today I attempted to load with condition, but I still had problems. I will do hide reference and condition tomorrow, hope that is the solution - it could be the case since even disabling TcUnit in the library project, when imported as a library to a third project and looking at the references, TcUnit was enabled (the reference in the imported library) and looking as if it were a normal reference in the release. Sorry for the weird phrasing, I am not really sure how to write this 🙉

Otherwise I will probably go ahead and make a script to create a release by deleting TcUnit reference, generating the library and adding it once again.

[ZigaJavornik]

@sagatowski I have finally got around to being on Beckhoff again :) I think I can confirm that under no circumstances can you remove the TcUnit from being included - I used compiler defines and made it disabled that way as well as hide reference. This still resulted in the GVL data to be generated, I can confirm this from target browser and not a single library in my project displays TcUnit (this is most likely how globals work with Beckhoff?). I belive the only way to really not have any data from TcUnit is to delete it before release and only add it when producing the libraries. If i delete TcUnit in my library before I create the release, the GVL data from TcUnit is deleted when i download again.

[ZigaJavornik]

With the help of some smart Beckhoff people, I have come to an acceptable solution for this. There was also some word about using compiler defines and within the TEST_PRG using {IF DEFINED} to really exclude any build occuring on the test FBs, but I will just include TcUnit and follow this solution.
The solution is as follows..

ABy default parameters are set to maximum and you can't exclude TcUnit. What is the best solution is, TcUnit is NOT excluded from build. Instead, the project that uses these libraries should have TcUnit as well, and then set these parameters to the highlighted values, this way you override the parameters from within the library, effectively setting the unit test count to 1:

This results in next to no memory usage by TcUnit in your project while having TcUnit enabled and working properly for library development (before setting tests to 0, GVL_TcUnit would use up around 100MB of space, ehnce problems with ARM devices);