IAR support broken with Pelion DM Client during ASM

[screamerbg]

Description

While trying to compile Pelion DM tests (and also example programs) for IAR, compilation fails with the following message:

$ mbed test -t IAR -m DISCO_L475VG_IOT01A -n simple-mbed-cloud-client-tests*dev* -v

[SNIP]

Macros: -DDEVICE_I2C=1 -D__MBED__=1 -DDEVICE_I2CSLAVE=1 -D__FPU_PRESENT=1 -DDEVICE_PORTOUT=1 -DUSBHOST_OTHER -DDEVICE_PORTINOUT=1 -DTARGET_RTOS_M4_M7 -DDEVICE_RTC=1 -DDEVICE_MPU=1 -D__CMSIS_RTOS -DTARGET_FF_ARDUINO -DDEVICE_USTICKER=1 -DFEATURE_BOOTLOADER=1 -DDEVICE_CAN=1 -DTARGET_CORTEX_M -DDEVICE_I2C_ASYNCH=1 -DTARGET_LIKE_CORTEX_M4 -DDEVICE_ANALOGOUT=1 -DTARGET_M4 -DTARGET_STM32L475xG -DDEVICE_QSPI=1 -DTARGET_STM32L4 -DDEVICE_SPI_ASYNCH=1 -DDEVICE_LPTICKER=1 -DTOOLCHAIN_IAR -DDEVICE_INTERRUPTIN=1 -DCOMPONENT_QSPIF=1 -DCOMPONENT_PSA_SRV_IMPL=1 -DTARGET_CORTEX -DTARGET_DISCO_L475VG_IOT01A -DTRANSACTION_QUEUE_SIZE_SPI=2 -D__CORTEX_M4 -DDEVICE_STDIO_MESSAGES=1 -DDEVICE_PWMOUT=1 -DTARGET_FAMILY_STM32 -DFEATURE_COMMON_PAL=1 -DDEVICE_PORTIN=1 -DTARGET_RELEASE -DTWO_RAM_REGIONS -DTARGET_STM -DDEVICE_SERIAL_FC=1 -DCOMPONENT_PSA_SRV_EMUL=1 -DMBED_BUILD_TIMESTAMP=1543622640.5 -DDEVICE_TRNG=1 -DTARGET_LIKE_MBED -D__MBED_CMSIS_RTOS_CM -DDEVICE_SLEEP=1 -DDEVICE_SPI=1 -DCOMPONENT_NSPE=1 -DTARGET_WIFI_ISM43362 -DDEVICE_SPISLAVE=1 -DDEVICE_ANALOGIN=1 -DDEVICE_SERIAL=1 -DDEVICE_FLASH=1 -DTARGET_STM32L475VG -DARM_MATH_CM4
Compile [  0.1%]: cmain.S
Compile: D:\Tools\IAR\IAR 7.8\arm\bin\iasmarm --cpu Cortex-M4F -DTRANSACTION_QUEUE_SIZE_SPI=2 -D__CORTEX_M4 -DARM_MATH_CM4 -D__FPU_PRESENT=1 -DUSBHOST_OTHER -D__MBED_CMSIS_RTOS_CM -D__CMSIS_RTOS -DTWO_RAM_REGIONS -f .\BUILD\tests\DISCO_L475VG_IOT01A\IAR\asm_macros_d292d317e7777fda6310dff5bcba2c50.xcl -o BUILD\tests\DISCO_L475VG_IOT01A\IAR\mbed-os\cmsis\TARGET_CORTEX_M\TOOLCHAIN_IAR\cmain.o .\mbed-os\cmsis\TARGET_CORTEX_M\TOOLCHAIN_IAR\cmain.S
[DEBUG] Return: 3
Failed to build library
Memory map breakdown for built projects (values in Bytes):
| name | target | toolchain | static_ram | total_flash |
|------|--------|-----------|------------|-------------|

Build failures:
  * DISCO_L475VG_IOT01A::IAR::MBED-BUILD

[mbed] ERROR: "d:\python\python.exe" returned error.
       Code: 1
       Path: "D:\mbed\pelion-enablement"
       Command: "d:\python\python.exe -u D:\mbed\pelion-enablement\mbed-os\tools\test.py -t IAR -m DISCO_L475VG_IOT01A --source . --build .\BUILD\tests\DISCO_L475VG_IOT01A\IAR --test-spec .\BUILD\tests\DISCO_L475VG_IOT01A\IAR\test_spec.json --build-data .\BUILD\tests\DISCO_L475VG_IOT01A\IAR\build_data.json -n simple-mbed-cloud-client-tests*dev* -v --greentea"
       Tip: You could retry the last command with "-v" flag for verbose output

Upon further investigation, running the IAR's iasmarm command "D:\Tools\IAR\IAR 7.8\arm\bin\iasmarm" --cpu Cortex-M4F -DTRANSACTION_QUEUE_SIZE_SPI=2 -D__CORTEX_M4 -DARM_MATH_CM4 -D__FPU_PRESENT=1 -DUSBHOST_OTHER -D__MBED_CMSIS_RTOS_CM -D__CMSIS_RTOS -DTWO_RAM_REGIONS -f .\BUILD\tests\DISCO_L475VG_IOT01A\IAR\asm_macros_d292d317e7777fda6310dff5bcba2c50.xcl -o BUILD\tests\DISCO_L475VG_IOT01A\IAR\mbed-os\cmsis\TARGET_CORTEX_M\TOOLCHAIN_IAR\cmain.o .\mbed-os\cmsis\TARGET_CORTEX_M\TOOLCHAIN_IAR\cmain.S returns:

Error in command line: Invalid argument to option '-D'

I'm attaching the asm_macros_d292d317e7777fda6310dff5bcba2c50.xcl for convenience -
asm_macros_d292d317e7777fda6310dff5bcba2c50.zip

Upon even further investigation, removing the aforementioned file from the iasmarm command allows the file to be compiled successfully, e.g.

"D:\Tools\IAR\IAR 7.8\arm\bin\iasmarm" --cpu Cortex-M4F -DTRANSACTION_QUEUE_SIZE_SPI=2 -D__CORTEX_M4 -DARM_MATH_CM4 -D__FPU_PRESENT=1 -DUSBHOST_OTHER -D__MBED_CMSIS_RTOS_CM -D__CMSIS_RTOS -DTWO_RAM_REGIONS -o BUILD\tests\DISCO_L475VG_IOT01A\IAR\mbed-os\cmsis\TARGET_CORTEX_M\TOOLCHAIN_IAR\cmain.o .\mbed-os\cmsis\TARGET_CORTEX_M\TOOLCHAIN_IAR\cmain.S

    IAR Assembler V7.80.4.12462/W32 for ARM
    Copyright 1999-2017 IAR Systems AB.

Errors:   0
Warnings: 0
Bytes: 44

Continuing to try to compile the tests and manually removing the .xcl file from the compile helps me to finally compile + link the tests, and then successfully pass greentea run:

+-------------------------+---------------------+-------------------------------------------------+----------------------------+--------+--------+--------+--------------------+
| target                  | platform_name       | test suite                                      | test case                  | passed | failed | result | elapsed_time (sec) |
+-------------------------+---------------------+-------------------------------------------------+----------------------------+--------+--------+--------+--------------------+
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Connect to WiFi            | 1      | 0      | OK     | 4.23               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Format LFS                 | 1      | 0      | OK     | 0.22               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Initialize QSPIF+LFS       | 1      | 0      | OK     | 0.05               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Initialize Simple PDMC     | 1      | 0      | OK     | 2.86               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Pelion DM Bootstrap & Reg. | 1      | 0      | OK     | 13.68              |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Pelion DM Directory        | 1      | 0      | OK     | 6.18               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Pelion DM Re-register      | 1      | 0      | OK     | 5.93               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Post-reset Identity        | 1      | 0      | OK     | 2.4                |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Resource LwM2M GET         | 1      | 0      | OK     | 0.64               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Resource LwM2M POST        | 1      | 0      | OK     | 0.57               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Resource LwM2M PUT         | 1      | 0      | OK     | 0.68               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-connect | Resource LwM2M SET         | 1      | 0      | OK     | 0.37               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Connect to WiFi            | 1      | 0      | OK     | 5.75               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Firmware Download          | 1      | 0      | OK     | 68.65              |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Firmware Prepare           | 1      | 0      | OK     | 0.19               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Firmware Update            | 1      | 0      | OK     | 13.25              |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Format LFS                 | 1      | 0      | OK     | 0.21               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Initialize QSPIF+LFS       | 1      | 0      | OK     | 0.04               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Initialize Simple PDMC     | 1      | 0      | OK     | 2.87               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Pelion DM Bootstrap & Reg. | 1      | 0      | OK     | 13.73              |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Pelion DM Directory        | 1      | 0      | OK     | 6.55               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Pelion DM Re-register      | 1      | 0      | OK     | 5.77               |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Post-update Erase          | 1      | 0      | OK     | 0.0                |
| DISCO_L475VG_IOT01A-IAR | DISCO_L475VG_IOT01A | simple-mbed-cloud-client-tests-dev_mgmt-update  | Post-update Identity       | 1      | 0      | OK     | 2.52               |
+-------------------------+---------------------+-------------------------------------------------+----------------------------+--------+--------+--------+--------------------+
mbedgt: test case results: 24 OK
mbedgt: completed in 258.03 sec

This makes me think that there's something fundamentally broken in the IAR support in Mbed OS 5.10.4 / master, where the toolchain assembling receives application level macros/config (some of which look invalid for IAR), while it's expected that assembling should only receive the target level config variables.

Note that this impacts all Pelion examples and fails with both mbed-os-5.10.4 and mbed-os master branch.

CC @theotherjimmy @cmonr @bridadan

How to reproduce:

1. mbed import pelion-enablement
2. Follow the setup instructions at https://github.com/ARMmbed/pelion-enablement
3. mbed test -t IAR -m DISCO_L475VG_IOT01A -n simple-mbed-cloud-client-tests*dev* -v

@ashok-rao Please label this in JIRA.
FYI @MarceloSalazar regarding Mbed OS 5.11 OOB.

Issue request type

[ ] Question
[ ] Enhancement
[X] Bug

[cmonr]

Interesting.

Looking at this problem, it looks like it stretches back all the way back to 5.10.

Doing the following results in the same error:

cd mbed-os
git checkout mbed-os-5.10
cd ..
<test cmd>

Running the command any sooner than 5.10 doesn't work because the BOOTLOADER feature wasn't introduced until them.

[screamerbg]

If you look at ARM/IAR/GCC_ARM toolchains, the assemble() routine calls self.get_symbols(True), which filters out only the target and generic macros (for_asm=True):

• Here https://github.com/ARMmbed/mbed-os/blob/master/tools/toolchains/arm.py#L205
• And then here https://github.com/ARMmbed/mbed-os/blob/master/tools/toolchains/__init__.py#L172

If you look at the latter file, many macros are filtered out for when doing assemble, yet for whatever reason, the app level config macros are being passed to IAR iasmarm.

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-235

[kjbracey]

Passing of any macros to IAR was originally missing: #6670
Macro passing was added here: #6752

That was required to solve various problems, but it seems that didn't filter the same as the other toolchains?

Last problem I saw in this area was incorrect escaping of string literals (like the error URL) causing failure. Found here: #8023, fix/workaround commit added to that PR: 62a55a6 (filter all string literal macros to IAR)

If you know the other 2 toolchains are filtering stuff, then applying same filtering to IAR makes sense. If it makes the problem go away, good, but would still be nice to know what the actual tool complaint is. Is there an incorrect escaping, or are we triggering a variant of the IAR 7.80 bug?

(I don't see any reason in principle why you shouldn't be able to pass mbed conf options to the assembler, but clearly it's causing practical issues).

[adbridge]

@theotherjimmy please take a look at this

[bridadan]

I believe I've found the issue causing the IAR assembler to error. A macro was being repeated in the .xcl file, specifically "-DPAL_DTLS_PEER_MIN_TIMEOUT=5000". It doesn't look like the IAR assembler likes repeated macros.

This macro appears twice in this case because it is set as a macro in the mbed_app.json of the pelion-enablement repository as well as an actual parameter set by the simple-mbed-cloud-client library.

For consistency reasons, I'd recommend removing the macro definition from the pelion-enablement mbed_app.json file and leave the macro to be set by the parameter. That being said, the config system should probably be able to handle this case gracefully, so I've added a fix in #8982.