Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GNU ARM Eclipse exporter #3561

Merged
merged 31 commits into from
Feb 17, 2017
Merged

Add GNU ARM Eclipse exporter #3561

merged 31 commits into from
Feb 17, 2017

Conversation

ilg-ul
Copy link
Contributor

@ilg-ul ilg-ul commented Jan 10, 2017

NOTICE: the GNU ARM Eclipse exporter went public with mbed version 5.4.

Preliminary documentation is available from:

https://docs.mbed.com/docs/mbed-os-handbook/en/latest/dev_tools/third_party/

For further problems, please open new issues.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 10, 2017

Hi @theotherjimmy, @0xc0170

This is the preliminary version of the exporter, not yet functional.

The following problems were identified:

  • the self.resources.objects was disabled, since it generates a duplicate main.o
  • the mbed-os folder was excluded from the build, since itself is not a source folder
  • without the sources in the mbed-os folder, the link obviously fails
  • I had to add #include "mbed_config.h" in platform/platform.h, otherwise compiling main.c unresolved macro MBED_CONF_PLATFORM_DEFAULT_SERIAL_BAUD_RATE

It looks like I'm missing something, or the current configuration might have some problems.

For reference, I issued the following commands:

mbed import mbed-os-example-blinky
cd mbed-os-example-blinky
rm -rf mbed-os
ln -s /Users/ilg/My\ Files/MacBookPro\ Projects/GNU\ ARM\ Eclipse/src/mbed-os.git mbed-os
mbed export -m K64F -i gnuarmeclipse

The full console log is:

ilg-mbp:mbed ilg$ mbed import mbed-os-example-blinky
[mbed] Importing program "mbed-os-example-blinky" from "https://github.com/ARMmbed/mbed-os-example-blinky" at latest revision in the current branch
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #2885c1b41e63
ilg-mbp:mbed ilg$ ls -l
total 0
drwxr-xr-x  11 ilg  staff  374 Jan 10 21:32 mbed-os-example-blinky
ilg-mbp:mbed ilg$ cd mbed-os-example-blinky
ilg-mbp:mbed-os-example-blinky ilg$ ls -l
total 32
-rw-r--r--   1 ilg  staff  2737 Jan 10 21:32 README.md
drwxr-xr-x   3 ilg  staff   102 Jan 10 21:32 img
-rw-r--r--   1 ilg  staff   213 Jan 10 21:32 main.cpp
drwxr-xr-x  26 ilg  staff   884 Jan 10 21:32 mbed-os
-rw-r--r--   1 ilg  staff    77 Jan 10 21:32 mbed-os.lib
-rw-r--r--   1 ilg  staff  1329 Jan 10 21:32 mbed_settings.py
ilg-mbp:mbed-os-example-blinky ilg$ rm -rf mbed-os
ilg-mbp:mbed-os-example-blinky ilg$ ln -s /Users/ilg/My\ Files/MacBookPro\ Projects/GNU\ ARM\ Eclipse/src/mbed-os.git mbed-os
ilg-mbp:mbed-os-example-blinky ilg$ ls -l
total 40
-rw-r--r--  1 ilg  staff  2737 Jan 10 21:32 README.md
drwxr-xr-x  3 ilg  staff   102 Jan 10 21:32 img
-rw-r--r--  1 ilg  staff   213 Jan 10 21:32 main.cpp
lrwxr-xr-x  1 ilg  staff    71 Jan 10 21:33 mbed-os -> /Users/ilg/My Files/MacBookPro Projects/GNU ARM Eclipse/src/mbed-os.git
-rw-r--r--  1 ilg  staff    77 Jan 10 21:32 mbed-os.lib
-rw-r--r--  1 ilg  staff  1329 Jan 10 21:32 mbed_settings.py
ilg-mbp:mbed-os-example-blinky ilg$ mbed export -m K64F -i gnuarmeclipse
/Library/Python/2.7/site-packages/fuzzywuzzy/fuzz.py:35: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Scan: .
Scan: FEATURE_BLE
Scan: FEATURE_COMMON_PAL
Scan: FEATURE_LWIP
Scan: FEATURE_UVISOR
Scan: FEATURE_LOWPAN_BORDER_ROUTER
Scan: FEATURE_LOWPAN_HOST
Scan: FEATURE_LOWPAN_ROUTER
Scan: FEATURE_NANOSTACK
Scan: FEATURE_NANOSTACK_FULL
Scan: FEATURE_THREAD_BORDER_ROUTER
Scan: FEATURE_THREAD_END_DEVICE
Scan: FEATURE_THREAD_ROUTER
Scan: FEATURE_STORAGE
ilg-mbp:mbed-os-example-blinky ilg$ ls -l
total 56
drwxr-xr-x  3 ilg  staff   102 Jan 10 21:34 BUILD
-rw-r--r--  1 ilg  staff  2737 Jan 10 21:32 README.md
drwxr-xr-x  3 ilg  staff   102 Jan 10 21:32 img
-rw-r--r--  1 ilg  staff   213 Jan 10 21:32 main.cpp
lrwxr-xr-x  1 ilg  staff    71 Jan 10 21:33 mbed-os -> /Users/ilg/My Files/MacBookPro Projects/GNU ARM Eclipse/src/mbed-os.git
-rw-r--r--  1 ilg  staff    77 Jan 10 21:32 mbed-os.lib
-rw-r--r--  1 ilg  staff  1597 Jan 10 21:34 mbed_config.h
-rw-r--r--  1 ilg  staff  1329 Jan 10 21:32 mbed_settings.py
-rw-r--r--  1 ilg  staff   831 Jan 10 21:34 mbed_settings.pyc
ilg-mbp:mbed-os-example-blinky ilg$ cat mbed-os.lib
https://github.com/ARMmbed/mbed-os/#2885c1b41e63158cb6faf5f107cd821ae06ef26c
ilg-mbp:mbed-os-example-blinky ilg$ cat mbed_config.h
// Automatically generated configuration file.
// DO NOT EDIT, content will be overwritten.

#ifndef __MBED_CONFIG_DATA__
#define __MBED_CONFIG_DATA__

// Configuration parameters
#define MBED_CONF_LWIP_TCP_SERVER_MAX               4    // set by library:lwip
#define MBED_CONF_PLATFORM_DEFAULT_SERIAL_BAUD_RATE 9600 // set by library:platform
#define MBED_CONF_LWIP_IPV4_ENABLED                 1    // set by library:lwip
#define MBED_CONF_LWIP_TCP_SOCKET_MAX               4    // set by library:lwip
#define MBED_CONF_EVENTS_PRESENT                    1    // set by library:events
#define MBED_CONF_RTOS_PRESENT                      1    // set by library:rtos
#define CFSTORE_STORAGE_DISABLE                     0    // set by library:configuration-store
#define MBED_CONF_PLATFORM_STDIO_FLUSH_AT_EXIT      1    // set by library:platform
#define MBED_CONF_LWIP_UDP_SOCKET_MAX               4    // set by library:lwip
#define MBED_CONF_NSAPI_PRESENT                     1    // set by library:nsapi
#define MBED_CONF_LWIP_ADDR_TIMEOUT                 5    // set by library:lwip
#define MBED_CONF_LWIP_IP_VER_PREF                  4    // set by library:lwip
#define MBED_CONF_PLATFORM_STDIO_CONVERT_NEWLINES   0    // set by library:platform
#define MBED_CONF_PLATFORM_STDIO_BAUD_RATE          9600 // set by library:platform
#define MBED_CONF_LWIP_SOCKET_MAX                   4    // set by library:lwip
#define MBED_CONF_LWIP_IPV6_ENABLED                 0    // set by library:lwip
// Macros
#define UNITY_INCLUDE_CONFIG_H                           // defined by library:utest

#endif
ilg-mbp:mbed-os-example-blinky ilg$ 

Please advise.

Copy link
Contributor

@theotherjimmy theotherjimmy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general it looks like you are not passing any of the flags generated from the toolchain profile.

I'm really excited about this exporter.

except AttributeError:
# TODO filter out projects with toolchain core not supported,
# instead of raising an exception.
raise Exception('Target core {0} not supported.'.format(core))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use NotSupportedException here? it's in tools/utils.py

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 10, 2017

In general it looks like you are not passing any of the flags generated from the toolchain profile.

sorry, but I don't know where to get these flags from. :-(

can you provide a link to another exporter that does this?

@theotherjimmy
Copy link
Contributor

the self.resources.objects was disabled, since it generates a duplicate main.o

That's very strange. I can't think of anything off the top of my head that could be causing that.

the mbed-os folder was excluded from the build, since itself is not a source folder

That should not be a problem, because it's sub folders should not be excluded.

without the sources in the mbed-os folder, the link obviously fails

This implies that my previous assumption is false.

I had to add #include "mbed_config.h" in platform/platform.h, otherwise compiling main.c unresolved macro MBED_CONF_PLATFORM_DEFAULT_SERIAL_BAUD_RATE

This is something that I can immediately help with. You are missing the -include mbed_config.h flag on the command line. In general you are missing all of the flags we normally provide on the command line. You can gen the flags in the dict self.flags e.g. https://github.com/ARMmbed/mbed-os/blob/master/tools/export/uvision/__init__.py#L157

can you provide a link to another exporter that does this?

I was typing this response as this came up. Glad we're on the same page.

@theotherjimmy
Copy link
Contributor

The format is:

{
 "c": [ "flags"],
 "cxx": [ "flags"],
 "asm": [ "flags"],
 "ld": ["flags"]
}

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 10, 2017

The second commit fixed the NotSupportedException issue.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

I was a bit busy fixing another GNU ARM Eclipse issue, but now I'm back.

... the mbed-os folder ...

I'm afraid we have a big misunderstanding here. simplifying things, the Eclipse CDT managed build, require a list of source folders, a list of include folders, a list of symbols and some compile options for the used tools. when starting the build, the source folders are scanned for .c, .cpp, .cxx, .cc, .S files, and all files identified are added to the generated make files, then the make is invoked.

if the mbed-os is in the source folders list, CDT will try to compile absolutely all files it can compile from this folder. as of now, the mbed-os folder contains 800MB in 12.553 files. on my fast SSD machine, Eclipse needs quite some time (many tens of seconds) to scan this folder.

but this is just annoying. the real problem is that many of the files, even if they look like legal source files (.c, .cpp, etc) should not be part of the build (the best example are test files).

so, in order to perform a build in Eclipse, I need to know exactly which are the folders with source files, keeping the entire mbed-os folder in the build is not a realist option. actually, since the mBed project structure is simplistic and do not use a top src folder but wants to build everything in the top folder, the mbed-os folder will be included in the build, so the logic need to be reversed, and all folders inside mbed-os that do not need to be included in the build, need to be explicitly excluded. and all means really all.

you are missing all of the flags we normally provide on the command line

right, I ignored the flags variable.

I checked it and, as it is prepared by the Exporters class, is of little use, since it includes again the symbols, which I already process separately; I had to overwrite it with a version that does not add the symbols.

however, integrating this with GNU ARM Eclipse is a bit tedious, since my plug-ins have quite an elaborate configuration mechanism, with most of the options having separate GUI entries, so options need to be identified in your list and specific settings need to be passed in the CDT configuration.

-include mbed_config.h

I added this and now main.cpp builds (even if the compile options are not accurate).

however, although legal, using -include is generally discouraged, since it is very confusing.

FYI, a preliminary build of main.cpp looks like this:

Building file: ../main.cpp
Invoking: Cross ARM C++ Compiler
arm-none-eabi-g++ -mcpu=cortex-m4 -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 -Og -fmessage-length=0 -fsigned-char -ffunction-sections -fdata-sections -fno-move-loop-invariants -Wall -Wextra  -g3 -DDEBUG -DFEATURE_LWIP=1 -D__MBED__=1 -DDEVICE_I2CSLAVE=1 -D__FPU_PRESENT=1 -DTARGET_Freescale -DDEVICE_PORTINOUT=1 -DTARGET_RTOS_M4_M7 -DDEVICE_LOWPOWERTIMER=1 -DDEVICE_RTC=1 -DTOOLCHAIN_object -DDEVICE_SERIAL_ASYNCH=1 -D__CMSIS_RTOS -DFSL_RTOS_MBED -DDEVICE_STORAGE=1 -DTARGET_KPSDK_MCUS -DTOOLCHAIN_GCC -DTARGET_CORTEX_M -DTARGET_KSDK2_MCUS -DTARGET_LIKE_CORTEX_M4 -DDEVICE_ANALOGOUT=1 -DTARGET_M4 -DTARGET_UVISOR_UNSUPPORTED -DTARGET_K64F -DDEVICE_SPI_ASYNCH=1 -DDEVICE_PWMOUT=1 -DDEVICE_INTERRUPTIN=1 -DDEVICE_I2C=1 -DDEVICE_PORTOUT=1 -D__CORTEX_M4 -DMBED_BUILD_TIMESTAMP=1484154238.68 -DDEVICE_STDIO_MESSAGES=1 -DCPU_MK64FN1M0VMD12 -DTARGET_LIKE_MBED -DTARGET_FF_ARDUINO -DTARGET_KPSDK_CODE -DTARGET_RELEASE -DDEVICE_SERIAL_FC=1 -DFEATURE_STORAGE=1 -DDEVICE_TRNG=1 -D__MBED_CMSIS_RTOS_CM -DDEVICE_SLEEP=1 -DTOOLCHAIN_GCC_ARM -DTARGET_FRDM -DTARGET_MCUXpresso_MCUS -DDEVICE_SPI=1 -DDEVICE_ERROR_RED=1 -DDEVICE_SPISLAVE=1 -DDEVICE_ANALOGIN=1 -DDEVICE_SERIAL=1 -DDEVICE_PORTIN=1 -DTARGET_MCU_K64F -DARM_MATH_CM4 -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/." -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./Debug" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./img" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/cmsis" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/cmsis/TOOLCHAIN_GCC" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/docs" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/drivers" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/events" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/events/equeue" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/greentea-client" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/greentea-client/greentea-client" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/greentea-client/source" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/unity" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/unity/source" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/unity/unity" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/utest" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/utest/source" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/frameworks/utest/utest" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls/importer" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls/inc" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls/inc/mbedtls" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls/platform" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls/platform/inc" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls/platform/src" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/mbedtls/src" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/nanostack" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/netsocket" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/hal" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/hal/storage_abstraction" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/platform" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/rtos" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/rtos/rtx" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/rtos/rtx/TARGET_CORTEX_M" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/rtos/rtx/TARGET_CORTEX_M/TARGET_RTOS_M4_M7" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/rtos/rtx/TARGET_CORTEX_M/TARGET_RTOS_M4_M7/TOOLCHAIN_GCC" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/api" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/device" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/device/TOOLCHAIN_GCC_ARM" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/drivers" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/TARGET_FRDM" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore/configuration-store" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore/doc" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore/doc/design" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore/doc/design/pics" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore/doc/design/umlet" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore/doc/project" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/cfstore/source" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/flash-journal" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/flash-journal/flash-journal-strategy-sequential" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/storage-volume-manager" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/storage-volume-manager/source" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/storage/FEATURE_STORAGE/storage-volume-manager/storage-volume-manager" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/api" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/core" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/core/ipv4" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/core/ipv6" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/include" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/include/lwip" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/include/lwip/priv" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/include/netif" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/include/netif/ppp" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/include/netif/ppp/polarssl" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip/src/netif" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip-eth" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip-eth/arch" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip-eth/arch/TARGET_Freescale" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip-sys" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/./mbed-os/features/FEATURE_LWIP/lwip-interface/lwip-sys/arch" -I"/Users/ilg/tmp/mbed/mbed-os-example-blinky/" -includembed_config.h -std=gnu++11 -fabi-version=0 -fno-exceptions -fno-rtti -fno-use-cxa-atexit -fno-threadsafe-statics -MMD -MP -MF"main.d" -MT"main.o" -c -o "main.o" "../main.cpp"
Finished building: ../main.cpp

In my opinion, the list of includes and defines is excessively long. FYI, such a long line (>9300 bytes), on Windows, which has a 8K limit on command line length, would not be allowed.

next I'll work on the procedures to properly pass options.

please advice on how to proceed with the mbed-os folder issue.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

a duplicate main.o
That's very strange.

print self.resources.objects
['./Debug/main.o']

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

I constantly get this message:

/Library/Python/2.7/site-packages/fuzzywuzzy/fuzz.py:35: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')

if python-Levenshtein is needed, why isn't it installed automatically as a dependency and I have to do it manually?

The initial {{uid}} was evaluated once. To force an evaluation for
each call, use an object property {{u.id}}.
@theotherjimmy
Copy link
Contributor

theotherjimmy commented Jan 11, 2017

the Eclipse CDT managed build, require a list of source folders, a list of include folders, a list of symbols and some compile options for the used tools.

Okay, here is how I would construct these things:

  • list of source folders - set(dirname(src) for src in self.resources.c_sources + self.resources.cpp_sources + self.resources.s_sources)
  • list of includes - self.resources.inc_dirs
  • list of symbols - self.toolchain.get_symbols()
  • compile options
    • C self.toolchain.flags["c"] + self.toolchain.flags["common"] + self.toolchain.get_config_option(self.toolchain.get_config_header())
    • C++ self.toolchain.flags["cxx"] + self.toolchain.flags["common"] + self.toolchain.get_conifg_option(self.toolchain.get_config_header())
    • ASM self.toolchain.flags["asm"]
    • Linker self.toolchain.flags["ld"]
print self.resources.objects
['./Debug/main.o']

Where did you get a Debug folder from? I don't think there is one generated by the tools, or checked into the repo.

if python-Levenshtein is needed, why isn't it installed automatically as a dependency and I have to do it manually?

It's not required, it is optional. I have been considering a way to remove that optional dependency. It may come in another PR.

@theotherjimmy
Copy link
Contributor

theotherjimmy commented Jan 11, 2017

however, although legal, using -include is generally discouraged, since it is very confusing.

The use of -include is generally for "configuration", which is what we use it for.

In my opinion, the list of includes and defines is excessively long. FYI, such a long line (>9300 bytes), on Windows, which has a 8K limit on command line length, would not be allowed.

Yep. You are welcome to push all of the symbols into another header file and -include that. Alternatively, you could put all of the -I into a file and use @<filename> to include those options on the command line.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

Where did you get a Debug folder from?

the Debug folder is created by CDT. usually CDT projects have two build configurations, Debug & Release. each build uses one of these folders to create the binary files and everything else necessary during the build.

so, in this case, the question is why the exporter enumerates these folders?

do you have a better strategy to get the list of object files I need to add to the link step?

@theotherjimmy
Copy link
Contributor

so, in this case, the question is why the exporter enumerates these folders?

The tools iterate through all of the files and directories present in a project, skipping:

  • directories and files starting with .
  • directories starting with FEATURE_, TOOLCHAIN_, and TARGET_ unless the respective feature, toolchain, or target is being used
  • directories and files ignored by .mbedignore Here is how it works

do you have a better strategy to get the list of object files I need to add to the link step?

No

You could work around this in several ways:

  • You could use a .mbedignore file containing * within your Debug and other folders
  • You could change the Debug and other folders to start with a .

I'm sure there are more, but off of the top of my head, these are it

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

You could use a .mbedignore file containing * within your Debug and other folders
You could change the Debug and other folders to start with a .

unfortunately the user can create any number of configurations, and name them as he wishes, I have no control over these names.

the problem occurs if export is run again, after performing Eclipse builds, which create these folders.

it looks like, for the moment, we have to keep resources.objects not used.

@theotherjimmy
Copy link
Contributor

theotherjimmy commented Jan 11, 2017

it looks like, for the moment, we have to keep resources.objects not used.

@ilg-ul I'm sorry, but that is not an option.

I suggest an alternative: by default do something that works with both tool sets. If a user modifies the configuration provided by the exporter they are 'on their own' in a sense. The sense is that they might not be able to use mbed export or mbed compile anymore without cleaning first.

@sarahmarshy
Copy link
Contributor

sarahmarshy commented Jan 11, 2017

Additionally, they probably will not run export again. If they use the Eclipse exporter like a user (rather than a developer exporting many times), then they will use Eclipse to manage/add files after the initial project creation.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

will only export once

do not count on this. user inventiveness is limitless.

@theotherjimmy
Copy link
Contributor

do not count on this. user inventiveness is limitless.

Agreed. But our ability to support arbitrary inventiveness is limited.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

set(dirname(src) for src in self.resources.c_sources + self.resources.cpp_sources + self.resources.s_sources)

unfortunately the entries in this set cannot be used directly, because CDT recursively searches the folders, and there are many parent folders and subfolders, with parent folders also containing folders that must not be included in the build.

the logic to compute exclusions requires some complicated tree processing. :-(

@theotherjimmy
Copy link
Contributor

the logic to compute exclusions requires some complicated tree processing. :-(

Yeah. That's why we have been using a "specify the files exactly" strategy in the exporters. I'm sorry, but we are unlikely to reorganize the directory structure to accommodate the invariant that directories included also guarantee that all of their sub-directories are also included.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 11, 2017

we have been using a "specify the files exactly" strategy in the exporters

hmmm... this raises another question: assuming I manage to write the tree exclusion logic and I identify the folders to build, can you guarantee that all source files in those folders really should be included in the build? all .c/.cpp/.cxx/.S files? because Eclipse, unless there are explicit exclusions, will try to compile all those files.

if you have the list of source files, perhaps safer would be to process these entries, not the folders.

@theotherjimmy
Copy link
Contributor

theotherjimmy commented Jan 11, 2017

assuming I manage to write the tree exclusion logic and I identify the folders to build, can you guarantee that all source files in those folders really should be included in the build? all .c/.cpp/.cxx/.S files?

Yes. Wait no. Re-reading through the mbedignore documentation leads me to believe that you could do something as silly as *.s in a .mbedignore file and exclude all assembly in that sub-tree. I have yet to see it, but it could be done.

if you have the list of source files, perhaps safer would be to process these entries, not the folders.

This seems so much easier. It's resources.c_sources + resources.s_sources + resources.cpp_sources

@bulislaw
Copy link
Member

One thing that worries me here is proliferation of eclipse exports, what's the difference between eclipse_gcc_arm and gnuarmeclipse? Should that be documented somewhere?

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Jan 12, 2017

One thing that worries me here is proliferation of eclipse exports,

did you manage to export and build the mbed-os-example-blinky with the existing exporters?

what's the difference between eclipse_gcc_arm and gnuarmeclipse?

eclipse_gcc_arm exports a CDT project, which builds with the host compiler (gcc on Linux) and generates a x86 executable

eclipse_gcc_arm exports an unmanaged CDT project

gnuarmeclipse exports a GNU ARM Eclipse managed project, which builds with arm-none-eabi-gcc (usually) and generates an ARM executable.

Should that be documented somewhere?

I'll try to add a section to the exporters page, but let's first make it work properly.

@bulislaw
Copy link
Member

eclipse_gcc_arm exports a CDT project, which builds with the host compiler (gcc on Linux) and generates a x86 executable

that doesn't make any sense, if it works like that it's a bug and should be changed -> your 'gnuarmeclipse' changes should replace the eclipse_gcc_arm

@theotherjimmy correct me if i'm wrong

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Feb 17, 2017

thank you all!

now we need to do something to update the documentation. any suggestions where to start?

what is the 5.4 ETA? how much time do we have for the documentation?

@theotherjimmy
Copy link
Contributor

We have 2-3 weeks, I think. The documentation is in the handbook. https://github.com/ARMmbed/Handbook @AnotherButler, could you help out with getting the documentation for GNU ARM Eclipse in the right place?

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Feb 17, 2017

how can we regenerate the tools/export/README.md to include the new exporter?

@bridadan
Copy link
Contributor

Not sure how to generate tools/export/README.md, but I'd like to add that this PR now currently holds the record for "most commented PR" 😄

@AnotherButler
Copy link
Contributor

The related existing page is https://github.com/ARMmbed/Handbook/blob/5.3/docs/dev_tools/third_party.md. We could add to this or create a new page if there's enough information to fill one.

@theotherjimmy
Copy link
Contributor

I'd rather just delete tools/export/README.md. It's kinda useless...

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Feb 17, 2017

It's kinda useless

+1

what do we do with the exporters that do not generate functional projects? from my past experience, these exporters only manage to frustrate users.

my initial proposal was to temporarily disable them until fixed.
to this I would add now to temporarily remove them from the documentation.

@theotherjimmy
Copy link
Contributor

#3797 Is the PR to remove it BTW.

@theotherjimmy
Copy link
Contributor

@ilg-ul I was thinking of moving some of the code you have to the exporter base classes. In particular, I was thinking of moving the compute directory exclude logic up. I may move a few more up. I'll mention you on the PR so that you can review.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Feb 17, 2017

moving some of the code

sure, no problem.

perhaps you can also think of a better solution to process all profiles.

@theotherjimmy
Copy link
Contributor

perhaps you can also think of a better solution to process all profiles.

Working on this now. I'm going to change the toolchain member of Exporter to a dict of profile names to toolchain instances. Should be pretty reasonable.

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 14, 2017

Is 5.4 out? I noticed that the GNU ARM Eclipse exporter is mentioned by the documentation, but I could not find it in the web compiler exporters list.

@CarloMara
Copy link

@ilg-ul Could you guide me on how to use the exporter, and how should I import the files in eclipse?

Here is what i have done:

  1. Configured a brand new eclipse using the tutorial of GNU ARM Eclipse (this)
  2. mbed import mbed-os-example-blinky
  3. mbed-os mbed export -m K64F -i gnuarmeclipse I'm not sure about this. Why did you rm mbed-os? Still it didn't trow any errors.
  4. Went into eclipse, and imported a new project by clicking on the import button and choosing the correct folder. I'm not sure about this step, for two reasons: the first one begin that the c++ indexer takes forever, and the second one begin that if I go into project proprieties the compile command is make -j.

FYI I have an i7 machine and if I use mbed compile -t GCC_ARM -m K64F everything works fine and compile goes smooth.

Thanks again for your time,
Carlo

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 17, 2017

@theotherjimmy, I tried to reproduce Carlo's problem and I encountered this:

$ mbed import mbed-os-example-blinky
[mbed] Importing program "mbed-os-example-blinky" from "https://github.com/ARMmbed/mbed-os-example-blinky" at latest revision in the current branch
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #f4864dc6429e
$ cd mbed-os-example-blinky/
ilg-mbp:mbed-os-example-blinky ilg$ mbed export -m K64F -i gnuarmeclipse
Scan: .
Traceback (most recent call last):
  File "/Users/ilg/tmp/mb/mbed-os-example-blinky/mbed-os/tools/project.py", line 248, in <module>
    main()
  File "/Users/ilg/tmp/mb/mbed-os-example-blinky/mbed-os/tools/project.py", line 244, in main
    build_profile=profile)
  File "/Users/ilg/tmp/mb/mbed-os-example-blinky/mbed-os/tools/project.py", line 94, in export
    build_profile=build_profile, silent=silent)
  File "/Users/ilg/tmp/mb/mbed-os-example-blinky/mbed-os/tools/project_api.py", line 222, in export_project
    macros=macros)
  File "/Users/ilg/tmp/mb/mbed-os-example-blinky/mbed-os/tools/project_api.py", line 89, in generate_project_files
    exporter.generate()
  File "/Users/ilg/tmp/mb/mbed-os-example-blinky/mbed-os/tools/export/gnuarmeclipse/__init__.py", line 129, in generate
    raise NotSupportedException("No linker script found.")
tools.utils.NotSupportedException: No linker script found.
[mbed] ERROR: "python" returned error code 1.
[mbed] ERROR: Command "python -u /Users/ilg/tmp/mb/mbed-os-example-blinky/mbed-os/tools/project.py -i gnuarmeclipse -m K64F --source ." in "/Users/ilg/tmp/mb/mbed-os-example-blinky"
---
$ ls -l
total 48
drwxr-xr-x   3 ilg  staff   102 Mar 17 22:45 BUILD
-rw-r--r--   1 ilg  staff  2626 Mar 17 22:44 README.md
drwxr-xr-x   3 ilg  staff   102 Mar 17 22:44 img
-rw-r--r--   1 ilg  staff   168 Mar 17 22:44 main.cpp
drwxr-xr-x  25 ilg  staff   850 Mar 17 22:44 mbed-os
-rw-r--r--   1 ilg  staff    77 Mar 17 22:44 mbed-os.lib
-rw-r--r--   1 ilg  staff   160 Mar 17 22:45 mbed_config.h
-rw-r--r--   1 ilg  staff  1329 Mar 17 22:44 mbed_settings.py
-rw-r--r--   1 ilg  staff   829 Mar 17 22:45 mbed_settings.pyc
$ mbed --version
1.0.0
$

any idea what went wrong?

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 17, 2017

@CarloMara, could you check your bullet no. 3? what do you mean by 'Why did you rm mbed-os? '

step no. 4 seems ok, but it looks like something wrong happened with mbed, since I also had problems, even worse than your.

@CarloMara
Copy link

Dear lord.....

On the very first post you rm'd the whole mbed-os folder with:

rm -rf mbed-os

I didn't execute that, because I couldn't find a plausible motivation for doing so.

I will try now to redo what I have done, to see if I get your errors.

Cheers,
Carlo

@CarloMara
Copy link

yep, same error

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 17, 2017

On the very first post you rm'd the whole mbed-os folder with

ah, right, but that message had a completely different purpose, to replace the stable version of mbed-os downloaded with the blink project with a development version cloned from git.

the current documentation for the exporter is available from:

https://docs.mbed.com/docs/mbed-os-handbook/en/latest/dev_tools/third_party/

(this location is not very fortunate, it needs some more work)

@CarloMara
Copy link

So, yhea my guess was right. That wasn't needed. When I will figure out how to do this, I will add some pics to the GNUARM section of the docs, so it's even easier for beginners.

Still, now there is a new bug somewhere. Go figure (;

Carlo

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 17, 2017

#3964

@CarloMara
Copy link

Since the patch has been merged on master, I tried again to export the project using mbed cli luckily without encountering any of problems that @ilg-ul did find.

Even with this small "road block" fixed my main issue remains, I can't compile the project from eclipse.
As I said before the steps that I have taken are:

  1. Get a simple mbed project using mbed import mbed-os-example-blinky
  2. Replace mbed-os with a symlink to my development copy of mbed. I have done this in order to be able to use the master head, with the fixes on the exporters
  3. a) Invoke the exporter using mbed export -m K64F -i gnuarmeclipse
    b) To make sure that everything is working correctly, I checked in the project folder that the files .cproject and .project are there.
  4. Opened eclipse, choose from the File --> Import
  5. When the import menu pops up, the option Existing code as Makefile Project is chosen
  6. I than proceed to choose the path of the project, and to select the from the Tool-chain indexer Settings the option Cross ARM GCC
  7. Eclipse than imports the project, and the CDT Indexer fires up.
  8. From my understanding, when I click on the hammer icon Eclipse should build the project, and give me a binary file for my uC. That for some reasons it's not happening, even if in the Console tab the build is reported as finished (without implying any errors).
    If I try to launch the simulator, by pressing the play button, it complains about the fact that it can't find a binary. The BUILD folder is empty.

On a side note, I'm planning to use a Nucleo F303K8 board from ST, but from what I have seen the standard for test commands is K64F, so I'm sticking with it until I can get this working.

Thanks again 4 your time,
Carlo

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 23, 2017

when I click on the hammer icon Eclipse should build the project, and give me a binary file for my uC. That for some reasons it's not happening, even if in the Console tab the build is reported as finished (without implying any errors).

where do you search for the result? check for a Debug folder, possibly after a refresh.

If I try to launch the simulator, by pressing the play button,

what simulator?

it complains about the fact that it can't find a binary. The BUILD folder is empty.

the BUILD folder is not used by Eclipse.

@CarloMara
Copy link

I can't find any debug folder, nor in eclipse nor via command line. Here it's what I see in the folder after running build:

~/progetti/arm_eclispe/test_2/mbed-os-example-blinky$ ls -al
total 96
drwxr-xr-x 6 carlo carlo  4096 mar 23 22:35 .
drwxr-xr-x 3 carlo carlo  4096 mar 18 18:42 ..
drwxr-xr-x 2 carlo carlo  4096 mar 23 22:34 BUILD
-rw-r--r-- 1 carlo carlo 25410 mar 23 22:35 .cproject
drwxr-xr-x 8 carlo carlo  4096 mar 18 18:35 .git
-rw-r--r-- 1 carlo carlo    32 mar 18 18:35 .gitignore
drwxr-xr-x 2 carlo carlo  4096 mar 18 18:35 img
-rw-r--r-- 1 carlo carlo   168 mar 18 18:35 main.cpp
-rw-r--r-- 1 carlo carlo   288 mar 23 22:34 makefile.targets
-rw-r--r-- 1 carlo carlo    19 mar 23 22:34 .mbed
-rw-r--r-- 1 carlo carlo   948 mar 23 22:34 mbed_config.h
-rw-r--r-- 1 carlo carlo    32 mar 23 22:34 .mbedignore
lrwxrwxrwx 1 carlo carlo    41 mar 23 22:33 mbed-os -> /home/carlo/progetti/arm_eclispe/mbed-os/
-rw-r--r-- 1 carlo carlo    77 mar 18 18:35 mbed-os.lib
-rw-r--r-- 1 carlo carlo  1329 mar 18 18:36 mbed_settings.py
-rw-r--r-- 1 carlo carlo   851 mar 23 22:34 mbed_settings.pyc
-rw-r--r-- 1 carlo carlo  1025 mar 23 22:34 .project
-rw-r--r-- 1 carlo carlo  2626 mar 18 18:35 README.md
drwxr-xr-x 2 carlo carlo  4096 mar 23 22:35 .settings

As you can see, there is no Debug folder....

Regarding the simulator, I was referring to QEMU which, from my understanding, runs when I click on the play button. This kinda makes sense, because when the play button is clicked eclipse complains about not finding the binaries...

I was guessing that the build folder was used exclusively by mbed, so thanks for confirming that.

On a side note, how can we increase the verbosity of the compiler output in an mbed project?

Thanks again,
Carlo

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 24, 2017

Here it's what I see in the folder after running build: ... increase the verbosity of the compiler output

do a clean and a new build, then copy the first and last lines of the console. eclipse is quite verbose, displays all compiler/linker commands. if you do not see those lines, you have a problem.

there is no Debug folder

please check what configurations you have in your project, and which one is active when you hit the hammer button. if I remember right, the exporter create 3 configurations, one for each mbed profile.

building each configuration should contribute a folder with the same name in the project root.

Regarding the simulator, I was referring to QEMU which, from my understanding, runs when I click on the play button.

no, it is not that easy, you need to create debug configurations, for qemu as for any other debugger.

I suggest you follow the GNU ARM Eclipse blinky tutorial 'by the book', but only after you install all GNU ARM Eclipse components, exactly as documented.

after you complete this tutorial, perhaps you'll understand how Eclipse works, and what to expect from the mbed exporter.

@CarloMara
Copy link

Ok, I checked and I can see only the default build configuration so I guess that that's the problem.

@ilg-ul What's the best way to proceed? Should I reinstall everything?

Cheers,
Carlo

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 27, 2017

I can see only the default build configuration so I guess that that's the problem

what problem? if you asked Eclipse to build the default configuration, that's exactly what happened, you got a folder with the result of the build.

if I remember right mbed has three profiles, so you got three Eclipse configurations in your project, you can build them one by one, or build them all, and you'll finally get three folders with the results of the build.

what did you expect to happen?

if you have specific problems, please use separate issues, this thread is already too long and is closed.

@CarloMara
Copy link

CarloMara commented Mar 27, 2017

I'm sorry I wasn't clear. If I go into the project configuration the only build config I see it's default.
If I try to add one the only option I have is to make one from scratch.

Thanks again,
Carlo

EDIT:
#4038

@ilg-ul
Copy link
Contributor Author

ilg-ul commented Mar 27, 2017

please open separate issues, and explain what you expected, what actually happened, and the procedure to reproduce the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.