Install via Board manager

[beriberikix]

With the 1.6.2 IDE release, there is now a way to download new boards directly. Will this implementation be able to support the manager? It would be nice to be able to use a single IDE rather than a fork.

[per1234]

I don't think you can install via Boards Manager, but this claims to allow you to use ESP8266 with the stock Arduino IDE. I haven't had a chance to try it out yet. https://github.com/rogerclarkmelbourne/arduino-esp8266/

[igrr]

@beriberikix Ultimately, that is the plan. But right now the board manager does not support installation from a .zip archive. The list of available boards gets downloaded from Arduino.cc servers. So we need to get listed there, which requires us to do some talking... But we will get there, eventually.

For now I will add a release archive packaged for installation to the user's hardware folder.

[beriberikix]

👍

[rogerclarkmelbourne]

I'm looking at how to do the installation via IDE 1.6.2 This is a very new feature

At the moment I can't see a way to add external repo's to the boards manager, it only seems to show the official Arduino boards.

Does either of you know where it gets the list from or what the package for the board needs to look like ?

It appears to download multiple files e.g. the core and the tools seem to be separate

I guess I could read the Java to try to work out if there is a way to patch in additional boards e.g. by updating an XML somewhere ??

[igrr]

IDE downloads the list of platforms from the following URL:
http://downloads.arduino.cc/packages/package_index.json
(that's in arduino-core/src/cc/arduino/contributions/packages/ContributionInstaller.java)

[rogerclarkmelbourne]

Thanks

This change has broken my STM32 repo :-( :-(

Just trying to fix it now

because my STM32 repo relies on the ATM compilor used to the DUE to be installed

The latest version of 1.6.2 release yesterday (or perhaps today) has moved the compilors to

%appdata%/Arduino15

So my compile recipe for STM32
{runtime.ide.path}/hardware/tools/{build.gcc_ver}/bin/

No longer works

I will need to find the new equivalents to {runtime.ide.path} for Due

This won't effect ESP8266 as the tools are in the hardware folder

Perhaps this change is a result of the internal dispute between Arduino LLC and the board manufacturing side of the business

Either way, it will break a lot of things that use the IDE :-(

[rogerclarkmelbourne]

OK

I did some more research, and the package data gets downloaded from Arduino.cc every time the popup is opened :-(

There is no way to just modify / patch the package file, as it gets overwritten (I have tried )

The Boards Manager needs a new feature

I don't know if Arduino.lcc developers will allow a feature to add external sources

However, even if they don't want to add it, I think we should

I have forked the main Arduino IDE repo and cloned it to my local machine

I will investigate how to add a feature where the Boards Manager can have multiple package data files from multiple external locations

If Arduino Won't add this via pull request, we should add this, and I will tell anyone using my STM32 hardware files etc, to use esp8266/arduino

However really we need to removed the esp8266 etc stuff from the release and add it as a separate package file, i.e may need to fork the repo and have another one that just contains the hardware cores files, taken as a sub project of this main repo ???

[igrr]

I think the idea was to allow only some "verified" cores to be installed via Board Manager.
So you really need to get listed by Arduino.cc if you want your core to be installed this way.

Also it's not entirely clear to me why we need to split the repository? We can very well create whatever package necessary from this one without all the forking and submodules.

[rogerclarkmelbourne]

Ivan.

OK. But basically that just won't happen.

I've found it very had to get any STM32 specific stuff into any of the core libraries.

e.g. Paul Stoffregen who is one of the lead developers on the Arduino IDE team ( https://github.com/PaulStoffregen ) won't even action a pull request to add generic support to the One Wire library (which isn't even specific to the STM32)

So I think the chances of anyone getting their repo added to the list is unlikely, unless they are for example Expressif or MicroChip or ST themselves

One other person has replied to my posting on the IDE developers mailing list asking for the same thing, but as this change has only happened in the last 2 or 3 days, I don't think a lot of people have noticed
e.g. there are a lot of boards which rely on the ARM compiler being installed (which is no longer is), and even the AVR boards e.g Mega1280p will have issues because the AVR compiler is no longer in the Program Files (x86) hardware folder, its in the AppData folder

I've re-activated a thread on the Arduino forum for Mega1280p users to alert them, and there is also another thread about the new library manager - which is basically along the same lines

So perhaps over the next few days, more people will contribute

PS. Sorry, I've not be able to update my binaries (long story, but I have an internet data quota per month, and I only have a small amount left - so I don't have enough quota to upload the new binaries :-( )

I should really remove them and use the distribution system instead, but I've been very busy with the Board Manager thing this morning, and this afternoon I've had repairs to do on my house ;-)

If I can find a copy of JDK on a CDI will attempt to build the IDE from source, and look at adding new packages etc

[rogerclarkmelbourne]

Edit

Sorry. Re packages

Looks like the tools and the hardware folders are in separate zips

I may try turning off my internet so that the IDE can't update the package description, and then use a local web server to test how to build the zips

[rogerclarkmelbourne]

See arduino/Arduino#2856
You may want to chip in and add a call for the Board Manager to support external sources

[rogerclarkmelbourne]

Ivan

I did a bit of testing, by changing my hosts file to prevent the IDE updating the package file

And I can get it to install a new hardware files, but I have not tried to get the tools working yet,

One issue I did encounter is that the checksum in the package Must be lower case.
When I used QuickHash to generate the hashes, the Board manager reported that there was a CRC error, but I tracked it down to not supporting upper case, which frankly is a bug.

I did have success by adding a new block of data inside the first json array item (which is arduino products), but when I tried to add a separate section e.g. at the top level in packages, I get an error because the IDE can't access the Internet

I'm not sure if the error is really because it can't contact the Internet, or if I have made a mistake with the package structure.

I will continue tomorrow as its getting too late for me to continue today.

[sandeepmistry]

@rogerclarkmelbourne nice work! There's also a way to override the package file URL via setting PACKAGE_INDEX_URL as a Java property. See: https://github.com/arduino/Arduino/blob/master/arduino-core/src/cc/arduino/contributions/packages/ContributionInstaller.java#L60

[rogerclarkmelbourne]

Hi Sandeep

I've posted this elsewhere as well, but this is how to (at least on Windows) change this path.

On widows, you edit the file C:\Program Files (x86)\Arduino\arduino.l4j.ini and add in an extra line which specifies the address of the package_index.json file

So the ini file could look like this

-Xms128M
-Xmx512M
-DPACKAGE_INDEX_URL=http://www.rogerclark.net/downloads/package_index.json

I put a copy of package_index.json on my personal website, you can specify another address or even a local webs server to test package deployment

I have not had time to build the tools and cores packages that would be needed, and also it appears that the Arduino IDE team have really messed things up and you can not install Due and the ARM compiler any more :-(

I have a backup of the json file that allows installation of ARM etc, if you want to see how the ARM / Due cores are setup in the packages file.

http://www.rogerclark.net/downloads/package_index.json.old

However I think the Arduino team have changed version numbers on the AVR files and I'm not sure if the ARM files in the old package_index are still available

I think we may need to leave this until the IDE team sorts out this current mess.

[beriberikix]

Looks like Arduino LLC is opening up the Board Manager in the next release: http://blog.arduino.cc/2015/04/28/opening-up-the-arduino-ide/

[beriberikix]

1.6.4 is now live and supports installing unofficial boards by adding a URL in the preference section.

[plinioseniore]

Looks like that ESP8266 support developed by this community has been added and hosted on Adafruit servers. From Mobile.

[toddtreece]

i packaged up an earlier version of the core and tools in this repo (mentioned in #57 & #204) for Adafruit's v1.6.3 board manager proxy, and just found @sandeepmistry's repo today. i made some modifications to make it easier to package for the 1.6.4 board manager. i made the modifications on OS X, and added a package.sh helper script that might work on linux boxes, but the helper script has only been tested on OS X.

here's the fork: https://github.com/toddtreece/esp8266-Arduino

i'm going to add the new package to Adafruit's main package index after some testing, but if anyone would like to test, you can add this to your v1.6.4 board manager url preferences and install the ESP8266 package from the board manager: https://github.com/toddtreece/esp8266-Arduino/releases/download/1.6.4/package_esptest_index.json

[gerardwr]

@toddtreece

I tried the previous 1.6.3 version and that worked fine. I really like the ease of updating via the board manager.

I just looked at updating to the latest version:

• @igrr original is (often) daily updated, latest update is from 13 hours ago
• @sandeepmistry version is from 7 days ago
• your fork datestamp list that the latest update is from 15 hours ago, that's confusing because it's based on an update from 7 days ago, right?

So why not base your fork on igrr's original your version, it will assure always the latest version.

BTW : I run the sandeepmistry version currently too, because it's an easy download/copy, much easier than igrr's complete build.

Thx for the effort!

[IanWalker]

Hi

Installed on a fresh install of Arduino 1.6.4 on a windows 7 64 bit machine.
Error on compile of web server example - C:\Program Files (x86)\Arduino\Arduino15\packages\esp8266\hardware\esp8266\1.6.4\cores\esp8266/Arduino.h:223:26: fatal error: pins_arduino.h: No such file or directory
#include "pins_arduino.h".
Also I dont see the ESPToll in the programmer list.

Also did a install from the official link to the Adafruit boards - On that install it would go straight to a compiler error.

[toddtreece]

@gerardwr thanks for the info. i'm trying to help out by restructuring so the repo for arduino 1.6.4 so it's a bit easier for this project to release without building a separate IDE, and that will probably be easier for users as well since they don't have to download multiple arduino IDEs. it has been a bit hard to track all of the changes with this project, the changes to the official arduino IDE, and still have time for normal work.

it seemed like @sandeepmistry's fork was a good starting place, since when i packaged up the adafruit version it was based on the 1.6.1 release from this project. the current version in the adafruit board index doesn't work for arduino 1.6.4 on windows, but it did on 1.6.3 for some reason. that's what @IanWalker is probably seeing, and that's why i started down this update path yesterday.

i'd be happy to fork and adapt any repo that's considered the canonical version of this project, but i didn't start with this one since i saw a comment saying master was fairly unstable currently. it'd almost make sense to start a new repo with just the esp8266 core and tools so the upstream Arduino IDE git history is cleared out. the files in the IDE repo aren't needed for releasing 1.6.4 third party board packages, and decoupling the esp8266 core version from the arduino IDE version seems to make sense to me as well.

[gerardwr]

@toddtreece

Thanks for the extensive explanation.

I fully agree:

• that the ESP extensions should be structured as an "addon" to an original Arduino IDE, and that the route you have chosen is ( in my opinion ) the most elegant.
• the route that @igrr has chosen works great if it was being built regularly/daily, but updated in complex to do.
• that the @sandeepmistry fork is currently closest to the ideal solution. I use it now too with Arduino1.6.3. Unfortunately it's being updated irregularly from the master.

My point from a user perspective is that your approach using the Board Manager + a stable version of @igrr 's master seems the optimal solution.

Would be excellent if @igrr would restructure his repo according to the @sandeepmistry repo, so you could use this to build upon. Better yet, if the three of you could team up!

That would save a lot of valuable specialist and user time. I understand it would take time from @igrr and he has probably a million things on his todo list :-)

A lot of user problems are caused by outdated ESP extensions, already solved in the latest versions. This software is great, and it deserves a rock solid release base. Thanks for supporting the evolution for that.

[IanWalker]

@toddtreece - Thank You for taking the time to do this. For newbies the additional board manager is I think the easiest solution.

Ok so I went back to a fresh install of 1.6.4 and selected the Adafruit packages as per the arduino GitHub
https://github.com/arduino/Arduino/wiki/Unofficial-list-of-3rd-party-boards-support-urls

One of the open issues mentioned a missing libiconv-2.dll. - I had a seperate install of this repo of @iggr and found that file there and copied it the new 1.6.4 install and compilation / downloading of the Hello Server worked fine.

It looks like Adafruit - You? are aware of this and working on a sollution.
https://forums.adafruit.com/viewtopic.php?f=22&t=73280&p=371965&hilit=esp8266#p371965 but I thought I would post the temporary solution for those having issues.

Thanks again to everyone that is making arduino compatibility for these great little modules possible.

[toddtreece]

@IanWalker yep. i have a test package index file for the esp windows fix here: https://github.com/adafruit/arduino-board-index/releases/download/esptool-0.4.2-winfix/package_esptest_index.json

i've been testing it on clean installs, and it seems to fix the issue. let me know if you get a chance to try it, and i can deploy the fix to the adafruit package index.

[IanWalker]

@toddtreece - Working this side as well. ESPTool does not show in programmer selection but uploads fine.

[Testato]

for the ESPool i read, if i remember on @sandeepmistry repo, that this is a new function.
ESPTool it is not needed under Programmer section, normally the Arduino IDE use this Programmer section for select the external rpogrammer (real programmer, not bootloader) and use it for two thinks:

• burn bootloader on a new mcu
• upload the sketch via ICSP without needing the bootloader on MCU

So in the repository merging a pull request it is removed the ESPTool

[gerardwr]

@toddtreece

Success on Mac Os X 10.10.3.

• fresh install of Arduino 1.6.4
• json file from https://github.com/arduino/Arduino/wiki/Unofficial-list-of-3rd-party-boards-support-urls added in "Arduino/Preferences"
• ESP8266 installed in "Boards Manager"
• Generic ESP8266 selected as "Board"
• compile OK, sketch on ESP runs OK.

Job well done, thx !

[toddtreece]

the esp8266 package in the main adafruit package index (https://adafruit.github.io/arduino-board-index/package_adafruit_index.json) has been updated to 1.6.2 with the windows libiconv fix.

thanks for the help testing!

there's more info about adding third party boards using the official arduino 1.6.4 IDE here: https://learn.adafruit.com/add-boards-arduino-v164

[sandeepmistry]

@toddtreece nice work! I totally agree with:

... decoupling the esp8266 core version from the arduino IDE version seems to make sense to me as well.

It probably doesn't make sense to have 3 repos to maintain long term (this one, my fork, and your fork). Sound like we should work with @igrr to see if we can have a build steps to produce:

1. ESP8266 IDE

2. ESP8266 Core

3. ESP8266 tool

4. and 3) can probably be used in 1) via a Git submodule, so maybe each live in their own repo? Thoughts?

[toddtreece]

@sandeepmistry that sounds like a good approach to me. really all that's needed from my repo is the package shell script, and the JSON template folder for building the JSON to add to the package index.

there are also some minor modifications to the platform.txt file to support the folder structure of board manager installed packages & tools.

we could probably point the esptool builds from @igrr, but they are missing xtensa-lx106-elf. that could be fixed by packaging that up as a separate tool, and pointing to the xtensa-lx106-elf & esptool archives as tool dependencies in the package index. that would mostly get rid of the need for submodules. the esp8266 IDE build could use a submodule or subtree and point to a esp8266 core repo, or it could download releases as well since the cores need to be put into archives for the board manager.

[Thro42]

@toddtreece I use your JSON and Install 1.6.4. It works fine. But I have problems with this version if I use DHT11 library.
#include <DHT.h>
void setup() {
Serial.begin(115200);
Serial.println("Setup... ");
}
void loop() {
}
When I use this sample the ESP dump with
Fatal exception (28):
epc1=0x40241d6f, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000084, depc=0x00000000

[duncan-a]

There are multiple DHT11 libraries on GitHub...

I think the 'ESP-compatible' library you want is:

https://github.com/adafruit/DHT-sensor-library

but I'm new to the ESP................

On 13 May 2015 at 20:20, Thro42 notifications@github.com wrote:

@toddtreece https://github.com/toddtreece I use your JSON and Install
1.6.4. It works fine. But I have problems with this version if I use DHT11
library.
#include
void setup() {
Serial.begin(115200);
Serial.println("Setup... ");
}
void loop() {
}
When I use this sample the ESP dump with
Fatal exception (28):
epc1=0x40241d6f, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000084,
depc=0x00000000

—
Reply to this email directly or view it on GitHub
#13 (comment).

[Thro42]

@duncan-a Yes I use https://github.com/adafruit/DHT-sensor-library . But after I type in my coding include DHT.h, it stops working.:(

[igrr]

Now we have a board manager package as well. Readme has been updated with instructions to use it.

[CptanPanic]

What version of release is included with board manager? Latest tagged release, or just latest?

[igrr]

The version shown by the Boards Manager contains the commit id — 545ffde in this case.
I will add something like stable/development branches in the next few days.

[Testato]

I added your json url to the Arduino 3rd party boards support urls
https://github.com/arduino/Arduino/wiki/Unofficial-list-of-3rd-party-boards-support-urls