Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WEMOS D1 mini Pro - 16M supporting #2581

Closed
4refr0nt opened this issue Oct 6, 2016 · 120 comments
Closed

WEMOS D1 mini Pro - 16M supporting #2581

4refr0nt opened this issue Oct 6, 2016 · 120 comments
Assignees
Milestone

Comments

@4refr0nt
Copy link

4refr0nt commented Oct 6, 2016

Hardware

Hardware: WEMOS D1 mini Pro - 16M

What's new with D1 mini Pro

  • 16M bytes (128M bit) Flash
  • External antenna connector
  • Built-in ceramic antenna
  • New CP2104 USB-TO-UART IC
  • Same size as D1 mini, but more light

View D1 mini Pro on WeMos Aliexpress store

@WereCatf
Copy link
Contributor

WereCatf commented Oct 6, 2016

This is not an issue and this is not the place to advertise your affiliate-links.

@TotallyInformation
Copy link

Hi, I've got 2 of these now and would like to understand how to update the boards.txt to make use of the 16M memory?

Not as simple as I'd hoped. I'm guessing that these are a copy of the D1 Mini but with the larger FLASH chips attached.

@WereCatf
Copy link
Contributor

@TotallyInformation It'd be best to make a proper variant out of it. I can take a stab at it in a few hours.

@TotallyInformation
Copy link

TotallyInformation commented Oct 29, 2016

Fab, thanks WereCatf.

https://www.wemos.cc/product/d1-mini-pro.html

@WereCatf
Copy link
Contributor

@TotallyInformation Okay, copy https://github.com/WereCatf/Arduino/blob/WemosD1MiniPro/boards.txt over your existing one and that should be it. I do not have one of these boards, so I can't actually verify that it works, so it would be nice if you'd chime back here if it works as it should.

@pieman64
Copy link

@WereCatf I have some of the Pro's and did some manual adjustments to boards.txt. Am I right in thinking we still can't access the memory from 4M to 16M?

@WereCatf
Copy link
Contributor

@pieman64 I don't know, I ain't got any 16MB-boards. If you've got the pros then you could just try.

@wfraga18
Copy link

why WEMOS never answer the forum questions ?? launch a product, but don't have support !

HOW TO USE 16MB ???

@WereCatf
Copy link
Contributor

@wfraga18 Ask them, not us. We don't work for Wemos. As for using it: how about you actually read the replies here, instead of raging? I provided a new boards.txt above you can try.

@TotallyInformation
Copy link

Thanks @WereCatf, I'm at a conference for a few days, I'll try after that. Good work.

@4refr0nt
Copy link
Author

@WereCatf, I'm try flash one of examples to Wemos D1 mini Pro with new boards.txt without any issues (spiffs not tested)

@martinayotte
Copy link
Contributor

martinayotte commented Nov 9, 2016

I've received my Mini Pro today. The sketch is running fine with 16MB settings, but SPIFFS and OTA becomes broken. It is not broken if I switch back to 4MB settings.
So, for now, I think the additional free Flash is only usable using direct calls to Esp.flashWrite() and Esp.flashRead(), although I didn't try it yet ...

EDIT : I think using Esp.flashWrite() and Esp.flashRead() as is won't work either, but doable if hacked with the following method :
http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html

@martinayotte
Copy link
Contributor

Using the hack details here http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html, I was able to get SPIFFS working on 16MB, but there still issues : SDK still try to access WiFi configs like 4MB, so I had to move SPIFFS_START after the first 4MB, with 12MB remaining. Also, I still face issue with OTA, when e-boot is call, I get "need boot 1.4+", and hangs forever until doing a erase_flash with esptool.py.

@pbecchi
Copy link

pbecchi commented Apr 11, 2017

Any update on over 4 mB support?
I read that should be included in next Arduino Code release 2.4!
It is already supported by the GitHub version?

@martinayotte
Copy link
Contributor

Take a look at #2351 (comment)
Of course, you need to tweak Arduino code to get those hacks merged manually.

@pbecchi
Copy link

pbecchi commented Apr 12, 2017

Thanks!
I understand that your mod and packing.org one are not merged yet.
Since I have some Esp8266 100 with 8Mb loosing 3 out of 8 isn't the best solution.
I was thinking to add a second Spiffs with SpiffsStart after 4Mb.Do you think it is feasible?

@martinayotte
Copy link
Contributor

Their a lot of work to be done to have 2 SPIFFS at the same time ...
But you can still use the unused part of the Flash as a raw flash storage, or you can reduce the 4MB part to 2MB and have a 6MB SPIFFS.

@pbecchi
Copy link

pbecchi commented Apr 13, 2017

This may be the best solution ....Some other value set to reduce 4Mb to 2?

@martinayotte
Copy link
Contributor

Yes, the SPIFFS location is defined in LD scripts, and you choose which one from Arduino Menu.

@capedra
Copy link

capedra commented Apr 23, 2017

@martinayotte I can't make SPIFFS from FS.h to work with the 12Mb of flash.

@martinayotte
Copy link
Contributor

In my setup, I moved the start above the 4M boundary, because ROM still access WiFi configs at the end of this first 4M.
PROVIDE ( _SPIFFS_start = 0x40600000 );
@capedra , did you modified the EspClass in Esp.cpp with the hack mentioned in platformio/platform-espressif8266#8 (comment) ?

@capedra
Copy link

capedra commented Apr 24, 2017

@martinayotte Yes, but these changes do not work with the functions from "FS.h" file.

@jonhp
Copy link
Contributor

jonhp commented May 5, 2017

I have read all of the posts on this issue. I think most of what is here is beyond the "usual" Arduino user. So, at the risk of collecting some developer ire, I would gently ask if added D1 mini Pro support is a goal and on what time-frame that might happen?
BTW I replaced the boards.txt with the one that has D1 mini Pro definitions. I still get Arduino IDE messages about the maximum size is 1MB.

@devyte
Copy link
Collaborator

devyte commented May 5, 2017 via email

@igrr
Copy link
Member

igrr commented May 8, 2017

SDK update to 2.1 will be tracked here: #3215

@igrr
Copy link
Member

igrr commented May 23, 2017

#3278 has preliminary support for 8MB and 16MB flash size

@capedra
Copy link

capedra commented Jun 29, 2017

@igrr any progress?

@igrr
Copy link
Member

igrr commented Jun 29, 2017

You can try the branch in #3278. As far as having SPIFFS on a 16 MB flash goes, it works.

@pieman64
Copy link

@igrr so 16MB SPIFFS support is still not available in the master branch, right?
Any idea when it might be?

@capedra
Copy link

capedra commented Jul 4, 2017

@pieman64
@igrr
@projectgus
@Spritetm

Ivan, I've been using SDK 2.4.0-RC1 because you said that the new fixes were more important than this 16 megabytes upgrade. I don't even know how to install the branch #3278 on Windows. Also, does this branch include the new changes of 2.4.0-RC1 ?

When will all the checkpoints become checked there on that branch? Are you going to finish that or is your main focus on ESP-32 from now on?

Maybe SDK 2.4.1 would sound better than 2.4.0-RC2?

Best Regards.

@Paraphraser
Copy link
Contributor

In my case, I ordered three WeMos D1 Mini Pros, across two separate orders. The first order was for two 32MB boards. I thought I was ordering a third 32MB board but I made a mistake and wound up with an 8MB board.

I have had no trouble with the 8MB board. I can upload a "data" directory and/or let SPIFFS.begin() create the partition automatically.

Conversely, I had a lot of trouble with the 32MB boards. Choosing "Tools > ESP8266 Sketch Data Upload" seemed to work but the sketch would not progress beyond SPIFFS.begin(). With debugging enabled, the code coming back was -10025 (SPIFFS_ERR_NOT_A_FS). What was happening was that SPIFFS.begin() was not seeing the partition created by "Sketch Data Upload" and was attempting to auto-format a fresh partition but was getting it wrong. Part of "getting it wrong" appeared to involve dribbling over the top of the sketch.

There is a fair bit of advice on using esptool to work around this kind of problem but none of the examples I found made any difference. Always a -10025 error as above.

The solution I stumbled across and which may be useful for others was to tell the IDE that the board was a "WeMos D1 R2 & Mini" rather than a "WeMos D1 mini Pro". When told that the 32MB board is a Mini rather than a Mini Pro, both uploading a "data" directory or allowing SPIFFS.begin() to auto-format the partition works.

While I have not checked whether this is actually the case, it seems likely that the board will be configured as if it were 4MB so, yes, a lot of flash is sitting there doing nothing. But, right now, I don't have the need for a huge amount of flash so this is an effective workaround.

@bill-orange
Copy link

I had precisely the same experience. I don't know if a solution to this problem is in work or not. In any case, extra SPIFFS will not do much for me. What I need is variable storage space. I am reading in large JSON strings (46k per string). I can get the code to run on ESP32 but not ESP8266.

@Paraphraser
Copy link
Contributor

What I wrote above has one factual error. I said that the third mini pro board had 8MB but it actually has 16MB.

I have also done some more experimenting:

  1. With the IDE configured to "WeMos D1 mini pro":
  • the IDE will upload a "data" directory to both the 16MB & 32MB boards. Regardless of actual board size, the claimed size of the partition is 15MB. In other words, the IDE does not seem to be able to work out that the 32MB boards have 32MB.
  • any sketch running on the 16MB board will successfully re-initialise the file system when it encounters SPIFFS.begin(). In other words, the data directory upload will be lost but files can be written to and read back from the partition once it has been re-initialised. The reported size of the re-initialised partition is slightly over 14MB.
  • any sketch running on the 32MB boards will try to re-initialise the file system when it encounters SPIFFS.begin() but will eventually dribble over the sketch. In other words, both the data directory upload and the sketch will be lost. There seems to be no way of mimicking the 16MB board behaviour.
  1. With the IDE configured to "WeMos D1 R2 & Mini":
  • the IDE will upload a "data" directory to both the 16MB & 32MB boards. The claimed size of the partition is always 1MB.
  • any sketch running on either 16MB or 32MB board will successfully read files that were uploaded without attempting to reinitialise the file system. In all cases, the reported size of the partition is slightly under 1MB.
  1. I also performed the following steps on both sizes of board:
  • Set the IDE to "WeMos D1 mini pro".
  • Upload the data directory. This (apparently) results in a 15MB partition being created.
  • Set the IDE to "WeMos D1 R2 & Mini" and run the sketch.
  1. Irrespective of board size, the result of running the sketch was:
  • SPIFFS.begin() returned immediately. In other words, it sensed a valid SPIFFS partition and did not attempt to re-initialise.
  • SPIFFS reported the partition size as being slightly under 1MB.
  • SPIFFS was unable to find any of the files uploaded via the "data" directory process.
  1. To me, this sounds incredibly hairy and a formula for data loss. My advice to anyone reading this is to stick to calling a mini pro a "WeMos D1 R2 & Mini" and live with the 1MB SPIFFS limitation until this bug is fixed.

@klaasdc
Copy link

klaasdc commented Jun 16, 2018

Just to confirm that I have exactly the same issue with a Wemos D1 mini Pro 16MB. After uploading sketch data, calling Spiffs.begin() will reformat, after raising error -10025.

But, I've had a few occasions (1 in 20), where it actually did work after uploading...

@capedra
Copy link

capedra commented Jun 16, 2018

In my humble opinion, I rather stay with TTGO T1 (cheaper than Wemos Pro: only US$ 5.00) or LOLIN D32 PRO V2.0.0 instead of getting my files corrupted in a very slow flash SPI memory.

You guys could tell me that it will be more expensive this way because you also need to buy a SD card, but it's the price that you pay for a decent reliability.

@jschlyter
Copy link

I can confirm this issue and that it can be mitigated by calling a "Wemos Mini Pro" a "WeMos D1 R2 & Mini" (with resulting 1MB SPIFFS limitation). Please advise.

@mike-s123
Copy link

I'll just add a bit to this. I've got a "pro" working with 14 MB SPIFFS. The only problem I have with it is that uploading is inconsistent - with no other changes, SPIFFS will be OK after some uploads, but not others. It seems to be better when the upload speed is slower (I used 115200 on the last successful upload, and waited forever), but I haven't done it enough to be sure. But once I get a "good" SPIFFS upload, things seem to be fine. I'm doing 99% reading, very little writing.

I suppose the SPIFFS image could be being built differently each time, but that's doubtful because I'm making no changes to the "data" folder between attempts.

@devyte
Copy link
Collaborator

devyte commented Mar 20, 2019

Is that with latest git or with 2.5.0?

@mike-s123
Copy link

Sorry for not being more complete.

ESP Core version 2_5_0, SDK version 3.0.0-dev(c0f7b44), Arduino IDE 1.8.8. Flash Chip ID 0x1840c8 (Winbond 25Q128, I think), on a CCCC (Cheap Chinese Crap Clone) of a v1.0.0 WEMOS D1 mini pro. If you would like any more detail, just ask and tell me how to get it.

It seems to act the same with both 0.3.0 and the very recent 0.4.0 of the uploader (arduino-esp8266fs-plugin). Whatever the root cause, if I upload enough times, eventually it works, which makes me suspicious of the upload process. Even when there's a good upload, SPIFFS.begin() takes a while on the first reboot after uploading (15-30 seconds?, less than if it's a bad upload). The equivalent of a *nix fsck, perhaps? On further reboots, it passes SPIFFS.begin() quickly, so there may be a minor issue with building the SPIFFS image.

I'd think there would be good error checking on the upload, but perhaps not.

YMMV. I thought I was wrong once, but I was mistaken. :)

@devyte
Copy link
Collaborator

devyte commented Mar 20, 2019

Try installing latest git and retry. There was a PR migrating from esptool-ck to latest esptool.py after 2.5.0 was released. Please report back here.
Don't forget to uninstall 2.5.0 before installing latest git.

@mike-s123
Copy link

Uhm, OK. I'm a rookie (Google is my friend). I'll work on figuring that out, but I will note that the "Latest version of ESP8266 Arduino [0.4.0, arduino-esp8266fs-plugin] uses esptool.py for uploads.", and I've installed that independently.

@devyte
Copy link
Collaborator

devyte commented Mar 20, 2019

I don't think installing esptool.py by itself will do anything, unless you're manually using it to flash the bin from command line. The core installation installs its own esptool.py and sets up to use it from the IDE.

@Paraphraser
Copy link
Contributor

Several comments on this.

First, I've found that using:

    Tools > Board > [ESP8266 Boards] LOLIN (WEMOS) D1 R2 & mini

is the only way to get anywhere. This is irrespective of the actual board type and actual SPIFFS size. I acknowledge that this wastes a lot of flash on "pro" boards but I prefer reliability.

Second, I've found that all flavours of WeMos D1 are much better behaved by having a jumper between D0 and RST, and then including this code in setup()

    // were we just reset by the IDE?
    if (ESP.getResetInfoPtr()->reason == REASON_EXT_SYS_RST) {
        
        // yes! perform a 1-second timed sleep to fake a reset
        ESP.deepSleep(1E6);

    }

The kinds of misbehaviour I'm talking about definitely include odd SPIFFS problems such as have been mentioned in the recent posts.

As a by-product of this setup, any code elsewhere in a sketch that runs into difficulties can trigger a restart - which simplifies a lot of design decisions.

Third, several times after using a different type of board and then coming back to a WeMos board I've fallen into the trap of opening the sketch and forgetting to check the Tools > Flash Size setting. If I upload any SPIFFS-dependent sketch while Tools > Flash Size is set to the default of "No SPIFFS" then everything turns to custard. It does not matter whether the upload is by wire or OTA. Once this happens, it usually requires a USB connection, a re-upload of the "data" directory (sometimes several attempts), and then the sketch, before normal service is resumed.

You can protect against compiling for the wrong board using something like:

    #if (!ARDUINO_ESP8266_WEMOS_D1MINI)
        #error "Board may not be LOLIN (WEMOS) D1 R2 & mini (check SPIFFS)"
    #endif

but I have yet to find any way of chucking a compile-time error if SPIFFS is not set correctly (if anyone reading this has solved that problem, I'd be grateful if you'd share the knowledge).

@bill-orange
Copy link

bill-orange commented Mar 20, 2019 via email

@mike-s123
Copy link

mike-s123 commented Mar 20, 2019

Try installing latest git and retry. There was a PR migrating from esptool-ck to latest esptool.py after 2.5.0 was released. Please report back here.
Don't forget to uninstall 2.5.0 before installing latest git.

@devyte, that seems to work swell. I uploaded 14MB SPIFFS four times, with two different boards, all without error. Much faster, too, with the compression.

Wrote 14659584 bytes (940371 compressed) at 0x00200000 in 65.0 seconds (effective 1803.6 kbit/s)...
Hash of data verified.

@Paraphraser
Copy link
Contributor

Paraphraser commented Mar 20, 2019

Hi bill-orange

You are quite right. I should have explained what I meant when I wrote "any code elsewhere in a sketch that runs into difficulties can trigger a restart". Sorry about that.

One of the problems I hit with the ESP8266 is that ESP.restart() does not work properly if the most-recent reboot was from the IDE. The pattern is that the restart fails, the watchdog timer fires, and faff-all happens after that.

It is easy to work around this problem by manually hitting the reset button after an upload but you have to remember to do that every time. Once a project becomes a production system, it's easy to forget to reset it by hand after a code change.

The code in my earlier post checking for REASON_EXT_SYS_RST mimics pressing the reset button (because of the way D0 pulls RST LOW at the end of the sleep).

Further, once you accept that D0+RST should be tied together to solve the original problem, there's no actual reason to use ESP.restart() anywhere.

So, the code I actually use is:

void reboot () {

    // blink the LED rapidly
    for (int i = 0; i < 20; i++) {

        digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
        delay(50);
        
    }
    
    // go into deep sleep for a little while (one second in this case)
    ESP.deepSleep(1E6);

}

and in setup() my code actually looks like this:

    // were we just reset by the IDE?
    if (ESP.getResetInfoPtr()->reason == REASON_EXT_SYS_RST) {

        // yes! perform a timed sleep/wake to cure this problem
        reboot();

    }

Elsewhere, any condition that might benefit from a prophylactic restart could go something like this (using WiFi as an example):

    // 10-second timeout assuming 1/2-second delays
    int timeout = 20;

    // start WiFi
    WiFi.begin(ssid,password);

    // wait for connection
    while (WiFi.status() != WL_CONNECTED) {
        
        delay(500);
        
        digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
        
        timeout--;

        if (timeout < 1) reboot();
        
    }

    digitalWrite(LED_BUILTIN, LOW);

Best case is that the reboot will cure whatever ails WiFi. Worst case is that the ESP8266 will keep restarting but the LED pattern (10 slow blinks followed by 10 rapid blinks) will be a giant clue that something is wrong.

And, to tie this back to the SPIFFS issue, if I forget to set the flash size menu item to 1M SPIFFS and an upload then corrupts SPIFFS (or makes it inaccessible), because my WiFi credentials are stored in SPIFFS the result is that the above code will exhibit this continual slow/fast blink reboot pattern. Cue sagged shoulders, slap of the forehead, and resolving to do better next time.

@bill-orange
Copy link

bill-orange commented Mar 20, 2019 via email

@devyte
Copy link
Collaborator

devyte commented Mar 21, 2019

@mike-s123 great news, thanks for reporting back!

@d-a-v
Copy link
Collaborator

d-a-v commented Aug 29, 2019

Have you recently tried to use again the full 16MB chip with LittleFS or esptool.py ?

@bill-orange
Copy link

I have not.

@Paraphraser
Copy link
Contributor

Neither have I. Treating all Mini Pros as D1/R2 with 1MB SPIFFS is ample. Not needing more storage means no need to keep trying to find a better solution.

Truth to tell, I have no idea what LittleFS is so I can't even make a sensible guess as to whether I have been using it or not - assuming that it would be plausible for it to be buried within the IDE and for me to be using it unawares.

As to esptool.py, implicitly via the IDE, yes, but subject to the caveat of D1/R2 + 1MB SPIFFS. Via command line, no.

@d-a-v
Copy link
Collaborator

d-a-v commented Aug 29, 2019

LittleFS is a better FS replacement in place of SPIFFS (faster in many cases - slowness/wdt was one of the issues with SPIFFS and 16MB flash chips - and sub-directory support).

esptool.py allows to flash the FS independently from the sketch.

Closing as 16MB flash chips are supposed to be working on every boards now.
Please open a new issue if something is wrong with git-master code.

@d-a-v d-a-v closed this as completed Aug 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests