SD card stops working after updating arduino-esp32 from ver. 1.0.6 to ver 2.0.2

[paoloinverse]

Board

ESP32 Dev Module

Device Description

Classic 38 pins DevKitC v4 module with ESP-WROOM32 and onboard antenna

Hardware Configuration

As shown by the many pinout diagrams available: VSPI standard pins attached to the SD card reader, card is used in SPI mode.

Version

v2.0.2

IDE Name

Arduino IDE

Operating System

Linux Ubuntu 20.04.3 LTS (Focal)

Flash frequency

80MHz

PSRAM enabled

no

Upload speed

921600

Description

The SD card test example sketch is not working when using version 2.0.2 of arduino-esp32.
Also, even in any other sketch using the SD card (SD.h library), the SD.begin() calls fail consistently.

The same sketch works flawlessly with version 1.0.6

Sketch

/*
 * Connect the SD card to the following pins:
 *
 * SD Card | ESP32
 *    D2       -
 *    D3       SS
 *    CMD      MOSI
 *    VSS      GND
 *    VDD      3.3V
 *    CLK      SCK
 *    VSS      GND
 *    D0       MISO
 *    D1       -
 */
#include "FS.h"
#include "SD.h"
#include "SPI.h"

void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("Failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if(levels){
                //listDir(fs, file.path(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void createDir(fs::FS &fs, const char * path){
    Serial.printf("Creating Dir: %s\n", path);
    if(fs.mkdir(path)){
        Serial.println("Dir created");
    } else {
        Serial.println("mkdir failed");
    }
}

void removeDir(fs::FS &fs, const char * path){
    Serial.printf("Removing Dir: %s\n", path);
    if(fs.rmdir(path)){
        Serial.println("Dir removed");
    } else {
        Serial.println("rmdir failed");
    }
}

void readFile(fs::FS &fs, const char * path){
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if(!file){
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while(file.available()){
        Serial.write(file.read());
    }
    file.close();
}

void writeFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }
    if(file.print(message)){
        Serial.println("File written");
    } else {
        Serial.println("Write failed");
    }
    file.close();
}

void appendFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if(!file){
        Serial.println("Failed to open file for appending");
        return;
    }
    if(file.print(message)){
        Serial.println("Message appended");
    } else {
        Serial.println("Append failed");
    }
    file.close();
}

void renameFile(fs::FS &fs, const char * path1, const char * path2){
    Serial.printf("Renaming file %s to %s\n", path1, path2);
    if (fs.rename(path1, path2)) {
        Serial.println("File renamed");
    } else {
        Serial.println("Rename failed");
    }
}

void deleteFile(fs::FS &fs, const char * path){
    Serial.printf("Deleting file: %s\n", path);
    if(fs.remove(path)){
        Serial.println("File deleted");
    } else {
        Serial.println("Delete failed");
    }
}

void testFileIO(fs::FS &fs, const char * path){
    File file = fs.open(path);
    static uint8_t buf[512];
    size_t len = 0;
    uint32_t start = millis();
    uint32_t end = start;
    if(file){
        len = file.size();
        size_t flen = len;
        start = millis();
        while(len){
            size_t toRead = len;
            if(toRead > 512){
                toRead = 512;
            }
            file.read(buf, toRead);
            len -= toRead;
        }
        end = millis() - start;
        Serial.printf("%u bytes read for %u ms\n", flen, end);
        file.close();
    } else {
        Serial.println("Failed to open file for reading");
    }


    file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }

    size_t i;
    start = millis();
    for(i=0; i<2048; i++){
        file.write(buf, 512);
    }
    end = millis() - start;
    Serial.printf("%u bytes written for %u ms\n", 2048 * 512, end);
    file.close();
}

void setup(){
    Serial.begin(115200);
    delay(2000);
    Serial.println("ESP-IDF version is: " + String(esp_get_idf_version()));
    Serial.println("SPI initialization start");
      
    if(!SD.begin()){
        Serial.println("Card Mount Failed");
        return;
    }
    uint8_t cardType = SD.cardType();

    if(cardType == CARD_NONE){
        Serial.println("No SD card attached");
        return;
    }

    Serial.print("SD Card Type: ");
    if(cardType == CARD_MMC){
        Serial.println("MMC");
    } else if(cardType == CARD_SD){
        Serial.println("SDSC");
    } else if(cardType == CARD_SDHC){
        Serial.println("SDHC");
    } else {
        Serial.println("UNKNOWN");
    }

    uint64_t cardSize = SD.cardSize() / (1024 * 1024);
    Serial.printf("SD Card Size: %lluMB\n", cardSize);

    listDir(SD, "/", 0);
    createDir(SD, "/mydir");
    listDir(SD, "/", 0);
    removeDir(SD, "/mydir");
    listDir(SD, "/", 2);
    writeFile(SD, "/hello.txt", "Hello ");
    appendFile(SD, "/hello.txt", "World!\n");
    readFile(SD, "/hello.txt");
    deleteFile(SD, "/foo.txt");
    renameFile(SD, "/hello.txt", "/foo.txt");
    readFile(SD, "/foo.txt");
    testFileIO(SD, "/test.txt");
    Serial.printf("Total space: %lluMB\n", SD.totalBytes() / (1024 * 1024));
    Serial.printf("Used space: %lluMB\n", SD.usedBytes() / (1024 * 1024));
}

void loop(){

}

Debug Message

ESP-IDF version is: v4.4-beta1-189-ga79dc75f0a
SPI initialization start
[  2333][W][sd_diskio.cpp:104] sdWait(): Wait Failed
[  2333][E][sd_diskio.cpp:126] sdSelectCard(): Select Failed
[  2333][W][sd_diskio.cpp:510] ff_sd_initialize(): GO_IDLE_STATE failed
[  2338][E][sd_diskio.cpp:795] sdcard_mount(): f_mount failed: (3) The physical drive cannot work
[  2647][W][sd_diskio.cpp:104] sdWait(): Wait Failed
[  2647][E][sd_diskio.cpp:126] sdSelectCard(): Select Failed
Card Mount Failed

Other Steps to Reproduce

Please use the sketch I made available. It's a modified version of SD_Test example that works perfectly in version 1.0.6 of arduino-esp32 and fails consistently when using version 2.0.2 of arduino-esp32

You need one SD card reader, connect GND, V+ and of course the VSS, VMISO, VMOSI, VCLK to the pertaining card reader pins. Usage of the card is in SPI mode.

steps:

• Make sure to check that you have SD_MMC.h and SD.h available
• from the arduino board mamanger install arduino-esp32 1.0.6, compile and upload my sketch
• from the arduino board mamanger install arduino-esp32 2.0.2, compile and upload my sketch <- this time it won't initialize the SD card

Only two things I have not tried yet:

1. starting from a clean new user profile, with a clean Arduino IDE install
2. manually installing the latest arduino-esp32 master branch.

This problem seems to be somewhat widespread, and personally I'd like to understand why it appeared and if it can be software fixed within the sketch rather than being worked around by wiping the slate clean.
Sorry, I might be biased, but wiping the slate does not sound acceptable if the problem arises under normal usage of the framework.

There is one pre-existing issue that was closed with a workaround without identifying what the actual problem is, because reproducing it proved somewhat difficult: #5701

I think this time I can provide a way to reliably reproduce the issue.

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[paoloinverse]

One more thing, it goes without saying that you need an SD card reader AND an actual SD card to be read. either micro SD or full size SD will do, depending on your reader.

[P-R-O-C-H-Y]

Hello @paoloinverse, I was now replying to you in the issue you commented. Actually this is a duplicate for mounting issue.

If you want to try Arduino-esp32 latest commits you need to download master branch in a zip file and unzip it in the Arduino IDE folder, where are the board data stored. What OS do you have? Can help you with that :) or just google it.

In 1.0.6 there was actually a missing part which gets fixed in 2.0.0 and that fix causes mount issues with some SD cards / boards. Still don't know where exactly is a problem but there is a fix coming in PR #6162. So if you now download master branch paste it into Arduino IDE, you can after that copy changes from the PR (change in file sd_diskio.cpp and SD cards shroud mount and work fine :) Actually will help us to test the changes :)

[paoloinverse]

@P-R-O-C-H-Y thank you. I'm more than willing to help with testing.
I'm on ubuntu 20.04.
For the moment, my project will officially recommend 1.0.6 as I cannot cause big incoveniences to anyone willing to use it,
In the meanwhile I'll be glad to test and help port the fix in the next stable 2.0.x release

[P-R-O-C-H-Y]

@paoloinverse Thats welcomed :)
So please download master branch zip and apply fix in mentioned PR in comment above.

There is a manual to how to install it on Linux link

You can find the path where is Arduino-esp32 stored:

• Open Arduino IDE
• Preferences
• You can see path down in the window IMAGE
• Go to path and there will be packages folder, go in.
• You will see there will be esp32 and version. So you can unpack downloaded zip to the 2.0.2 version.

[paoloinverse]

@protttoy

Use another pin as CS. or try

Thank you but that is not the case. It seems the matter was a timeout of the first SD card hardware reply that is now fixed as the ESP32 framework correctly takes into account the possibility of an initial failure and handles it gracefully.
Some card readers and microSD cards do it, while others work immediately on first contact.

@P-R-O-C-H-Y I can gladly confirm the fix in https://github.com/P-R-O-C-H-Y/arduino-esp32/tree/SD-fix works as expected, I just copied this over the master dir and finalized the install with get.py as per the official instructions.
The SD functionality is back!

One final note, this fix is expected get officially into next version: 2.0.3 of arduino-esp32

You can however still apply it manually to the current 2.0.2 (at the time of writing this reply)

[protttoy]

@paoloinverse sorry. I was exactly facing the same issues and while trying things used core 1.0.6 and it worked. I was trying out a lot of things to make it work, and I thought changing CS solved it, but actually it was the downgrading ! After I figured out that it was the issue of the new version I deleted my comment.

[paoloinverse]

@paoloinverse sorry. I was exactly facing the same issues and while trying things used core 1.0.6 and it worked. I was trying out a lot of things to make it work, and I thought changing CS solved it, but actually it was the downgrading ! After I figured out that it was the issue of the new version I deleted my comment.

To be totally fair, I've not tried your other suggestion:

while(!SD.begin()){
Serial.println("Card Mount Failed");
return;
}

which actually sounded very reasonable. I leave it here just in case. (but again, there's an incoming permanent fix to the matter)

[paoloinverse]

I'm now requesting to close the issue, thank you all!

[P-R-O-C-H-Y]

@paoloinverse You are welcome :)
The fix for the SD card is planned to be in next version -> 2.0.3

[marmou-fritzing]

Hello.
I solving last three days problem with SD on 2.0.2 too. Exactly - I remake my sketch and it already works on 1.0.6 and also on 2.0.2, but in oposite to 1.0.6 the results from FS.name() are little different BUT (And this is most important) the speed of SD work on 2.0.2 is MUCH, MUCH SLOWER than 1.0.6.
For example - list of folders - the same sketch and the same device NODEMCU ESP-32S:
1.0.6: Cca 4s, 2.0.2 Cca 15 s !
I tried find the problem in my sketch first, but afer 3 days I got an idea try to go back to oldest library 1.0.6 - and BINGO!
I don't know exactly where is the problem - SPI, FS, SD...

[mrengineer7777]

@marmou-fritzing You are commenting on a closed issue. No one is going to see it. Please open a new issue.

[VojtechBartoska]

@marmou-fritzing and @mrengineer7777...I'm tagging @P-R-O-C-H-Y to take a look on the commet above. We can stay here, no new issue is needed for now. We will check it. :) Thanks!

[P-R-O-C-H-Y]

@marmou-fritzing Can you try to download latest version directly from GitHub? There are already fixes in master branch for SD card. We know that in 2.0.2 its not working very well. For now you can use master branch or stick to 1.0.6 and wait for 2.0.3 release :)