pico-sdk only rp2040 implementation

[bog-dan-ro]

Hello there,

I'd like to add support for rp2040 but based only on pico-sdk, I'd love to upstream it, so I'd like to ask if such a contribution is something that the maintainer(s) will accept.

Why pico-sdk only? Mostly because the arduino-pico can't be used with cmake projects .

[greiman]

I have no interest in supporting SdFat on new platforms. I will not being doing much much new development of SdFat.

A file system should be integrated into the operating system not a library like SdFat. When I started SdFat in 2008 for an Arduino with 1 KB of RAM, I assumed it would be replaced with a proper filesystem.

The pico-sdk should have an integrated file system with SD being one of the file devices.

It would be fairly easy to port SdFat to pico-sdk. I use SdFat on the ChibiOS/RT RTOS. You need to replace a couple of calls to millis(), digitalWrite(), and pinMode().

You need a wrapper for SPI like in this example.

I have a very fast pio SDIO driver for rp2040 that can do over 20 MB/sec. I will post it as an example eventually.

[bog-dan-ro]

Thanks a lot for your quick reply!

have no interest in supporting SdFat on new platforms. I will not being doing much much new development of SdFat.

In this case I'll not waste time to try to isolate the Arduino stuff and I'll just remove it or replace it with pico-sdk.

The pico-sdk should have an integrated file system with SD being one of the file devices.

It should, but ... Right now I'm using https://github.com/carlk3/no-OS-FatFS-SD-SDIO-SPI-RPi-Pico but is buggy.

Thanks a lot for the link. Regarding PIO SDIO implementation, how big is it? I'm asking as I might not be able to use it as I'm a bit short on PIO space (on both of PIOs :) ).

[bog-dan-ro]

@greiman did you manage to post the pio SDIO driver?
Thanks in advance :) !

[greiman]

I looked at the SDIO driver but became busy with other things.

You probably can't use it since it uses almost all memory in one PIO. I implemented it to be fast for small transfers like the SdFat dedicated SPI mode but that takes more memory.

I also need to decide whether to do more development. There are lots of possible configuration options so I need to make it easy to choose the pins and speed options. This can't be done at run time since it must be compiled into the PIO code.

I also need to decide whether to develop a second mode. SD cards start in the very old "Standard mode" which was limited to 25 MHz. Modern cards can run much faster in that mode. I find all cards I have tested can run at 62.6 MHz with 250 MHz CPU speed.

Implementing "High Speed" mode is a big job since it has totally different bus timing but the SD start in "Standard" mode. So a reload of the PIO code is required during the initialization process. 25 years ago 50 MHz bus speed required an unusual timing.

I also don't want to increase the amount time I spend on maintenance of SdFat. I don't enjoy development on the Arduino platform. I had high hope for Arduino15 years ago when I released the first version of SdFat. I now don't use Adruino for my projects.

[bog-dan-ro]

First and foremost, I'd like to say thanks a lot Bill for all your effort on this project. As a FOSS contributor myself, I know it is hard to maintain a project that is used by so many people even if we like it or not ;-).

I managed to free one of the PIOs, next I'll try to see if I can adapt ZuluSCSI's SDIO implementation. Is your implementation much different?

[greiman]

Is your implementation much different?

My implementation is very different. I started from scratch, I didn't copy any existing code.

Modern SD cards require vary large transfers for maximum speed. I do this for both read and write by keeping the SD in read or write mode as long as possible. To do this I stop the SDIO clock between 512 byte transfers.

Here is a typical result of the bench example for 512 byte reads and writes. This is overclocked at 250 MHz CPU.

This is on an Adafruit Metro RP2040. I also test on a Pico.

Type is exFAT
Card size: 128.18 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: YD4QD
Revision: 3.0
Serial number: 0XCBDD66A0
Manufacturing date: 9/2023

FILE_SIZE_MB = 50
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
24366.41,48,19,20
24366.41,37,19,20

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
24402.08,77,20,20
24402.08,169,20,20

It is slower for smaller transfers. In this case I can't transfer directly to the user buffer and must cache the 512 byte block and do an extra copy using the CPU and memcpy.

FILE_SIZE_MB = 50
BUF_SIZE = 64 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
13488.00,34,1,4
13484.36,34,1,4

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
14425.85,75,1,4
14421.69,204,1,4

[greiman]

I looked at the current state of my RP2040 SDIO. The source needs to be cleaned up since I was still playing with different approaches.

I looked at the High Speed mode again. Card output valid timing has a long delay from the rising edge of clock. I might try it again at some point.

Here is High Speed mode timing:

I have a 350 MHz mixed signal scope and a fast logic analyzer so I can debug the PIO.

[androda]

You probably can't use it since it uses almost all memory in one PIO. I implemented it to be fast for small transfers like the SdFat dedicated SPI mode but that takes more memory.

Existing RP2040 SDIO systems like the aforementioned https://github.com/carlk3/no-OS-FatFS-SD-SDIO-SPI-RPi-Pico already use most (or all) of the instruction memory in one of the PIO units. It's not a downside for yours to use most of the instruction memory, rather it seems to be the status quo. :)

[greiman]

@androda

I had hoped to post a version of SdFat with RP2040 SDIO some time ago but have been very busy and away from home so not had time to finish testing all the mods in the new version.

[androda]

@greiman
If you'd like another set of eyes on it, let me know. I have an Agilent logic analyzer frame with 16717A card which ought to be more than up to the task of checking SD waveforms.

[greiman]

Here is a test version of SdFat with RP2040 SDIO. Try it and let me know the result.

SdFatRp2040.zip

It is setup with two examples, Rp2040SdioSetup and bench. I test with an Adafruit RP2040 Metro and a Pi Pico.

I use the AdaFruit PiCowBell on Pico. The blue proto board has pins for a logic analyzer.

I tested at 133 MHz and 250 MHz. Here is my results for the bench example:

At 133MHz:

Type is exFAT
Card size: 128.18 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: YD4QD
Revision: 3.0
Serial number: 0XF024666C
Manufacturing date: 7/2023

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12819.69,2005,38,38
12657.42,2414,38,38

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12593.65,88,39,39
12562.01,83,39,39

At 250 MHz:

Type is exFAT
Card size: 128.18 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: YD4QD
Revision: 3.0
Serial number: 0XF024666C
Manufacturing date: 7/2023

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23472.68,1181,20,21
23472.68,1220,20,21

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23808.00,45,20,21
23808.00,49,20,21

[greiman]

@androda

Have you looked at boards like the Pico 2 with the new RP2350 family of chips?

I have become bored with most new MCU boards but Raspberry PI has another interesting processor.

[androda]

@greiman
Yes, pre-ordered some of the Pico 2. I'm also excited about the RP2350B, given the additional IO pins. We should see some really interesting projects with this new chip.

[androda]

Got things set up this morning for building the RP2040 library, here are the first results (no clock speed changes yet, just compiled as-is and run).

FreeStack: 253848
Type is exFAT
Card size: 31.91 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: WX32G
Revision: 8.0
Serial number: 0X830F12B8
Manufacturing date: 3/2023

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12406.15,3134,37,39
12164.67,6934,38,39

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12593.65,84,39,39
12562.01,68,39,39

Done
Type any character to start

edit: Results at a 500MB file are nearly the same

FILE_SIZE_MB = 500
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12348.42,47941,37,40
12280.78,52137,37,41

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12591.28,55,39,39
12591.28,66,39,39

edit the 2nd (son of edit):
At 250MHz my speeds aren't quite as high as yours but this is a WD Purple card, not really "made for speed".

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
22192.62,46673,20,22
21952.92,51420,20,22

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23771.03,30,21,21
23771.03,28,21,21

edit the third (son of edit returns!):
These tests were executed on a BlueSCSI V2 PCB.

[greiman]

Edited: had ms where μs should have been.

Write is often slower for big writes since there will be occasional large latencies while the card programs flash.

Your card has max latencies of about 50 ms. For 133 MHz the average latency was 3-4 μs greater than the min latency.

For the card I used with a 500 MB file, the max latency is 8-10 ms and average latency is one μs greater than the min.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12961.08,10603,37,38
12952.69,8179,37,38

For some apps occasion large write latency is very undesirable.

Here is my card for 500 MB at 250 MHz. Max of 8 ms write latency.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
24273.01,8285,20,20
24271.83,8275,20,20

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23776.68,85,20,21
23774.42,197,20,21