Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I cannot boot linux. [LITEX-TERM] Got unexpected response from device 'b'E'' #354

Closed
nohahanon opened this issue Sep 17, 2023 · 34 comments
Closed

Comments

@nohahanon
Copy link

nohahanon commented Sep 17, 2023

Hi there!

I'm trying to boot with a Linux image created locally on a tang nano 20k, but I'm encountering an error that says "[LITEX-TERM] Got unexpected response from device 'b'E'" and the log stops without booting. I have attached the detailed log below.
Please provide any answer. Thank you!

{
    "Image": "0x40000000",
    "sipeed_tang_nano_20k.dtb": "0x40ef0000",
    "rootfs.cpio": "0x41000000",
    "fw_jump.bin": "0x40f00000"
}
  • log
$ sudo openFPGALoader -f -c ft2232 ~/linux-on-litex-vexriscv/build/sipeed_tang_nano_20k/gateware/sipeed_tang_nano_20k.fs
write to flash
Jtag frequency : requested 6.00MHz   -> real 6.00MHz  
Parse file Parse /home/nohara/linux-on-litex-vexriscv/build/sipeed_tang_nano_20k/gateware/sipeed_tang_nano_20k.fs: 
Done
DONE
Jtag frequency : requested 2.50MHz   -> real 2.00MHz  
Jtag frequency : requested 10.00MHz  -> real 6.00MHz  
erase SRAM Done
Detected: Winbond W25Q64 128 sectors size: 64Mb
Detected: Winbond W25Q64 128 sectors size: 64Mb
RDSR : 00
WIP  : 0
WEL  : 0
BP   : 0
TB   : 0
SRWD : 0
00000000 00000000 00000000 00
Erasing: [==================================================] 100.00%
Done
Writing: [==================================================] 100.00%
Done
$ sudo chmod 666 /dev/ttyUSB*
$ litex_term --serial-boot --images ~/TangNano-20K-example/linux/image5/boot.json /dev/ttyUSB1


litex> reboot


        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS CRC passed (5386dd7d)

 LiteX git sha1: 36ce71d5

--=============== SoC ==================--
CPU:		VexRiscv SMP-LINUX @ 48MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		64.0KiB
SRAM:		6.0KiB
L2:		128B
SDRAM:		8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2)
MAIN-RAM:	8.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 18.9MiB/s
   Read speed: 21.9MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading /home/nohara/TangNano-20K-example/linux/image5/Image to 0x40000000 (7726264 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (9.9KB/s).
[LITEX-TERM] Uploading /home/nohara/TangNano-20K-example/linux/image5/sipeed_tang_nano_20k.dtb to 0x40ef0000 (2115 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Got unexpected response from device 'b'E''


@nohahanon
Copy link
Author

I attempted to use the SD card containing the above files to boot sdcardboot, but it stopped as follows. Dtb file is not read.

litex> sdcardboot
    Booting from SDCard in SD-Mode...
    Booting from boot.json...
    Copying Image to 0x40000000 (7726264 bytes)...
    [########################################]
    Copying rv32.dtb to 0x40ef0000 (2689 bytes)...
    [  ]

@nohahanon
Copy link
Author

I have identified that the processing is halted at the following line in ff.c within the Litex output:
if (FfDiskOps->disk_read(fs->pdrv, rbuff, sect, cc) != RES_OK) ABORT(fs, FR_DISK_ERR);
I commented out this section, and I was able to successfully move the file to RAM. However, the processing halted at the liftoff stage.

litex> sdcardboot

Booting from SDCard in SD-Mode...
Booting from boot.json...
Copying Image to 0x40000000 (7726264 bytes)...
[########################################]
Copying rv32.dtb to 0x40ef0000 (2689 bytes)...
[########################################]
Copying rootfs.cpio to 0x41000000 (5376512 bytes)...
[########################################]
Copying opensbi.bin to 0x40f00000 (263652 bytes)...
[########################################]
Executing booted program at 0x40f00000

--============= Liftoff! ===============--



@avkghost
Copy link

avkghost commented Oct 2, 2023

Hi @nohahanon
I have the same problem with booting via serial/sdcard boot.

The short answer: It's need to create a customized project for small memory boards.

TL;DR I've found that the build system generated an incorrect device tree and system configuration. The board has only 8 MB of SDRAM, as shown during self-diagnosis, but the addresses generated in the device tree and the base address for opensbi are incorrect and lie beyond the 8 MB limit. The address 0x40f00000 corresponds to 15 MB. You can try to read addresses 0x4000000 and 0x40f00000 via mem_read and got different results for a first and a second address (second is incorrect address for the device).

make.py

part of device tree generated by make.py

        memory@40000000 {
            device_type = "memory";
            reg = <0x40000000 0x800000>;
        };

        reserved-memory {
            #address-cells = <1>;
            #size-cells    = <1>;
            ranges;

            opensbi@40f00000 {
                reg = <0x40f00000 0x80000>;
            };

        };

boot.json

{
	"Image":       "0x40000000",
	"rv32.dtb":    "0x40ef0000",
	"rootfs.cpio": "0x41000000",
	"opensbi.bin": "0x40f00000"
}

A tang nano 20k linux example

part of device tree from project contains right adresses in memory below 8 MB limit

        memory@40000000 {
            device_type = "memory";
            reg = <0x40000000 0x800000>;
        };

        reserved-memory {
            #address-cells = <1>;
            #size-cells    = <1>;
            ranges;

            opensbi@407c0000 {
                reg = <0x407c0000 0x20000>;
            };

        };

boot.json

{
	"Image":			"0x40000000",
	"sipeed_tang_nano_20k.dtb":	"0x40780000",
	"opensbi.bin.tangnano20k":	"0x407c0000"
}

The problem is in a litex/soc/cores/cpu/vexriscv_smp/core.py

soc.bus.add_region("opensbi", SoCRegion(origin=self.mem_map["main_ram"] + 0x00f0_0000, size=0x8_0000, cached=True, linker=True))

My suggestions:

  • Maybe previous versions litex allow to create a correct device tree for 8 MB boards.
  • Maybe sipeed created own version of litex/linux-on-litex-vexriscv for the tang nano 20k.

@nohahanon
Copy link
Author

nohahanon commented Oct 3, 2023

Hello @avkghost !
Thank you for the advice! I see that there are errors in the generated device tree.
I sent an inquiry email to Sipeed last week asking about how to create the Image and dtb. I'm currently waiting for a response from Sipeed (although my expectations are low).

And, I found a successful example of running Linux on the Tang Primer board. It seems that reducing the size of the Linux image is essential, so I will continue my research.
https://www.luffca.com/ja/2022/10/linux-tang-primer/

@nohahanon
Copy link
Author

I'm currently trying to modify the BusyBox configuration to see if I can create a smaller Linux image. I've outlined the specific steps below. It seems like I can create a smaller image by removing unnecessary commands, but I'm struggling to get it below 7.7MB. If anyone has any suggestions, I would appreciate it.

(get buildroot, extract and cd there)
$ sudo make BR2_EXTERNAL=../linux-on-litex-vexriscv/buildroot/ litex_vexriscv_defconfig
$ sudo make busybox-menuconfig
(select and delete commands.)
$ sudo make

@Dolu1990
Copy link
Contributor

Dolu1990 commented Oct 3, 2023

Enabline RVC compressed instruction may help reaching it (need to be enable on both litex arguments and linux / busybody.
Generaly, it reduce the kernel about 30 %

@peiceliu
Copy link

peiceliu commented Oct 4, 2023

Hello,bro @nohahanon , I have also encountered such a problem. Has your problem been resolved so far?

@peiceliu
Copy link

peiceliu commented Oct 4, 2023

Hello @Dolu1990 , I would like to ask if you have any good solutions for processing halted at the liftoff stage.I am looking forward to your reply .Thanks.

@nohahanon
Copy link
Author

nohahanon commented Oct 4, 2023

Hello, bro @peiceliu ,

  • I confirmed that by changing the # Ethernet block in linux-on-litex-vexriscv/buildroot/board/litex_vexriscv/linux.config as follows, the size of the Image becomes approximately 5.9MB.(7.7MB->5.9MB)
like this

before

# Ethernet
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_PACKET_DIAG=y
CONFIG_INET=y
CONFIG_NETDEVICES=y
CONFIG_NET_VENDOR_LITEX=y
CONFIG_LITEX_LITEETH=y

after

# Ethernet
CONFIG_NET=n
CONFIG_PACKET=n
CONFIG_PACKET_DIAG=n
CONFIG_INET=n
CONFIG_NETDEVICES=n
CONFIG_NET_VENDOR_LITEX=n
CONFIG_LITEX_LITEETH=n
  • I am unable to remove unnecessary commands from busybox. Even after executing 'make busybox-menuconfig' and removing some commands, when I build it, the resulting Image is still the same size. Additionally, specifying 'buildroot-minimal.config' in the .config file located directly under the buildroot directory, obtained by extraction, did not change the size of the Image when compared to specifying 'buildroot.config'.

@peiceliu
Copy link

peiceliu commented Oct 4, 2023

Thanks for your advice @nohahanon,after i change y to n,i still can not boot linux successfully. Should i change other files?Such as [litex/soc/cores/cpu/vexriscv_smp/core.py]. If you could give me appropriate advice to make linux to boot successfully on my fpga board, I would be very grateful.I am looking forward to your reply,bro.

@avkghost
Copy link

avkghost commented Oct 5, 2023

Hello @nohahanon,

I've seen the link you provided, thanks!
However, I had neither found a solution for the Linux kernel configuration, nor a solution for litex for a board configuration. So, the example contains information about Tang Primer which has a different FPGA (Anlogic instead of Gowin). In addition, the example shows that an author can create a multicore VexRiscv bitstream and program the FPGA with it.

  • Kernel configuration - I've tried to get the kernel configuration from itself, but unfortunately, the kernel didn't have it to reduce its size.
  • SoC configuration - I've also tried to generate a dual-core soc for Tang Nano 20k, but unfortunately, all my configurations did not fill in 20K LUT. The minimal amount of LUT's in for the dual-core that I've got was 51272, but the GW2AR has only 20k. When I used the SoC generator from litex-boards it generated a configuration with more LUTs (~60+k).

As I investigated the readme from TangNano-20K-example I've found that the Linux kernel, device tree, and boot loader provided by Icenowy.
I'm going to ask her for details about the kernel configuration.

So, I'm going to create a debug version of a VexRiscv to debug the booting process. First of all, it would help me to understand a configuration difference between my own and @Icenowy 's.

Update:

@nohahanon
Copy link
Author

Thanks @avkghost !!
I'm looking forward to it!

@Icenowy
Copy link

Icenowy commented Oct 6, 2023

@avkghost The LiteX port is by me, but booting Linux is done by another person, Jisheng Zhang .

@avkghost
Copy link

avkghost commented Oct 6, 2023

Hi @Icenowy.

Thanks for joining our discussion.
There are my experiments with litex-boards and linux-on-litex-vexriscv.

litex-boards

python -m litex_boards.targets.sipeed_tang_nano_20k --cpu-type vexriscv --cpu-variant linux --csr-csv csr.csv --uart-baudrate 3000000 --build
It produces csr.csv
csr_base,buttons,0xf0000000,,
csr_base,ctrl,0xf0000800,,
csr_base,identifier_mem,0xf0001000,,
csr_base,leds,0xf0001800,,
csr_base,sdram,0xf0002000,,
csr_base,timer0,0xf0002800,,
csr_base,uart,0xf0003000,,
csr_register,buttons_in,0xf0000000,1,ro
csr_register,ctrl_reset,0xf0000800,1,rw
csr_register,ctrl_scratch,0xf0000804,1,rw
csr_register,ctrl_bus_errors,0xf0000808,1,ro
csr_register,leds_out,0xf0001800,1,rw
csr_register,sdram_dfii_control,0xf0002000,1,rw
csr_register,sdram_dfii_pi0_command,0xf0002004,1,rw
csr_register,sdram_dfii_pi0_command_issue,0xf0002008,1,rw
csr_register,sdram_dfii_pi0_address,0xf000200c,1,rw
csr_register,sdram_dfii_pi0_baddress,0xf0002010,1,rw
csr_register,sdram_dfii_pi0_wrdata,0xf0002014,1,rw
csr_register,sdram_dfii_pi0_rddata,0xf0002018,1,ro
csr_register,timer0_load,0xf0002800,1,rw
csr_register,timer0_reload,0xf0002804,1,rw
csr_register,timer0_en,0xf0002808,1,rw
csr_register,timer0_update_value,0xf000280c,1,rw
csr_register,timer0_value,0xf0002810,1,ro
csr_register,timer0_ev_status,0xf0002814,1,ro
csr_register,timer0_ev_pending,0xf0002818,1,rw
csr_register,timer0_ev_enable,0xf000281c,1,rw
csr_register,uart_rxtx,0xf0003000,1,rw
csr_register,uart_txfull,0xf0003004,1,ro
csr_register,uart_rxempty,0xf0003008,1,ro
csr_register,uart_ev_status,0xf000300c,1,ro
csr_register,uart_ev_pending,0xf0003010,1,rw
csr_register,uart_ev_enable,0xf0003014,1,rw
csr_register,uart_txempty,0xf0003018,1,ro
csr_register,uart_rxfull,0xf000301c,1,ro
constant,config_clock_frequency,48000000,,
constant,config_cpu_has_interrupt,None,,
constant,config_cpu_reset_addr,0,,
constant,config_cpu_has_dcache,None,,
constant,config_cpu_has_icache,None,,
constant,config_cpu_type_vexriscv,None,,
constant,config_cpu_variant_linux,None,,
constant,config_cpu_human_name,vexriscv_linux,,
constant,config_cpu_nop,nop,,
constant,config_l2_size,128,,
constant,config_csr_data_width,32,,
constant,config_csr_alignment,32,,
constant,config_bus_standard,wishbone,,
constant,config_bus_data_width,32,,
constant,config_bus_address_width,32,,
constant,config_bus_bursting,0,,
constant,timer0_interrupt,1,,
constant,uart_interrupt,0,,
memory_region,rom,0x00000000,131072,cached
memory_region,sram,0x10000000,8192,cached
memory_region,main_ram,0x40000000,8388608,cached
memory_region,csr,0xf0000000,65536,io
As I see, the script produces configuration with wrong addresses.

make.py from linux-on-litex-vexriscv

 ./make.py --board sipeed_tang_nano_20k --uart-baudrate 3000000 --build
produces following csr.csv
csr_base,buttons,0xf0000000,,
csr_base,ctrl,0xf0000800,,
csr_base,uart,0xf0001000,,
csr_base,timer0,0xf0001800,,
csr_base,identifier_mem,0xf0002000,,
csr_base,leds,0xf0002800,,
csr_base,sdcard_block2mem,0xf0003000,,
csr_base,sdcard_core,0xf0003800,,
csr_base,sdcard_irq,0xf0004000,,
csr_base,sdcard_mem2block,0xf0004800,,
csr_base,sdcard_phy,0xf0005000,,
csr_base,sdram,0xf0005800,,
csr_register,buttons_in,0xf0000000,1,ro
csr_register,ctrl_reset,0xf0000800,1,rw
csr_register,ctrl_scratch,0xf0000804,1,rw
csr_register,ctrl_bus_errors,0xf0000808,1,ro
csr_register,uart_rxtx,0xf0001000,1,rw
csr_register,uart_txfull,0xf0001004,1,ro
csr_register,uart_rxempty,0xf0001008,1,ro
csr_register,uart_ev_status,0xf000100c,1,ro
csr_register,uart_ev_pending,0xf0001010,1,rw
csr_register,uart_ev_enable,0xf0001014,1,rw
csr_register,uart_txempty,0xf0001018,1,ro
csr_register,uart_rxfull,0xf000101c,1,ro
csr_register,timer0_load,0xf0001800,1,rw
csr_register,timer0_reload,0xf0001804,1,rw
csr_register,timer0_en,0xf0001808,1,rw
csr_register,timer0_update_value,0xf000180c,1,rw
csr_register,timer0_value,0xf0001810,1,ro
csr_register,timer0_ev_status,0xf0001814,1,ro
csr_register,timer0_ev_pending,0xf0001818,1,rw
csr_register,timer0_ev_enable,0xf000181c,1,rw
csr_register,leds_out,0xf0002800,1,rw
csr_register,sdcard_block2mem_dma_base,0xf0003000,2,rw
csr_register,sdcard_block2mem_dma_length,0xf0003008,1,rw
csr_register,sdcard_block2mem_dma_enable,0xf000300c,1,rw
csr_register,sdcard_block2mem_dma_done,0xf0003010,1,ro
csr_register,sdcard_block2mem_dma_loop,0xf0003014,1,rw
csr_register,sdcard_block2mem_dma_offset,0xf0003018,1,ro
csr_register,sdcard_core_cmd_argument,0xf0003800,1,rw
csr_register,sdcard_core_cmd_command,0xf0003804,1,rw
csr_register,sdcard_core_cmd_send,0xf0003808,1,rw
csr_register,sdcard_core_cmd_response,0xf000380c,4,ro
csr_register,sdcard_core_cmd_event,0xf000381c,1,ro
csr_register,sdcard_core_data_event,0xf0003820,1,ro
csr_register,sdcard_core_block_length,0xf0003824,1,rw
csr_register,sdcard_core_block_count,0xf0003828,1,rw
csr_register,sdcard_irq_status,0xf0004000,1,ro
csr_register,sdcard_irq_pending,0xf0004004,1,rw
csr_register,sdcard_irq_enable,0xf0004008,1,rw
csr_register,sdcard_mem2block_dma_base,0xf0004800,2,rw
csr_register,sdcard_mem2block_dma_length,0xf0004808,1,rw
csr_register,sdcard_mem2block_dma_enable,0xf000480c,1,rw
csr_register,sdcard_mem2block_dma_done,0xf0004810,1,ro
csr_register,sdcard_mem2block_dma_loop,0xf0004814,1,rw
csr_register,sdcard_mem2block_dma_offset,0xf0004818,1,ro
csr_register,sdcard_phy_card_detect,0xf0005000,1,ro
csr_register,sdcard_phy_clocker_divider,0xf0005004,1,rw
csr_register,sdcard_phy_init_initialize,0xf0005008,1,rw
csr_register,sdcard_phy_dataw_status,0xf000500c,1,ro
csr_register,sdram_dfii_control,0xf0005800,1,rw
csr_register,sdram_dfii_pi0_command,0xf0005804,1,rw
csr_register,sdram_dfii_pi0_command_issue,0xf0005808,1,rw
csr_register,sdram_dfii_pi0_address,0xf000580c,1,rw
csr_register,sdram_dfii_pi0_baddress,0xf0005810,1,rw
csr_register,sdram_dfii_pi0_wrdata,0xf0005814,1,rw
csr_register,sdram_dfii_pi0_rddata,0xf0005818,1,ro
constant,config_clock_frequency,48000000,,
constant,config_cpu_has_interrupt,None,,
constant,config_cpu_reset_addr,0,,
constant,config_cpu_count,1,,
constant,config_cpu_isa,rv32i2p0_ma,,
constant,config_cpu_mmu,sv32,,
constant,config_cpu_dcache_size,4096,,
constant,config_cpu_dcache_ways,1,,
constant,config_cpu_dcache_block_size,64,,
constant,config_cpu_icache_size,4096,,
constant,config_cpu_icache_ways,1,,
constant,config_cpu_icache_block_size,64,,
constant,config_cpu_dtlb_size,4,,
constant,config_cpu_dtlb_ways,4,,
constant,config_cpu_itlb_size,4,,
constant,config_cpu_itlb_ways,4,,
constant,config_cpu_type_vexriscv_smp,None,,
constant,config_cpu_variant_linux,None,,
constant,config_cpu_human_name,vexriscv smp-linux,,
constant,config_cpu_nop,nop,,
constant,config_bios_no_build_time,None,,
constant,config_l2_size,128,,
constant,config_csr_data_width,32,,
constant,config_csr_alignment,32,,
constant,config_bus_standard,wishbone,,
constant,config_bus_data_width,32,,
constant,config_bus_address_width,32,,
constant,config_bus_bursting,0,,
constant,sdcard_irq_interrupt,3,,
constant,timer0_interrupt,2,,
constant,uart_interrupt,1,,
memory_region,opensbi,0x40f00000,524288,cached+linker
memory_region,plic,0xf0c00000,4194304,io
memory_region,clint,0xf0010000,65536,io
memory_region,rom,0x00000000,65536,cached
memory_region,sram,0x10000000,6144,cached
memory_region,main_ram,0x40000000,8388608,cached
memory_region,csr,0xf0000000,65536,io
It looks (almost?) similar to the device tree provided by [TangNano-20K-example](https://github.com/sipeed/TangNano-20K-example/blob/main/linux/image/sipeed_tang_nano_20k.dts) But when I was trying to boot this an Image with bootloader I got "Liftoff!". As I understand the configuration that I've built is different than your and I need some corrections in a generator flags or maybe create own configuration with corrected addresses.

Could you please answer a few questions? See below.

  1. Do I need to modify a source code for litex-boards and/or litex?
  2. Do I need to use any additional generator flags rather than I provided above?
  3. Could you please provide an example one for litex-board or linux-on-litex-vexriscv that allow to boot linux kernel?

Thanks for the advice.

@Dolu1990
Copy link
Contributor

Dolu1990 commented Oct 6, 2023

peiceliu

I would say running it in a simulation would give full visibility on what the CPU does, that's the way in generaly always go.

Else there is the possibility to use the JTAG debug, and since recently you can even use the official RISC-V debug spec, which support linux kernel debugging aswell :)

@avkghost
Copy link

avkghost commented Oct 6, 2023

Update:

@nohahanon FYI

I've successfully run linux on a Tang Nano 20k with the kernel provided in a TangNano-20K-example repository.

I use the linux-on-litex-vexriscv with following parameters/flags:

./make.py --board sipeed_tang_nano_20k --with-rvc --uart-baudrate 3000000 --build
There is a boot log
        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS CRC passed (b7fd5a52)

 LiteX git sha1: c0d37665

--=============== SoC ==================--
CPU:		VexRiscv SMP-LINUX @ 48MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		64.0KiB
SRAM:		6.0KiB
L2:		128B
SDRAM:		8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2)
MAIN-RAM:	8.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 18.9MiB/s
   Read speed: 21.2MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading ./image/Image to 0x40000000 (1818868 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (128.2KB/s).
[LITEX-TERM] Uploading ./image/sipeed_tang_nano_20k.dtb to 0x40780000 (2115 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (92.3KB/s).
[LITEX-TERM] Uploading ./image/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (126.3KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imacs
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
[    0.000000] Linux version 6.4.0-rc1+ (zjs@CL-A-00022) (riscv64-linux-gnu-gcc (Debian 12.2.0-13) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #10 Tue May 16 09:20:49 CST 2023
[    0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'console'
[    0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[    0.000000] printk: bootconsole [liteuart0] enabled
[    0.000000] OF: reserved mem: 0x407c0000..0x407dffff (128 KiB) map non-reusable opensbi@407c0000
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] riscv: base ISA extensions aim
[    0.000000] riscv: ELF capabilities aim
[    0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2032
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 6112K/8192K available (1084K kernel code, 82K rwdata, 110K rodata, 495K init, 49K bss, 2080K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-1, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[    0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000083] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.008170] Console: colour dummy device 80x25
[    0.011115] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=480000)
[    0.014850] pid_max: default: 4096 minimum: 301
[    0.019099] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.021834] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.092709] ASID allocator using 9 bits (512 entries)
[    0.123244] devtmpfs: initialized
[    0.233157] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.235822] futex hash table entries: 16 (order: -5, 192 bytes, linear)
[    0.288091] platform soc: Fixed dependency cycle(s) with /soc/interrupt-controller@f0c00000
[    0.368089] clocksource: Switched to clocksource riscv_clocksource
[    1.366525] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    2.502945] LiteX SoC Controller driver initialized
[    3.099011] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
[    3.102848] printk: console [liteuart0] enabled
[    3.102848] printk: console [liteuart0] enabled
[    3.105098] printk: bootconsole [liteuart0] disabled
[    3.105098] printk: bootconsole [liteuart0] disabled
[    3.315434] clk: Disabling unused clocks
[    3.319630] Warning: unable to open an initial console.
[    3.361744] Freeing unused kernel image (initmem) memory: 492K
[    3.363448] Kernel memory protection not selected by kernel config.
[    3.365094] Run /init as init process
starting pid 18, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100'

/ # 

At the moment I'm going to experiment with a Linux kernel (reduce size and remove unneeded modules/subsystems).

@Icenowy Could Jisheng Zhang help us with a kernel configuration for the board?

@peiceliu
Copy link

peiceliu commented Oct 6, 2023

Hello @avkghost , I tried the suggestions you provided but I was not able to successfully boot Linux on my sipeed_tang_primer_20k . Here is my log.Could you please provide me with more details about the issue?

  1. ~/Desktop/linux-on-litex-vexriscv$ ./make.py --board sipeed_tang_primer_20k --with-rvc --uart-baudrate 3000000 --build --load
  2. ~/Desktop/linux-on-litex-vexriscv$ litex_term --speed 3000000 --serial-boot --images=images/boot.json /dev/ttyUSB1
    **NOTE:**I had replaced the files in the images folder with the boot.json and other files of tang_primeri_20k example.But my computer does not respond to this command /Desktop/linux-on-litex-vexriscv$ litex_term --speed 3000000 --serial-boot --images=images/boot.json /dev/ttyUSB1 and it stop here. How can i encounter this problem?Please provide any answer. Thank you!

@avkghost
Copy link

avkghost commented Oct 6, 2023

Hello @peiceliu,
Have you tried accessing a litex bootloader console? Is it possible to write any command in the console?
If yes, please use serialboot or reboot command.

Unfortunately, now I do not have the ability to test Tang Primer build. The power went out while I was flashing the board. It is currently not possible to write anything to FPGA memory or software SPI flash.

@peiceliu
Copy link

peiceliu commented Oct 6, 2023

Hello @avkghost ,After the second command,I can not boot litex console.In addition,after the first command,my serial tool displayed messy code instead of boot log .

@avkghost
Copy link

avkghost commented Oct 6, 2023

@peiceliu
I've successfully loaded Tang Primer 20k with Linux. So, I restored my board under the windows (using Gowin programmer). The second problem was with a bitstream. I was not able to program it with openFPGALoader. In some cases, the upload was freezing while flashing. When the bitstream uploaded successfully and FPGA was re-booted it did not start a Litex BIOS.

There is a step-by-step solution that I used for my own board, please take a look below.

  1. Create a software and a gateware by following command line. In my variant flashing and loading into SRAM didn't allow programming the bitstream via openFPGALoader.
./make.py --board sipeed_tang_primer_20k --with-rvc --uart-baudrate 3000000 --build 
  1. Transfer all needed gateware files to windows. (sipeed_tang_nano_20k.cst sipeed_tang_nano_20k_mem.init sipeed_tang_nano_20k.sdc sipeed_tang_nano_20k.v sipeed_tang_nano_20k.fs sipeed_tang_nano_20k_rom.init sipeed_tang_nano_20k_sram.init)
  2. Create a new project and add these files.
  3. Build the project.
  4. Flash it. Loading into SRAM is possible, but after turning off the power, SRAM will be cleared.
  5. Create a new directory and copy to it following files: Image, boot.json, root.cpio
  6. Copy sipeed_tang_primer_20k.dtb to the directory and rename it to rv32.dtb
  7. Run litex_term with the flags you provided.
litex_term --speed 3000000 --serial-boot --images=images/boot.json /dev/ttyUSB1
There is my boot log
litex> reboot


        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS CRC passed (82ac047a)

 LiteX git sha1: c0d37665

--=============== SoC ==================--
CPU:		VexRiscv SMP-LINUX @ 48MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		64.0KiB
SRAM:		6.0KiB
L2:		512B
SDRAM:		128.0MiB 16-bit @ 192MT/s (CL-6 CWL-5)
MAIN-RAM:	128.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Read leveling:
  m0, b00: |00000000| delays: -
  m0, b01: |00000000| delays: -
  m0, b02: |01110000| delays: 02+-01
  m0, b03: |00000000| delays: -
  best: m0, b02 delays: 02+-01
  m1, b00: |00000000| delays: -
  m1, b01: |00000000| delays: -
  m1, b02: |01110000| delays: 02+-01
  m1, b03: |00000000| delays: -
  best: m1, b02 delays: 02+-01
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 11.7MiB/s
   Read speed: 15.6MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading ./images/Image to 0x40000000 (7531468 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (134.9KB/s).
[LITEX-TERM] Uploading ./images/rv32.dtb to 0x40ef0000 (2617 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 160.00us, length: 64)
[LITEX-TERM] Upload complete (165.4KB/s).
[LITEX-TERM] Uploading ./images/rootfs.cpio to 0x41000000 (3781632 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (131.9KB/s).
[LITEX-TERM] Uploading ./images/opensbi.bin to 0x40f00000 (53640 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (121.4KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x40f00000

--============= Liftoff! ===============--

OpenSBI v0.8-1-gecf7701
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imacs
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x40f00000
Firmware Size       : 124 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
[    0.000000] Linux version 5.14.0 (florent@panda) (riscv32-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.08-381-g279167ee8d) 10.3.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP Tue Sep 21 12:57:31 CEST 2021
[    0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'console'
[    0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[    0.000000] printk: bootconsole [liteuart0] enabled
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000040000000-0x0000000047ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x0000000047ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x0000000047ffffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] SBI v0.2 HSM extension detected
[    0.000000] riscv: ISA extensions acimp
[    0.000000] riscv: ELF capabilities acim
[    0.000000] percpu: Embedded 8 pages/cpu s11340 r0 d21428 u32768
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 32512
[    0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0
[    0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
[    0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 113620K/131072K available (5685K kernel code, 572K rwdata, 883K rodata, 209K init, 221K bss, 17452K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[    0.000000] random: get_random_bytes called from start_kernel+0x4ac/0x63c with crng_init=0
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000124] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.015034] Console: colour dummy device 80x25
[    0.023803] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000)
[    0.035914] pid_max: default: 32768 minimum: 301
[    0.064347] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.073816] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.271613] ASID allocator using 9 bits (512 entries)
[    0.298566] rcu: Hierarchical SRCU implementation.
[    0.356230] smp: Bringing up secondary CPUs ...
[    0.361589] smp: Brought up 1 node, 1 CPU
[    0.413326] devtmpfs: initialized
[    0.675550] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.687041] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[    0.764644] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    2.286756] pps_core: LinuxPPS API ver. 1 registered
[    2.291895] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    2.303005] PTP clock support registered
[    2.337553] FPGA manager framework
[    2.465054] clocksource: Switched to clocksource riscv_clocksource
[    4.354761] NET: Registered PF_INET protocol family
[    4.376303] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    4.447113] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
[    4.462334] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    4.474843] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    4.483897] TCP: Hash tables configured (established 1024 bind 1024)
[    4.506406] UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
[    4.518596] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
[    4.802981] Unpacking initramfs...
[    5.098350] workingset: timestamp_bits=30 max_order=15 bucket_order=0
[    7.675652] io scheduler mq-deadline registered
[    7.682910] io scheduler kyber registered
[   11.310904] LiteX SoC Controller driver initialized
[   26.476123] Initramfs unpacking failed: invalid magic at start of compressed archive
[   28.075116] Freeing initrd memory: 8192K
[   32.591960] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
[   32.606940] printk: console [liteuart0] enabled
[   32.606940] printk: console [liteuart0] enabled
[   32.614654] printk: bootconsole [liteuart0] disabled
[   32.614654] printk: bootconsole [liteuart0] disabled
[   33.018394] i2c_dev: i2c /dev entries driver
[   33.175242] mmc_spi spi0.0: SD/MMC host mmc0, no WP, no poweroff, cd polling
[   33.584093] NET: Registered PF_INET6 protocol family
[   34.551630] Segment Routing with IPv6
[   34.562796] In-situ OAM (IOAM) with IPv6
[   34.578854] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[   34.727777] NET: Registered PF_PACKET protocol family
[   35.048255] Freeing unused kernel image (initmem) memory: 204K
[   35.058976] Kernel memory protection not selected by kernel config.
[   35.070574] Run /init as init process
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Saving random seed: [   89.918881] random: dd: uninitialized urandom read (512 bytes read)
OK
Starting network: OK

Welcome to Buildroot
buildroot login: root
                   __   _
                  / /  (_)__  __ ____ __
                 / /__/ / _ \/ // /\ \ /
                /____/_/_//_/\_,_//_\_\
                      / _ \/ _ \
   __   _ __      _  _\___/_//_/         ___  _
  / /  (_) /____ | |/_/__| | / /____ __ / _ \(_)__ _____  __
 / /__/ / __/ -_)>  </___/ |/ / -_) \ // , _/ (_-</ __/ |/ /
/____/_/\__/\__/_/|_|____|___/\__/_\_\/_/|_/_/___/\__/|___/
                  / __/  |/  / _ \
                 _\ \/ /|_/ / ___/
                /___/_/  /_/_/
  32-bit RISC-V Linux running on LiteX / VexRiscv-SMP.

login[71]: root login on 'console'
root@buildroot:~# pwd
/root
root@buildroot:~# uname -a
Linux buildroot 5.14.0 #1 SMP Tue Sep 21 12:57:31 CEST 2021 riscv32 GNU/Linux

Maybe it would help you.

@peiceliu
Copy link

peiceliu commented Oct 7, 2023

Thanks bro @avkghost,now i boot linux on my fpga board successfully.

@nohahanon
Copy link
Author

@avkghost
@peiceliu
Big thanks for the great news and sharing it.
If you've had success with the Primer 20k, I hope I can achieve the same results with the Nano 20k...
However, when it comes to writing the fs file to the board using Gowin and booting with self-built Image, I encountered the same issue as before – the process stops. What steps should I take within the gowin project, including adding mem.init and such?
The synthesis process stops due to errors.

  • serialboot: [LITEX-TERM] Got unexpected response from device 'b'E''
  • sdcardboot: `litex> sdcardboot

Booting from SDCard in SD-Mode...
Booting from boot.json...
Copying Image to 0x40000000 (7726264 bytes)...
[########################################]
Copying rv32.dtb to 0x40ef0000 (2689 bytes)...
[########################################]
Copying rootfs.cpio to 0x41000000 (5376512 bytes)...
[########################################]
Copying opensbi.bin to 0x40f00000 (263652 bytes)...
[########################################]
Executing booted program at 0x40f00000

--============= Liftoff! ===============--`

@avkghost
Copy link

avkghost commented Oct 7, 2023

@nohahanon
As I wrote before, you are still using an incorrect memory map in a bootloader. Maybe you using the wrong address for opensbi/fw_jump for building and/or loading.
The second possible problem I see is disabling verification during load into RAM.
Your log file leds that rv32.dtb at 0x40ef0000, rootfs.cpio at 0x41000000, and placed at an unreachable memory region.
On a Tang Nano 20k is possible to access the memory region from address 0x40000000 to 0x40800000 (default and your variant). All your code/data needs to be placed in this region. For example, the original kernel+root.cpio, opensbi.bin.tangnano20k, and device tree are placed below the memory limit (0x40800000).

All addresses below the memory limit are included in the original example. Please pay attention to the addresses provided below.

{
	"Image":			"0x40000000",
	"sipeed_tang_nano_20k.dtb":	"0x40780000",
	"opensbi.bin.tangnano20k":	"0x407c0000"
}

So, I also booted Tang Nano 20k from SDCard.

A boot log from SDCard
$ litex_term --speed 3000000 /dev/ttyUSB1

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS CRC passed (b7fd5a52)

 LiteX git sha1: c0d37665

--=============== SoC ==================--
CPU:		VexRiscv SMP-LINUX @ 48MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		64.0KiB
SRAM:		6.0KiB
L2:		128B
SDRAM:		8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2)
MAIN-RAM:	8.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 18.9MiB/s
   Read speed: 21.2MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
Booting from SDCard in SD-Mode...
Booting from boot.json...
Copying Image to 0x40000000 (1818868 bytes)...
[########################################]
Copying sipeed_tang_nano_20k.dtb to 0x40780000 (2115 bytes)...
[########################################]
Copying opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[########################################]
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imacs
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
[    0.000000] Linux version 6.4.0-rc1+ (zjs@CL-A-00022) (riscv64-linux-gnu-gcc (Debian 12.2.0-13) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #10 Tue May 16 09:20:49 CST 2023
[    0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'console'
[    0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[    0.000000] printk: bootconsole [liteuart0] enabled
[    0.000000] OF: reserved mem: 0x407c0000..0x407dffff (128 KiB) map non-reusable opensbi@407c0000
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] riscv: base ISA extensions aim
[    0.000000] riscv: ELF capabilities aim
[    0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2032
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 6112K/8192K available (1084K kernel code, 82K rwdata, 110K rodata, 495K init, 49K bss, 2080K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-1, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[    0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000083] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.008171] Console: colour dummy device 80x25
[    0.011117] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=480000)
[    0.014853] pid_max: default: 4096 minimum: 301
[    0.019100] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.021835] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.092711] ASID allocator using 9 bits (512 entries)
[    0.123245] devtmpfs: initialized
[    0.233158] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.235823] futex hash table entries: 16 (order: -5, 192 bytes, linear)
[    0.288092] platform soc: Fixed dependency cycle(s) with /soc/interrupt-controller@f0c00000
[    0.368091] clocksource: Switched to clocksource riscv_clocksource
[    1.366526] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    2.502946] LiteX SoC Controller driver initialized
[    3.099012] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
[    3.102855] printk: console [liteuart0] enabled
[    3.102855] printk: console [liteuart0] enabled
[    3.105105] printk: bootconsole [liteuart0] disabled
[    3.105105] printk: bootconsole [liteuart0] disabled
[    3.315436] clk: Disabling unused clocks
[    3.319631] Warning: unable to open an initial console.
[    3.361745] Freeing unused kernel image (initmem) memory: 492K
[    3.363449] Kernel memory protection not selected by kernel config.
[    3.365095] Run /init as init process
starting pid 18, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100'

/ # 

To reduce the memory footprint for the kernel try to use make tinyconfigand then add the needed options.
For buildroot I recommend using uclibc instead of glibc to reduce the memory footprint.

@avkghost
Copy link

avkghost commented Oct 7, 2023

@nohahanon Small update
Below is my result of the kernel compilation and execution. I removed SMP, networking, and all compressions instead of GZIP. Enabled "Emit compressed instructions when building Linux" in the kernel config.

  • Kernel: commit ae80e67c6b48bbedcd13db753237a25b3dec8301 (HEAD -> 6.1.0, origin/litex-rebase, origin/6.1.0) buildroot.
  • .config: linux-on-litex-vexriscv/buildroot/board/litex_vexriscv/linux.config
  • opensbi.bin.tangnano20k - from example provide by Tang Nano 20k example repository.
  • sipeed_tang_nano_20k.dtb - A device tree generated by ./make.py provided above.

I had not used any additional scripts or ran make from buildroot. I've created a test build aiming to minimize the kernel.
Please pay attention to the addresses I used in boot.json. Unfortunately, the kernel did not boot properly. I've only created a kernel, not rootfs cpio and integrated it into the Image.

{
	"Image":			"0x40000000",
	"sipeed_tang_nano_20k.dtb":	"0x40780000",
	"opensbi.bin.tangnano20k":	"0x407c0000"
}
Boot log
        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS CRC passed (b7fd5a52)

 LiteX git sha1: c0d37665

--=============== SoC ==================--
CPU:		VexRiscv SMP-LINUX @ 48MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		64.0KiB
SRAM:		6.0KiB
L2:		128B
SDRAM:		8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2)
MAIN-RAM:	8.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 18.9MiB/s
   Read speed: 21.2MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading ./Image to 0x40000000 (3547064 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (131.6KB/s).
[LITEX-TERM] Uploading ./sipeed_tang_nano_20k.dtb to 0x40780000 (2689 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (122.0KB/s).
[LITEX-TERM] Uploading ./opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (130.1KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imacs
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
[    0.000000] Linux version 6.1.0-rc2-296014-gae80e67c6b48 (andy@laptop) (riscv64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #1 Sat Oct  7 23:39:42 CEST 2023
[    0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'console'
[    0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[    0.000000] printk: bootconsole [liteuart0] enabled
[    0.000000] INITRD: 0x41000000+0x00800000 is not a memory region - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] riscv: base ISA extensions aim
[    0.000000] riscv: ELF capabilities aim
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2032
[    0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 4364K/8192K available (2361K kernel code, 510K rwdata, 445K rodata, 142K init, 241K bss, 3828K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[    0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000081] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.011053] Console: colour dummy device 80x25
[    0.014633] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000)
[    0.017877] pid_max: default: 4096 minimum: 301
[    0.035452] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.038862] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.128176] ASID allocator using 9 bits (512 entries)
[    0.290047] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns

@Dolu1990 Could I ask you for help with kernel debugging? Maybe I removed something important from the kernel configuration or did something wrong.
Thanks.

@nohahanon
Copy link
Author

Big thanks, bro,
I'll try.

@avkghost
Copy link

avkghost commented Oct 8, 2023

@nohahanon
Another small update
I successfully booted up my own custom-built environment.

  • kernel - as described above
  • sipeed_tang_nano_20k.dtb - I have made some customizations to the device tree.
  • root.cpio - extracted from tang nano example kernel image
  • opensbi.bin.tangnano20k

boot.json

{
	"Image":			"0x40000000",
	"root.cpio":			"0x406ef000",
	"sipeed_tang_nano_20k.dtb":	"0x40780000",
	"opensbi.bin.tangnano20k":	"0x407c0000"
}
device tree
/dts-v1/;

/ {
        #address-cells = <1>;
        #size-cells    = <1>;


        chosen {
            bootargs = "console=liteuart earlycon=liteuart,0xf0001000 root=/dev/ram0";
            linux,initrd-start = <0x406EF000>;
            linux,initrd-end   = <0x4077FFFF>;
        };

        sys_clk: pll {
            compatible = "fixed-clock";
            #clock-cells = <0>;
            clock-frequency  = <48000000>;
        };

        cpus {
            #address-cells = <1>;
            #size-cells    = <0>;
            timebase-frequency = <48000000>;

            CPU0: cpu@0 {
                device_type = "cpu";
                compatible = "riscv";
                riscv,isa = "rv32i2p0_ma";
                mmu-type = "riscv,sv32";
                reg = <0>;
                clock-frequency = <48000000>;
                status = "okay";
                
                d-cache-size = <4096>;
                d-cache-sets = <1>;
                d-cache-block-size = <64>;

                i-cache-size = <4096>;
                i-cache-sets = <1>;
                i-cache-block-size = <64>;

                
                d-tlb-size = <4>;
                d-tlb-sets = <4>;

                i-tlb-size = <4>;
                i-tlb-sets = <4>;

                L0: interrupt-controller {
                    #address-cells = <0>;
                    #interrupt-cells = <0x00000001>;
                    interrupt-controller;
                    compatible = "riscv,cpu-intc";
                };
            };

            
        };

        memory@40000000 {
            device_type = "memory";
            reg = <0x40000000 0x800000>;
        };

        reserved-memory {
            #address-cells = <1>;
            #size-cells    = <1>;
            ranges;

            cpio@406ef000 {
                reg = <0x406ef000 0x91000>;
            };
            opensbi@407c0000 {
                reg = <0x407c0000 0x80000>;
            };

        };

        vreg_mmc: vreg_mmc {
            compatible = "regulator-fixed";
            regulator-name = "vreg_mmc";
            regulator-min-microvolt = <3300000>;
            regulator-max-microvolt = <3300000>;
            regulator-always-on;
        };

        soc {
            #address-cells = <1>;
            #size-cells    = <1>;
            compatible = "simple-bus";
            interrupt-parent = <&intc0>;
            ranges;

            soc_ctrl0: soc_controller@f0000800 {
                compatible = "litex,soc-controller";
                reg = <0xf0000800 0xc>;
                status = "okay";
            };

            intc0: interrupt-controller@f0c00000 {
                compatible = "sifive,fu540-c000-plic", "sifive,plic-1.0.0";
                reg = <0xf0c00000 0x400000>;
                #address-cells = <0>;
                #interrupt-cells = <1>;
                interrupt-controller;
                interrupts-extended = <
                    &L0 11 &L0 9>;
                riscv,ndev = <32>;
            };

            liteuart0: serial@f0001000 {
                compatible = "litex,liteuart";
                reg = <0xf0001000 0x100>;
                interrupts = <1>;
                status = "okay";
            };

            mmc0: mmc@f0005000 {
                compatible = "litex,mmc";
                reg = <0xf0005000 0x100>,
                      <0xf0003800 0x100>,
                      <0xf0003000 0x100>,
                      <0xf0004800 0x100>,
                      <0xf0004000 0x100>;
                reg-names = "phy", "core", "reader", "writer", "irq";
                clocks = <&sys_clk>;
                vmmc-supply = <&vreg_mmc>;
                bus-width = <0x04>;
                interrupts = <3>;
                status = "okay";
            };

            leds: gpio@f0002800 {
                compatible = "litex,gpio";
                reg = <0xf0002800 0x4>;
                gpio-controller;
                #gpio-cells = <2>;
                litex,direction = "out";
                status = "disabled";
            };

        };

        aliases {

                serial0 = &liteuart0;

        };

};

&leds {
        litex,ngpio = <4>;
        status = "okay";
};
boot log
litex_term --speed 3000000 --serial-boot --image ./boot.json /dev/ttyUSB1

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS CRC passed (b7fd5a52)

 LiteX git sha1: c0d37665

--=============== SoC ==================--
CPU:		VexRiscv SMP-LINUX @ 48MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		64.0KiB
SRAM:		6.0KiB
L2:		128B
SDRAM:		8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2)
MAIN-RAM:	8.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 18.9MiB/s
   Read speed: 21.2MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading ./Image to 0x40000000 (2993104 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (132.2KB/s).
[LITEX-TERM] Uploading ./root.cpio to 0x406ef000 (592224 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (136.6KB/s).
[LITEX-TERM] Uploading ./sipeed_tang_nano_20k.dtb to 0x40780000 (2725 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (115.8KB/s).
[LITEX-TERM] Uploading ./opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64)
[LITEX-TERM] Upload complete (137.4KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imacs
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
[    0.000000] Linux version 6.1.0-rc2-296014-gae80e67c6b48 (andy@laptop) (riscv64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #17 SMP Sun Oct  8 02:25:59 CEST 2023
[    0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'console'
[    0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[    0.000000] printk: bootconsole [liteuart0] enabled
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] SBI HSM extension detected
[    0.000000] riscv: base ISA extensions aim
[    0.000000] riscv: ELF capabilities aim
[    0.000000] percpu: Embedded 8 pages/cpu s10896 r0 d21872 u32768
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2032
[    0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 root=/dev/ram0  
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 4076K/8192K available (1876K kernel code, 516K rwdata, 379K rodata, 146K init, 232K bss, 4116K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=32 to nr_cpu_ids=1.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000100] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.014662] Console: colour dummy device 80x25
[    0.019356] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000)
[    0.022018] pid_max: default: 32768 minimum: 301
[    0.041167] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.045037] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.235037] ASID allocator using 9 bits (512 entries)
[    0.257621] rcu: Hierarchical SRCU implementation.
[    0.260791] rcu: 	Max phase no-delay instances is 1000.
[    0.317495] smp: Bringing up secondary CPUs ...
[    0.318840] smp: Brought up 1 node, 1 CPU
[    0.361797] devtmpfs: initialized
[    0.555579] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.559983] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[    0.695002] cpuidle: using governor ladder
[    1.883589] FPGA manager framework
[    1.900723] clocksource: Switched to clocksource riscv_clocksource
[    3.332691] Unpacking initramfs...
[    3.364459] workingset: timestamp_bits=30 max_order=10 bucket_order=0
[    6.694820] Initramfs unpacking failed: broken padding
[    6.764988] Freeing initrd memory: 576K
[    6.944560] LiteX SoC Controller driver initialized
[   14.876584] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
[   14.883637] printk: console [liteuart0] enabled
[   14.883637] printk: console [liteuart0] enabled
[   14.887804] printk: bootconsole [liteuart0] disabled
[   14.887804] printk: bootconsole [liteuart0] disabled
[   15.039121] i2c_dev: i2c /dev entries driver
[   15.059933] cpuidle-riscv-sbi: HSM suspend not available
[   15.188386] litex-mmc f0005000.mmc: LiteX MMC controller initialized.
[   15.351267] Freeing unused kernel image (initmem) memory: 140K
[   15.354801] Kernel memory protection not selected by kernel config.
[   15.356935] Run /init as init process
[   15.533254] mmc0: new SDXC card at address aaaa
starting pid 24, tty '': '/etc/init.d/rcS'
[   16.547433] tmpfs: Unknown parameter 'mode'
mount: mounting tmpfs on /dev/shm failed: Invalid argument
[   16.596831] tmpfs: Unknown parameter 'mode'
mount: mounting tmpfs on /tmp failed: Invalid argument
starting pid 28, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100'

/ # free
              total        used        free      shared  buff/cache   available
Mem:           4792        1776        2032           0         984        1868
Swap:             0           0           0

@nohahanon
Copy link
Author

@avkghost

Thank you for your reply. I edited linux.config like this and I successfully reduced the size to less than 8MB! In the comment two comments back, you were saying I removed SMP, networking, and all compressions instead of GZIP. Enabled "Emit compressed instructions when building Linux" in the kernel config., but did you mean that you edited the linux.config file like below?

  • CONFIG_RISCV_ISA_C=n -> y
  • CONFIG_SMP=y -> n
  • CONFIG_RD_BZIP2=y -> n
  • CONFIG_RD_LZMA=y -> n
  • CONFIG_RD_XZ=y -> n
  • CONFIG_RD_LZO=y -> n
  • CONFIG_RD_LZ4=y -> n
  • #Ethernet config -> n

In conclusion, I failed to boot linux using the following file. It stopped in the middle of the opensbi process.
By the way, you said root.cpio - extracted from tang nano example kernel image, could you please explain how to do that?

Regards.

linux.config
CONFIG_SECTION_MISMATCH_WARN_ONLY=y

# Architecture
CONFIG_ARCH_DEFCONFIG="arch/riscv/configs/defconfig"
CONFIG_NONPORTABLE=y
CONFIG_ARCH_RV32I=y
CONFIG_RISCV_ISA_M=y
CONFIG_RISCV_ISA_A=y
CONFIG_RISCV_ISA_C=y
CONFIG_SIFIVE_PLIC=y
CONFIG_FPU=n
CONFIG_SMP=n
CONFIG_STRICT_KERNEL_RWX=n
CONFIG_EFI=n
CONFIG_HVC_RISCV_SBI=y
CONFIG_NONPORTABLE=y

CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=n
CONFIG_RD_LZMA=n
CONFIG_RD_XZ=n
CONFIG_RD_LZO=n
CONFIG_RD_LZ4=n

# FPGA / SoC
CONFIG_FPGA=y
CONFIG_FPGA_MGR_LITEX=y
CONFIG_LITEX_SOC_CONTROLLER=y
CONFIG_LITEX_SUBREG_SIZE=4

# Time
CONFIG_PRINTK_TIME=y

# Clocking
CONFIG_COMMON_CLK=y
CONFIG_COMMON_CLK_LITEX=y

# Interrupts
CONFIG_IRQCHIP=y
CONFIG_OF_IRQ=y
CONFIG_HANDLE_DOMAIN_IRQ=y
CONFIG_LITEX_VEXRISCV_INTC=y

# Ethernet
CONFIG_NET=n
CONFIG_PACKET=n
CONFIG_PACKET_DIAG=n
CONFIG_INET=n
CONFIG_NETDEVICES=n
CONFIG_NET_VENDOR_LITEX=n
CONFIG_LITEX_LITEETH=n

# Serial
CONFIG_SERIAL_EARLYCON_RISCV_SBI=y
CONFIG_SERIAL_LITEUART=y
CONFIG_SERIAL_LITEUART_CONSOLE=y

# GPIO
CONFIG_GPIO_SYSFS=y
CONFIG_GPIOLIB=y
CONFIG_GPIO_LITEX=y

# PWM
CONFIG_PWM=y
CONFIG_PWM_LITEX=y

# SPI
CONFIG_SPI=y
CONFIG_SPI_LITESPI=y
CONFIG_SPI_SPIDEV=y

# I2C
CONFIG_I2C=y
CONFIG_I2C_LITEX=y
CONFIG_I2C_CHARDEV=y

# Hardware monitoring
CONFIG_HWMON=y
CONFIG_SENSORS_LITEX_HWMON=y

# Framebuffer
CONFIG_FB=y
CONFIG_FB_SIMPLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
CONFIG_LOGO=y
CONFIG_DRM=y
CONFIG_DRM_LITEVIDEO=y

# Flash
CONFIG_MTD=y
CONFIG_MTD_SPI_NOR=y
CONFIG_SPI_FLASH_LITEX=y

# MMC
CONFIG_MMC=y
CONFIG_MMC_SPI=y
CONFIG_MMC_LITEX=y

CONFIG_EXT2_FS=y
CONFIG_EXT3_FS=y
CONFIG_EXT4_FS=y

# .config in kernel
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y

# Filesystem
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_MSDOS_PARTITION=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NCPFS_SMALLDOS=y
CONFIG_NLS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
CONFIG_TMPFS=y

size of each file

  • Image: 4125780(0x3EF454), same as you
  • opensbi.bin.tangnano20k: 49636(0xC1E4), example's one
  • rootfs.cpio: 3712512(0x38A600), created simultaneously with the image
  • sipeed_tang_nano_20k.dtb: 2725(AA5), compiled version of the following .dts
  • total: 7890653 < 8MB!!!

boot.json

{
	"Image":			"0x40000000",
	"rootfs.cpio":			"0x40400000",
	"sipeed_tang_nano_20k.dtb":	"0x40780000",
	"opensbi.bin.tangnano20k":	"0x407c0000"
}
devise tree
/dts-v1/;

/ {
        #address-cells = <1>;
        #size-cells    = <1>;


        chosen {
            bootargs = "console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0";
            linux,initrd-start = <0x41000000>;
            linux,initrd-end   = <0x41800000>;
        };

        sys_clk: pll {
            compatible = "fixed-clock";
            #clock-cells = <0>;
            clock-frequency  = <48000000>;
        };

        cpus {
            #address-cells = <1>;
            #size-cells    = <0>;
            timebase-frequency = <48000000>;

            CPU0: cpu@0 {
                device_type = "cpu";
                compatible = "riscv";
                riscv,isa = "rv32i2p0_ma";
                mmu-type = "riscv,sv32";
                reg = <0>;
                clock-frequency = <48000000>;
                status = "okay";
                
                d-cache-size = <4096>;
                d-cache-sets = <1>;
                d-cache-block-size = <64>;

                i-cache-size = <4096>;
                i-cache-sets = <1>;
                i-cache-block-size = <64>;

                
                d-tlb-size = <4>;
                d-tlb-sets = <4>;

                i-tlb-size = <4>;
                i-tlb-sets = <4>;

                L0: interrupt-controller {
                    #address-cells = <0>;
                    #interrupt-cells = <0x00000001>;
                    interrupt-controller;
                    compatible = "riscv,cpu-intc";
                };
            };

            
        };

        memory@40000000 {
            device_type = "memory";
            reg = <0x40000000 0x800000>;
        };

        reserved-memory {
            #address-cells = <1>;
            #size-cells    = <1>;
            ranges;

            cpio@40400000 {
                reg = <0x40400000 0x91000>;
            };
            opensbi@407c0000 {
                reg = <0x407c0000 0x80000>;
            };

        };

        vreg_mmc: vreg_mmc {
            compatible = "regulator-fixed";
            regulator-name = "vreg_mmc";
            regulator-min-microvolt = <3300000>;
            regulator-max-microvolt = <3300000>;
            regulator-always-on;
        };

        soc {
            #address-cells = <1>;
            #size-cells    = <1>;
            compatible = "simple-bus";
            interrupt-parent = <&intc0>;
            ranges;

            soc_ctrl0: soc_controller@f0000800 {
                compatible = "litex,soc-controller";
                reg = <0xf0000800 0xc>;
                status = "okay";
            };

            intc0: interrupt-controller@f0c00000 {
                compatible = "sifive,fu540-c000-plic", "sifive,plic-1.0.0";
                reg = <0xf0c00000 0x400000>;
                #address-cells = <0>;
                #interrupt-cells = <1>;
                interrupt-controller;
                interrupts-extended = <
                    &L0 11 &L0 9>;
                riscv,ndev = <32>;
            };

            liteuart0: serial@f0001000 {
                compatible = "litex,liteuart";
                reg = <0xf0001000 0x100>;
                interrupts = <1>;
                status = "okay";
            };

            mmc0: mmc@f0005000 {
                compatible = "litex,mmc";
                reg = <0xf0005000 0x100>,
                      <0xf0003800 0x100>,
                      <0xf0003000 0x100>,
                      <0xf0004800 0x100>,
                      <0xf0004000 0x100>;
                reg-names = "phy", "core", "reader", "writer", "irq";
                clocks = <&sys_clk>;
                vmmc-supply = <&vreg_mmc>;
                bus-width = <0x04>;
                interrupts = <3>;
                status = "okay";
            };

            leds: gpio@f0002800 {
                compatible = "litex,gpio";
                reg = <0xf0002800 0x4>;
                gpio-controller;
                #gpio-cells = <2>;
                litex,direction = "out";
                status = "disabled";
            };

        };

        aliases {

                serial0 = &liteuart0;

        };

};

&leds {
        litex,ngpio = <4>;
        status = "okay";
};
boot log
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/Image to 0x40000000 (4125780 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 20.00us, length: 64)
[LITEX-TERM] Upload complete (260.0KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/rootfs.cpio to 0x40400000 (3712512 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (255.3KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/sipeed_tang_nano_20k.dtb to 0x40780000 (2733 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (143.8KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (189.8KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imas
BOOT HART Features  : time
BOOT HART PMP Count : 0
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b101

(halt...)

@avkghost
Copy link

avkghost commented Oct 8, 2023

@nohahanon

linux.config

An "Emit compressed instructions when building Linux" option is CONFIG_RISCV_ISA_C=y

CONFIG_SMP=n

Yes, it disables SMP.

Disable networking
CONFIG_NET=n or remove it.

Image: 4125780(0x3EF454), same as you

So close, but mine is 2993104 bytes.

root.cpio

In conclusion, I failed to boot linux using the following file. It stopped in the middle of the opensbi process.
By the way, you said root.cpio - extracted from tang nano example kernel image, could you please explain how to do that?

I used binwalk to extrat root.cpio from the example's kernel. binwalk -e Image. An extracted cpio was placed in _Image.extracted directory.

offtopic

I had not used buildroot for some reasons:

@nohahanon
Copy link
Author

nohahanon commented Oct 9, 2023

@avkghost
I hadn't know binwalk! Thank you very much.

Currently, I believe the only difference between your setup and mine is the Image. Could you please share how you created that Image?

By the way, I encountered the same error with the provided Buildroot recipe, but when I extracted the tar.gz available on buildroot.org and did the make process, I was able to obtain the Image without any errors. So, I am using the Image created using this method.

I'm sorry for taking up your time. Thank you.

report
Even with opensbi processing, it progressed quite far, but it still didn't reach the boot and stopped.

  • When I replaced the rootfs.cpio extracted using binwalk from the example Image and executed it, opensbi processing stopped in the middle.
boot.json
{
	"Image":			"0x40000000",
	"rootfs.cpio":			"0x4050FFFF",
	"sipeed_tang_nano_20k.dtb":	"0x40780000",
	"opensbi.bin.tangnano20k":	"0x407c0000"
}
.dts
/dts-v1/;

/ {
        #address-cells = <1>;
        #size-cells    = <1>;


        chosen {
            bootargs = "console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0";
            linux,initrd-start = <0x41000000>;
            linux,initrd-end   = <0x41800000>;
        };

        sys_clk: pll {
            compatible = "fixed-clock";
            #clock-cells = <0>;
            clock-frequency  = <48000000>;
        };

        cpus {
            #address-cells = <1>;
            #size-cells    = <0>;
            timebase-frequency = <48000000>;

            CPU0: cpu@0 {
                device_type = "cpu";
                compatible = "riscv";
                riscv,isa = "rv32i2p0_ma";
                mmu-type = "riscv,sv32";
                reg = <0>;
                clock-frequency = <48000000>;
                status = "okay";
                
                d-cache-size = <4096>;
                d-cache-sets = <1>;
                d-cache-block-size = <64>;

                i-cache-size = <4096>;
                i-cache-sets = <1>;
                i-cache-block-size = <64>;

                
                d-tlb-size = <4>;
                d-tlb-sets = <4>;

                i-tlb-size = <4>;
                i-tlb-sets = <4>;

                L0: interrupt-controller {
                    #address-cells = <0>;
                    #interrupt-cells = <0x00000001>;
                    interrupt-controller;
                    compatible = "riscv,cpu-intc";
                };
            };

            
        };

        memory@40000000 {
            device_type = "memory";
            reg = <0x40000000 0x800000>;
        };

        reserved-memory {
            #address-cells = <1>;
            #size-cells    = <1>;
            ranges;

            cpio@4050FFFF {
                reg = <0x4050FFFF 0x91000>;
            };
            opensbi@407c0000 {
                reg = <0x407c0000 0x20000>;
            };

        };

        vreg_mmc: vreg_mmc {
            compatible = "regulator-fixed";
            regulator-name = "vreg_mmc";
            regulator-min-microvolt = <3300000>;
            regulator-max-microvolt = <3300000>;
            regulator-always-on;
        };

        soc {
            #address-cells = <1>;
            #size-cells    = <1>;
            compatible = "simple-bus";
            interrupt-parent = <&intc0>;
            ranges;

            soc_ctrl0: soc_controller@f0000800 {
                compatible = "litex,soc-controller";
                reg = <0xf0000800 0xc>;
                status = "okay";
            };

            intc0: interrupt-controller@f0c00000 {
                compatible = "sifive,fu540-c000-plic", "sifive,plic-1.0.0";
                reg = <0xf0c00000 0x400000>;
                #address-cells = <0>;
                #interrupt-cells = <1>;
                interrupt-controller;
                interrupts-extended = <
                    &L0 11 &L0 9>;
                riscv,ndev = <32>;
            };

            liteuart0: serial@f0001000 {
                compatible = "litex,liteuart";
                reg = <0xf0001000 0x100>;
                interrupts = <1>;
                status = "okay";
            };

            mmc0: mmc@f0005000 {
                compatible = "litex,mmc";
                reg = <0xf0005000 0x100>,
                      <0xf0003800 0x100>,
                      <0xf0003000 0x100>,
                      <0xf0004800 0x100>,
                      <0xf0004000 0x100>;
                reg-names = "phy", "core", "reader", "writer", "irq";
                clocks = <&sys_clk>;
                vmmc-supply = <&vreg_mmc>;
                bus-width = <0x04>;
                interrupts = <3>;
                status = "okay";
            };

            leds: gpio@f0002800 {
                compatible = "litex,gpio";
                reg = <0xf0002800 0x4>;
                gpio-controller;
                #gpio-cells = <2>;
                litex,direction = "out";
                status = "disabled";
            };

        };

        aliases {

                serial0 = &liteuart0;

        };

};

&leds {
        litex,ngpio = <4>;
        status = "okay";
};
boot log
litex> serialboot

Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/Image to 0x40000000 (5305428 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (256.4KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/rootfs.cpio to 0x4050ffff (592224 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (254.6KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/sipeed_tang_nano_20k.dtb to 0x40780000 (2733 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (110.4KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (208.8KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imas
BOOT HART Features  : time
BOOT HART PMP Count : 0
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b101
  • When I changed CONFIG_RISCV_ISA_C=n and executed it, opensbi processing stopped at a position further than before! Errors like below are occured...
[    8.899980] litex-mmc f0005000.mmc: Command (cmd 8) error, status -110
[    8.924722] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    8.952200] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    8.976205] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    9.000210] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
  • boot.json: same as above

  • .dts: same as above

boot log
litex> serialboot

Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LITEX-TERM] Received firmware download request from the device.
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/Image to 0x40000000 (5305428 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (256.4KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/rootfs.cpio to 0x4050ffff (592224 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (254.6KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/sipeed_tang_nano_20k.dtb to 0x40780000 (2733 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (110.4KB/s).
[LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64)
[LITEX-TERM] Upload complete (208.8KB/s).
[LITEX-TERM] Booting the device.
[LITEX-TERM] Done.
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imas
BOOT HART Features  : time
BOOT HART PMP Count : 0
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b101
[    0.000000] Linux version 6.1.0-rc2 (root@nohara-MacBookPro) (riscv32-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.02.4) 11.4.0, GNU ld (GNU Binutils) 2.38) #5 Mon Oct  9 10:00:31 JST 2023
[    0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'console'
[    0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[    0.000000] printk: bootconsole [liteuart0] enabled
[    0.000000] INITRD: 0x41000000+0x00800000 is not a memory region - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] riscv: base ISA extensions aim
[    0.000000] riscv: ELF capabilities aim
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2032
[    0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 2180K/8192K available (3796K kernel code, 526K rwdata, 694K rodata, 159K init, 235K bss, 6012K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[    0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000099] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.009903] Console: colour dummy device 80x25
[    0.013447] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000)
[    0.015789] pid_max: default: 32768 minimum: 301
[    0.032100] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.034394] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.134036] ASID allocator using 9 bits (512 entries)
[    0.172003] devtmpfs: initialized
[    0.302573] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.305876] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.894239] FPGA manager framework
[    0.902612] clocksource: Switched to clocksource riscv_clocksource
[    2.013982] workingset: timestamp_bits=30 max_order=10 bucket_order=0
[    3.127995] io scheduler mq-deadline registered
[    3.129392] io scheduler kyber registered
[    3.237980] LiteX SoC Controller driver initialized
[    8.555817] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
[    8.560177] printk: console [liteuart0] enabled
[    8.560177] printk: console [liteuart0] enabled
[    8.562360] printk: bootconsole [liteuart0] disabled
[    8.562360] printk: bootconsole [liteuart0] disabled
[    8.664773] i2c_dev: i2c /dev entries driver
[    8.772213] litex-mmc f0005000.mmc: LiteX MMC controller initialized.
[    8.891705] Waiting for root device /dev/ram0...
[    8.899980] litex-mmc f0005000.mmc: Command (cmd 8) error, status -110
[    8.924722] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    8.952200] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    8.976205] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    9.000210] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110

@avkghost
Copy link

avkghost commented Oct 9, 2023

@nohahanon
I've successfully booted from SDCard and added SDCard support to the kernel.
As a root.cpio I continue using the old one from the example above. However, I've successfully built root.cpio with a buildroot, but not used it due to a big footprint and out-of-memory while booting.

A linux.config

Boot log
        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2023 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS CRC passed (b7fd5a52)

 LiteX git sha1: c0d37665

--=============== SoC ==================--
CPU:		VexRiscv SMP-LINUX @ 48MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		64.0KiB
SRAM:		6.0KiB
L2:		128B
SDRAM:		8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2)
MAIN-RAM:	8.0MiB

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 18.9MiB/s
   Read speed: 21.2MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
Booting from SDCard in SD-Mode...
Booting from boot.json...
Copying Image to 0x40000000 (3337720 bytes)...
[########################################]
Copying root-new.cpio.gz to 0x406ef000 (216670 bytes)...
[########################################]
Copying sipeed_tang_nano_20k.dtb to 0x40780000 (2737 bytes)...
[########################################]
Copying opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)...
[########################################]
Executing booted program at 0x407c0000

--============= Liftoff! ===============--

OpenSBI v0.8-2-ga9ce3ad
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : LiteX / VexRiscv-SMP
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv32imacs
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x407c0000
Firmware Size       : 120 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
[    0.000000] Linux version 6.1.0-rc2-296014-gae80e67c6b48 (andy@laptop) (riscv64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #34 PREEMPT Sun Oct  8 18:52:39 CEST 2023
[    0.000000] earlycon: liteuart0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'console'
[    0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '')
[    0.000000] printk: bootconsole [liteuart0] enabled
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] riscv: base ISA extensions aim
[    0.000000] riscv: ELF capabilities aim
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2032
[    0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/mmcblk0p2
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 3744K/8192K available (2057K kernel code, 518K rwdata, 569K rodata, 110K init, 228K bss, 4448K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts.
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000099] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.013174] Console: colour dummy device 80x25
[    0.016010] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000)
[    0.019874] pid_max: default: 32768 minimum: 301
[    0.038670] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.042434] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.180385] ASID allocator using 9 bits (512 entries)
[    0.194591] rcu: Hierarchical SRCU implementation.
[    0.195837] rcu: 	Max phase no-delay instances is 1000.
[    0.238191] devtmpfs: initialized
[    0.417723] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.420324] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    2.067058] FPGA manager framework
[    2.079990] clocksource: Switched to clocksource riscv_clocksource
[    3.505609] Unpacking initramfs...
[    3.565922] workingset: timestamp_bits=30 max_order=10 bucket_order=0
[    3.794059] Initramfs unpacking failed: no cpio magic
[    3.919034] Freeing initrd memory: 576K
[    5.226289] io scheduler mq-deadline registered
[    5.227708] io scheduler kyber registered
[    5.357527] LiteX SoC Controller driver initialized
[    6.046900] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
[    6.053728] printk: console [liteuart0] enabled
[    6.053728] printk: console [liteuart0] enabled
[    6.057051] printk: bootconsole [liteuart0] disabled
[    6.057051] printk: bootconsole [liteuart0] disabled
[    6.271572] null_blk: disk nullb0 created
[    6.274070] null_blk: module loaded
[    6.310911] i2c_dev: i2c /dev entries driver
[    6.437830] litex-mmc f0005000.mmc: LiteX MMC controller initialized.
[    6.529881] mmc0: new SDXC card at address aaaa
[    6.542703] Waiting for root device /dev/mmcblk0p2...
[    6.605616] mmcblk0: mmc0:aaaa SD64G 59.5 GiB 
[    6.757951]  mmcblk0: p1 p2
[    8.713423] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Quota mode: disabled.
[    8.718849] VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
[    8.755073] devtmpfs: mounted
[    8.770224] Freeing unused kernel image (initmem) memory: 108K
[    8.773296] Kernel memory protection not selected by kernel config.
[    8.775167] Run /sbin/init as init process
starting pid 27, tty '': '/etc/init.d/rcS'
mount: mounting devtmpfs on /dev failed: Resource busy
starting pid 31, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100'

/ # uname -a
Linux (none) 6.1.0-rc2-296014-gae80e67c6b48 #34 PREEMPT Sun Oct  8 18:52:39 CEST 2023 riscv32 GNU/Linux

/ # df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                57.8G    476.0K     54.8G   0% /
devtmpfs                  1.8M         0      1.8M   0% /dev
tmpfs                     2.2M         0      2.2M   0% /dev/shm
tmpfs                     2.2M         0      2.2M   0% /tmp

/ # 

@nohahanon
Copy link
Author

@avkghost
Is that true! I'm glad to hear that report. My goal is to bring files I've created locally onto that Linux running on tang nano 20k so I can execute a C program. However, I couldn't mount it on the SD card in the Linux image provided in the tang nano 20k example (it seems to be a very minimal Linux). So, I want to create my own, more feature-rich image and use it to boot Linux.

You mentioned that you haven't created an image with Buildroot, but how did you create it? I'd like to create an image similar to what you're using. Looking forward to your reply.

Thanks.

@avkghost
Copy link

avkghost commented Oct 9, 2023

@nohahanon
Initially, I had a problem creating images with build root. For now, I've successfully built it but I need to tune the size due to an out-of-memory error while Linux loading.

I prefer to use separate kernel configurations to minimize its footprint. I cloned the litex-linux repo and checked out 6.1.0 branch, and used the config provided above to build the kernel image.

To cross-compile build the kernel I used the following command.
make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu-

Below, is a device tree I've used
dt_nano_20k.zip

@avkghost
Copy link

avkghost commented Oct 9, 2023

@nohahanon
Below is an archive with my experiments.
nano_examples.zip
It is possible to boot them via serial or SD card.

The SD card needs to be partitioned into 2 or 3 partitions.

  • partition 1 (mmcblk0p1) - 200MB, FAT (For experiments I prefer to use 500MB). It is possible to use a small size.
  • partition 2 (mmcblk0p2) - as many as you need, /, Ext4 (I've added only ext4 support into the kernel).
  • partition 3 (mmcblk0p3) - as many as you need, swap. I do not use it.

A command line from the device tree sets mmcblk0p2 as the root partition.
The root partition I extracted from root.cpio.

I couldn't mount it on the SD card in the Linux image provided in the tang nano 20k example (it seems to be a very minimal Linux).

The kernel provided in the Tang Nano example did not have drivers for MMC, block devices, partition schemes, and file systems. It is impossible to mount any filesystem under the example's kernel. It is built as an experiment to show the possibility of running Linux on the board.

Now, I have no idea where I can use this configuration. In other words, I don't see any practical use for this configuration in my projects. In my opinion, there is the best way to use this board with bare metal OS, like FreeRTOS, etc, because in this case, Linux is an overhead for the board.
For me, it is interesting in the case of porting/running Linux to embedding systems with an architecture different than x86.

If you need to build and run your own program on a device I recommend you start with litex_bare_metal_demo
Build your own demo and run it on the board. For example:

  1. create a bitstream and bios if needed. When the script builds the BIOS it also creates a tiny library for I/O that possible to use in self-made projects.
  2. run litex_bare_metal_demo --with-cxx --build-path build/sipeed_tang_nano_20k in a directory contains build result. It creates a demo that you can run on a device
  3. run litex_term --kernel demo/demo.bin and see how it works.

Hope you find it helpful.

@nohahanon
Copy link
Author

Big thanks, bro,

I was able to successfully boot Linux using the provided image, but it seems that I still couldn't mount the SD card. While my initial goal was to run a program on Linux, I'm glad that I was able to run programs like Donut and Helloc on VexRiscv with litex_sim.
Although I didn't achieve the original goal of running a C program on Linux booted with the locally built Linux image, I will continue this on an Arty board where there is no need to reduce the size of the image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants