Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache disabled when copying from SPIFFS to SPIRAM (IDFGH-1798) #4010

Closed
HanVertegaal opened this issue Sep 4, 2019 · 7 comments
Closed

Cache disabled when copying from SPIFFS to SPIRAM (IDFGH-1798) #4010

HanVertegaal opened this issue Sep 4, 2019 · 7 comments

Comments

@HanVertegaal
Copy link

#3401 Environment

  • Module or chip used: Analog Lamb ALB32
  • IDF version: v3.3-rc-dirty (modified portmalloc.h)
  • Build System: Make
  • Compiler version: 1.22.0-98-g4638c4f
  • Operating System: Windows/WSL
  • Power Supply: Battery

Problem Description

When I copy data from SPIFFS to SPIRAM, the software panics.

Expected Behavior

No panic

Actual Behavior

`
Rebooting...
ets Jun 8 2016 00:22:57

rst:0x3 (SW_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:4248
load:0x40078000,len:10996
load:0x40080400,len:6112
entry 0x400806e0
I (208) psram: This chip is ESP32-D0WD
I (209) spiram: Found 32MBit SPI RAM device
I (209) spiram: SPI RAM mode: flash 80m sram 40m
I (211) spiram: PSRAM initialized, cache is in low/high (2-core) mode.
I (219) cpu_start: Pro cpu up.
I (222) cpu_start: Application information:
I (227) cpu_start: Project name: aether
I (232) cpu_start: App version: 1
I (236) cpu_start: Compile time: Sep 4 2019 10:31:10
I (243) cpu_start: ELF file SHA256: 953a24fc02961b39...
I (248) cpu_start: ESP-IDF: v3.3-rc-dirty
I (254) cpu_start: Starting app cpu, entry point is 0x400813e0
0x400813e0: call_start_cpu1 at /home/han/esp/esp-idf-v3.3-rc/components/esp32/cpu_start.c:267

I (0) cpu_start: App cpu up.
D (264) memory_layout: Checking 8 reserved memory ranges:
D (269) memory_layout: Reserved memory range 0x3f800000 - 0x3fc00000
D (276) memory_layout: Reserved memory range 0x3ffae000 - 0x3ffae6e0
D (282) memory_layout: Reserved memory range 0x3ffb0000 - 0x3ffb4690
D (289) memory_layout: Reserved memory range 0x3ffe0000 - 0x3ffe0440
D (295) memory_layout: Reserved memory range 0x3ffe3f20 - 0x3ffe4350
D (302) memory_layout: Reserved memory range 0x40070000 - 0x40078000
D (308) memory_layout: Reserved memory range 0x40078000 - 0x40080000
0x40080000: _WindowOverflow4 at /home/han/esp/esp-idf-v3.3-rc/components/freertos/xtensa_vectors.S:1779

D (314) memory_layout: Reserved memory range 0x40080000 - 0x4008ef63
0x40080000: _WindowOverflow4 at /home/han/esp/esp-idf-v3.3-rc/components/freertos/xtensa_vectors.S:1779

D (321) memory_layout: Building list of available memory regions:
D (327) memory_layout: Available memory region 0x3ffae6e0 - 0x3ffb0000
D (334) memory_layout: Available memory region 0x3ffb4690 - 0x3ffb8000
D (340) memory_layout: Available memory region 0x3ffb8000 - 0x3ffc0000
D (347) memory_layout: Available memory region 0x3ffc0000 - 0x3ffc2000
D (353) memory_layout: Available memory region 0x3ffc2000 - 0x3ffc4000
D (360) memory_layout: Available memory region 0x3ffc4000 - 0x3ffc6000
D (367) memory_layout: Available memory region 0x3ffc6000 - 0x3ffc8000
D (373) memory_layout: Available memory region 0x3ffc8000 - 0x3ffca000
D (380) memory_layout: Available memory region 0x3ffca000 - 0x3ffcc000
D (386) memory_layout: Available memory region 0x3ffcc000 - 0x3ffce000
D (393) memory_layout: Available memory region 0x3ffce000 - 0x3ffd0000
D (400) memory_layout: Available memory region 0x3ffd0000 - 0x3ffd2000
D (406) memory_layout: Available memory region 0x3ffd2000 - 0x3ffd4000
D (413) memory_layout: Available memory region 0x3ffd4000 - 0x3ffd6000
D (419) memory_layout: Available memory region 0x3ffd6000 - 0x3ffd8000
D (426) memory_layout: Available memory region 0x3ffd8000 - 0x3ffda000
D (433) memory_layout: Available memory region 0x3ffda000 - 0x3ffdc000
D (439) memory_layout: Available memory region 0x3ffdc000 - 0x3ffde000
D (446) memory_layout: Available memory region 0x3ffde000 - 0x3ffe0000
D (452) memory_layout: Available memory region 0x3ffe0440 - 0x3ffe3f20
D (459) memory_layout: Available memory region 0x3ffe4350 - 0x3ffe8000
D (466) memory_layout: Available memory region 0x3ffe8000 - 0x3fff0000
D (472) memory_layout: Available memory region 0x3fff0000 - 0x3fff8000
D (479) memory_layout: Available memory region 0x3fff8000 - 0x3fffc000
D (485) memory_layout: Available memory region 0x3fffc000 - 0x40000000
D (492) memory_layout: Available memory region 0x4008ef64 - 0x40090000
D (499) memory_layout: Available memory region 0x40090000 - 0x40092000
D (505) memory_layout: Available memory region 0x40092000 - 0x40094000
D (512) memory_layout: Available memory region 0x40094000 - 0x40096000
D (518) memory_layout: Available memory region 0x40096000 - 0x40098000
D (525) memory_layout: Available memory region 0x40098000 - 0x4009a000
D (532) memory_layout: Available memory region 0x4009a000 - 0x4009c000
D (538) memory_layout: Available memory region 0x4009c000 - 0x4009e000
D (545) memory_layout: Available memory region 0x4009e000 - 0x400a0000
I (551) heap_init: Initializing. RAM available for dynamic allocation:
D (559) heap_init: New heap initialised at 0x3ffae6e0
I (564) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
D (570) heap_init: New heap initialised at 0x3ffb4690
I (575) heap_init: At 3FFB4690 len 0002B970 (174 KiB): DRAM
I (581) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (587) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
D (594) heap_init: New heap initialised at 0x4008ef64
I (599) heap_init: At 4008EF64 len 0001109C (68 KiB): IRAM
I (605) cpu_start: Pro cpu start user code
I (610) spiram: Adding pool of 4096K of external SPI memory to heap allocator
D (625) clk: RTC_SLOW_CLK calibration value: 3274112
D (187) intr_alloc: Connected src 46 to int 2 (cpu 0)
D (187) intr_alloc: Connected src 57 to int 3 (cpu 0)
D (188) intr_alloc: Connected src 24 to int 9 (cpu 0)
I (193) cpu_start: Starting scheduler on PRO CPU.
D (0) intr_alloc: Connected src 25 to int 2 (cpu 1)
I (0) cpu_start: Starting scheduler on APP CPU.
D (209) heap_init: New heap initialised at 0x3ffe0440
D (219) heap_init: New heap initialised at 0x3ffe4350
D (229) intr_alloc: Connected src 16 to int 12 (cpu 0)
I (229) main: >> app_main
I (229) main: SPIFFS init
I (239) spiffs: Initializing SPIFFS
I (409) spiffs: Partition size: total: 1920401, used: 373739
I (409) main: Start copy
Guru Meditation Error: Core 0 panic'ed (Cache disabled but cached memory region accessed)
Core 0 register dump:
PC : 0x400896d8 PS : 0x00060134 A0 : 0x80085c2c A1 : 0x3ffb5a80
0x400896d8: memcpy at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:342

A2 : 0x3f80001c A3 : 0x3ffb5a98 A4 : 0x0000000b A5 : 0x3f800024
A6 : 0x9f3886a8 A7 : 0xa8bbfaba A8 : 0x9f3886a8 A9 : 0x3ffb5a50
A10 : 0x00000000 A11 : 0x00000001 A12 : 0x3ffb5a9c A13 : 0x00000000
A14 : 0xf0000040 A15 : 0x00000000 SAR : 0x00000008 EXCCAUSE: 0x00000007
EXCVADDR: 0x00000000 LBEG : 0x40089685 LEND : 0x400896b9 LCOUNT : 0xffffffff
0x40089685: memcpy at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:291

0x400896b9: memcpy at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:324

ELF file SHA256: ad0bd0baad0bd0baad0bd0baad0bd0baad0bd0baad0bd0baad0bd0baad0bd0ba

Backtrace: 0x400896d8:0x3ffb5a80 0x40085c29:0x3ffb5a90 0x40119053:0x3ffb5b00 0x400e7af1:0x3ffb5b30 0x40119181:0x3ffb5b60 0x400eae7d:0x3ffb5b80 0x400e6cea:0x3ffb5bd0 0x400e72fd:0x3ffb5c00 0x400e6a36:0x3ffb5c20 0x400ec596:0x3ffb5c40 0x4000bdbb:0x3ffb5c60 0x4008908a:0x3ffb5c80 0x400eea39:0x3ffb5ca0 0x400ef01a:0x3ffb5cc0 0x40119f11:0x3ffb5ce0 0x400f9622:0x3ffb5d00 0x400d6301:0x3ffb5d30 0x400d6405:0x3ffb5e60 0x400d1a72:0x3ffb5e80 0x4008bfc9:0x3ffb5ea0
0x400896d8: memcpy at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:342

0x40085c29: spi_flash_read at /home/han/esp/esp-idf-v3.3-rc/components/spi_flash/flash_ops.c:525

0x40119053: esp_partition_read at /home/han/esp/esp-idf-v3.3-rc/components/spi_flash/partition.c:242

0x400e7af1: spiffs_api_read at /home/han/esp/esp-idf-v3.3-rc/components/spiffs/spiffs_api.c:36

0x40119181: spiffs_phys_rd at /home/han/esp/esp-idf-v3.3-rc/components/spiffs/spiffs/src/spiffs_nucleus.c:69

0x400eae7d: spiffs_object_read at /home/han/esp/esp-idf-v3.3-rc/components/spiffs/spiffs/src/spiffs_nucleus.c:2037

0x400e6cea: spiffs_hydro_read at /home/han/esp/esp-idf-v3.3-rc/components/spiffs/spiffs/src/spiffs_hydrogen.c:1121

0x400e72fd: SPIFFS_read at /home/han/esp/esp-idf-v3.3-rc/components/spiffs/spiffs/src/spiffs_hydrogen.c:1121

0x400e6a36: vfs_spiffs_read at /home/han/esp/esp-idf-v3.3-rc/components/spiffs/esp_spiffs.c:327

0x400ec596: esp_vfs_read at /home/han/esp/esp-idf-v3.3-rc/components/vfs/vfs.c:972

0x4008908a: read at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/syscalls/../../../.././newlib/libc/syscalls/sysread.c:12

0x400eea39: std::__basic_file::xsgetn(char*, int) at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/build/build-cc-gcc-final/xtensa-esp32-elf/esp32-psram/libstdc++-v3/src/c++98/basic_file.cc:211

0x400ef01a: std::basic_filebuf<char, std::char_traits >::xsgetn(char*, int) at
/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/build/build-cc-gcc-final/xtensa-esp32-elf/esp32-psram/libstdc++-v3/include/fstream:758

0x40119f11: std::basic_streambuf<char, std::char_traits >::sgetn(char*, int) at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/build/build-cc-gcc-final/xtensa-esp32-elf/esp32-psram/libstdc++-v3/include/streambuf:436

0x400f9622: std::istream::read(char*, int) at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/build/build-cc-gcc-final/xtensa-esp32-elf/esp32-psram/libstdc++-v3/include/bits/streambuf_iterator.h:113

0x400d6301: CopyTest() at /mnt/c/Users/Han/Documents/Projects/esp32bug/main/main.cpp:46

0x400d6405: app_main at /mnt/c/Users/Han/Documents/Projects/esp32bug/main/main.cpp:27
0x400d1a72: main_task at /home/han/esp/esp-idf-v3.3-rc/components/esp32/cpu_start.c:529

0x4008bfc9: vPortTaskWrapper at /home/han/esp/esp-idf-v3.3-rc/components/freertos/port.c:435

`

Steps to repropduce

Code to reproduce this issue

Gist here: https://gist.github.com/HanVertegaal/cfd356161df072c3d2d5f3b65726989e

sdkconfig .txt
@github-actions github-actions bot changed the title Cache disabled when copying from SPIFFS to SPIRAM Cache disabled when copying from SPIFFS to SPIRAM (IDFGH-1798) Sep 4, 2019
@mahavirj
Copy link
Member

mahavirj commented Sep 6, 2019

@HanVertegaal Please refer to https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/external-ram.html#restrictions. SPIRAM access is not possible when flash cache disabled (due to flash read operation during SPIFFS read), and hence you will have to use internal memory as buffering scheme (i.e. to copy to internal memory first and then accumulate in external SPIRAM).

@negativekelvin
Copy link
Contributor

@mahavirj true but can vfs handle this better by checking pointer and returning an error?

@mahavirj
Copy link
Member

mahavirj commented Sep 6, 2019

@negativekelvin Yes, was about to mark for enhancement w.r.t documentation or better error handling.

@HanVertegaal
Copy link
Author

@mahavirj Thanks for pointing this out to me. I must say however that even after re-reading that section it is not obvious to me that I cannot read from the flash while accessing the external RAM - as opposed to writing to the flash. A more explicit statement of this restriction would indeed be helpful.

@mahavirj
Copy link
Member

mahavirj commented Sep 6, 2019

@HanVertegaal Reopening, until we have some explicit code checks and/or documentation improvement.

@mahavirj mahavirj reopened this Sep 6, 2019
@mahavirj
Copy link
Member

@HanVertegaal Apologies, seems like spi_flash_read can indeed support reading to external SPIRAM buffer (with the help of small buffer from internal memory) but it has issue when read size < 16. Can you please try attached fix (on top of release/v3.3 branch) and let me know if it helps to fix your issue?
0001-flash_ops-fix-spi_flash_read-with-source-buffer-not-.zip

@HanVertegaal
Copy link
Author

@mahavirj Yes, this fixes the problem. Thanks!

@igrr igrr closed this as completed in 56f73d7 Oct 25, 2019
igrr pushed a commit that referenced this issue Oct 27, 2019
@ajita02 @mahavirj
flash_ops: fix spi_flash_read with source buffer not from internal me…
32b8b60 
…mory and size < 16

Closes #4010
espressif-bot pushed a commit that referenced this issue Nov 19, 2019
@ajita02 @mahavirj
flash_ops: fix spi_flash_read with source buffer not from internal me…
622842a 
…mory and size < 16

Closes #4010
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mahavirj @HanVertegaal @negativekelvin