Uploading to ota_0 instead of factory partition

[paolosanchi]

I'm trying to upload the program to an esp32 devkitv1 and I get the following issue.

When I press the upload button in vscode the firmware is uploaded to the ota_ 0 partition instead of the factory partition.
The bootstrap tries to start the factory partition and rises an error, then the app starts from the ota_0 partition.
Prior to upload I did a flash erase.
What I expect is that the firmware is uploaded to the factory partition.

This is my custom partition table

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x19000,  0x5000,
otadata,  data, ota,     0x1E000,  0x2000,
factory,  app,  factory, 0x20000, 0x100000,
app0,     app,  ota_0,   0x120000, 0x150000,
app1,     app,  ota_1,   0x270000, 0x150000,
spiffs,   data, spiffs,  0x3C0000, 0x40000,

this is my platformio.ini:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino, espidf
monitor_speed = 115200
lib_deps = 
	C:\git\rfid
monitor_filters = esp32_exception_decoder
board_build.partitions = partitions_custom.csv

And this is the command displayed in the console:

platformio.exe run --target upload

and this the point that it says it's writing to ota0 (0x120000) instead that factory (0x20000)

Wrote 764096 bytes (471615 compressed) at 0x00120000 in 11.0 seconds (effective 554.5 kbit/s)...

This is the boostrapper log:

D (259) boot: Trying partition index -1 offs 0x20000 size 0x100000
D (265) esp_image: reading image header @ 0x20000
D (270) bootloader_flash: mmu set block paddr=0x00020000 (was 0xffffffff)
D (277) esp_image: image header: 0xff 0xff 0xff 0x0f ffffffff
E (283) esp_image: image at 0x20000 has invalid magic byte (nothing flashed here?)
E (291) boot: Factory app partition is not bootable
D (297) boot: Trying partition index 0 offs 0x120000 size 0x100000
D (303) esp_image: reading image header @ 0x120000
D (308) bootloader_flash: mmu set block paddr=0x00120000 (was 0x00020000)
D (315) esp_image: image header: 0xe9 0x06 0x02 0x02 40082094

This issue is a follow-up to this thread:
https://community.platformio.org/t/uploading-to-ota-0-instead-of-factory-partition/28951/9

[Jason2866]

I was running in this too. See my PR #785
We are using a factory firmware too handle a optimized OTA process to save flash space. We used flash extra images to specify where to flash the factory image https://github.com/arendst/Tasmota/blob/development/boards/esp32_4M.json
This functions can be used to flash more images!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Please provide more details or it will be closed if no further activity occurs. Thank you for your contributions.

[OptoStocks]

While I was having the same problem, I came accross your post.
You probably have found a solution since, but in the hope it'll help next people looking into it :

After digging in espressif' docs, the reason is simple : you MUST set a factory app at the address 0x10000.
No compromise here.
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/partition-tables.html#built-in-partition-tables

So, your 0x20000 address is not considered as a correct factory region, and the bootloader will select the next available app, which is "app1" in your case.

Your problem comes from the offset of your nvs region, which should be 0x9000 instead of 0x19000.
=> "nvs, data, nvs, 0x19000, 0x5000"

Unless, of course, you doubled the size of the bootloader ?
If that is the case, you may probably update the bootloader behavior to look at address 0x20000 for factory app as well.

Hope it helps.

[Jason2866]

@OptoStocks Yes, factory app is only possible at address 0x10000.
But with platformio as it is coded at the moment it is not possible to flash factory firmware there AND a app partition to a different address. See my PR to solve this.
It is used and works perfectly for project Tasmota.

[OptoStocks]

@Jason2866 Sure it is possible with VSCode+Platformio !!
Well, at least with espidf framework. I don't use arduino.

For reference :

1. uploading logs (serial monitor) :

Auto-detected Flash size: 4MB
Flash will be erased from 0x00001000 to 0x00007fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x0000e000 to 0x0000ffff...
Flash will be erased from 0x00010000 to 0x000ccfff...
Compressed 25296 bytes to 15414...
Writing at 0x00001000... (100 %)
Wrote 25296 bytes (15414 compressed) at 0x00001000 in 1.9 seconds (effective 103.8 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 152...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (152 compressed) at 0x00008000 in 0.1 seconds (effective 266.8 kbit/s)...
Hash of data verified.
Compressed 8192 bytes to 31...
Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (31 compressed) at 0x0000e000 in 0.1 seconds (effective 642.7 kbit/s)...
Hash of data verified.
Compressed 774032 bytes to 492142...
=> Writing at 0x00010000... (3 %)
Writing at 0x0001cb55... (6 %)
Writing at 0x0002565d... (9 %)

2. boot logs (serial monitor) :

I (29) boot: ESP-IDF 4.3.0 2nd stage bootloader
I (29) boot: compile time 16:16:45
I (29) boot: chip revision: 1
I (31) boot_comm: chip revision: 1, min. bootloader chip revision: 0
I (38) boot.esp32: SPI Speed : 40MHz
I (43) boot.esp32: SPI Mode : DIO
I (48) boot.esp32: SPI Flash Size : 4MB
I (52) boot: Enabling RNG early entropy source...
I (58) boot: Partition Table:
I (61) boot: ## Label Usage Type ST Offset Length
I (68) boot: 0 nvs WiFi data 01 02 00009000 00005000
I (76) boot: 1 otadata OTA data 01 00 0000e000 00002000
I (83) boot: 2 factory_app factory app 00 00 00010000 00100000
I (91) boot: 3 ota_0 OTA app 00 10 00110000 00100000
I (98) boot: 4 ota_1 OTA app 00 11 00210000 00100000
I (106) boot: 5 factory_storage Unknown data 01 82 00310000 00020000
I (113) boot: 6 ota_storage Unknown data 01 82 00330000 000ce000
I (121) boot: End of partition table
=> I (125) boot: Defaulting to factory image

As you can see :

Firmware is written at address 0x10000, and bootloader launches the "factory image" from this very same address.
Once your factory app is up and running, you can upload your "ota_0" app through OTA process.
We maybe do not speak about the same thing ?

[Jason2866]

@OptoStocks Yes, and it is not for Arduino. My PR is for Arduino.
IDF and Arduino builder are completly different code parts in platformio

[paolosanchi]

While I was having the same problem, I came accross your post. You probably have found a solution since, but in the hope it'll help next people looking into it :

After digging in espressif' docs, the reason is simple : you MUST set a factory app at the address 0x10000. No compromise here. https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/partition-tables.html#built-in-partition-tables

So, your 0x20000 address is not considered as a correct factory region, and the bootloader will select the next available app, which is "app1" in your case.

Your problem comes from the offset of your nvs region, which should be 0x9000 instead of 0x19000. => "nvs, data, nvs, 0x19000, 0x5000"

Unless, of course, you doubled the size of the bootloader ? If that is the case, you may probably update the bootloader behavior to look at address 0x20000 for factory app as well.

Hope it helps.

of course, I have a custom bootloader whose size is more than 10000. I just hoped that platformio was smart enough to search the factory partition position directly in the custom partition table file

[Jason2866]

@paolosanchi Platformio cant change the hard limits given from IDF.
Factory image is restricted to be at 0x10000. If you want to have this elswhere you have to open a feature request in IDF github.

[OptoStocks]

@paolosanchi Yes, @Jason2866 is right, the 0x10000 parameter appears to be from an unreleased part of code of espidf (or at least I couldn't easily find it).

Feature request might be your only option, besides reducing your bootloader down to 0xC0000 and still having some room (0x4000) for nvs.

[paolosanchi]

@Jason2866 , @OptoStocks
I'm not sure we are talking about the exactly same thing, or I am missing something, because I can set the factory partition position in the bootloader configuration (CONFIG_PARTITION_TABLE_OFFSET=0x18000), and I can burn the bins correctly with this command:

python.exe esptool.py --chip esp32 --port "COM3" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size 4MB 0x1000 rel\v1.0.0\bootloader.bin 0x18000 rel\v1.0.0\partitions.bin 0x1e000 rel\v1.0.0\ota_data_initial.bin 0x20000 rel\v1.0.0\firmware.bin

How can it can be an espidf issue if I can compile (with platformio commands) and burn correctly with the command?
To me, it simply seems that platformio doesn't generate correctly the esptool.py command string, event if it has the information of the factory partition position (in the custom partition table).

My issue is just this, the platformio button doesn't execute the correct burning command

[OptoStocks]

@paolosanchi Ohhh, sorry I missed that part.
You're right, and I couldn't find it in espidf because it clearly is hard coded in platformio espressif32 platform :
https://github.com/platformio/platform-espressif32/blob/cf076dd5fdb04083f5ebce84335c59e2caf02d56/builder/main.py
=> have a look and search for "0x10000".

You'll probably have some luck by updating this offset especially :
=> ESP32_APP_OFFSET=board.get("upload.offset_address", "0x10000")
(of course you'd better update the 3 occurrences of it)

[Jason2866]

Interesting, since it is against the documentation from espressif that the factory firmware can be only at 0x10000. In this case do not expect this not supported nor documented feature will be implemented. Before this espressif has to support official and change there documentation.

[Jason2866]

@OptoStocks @paolosanchi See my PR #785 could work here too.
Main goal was to make it possible to place factory firmware at 0x10000 for Arduino.
Since it removes the hardcoded offset and queries partition table it may work for this use case too.

[OptoStocks]

Yes probably @Jason2866 :)
Nice workaround you did there.

Would be preferable if the Factory App offset was dynamic and set according to the custom partition, thought.
It's the most simple, and logical, way of doing this.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Please provide more details or it will be closed if no further activity occurs. Thank you for your contributions.