[FEATURE] Introduce a "new" image format for espressif devices.

[Laczen]

Is your feature request related to a problem? Please describe.

Espressif devices now support different image formats in combination with a secondary bootloader/rom bootloader:

1. ESP-IDF format bootable by the esp-idf bootloader,
2. MCUBoot format bootable by mcuboot,
3. Simpleboot booteable by the rom bootloader,

The first image format is receiving less support and focus is put on the MCUBoot format and the Simpleboot format.

The simpleboot format is closely related to the ESP-IDF format but there are 3 main differences:

1. The Simpleboot image fixes it's location to a location known by the rom bootloader,
2. The Simpleboot hides the rom segments (placed after the iram/dram segments) for the rom bootloader,
3. The Simpleboot manages the mapping of irom/drom image segments at start,

A Simpleboot image can easily be made into an esp-idf bootable image by removing limitations 1 and 2:
introduce a location where the image will be placed and generate the image using esptool.py without the option --ram-only-header.

Describe the solution you'd like

To support the booting of Simpleboot images from a secondary bootloader I propose to introduce a Kconfig bool for SIMPLE_BOOT_SBLI (Simpleboot secondary bootloader image) and a variable SIMPLE_BOOT_SBLI_OFFSET (offset where the image will reside). Based on these two it is straight forward to modify the code and tools used to generate the image.

This change will allow the use of the esp-idf bootloader and other (secure) bootloaders that support the esp-idf bootloader format and allow NuttX to remain focussed on just the simpleboot and mcuboot format.

Describe alternatives you've considered

Leave it as is and be forced to use mcuboot as a secondary bootloader for secure images.

Verification

• I have verified before submitting the report.

[Laczen]

@eren-terzioglu @tmedicci @fdcavalcanti @almir-okato FYI

[xiaoxiang781216]

why not add them to apps/boot (e.g. espboot, simpleboot), which parse the specific image format to load the code to ram or find the entry point, and boardctl into kernel for jumping to it, like uboot/lk. And then you can add any option you want in the apps/boot/xxxx/Kconfig.

[Laczen]

why not add them to apps/boot (e.g. espboot, simpleboot), which parse the specific image format to load the code to ram or find the entry point, and boardctl into kernel for jumping to it, like uboot/lk. And then you can add any option you want in the apps/boot/xxxx/Kconfig.

I am not quite sure how this could be done. The main change is to allow setting the location of the image to a different location from what is expected by the rom bootloader, this location is set in code as part of the mmu mapping. The value is not a Kconfig value, but instead set by non NuttX include files (it is not a part of .config).

[xiaoxiang781216]

why not add them to apps/boot (e.g. espboot, simpleboot), which parse the specific image format to load the code to ram or find the entry point, and boardctl into kernel for jumping to it, like uboot/lk. And then you can add any option you want in the apps/boot/xxxx/Kconfig.

I am not quite sure how this could be done. The main change is to allow setting the location of the image to a different location from what is expected by the rom bootloader,

SIMPLE_BOOT_SBLI and SIMPLE_BOOT_SBLI_OFFSET is used only by image creation tool? whether this value is fixed by chip or board?

is the address is this location is set in code as part of the mmu mapping.

the mapping is set by rom bootloader or nuttx self?

The value is not a Kconfig value, but instead set by non NuttX include files (it is not a part of .config).

so, the code in NuttX doesn't need to know this value.

[Laczen]

why not add them to apps/boot (e.g. espboot, simpleboot), which parse the specific image format to load the code to ram or find the entry point, and boardctl into kernel for jumping to it, like uboot/lk. And then you can add any option you want in the apps/boot/xxxx/Kconfig.

I am not quite sure how this could be done. The main change is to allow setting the location of the image to a different location from what is expected by the rom bootloader,

SIMPLE_BOOT_SBLI and SIMPLE_BOOT_SBLI_OFFSET is used only by image creation tool? whether this value is fixed by chip or board?

At the moment when SIMPLE_BOOT is selected it is using CONFIG_BOOTLOADER_OFFSET_IN_FLASH (defined in one of the device specific espressif include files, varies by device) in code. Next to this it is using this as a switch to create an image that can be processed by the rom bootloader (hiding the rom segments).

To be able to override CONFIG_BOOTLOADER_OFFSET_IN_FLASH the bool SIMPLE_BOOT_SBLI is used and CONFIG_BOOTLOADER_OFFSET_IN_FLASH is replaced by SIMPLE_BOOT_SBLI_OFFSET.

is the address is this location is set in code as part of the mmu mapping.

the mapping is set by rom bootloader or nuttx self?

By NuttX.

The value is not a Kconfig value, but instead set by non NuttX include files (it is not a part of .config).

so, the code in NuttX doesn't need to know this value.

The code does need to know the value, it is just "hidden" from the NuttX configuration.

[almir-okato]

@Laczen, the initial motivation for implementing Simple Boot was to have an app running straight after ROM bootloader, that's why its image header format is similar to IDFs, and reduce "overhead" for the users that don't need all the features provided by a bootloader like OTA or security, so it saves a bit of time and resources, from building the project until having it running in the device.

From what I understood from your other PR and what you described here, also assuming you are using a custom second stage bootloader "bootable" as a Simple Boot image, it seems the issue is reusing part of map_rom_segments function, so this custom bootloader could boot another image with same Simple Boot header format.

My suggestion is changing the fetch_rom_segments_lma (see my comment in your PR) to receive the offset argument:

int fetch_rom_segments_lma(uint32_t *app_irom_start,
                           uint32_t *app_drom_start,
                           size_t offset)

Then on https://github.com/apache/nuttx/pull/19985/changes#diff-c704a8399b796e32bd41fdcb8090cd603fa5043fe30d40ca291c121247fccbf9R542-R548 you may keep the offset as CONFIG_BOOTLOADER_OFFSET_IN_FLASH for Simple Boot:

#ifdef CONFIG_ESPRESSIF_SIMPLE_BOOT
  if (fetch_rom_segments_lma(&app_irom_start, &app_drom_start, CONFIG_BOOTLOADER_OFFSET_IN_FLASH) != 0)
    {
      while (true);
    }
#endif

From your custom second stage bootloader (which may be implemented as a NuttX app, as @xiaoxiang781216 suggest), you may call fetch_rom_segments_lma with a configurable offset.

Notice that probably there's more to change, because the final app you are booting may share the same header format as a Simple Boot image, but you'll need to sync memory organization, code execution path (like having something as #elif CONFIG_ESPRESSIF_CUSTOM_IMAGE for not calling fetch_rom_segments_lma on the final app __esp_start, etc.) and other resources between the custom bootloader and final app.

[hardik08tripathi-svg]

why not add them to apps/boot (e.g. espboot, simpleboot), which parse the specific image format to load the code to ram or find the entry point, and boardctl into kernel for jumping to it, like uboot/lk. And then you can add any option you want in the apps/boot/xxxx/Kconfig.

I am not quite sure how this could be done. The main change is to allow setting the location of the image to a different location from what is expected by the rom bootloader,

SIMPLE_BOOT_SBLI and SIMPLE_BOOT_SBLI_OFFSET is used only by image creation tool? whether this value is fixed by chip or board?

is the address is this location is set in code as part of the mmu mapping.

the mapping is set by rom bootloader or nuttx self?

The value is not a Kconfig value, but instead set by non NuttX include files (it is not a part of .config).

so, the code in NuttX doesn't need to know this value

[Laczen]

@Laczen, the initial motivation for implementing Simple Boot was to have an app running straight after ROM bootloader, that's why its image header format is similar to IDFs, and reduce "overhead" for the users that don't need all the features provided by a bootloader like OTA or security, so it saves a bit of time and resources, from building the project until having it running in the device.

From what I understood from your other PR and what you described here, also assuming you are using a custom second stage bootloader "bootable" as a Simple Boot image, it seems the issue is reusing part of map_rom_segments function, so this custom bootloader could boot another image with same Simple Boot header format.

My suggestion is changing the fetch_rom_segments_lma (see my comment in your PR) to receive the offset argument:

int fetch_rom_segments_lma(uint32_t *app_irom_start,
                           uint32_t *app_drom_start,
                           size_t offset)

Then on https://github.com/apache/nuttx/pull/19985/changes#diff-c704a8399b796e32bd41fdcb8090cd603fa5043fe30d40ca291c121247fccbf9R542-R548 you may keep the offset as CONFIG_BOOTLOADER_OFFSET_IN_FLASH for Simple Boot:

#ifdef CONFIG_ESPRESSIF_SIMPLE_BOOT
  if (fetch_rom_segments_lma(&app_irom_start, &app_drom_start, CONFIG_BOOTLOADER_OFFSET_IN_FLASH) != 0)
    {
      while (true);
    }
#endif

From your custom second stage bootloader (which may be implemented as a NuttX app, as @xiaoxiang781216 suggest), you may call fetch_rom_segments_lma with a configurable offset.

Notice that probably there's more to change, because the final app you are booting may share the same header format as a Simple Boot image, but you'll need to sync memory organization, code execution path (like having something as #elif CONFIG_ESPRESSIF_CUSTOM_IMAGE for not calling fetch_rom_segments_lma on the final app __esp_start, etc.) and other resources between the custom bootloader and final app.

Hi @almir-okato, thank you for your comment.

The proposed image format is also closely related to the legacy esp-idf format. The proposed image format can be considered as a fixed location esp-idf format that is responsible for the rom mapping. As the image itself does the rom mapping it needs to know its location and this is exactly the flash offset.

There is not need for a bootloader to call fetch_rom segments_lma, it is the image that needs it so it needs to be set in the NuttX source. I can incorporate the proposed change, but this will be a constant for the image in the end. There are indeed some extra changes required: there is no need for the image to call bootloader_init(), and it needs to zero the .bss.

[Laczen]

Hi @hardik08tripathi-svg thanks for the comment.

why not add them to apps/boot (e.g. espboot, simpleboot), which parse the specific image format to load the code to ram or find the entry point, and boardctl into kernel for jumping to it, like uboot/lk. And then you can add any option you want in the apps/boot/xxxx/Kconfig.

I am not quite sure how this could be done. The main change is to allow setting the location of the image to a different location from what is expected by the rom bootloader,

SIMPLE_BOOT_SBLI and SIMPLE_BOOT_SBLI_OFFSET is used only by image creation tool? whether this value is fixed by chip or board?

These config values would be used to define the NuttX image that is generated. In the NuttX code they would be used to change part of the code (e.g. eliminate bootloader_init(),...). The result would be a NuttX image that can be placed at a specific location and started by copying ram segments and jumping to the entry address.

is the address is this location is set in code as part of the mmu mapping.

the mapping is set by rom bootloader or nuttx self?

NuttX self.

The value is not a Kconfig value, but instead set by non NuttX include files (it is not a part of .config).

so, the code in NuttX doesn't need to know this value

The code in NuttX needs to know it to generate the correct image.

[almir-okato]

There is not need for a bootloader to call fetch_rom segments_lma, it is the image that needs it so it needs to be set in the NuttX source. I can incorporate the proposed change, but this will be a constant for the image in the end. There are indeed some extra changes required: there is no need for the image to call bootloader_init(), and it needs to zero the .bss.

Hi @Laczen, correct me if I'm wrong, I assumed you are implementing the custom second stage bootloader as an application itself on nuttx-apps, like:

• Custom bootloader = implementation out of the main NuttX tree, built as a Simple Boot image, so yes, CONFIG_BOOTLOADER_OFFSET_IN_FLASH is still a constant for all Simple Boot images, including this one;
• Your custom compatible image = implementation out of the main NuttX tree, built as CONFIG_ESPRESSIF_CUSTOM_IMAGE, which probably this config would need to be added to the main tree, so the code can be branched as when to call the mentioned bootloader_init(), fetch_rom_segments_lma(), etc.. Since there's no "custom image" yet, those "calls, where and when" need to be discussed.

I think it's important we avoid mixing the "Simple Boot" concept with a custom image, although they may be using the same image format header.

[Laczen]

There is not need for a bootloader to call fetch_rom segments_lma, it is the image that needs it so it needs to be set in the NuttX source. I can incorporate the proposed change, but this will be a constant for the image in the end. There are indeed some extra changes required: there is no need for the image to call bootloader_init(), and it needs to zero the .bss.

Hi @Laczen, correct me if I'm wrong, I assumed you are implementing the custom second stage bootloader as an application itself on nuttx-apps, like:

• Custom bootloader = implementation out of the main NuttX tree, built as a Simple Boot image, so yes, CONFIG_BOOTLOADER_OFFSET_IN_FLASH is still a constant for all Simple Boot images, including this one;
• Your custom compatible image = implementation out of the main NuttX tree, built as CONFIG_ESPRESSIF_CUSTOM_IMAGE, which probably this config would need to be added to the main tree, so the code can be branched as when to call the mentioned bootloader_init(), fetch_rom_segments_lma(), etc.. Since there's no "custom image" yet, those "calls, where and when" need to be discussed.

I think it's important we avoid mixing the "Simple Boot" concept with a custom image, although they may be using the same image format header.

Hi @almir-okato thanks for the comment.

I agree that this would be a espressif custom image that would need to be added in NuttX tree, and I also don't want to mix the "Simple Boot" concept. The proposal here is exactly meant to discuss the "new" image type before integrating it in the tree. Of course the correctness of the integration will need to be discussed in PR's that introduce the new type in the NuttX tree.

This "new" image type is not really new, it is the esp-idf format where the startup is done from iram (including the rom mappings) instead of directly from irom. The image can be booted by the legacy esp-idf bootloader (which would setup the rom mappings twice, once in the bootloader and once in the image itself). The "new" image type would allow easier addition of newer devices as they are added with the simpleboot mechanism first.

Regarding the integration in nuttx-apps: I would add 2 things in nuttx-apps:

1. The image configuration for bootloader support: that defines the basic properties of the NuttX image (the location it is compiled for and the image size), but also the support routines that defines where upgrade images should be placed and how to switch between different images,
2. The bootloader itself that would use the configuration variables defined in the nuttx-apps section. The bootloader itself could be a nuttx image, but also a bare metal application.

The bootloader in nuttx-apps would not be limited to espressif devices but also target arm devices. I already have a prototype for this in https://codeberg.org/Laczen/sfboot.