E.memoryArea not working on ESP8266 (due to char-iterator ?)

I also believe these is a restriction of 1mb max, and that this is in the espurino core rather than the esp8266 implementation.

Well, it's not a restriction in Espruino itself - maybe just something about getFlash?

I also believe these is a restriction of 1mb max, and that this is in the espurino core rather than the esp8266 implementation.
Well, it's not a restriction in Espruino itself - maybe just something about getFlash?

I was going by this comment:
https://github.com/espruino/Espruino/blob/master/targets/esp8266/jshardware.c line 48

The restriction seems to be here:

// Save-to-flash uses 12KB at 0x78000
// The jshFlash functions use memory-mapped reads to access the first 1MB
// of flash and refuse to go beyond that. Writing uses the SDK functions and is also
// limited to the first MB.

#define FLASH_MAX (1024*1024)
#define FLASH_MMAP 0x40200000
#define FLASH_PAGE_SHIFT 12 // 4KB
#define FLASH_PAGE (1<<FLASH_PAGE_SHIFT)

Looking at the fetch:

void jshFlashRead(
    void *buf,     //!< buffer to read into
    uint32_t addr, //!< Flash address to read from
    uint32_t len   //!< Length of data to read
  ) {
  //os_printf("jshFlashRead: dest=%p, len=%ld flash=0x%lx\n", buf, len, addr);

  // make sure we stay with the flash address space
  if (addr >= FLASH_MAX) return;
  if (addr + len > FLASH_MAX) len = FLASH_MAX - addr;
  addr += FLASH_MMAP;

So really #define FLASH_MAX (1024*1024) should be determined by the type of Flash chip in the esp8266 board.

https://github.com/espruino/Espruino/blob/master/targets/esp8266/jswrap_esp8266.c

JsVar *jswrap_ESP8266_getFreeFlash() {
  JsVar *jsFreeFlash = jsvNewArray(NULL, 0);
  // Area reserved for EEPROM
  addFlashArea(jsFreeFlash, 0x77000, 0x1000);

  // need 1MB of flash to have more space...
  extern uint16_t espFlashKB; // in user_main,c
  if (espFlashKB > 512) {
    addFlashArea(jsFreeFlash, 0x80000, 0x1000);
    if (espFlashKB > 1024) {
      addFlashArea(jsFreeFlash, 0xf7000, 0x9000);
    } else {
      addFlashArea(jsFreeFlash, 0xf7000, 0x5000);
    }
  }

So this could be used rather than the #define:
extern uint16_t espFlashKB; // in user_main,c

Maybe E.memoryArea should use flash addresses (starting at 0) instead of memory mapped addresses?
In any case, it can only access the first 1MB of flash due to esp8266 memory mapping limitations.

Maybe E.memoryArea should use flash addresses (starting at 0) instead of memory mapped addresses?

I guess we'd need some ESP8266-specific tweaks in there for that, but it could be a plan.

In any case, it can only access the first 1MB of flash due to esp8266 memory mapping limitations.

@Wilberforce So jshFlashRead could potentially use more memory as it can use API calls, but memoryArea needs a pointer, so it will never be able to access more than the first 1MB.

Would it be possible/an option to move the defines from jsutils to jshardware ?

At the moment there's no per-platform .h file apart from platform_config.h - that could be an option?

I agree it's a shame to have loads of ESP8266 ifdefs in jsutils.h, but right now I think even if those moved, there would still be a bunch of stuff for jsException/jsError (which is used to get the error messages moved out of RAM).

Does using the api get around the memory mapped limitation of accessing > 1mb memory for flash?
https://github.com/espruino/Espruino/blob/9781bacee8185b58438604cb22672389bc3ff42d/targets/esp8266/jshardware.c
Line 1193

For writing the api call is used,but for reading it is just accessing memory. If the Api was used for the reading could we using the flash module for accessing > 1 mb?

I would imagine so, but there might be some gotchas. @tve might know?

Speed is the main gotcha, IIRC. Espruino uses 4-byte flash reads when loading stuff like the saved code from flash, so if each one turns into an SDK call which has to tweak the HW memory map back and forth to access the rest of flash it's gonna be real slow.
Why not add an esp8266-specific call to access the rest of flash?

Or perhaps a better option: if the flash region accessed is in the lower 1MB use the existing memory-mapped code, if it's higher use the SDK calls...

I think you'd be fine using the API calls for jshFlashRead, if that's what you were thinking of? jshFlashRead isn't (or shouldn't) be called that often so wouldn't hurt performance.

As @tve says, I wouldn't start changing internal stuff (eg. E.memoryArea) to use it, or performance will really suffer.

Looks like the conditional if < 1mb direct read otherwise api is the way to go.

Now that the flash module returns available flash should the esp8266 call be depreciated?

If I manage to get this stuff working, I'll have to extend the free areas to return the new available space too

The esp8266 call is deprecated - it'll say so in the documentation when the next version is released (the online documentation matches the current release).

The code compiles, but I've not had a chance to test yet:
https://github.com/espruino/Espruino/compare/master...wilberforce:master

https://github.com/espruino/Espruino/pull/827

Tested ok on 4mb board.