Search results for:

Thanks for the pointers, Gordon. Sounds like it "shouldn't be too difficult" to (a) move the source code strings to flash and just keep a string-ref in jsVars, and (b) place compiled functions into flash as well.

What this means to me right now is that for the esp8266 I should place the "save to flash" stuff in a memory mapped area while a SPIFFS filesystem can use the non-mapped area. There's a 2MB limit to what can be mapped.

WRT JS vs C, I'm getting the feeling that some things will have to be pushed down into C to save memory. What would be neat is if one could use the generic interfaces that make the JS stuff work from C. For example, for the network stuff it ought to be possible to call the functions exported via a JsNetwork (https://github.com/espruino/Espruino/blob/master/libs/network/network.h#L51-L73) from other C code and thereby make that somewhat target independent.

To me the whole point is to have bootup code that is not part of the sketch being uploaded via the IDE. This way I can have some init code that enables the IDE in the first place and that stays in place while I'm wildly editing and testing.

I've read these pages but there really isn't much there about how code is executed. Is the code stored as strings in JsVars or is that external to JsVars? One string or one per line or ...? Is each statement recursive-descent-parsed each time it is executed? What does the compiler produce?

I like the extensible event...

Related to this, I'm wondering how to implement a bit of a "supervisor" on Esprunio. This is in the context of the esp8266 and what I'd like to have is a small web server that allows me to configure the wifi, but also other settings, so I don't have to include that into each sketch. Basically a couple of web pages for scanning networks, associating, etc.

Maybe this has to be built-in in C, but it would be nice if it didn't have to be. But I also don't want to add the 100+ lines of code to every sketch. So some notion of being able to store more persistent code would be nice. Thoughts?

Aren't there currently separate Makefile targets and binaries for all the different amounts of flash?

Yes. See https://github.com/tve/Espruino/blob/master/Makefile#L463-L535
But FLASH_SAVED_CODE_START is set through the board config stuff, isn't it?

it'll only need the same area of flash memory in each

Correct, but at different start addresses. With a 512KB flash we'll have to squeeze the save area in there. With 4MB it can move up beyond the firmware partition, which will help with ensuring that upgrades don't wipe it out.

Whenever a function gets defined it'd go into RAM anyway

Do you have an high-level overview of how this works? You store the source in RAM? Is that in JsVars or somewhere else? Do you parse it into an AST or something and then interpret that? And what does the compiler produce?

Ah got it, I didn't understand your previous comment correctly.

I'm looking into the saving of code into flash. I see the stuff in jswrap_flash.c which uses a special area that is statically defined. Is there a way to make this a bit more dynamic because esp8266's come with different flash memory sizes and it would be a shame to have to define different boards just for this.

Related question, would it be possible to execute JS programs directly from "read-only" memory? In the esp8266 a big chunk of the flash is memory mapped, so the program code could be written to flash but be readable like normal memory. There are some 4-byte alignment constraints I'd have to look into. This would mean that programs don't have to be loaded into RAM and use space there before being executed.

I think getting ESP8266 actually working reliably with hardware peripheral support should really a priority before you start optimising

It's not because I ask about advanced features that I'm necessarily optimizing right now...

We could definitely add some kind of data_received and get_data_to_send callbacks at some point

I'm puzzled, isn't that what Socket.on('data' and Socket.on('drain' do?

Some comments:
wifi.setPower(onOrOff, function(err) { ... }) sounds like TX power level to me? We need a function to set the wifi mode, I believe, this could include: none (=off), station, ap, station+ap. Or do you want just on/off and then drive the mode by using connect/disconnect/createAP/deleteAP ?

For wifi.connect(ssid, key, [options], function(err) { ... }); I would prefer to see callbacks that get the current state. That allows the app to show whether it's waiting to associate or waiting to get a DHCP lease. Unless you want to roll all that into a global wifi state callback.

wifi.getAPs(function(err, aps) { ... }); I believe this is usually called scan, no? Or put differently, do you expect getAPs to return a list of already-known APs or do you expect it to initiate a scan? If it initiates a scan, can the callback be called multiple times as APs are discovered or only once when the scan finishes?

Why does wifi.getAPs(function(err, aps) { ... }); need a callback as opposed to just returning the info?

wifi.createAP(ssid, key, channel, enc, function(err) { ... }) I see a problem with the channel, which is that in sta+ap mode the ap has to be on the same channel as the sta. Maybe allowing channel 0 to stand for any can make this work.

Additional thoughts:

Being able to set the IP address and associated network parameters (gateway and mask) is needed.

A call is needed to set the hostname, which may be used in DHCP to update DNS and in other local host discovery protocols. The hostname needs to be set before connecting to an AP...

We need calls to disconnect and to shut down the AP.

How about a call to get RSSI and current data rate? Alternatively, getConnectedAP could return that info.

I wonder how wifi and low-power modes should interact. It would be good to have a wifiPowerMode call with an extensible set of values, starting with full power and something mapping to PS-Poll (using the DTIM intervals to power down Wifi). This way there's at least some portable way to state that it's desirable to use less power at the expense of throughput and latency.

One question with the above is what the state at boot should be. Systems like the esp8266 save the wifi config in flash and can automatically connect at boot time. Also, I'd really like to support connecting the IDE to the esprunio via TCP/Wifi and then the espruino has to connect at boot to something on its own (unless that's expected to be driven by startup JS files stored in flash).

I'm a bit disappointed by the way the network is hooked in from an event perspective. With the esp8266 each packets gets buffered and copied many times. As far as I can tell, for receive: LwIP buffers the packet, the SDK notices and probably copies it into a buffer of its own (not sure about that), then invokes a callback into user-code (the esp8266 esprunio "driver"), there it's copied into another buffer (required by the callback semantics) and the callback chain is dropped. Later, in the idle loop, espruino discovers "oh, look, there's some data there" and does a recv, which copies the data again, and I believe it's copied yet once more into a JS data structure?
I may not have all the details right, but is there not a way to queue an event for the "idle loop" instead of having it poll every socket all the time, which is also in the end what requires all the intermediate buffers?
On the sending side the buffer copying is not much better and the "you can send more" event is similarly dropped and the idle loop rediscovers it.

I see, so it's not something that is exposed to javascript, i.e., setTimeout()?

It might be there's some free timer hardware in there somewhere :)

Could you elaborate on what you expect, perhaps use-cases you expect to cover? In the end the issue is LwIP, which is not multithreading friendly. As a result, the Espressif SDK manages the application thread carefully. It would be easy to create a timer interrupt, but that will run on some random FreeRTOS thread and it won't be able to safely perform any network I/O.

My recommendation would be to use the timers provided by the SDK and see where that leads us. The step up would be to use interrupts in some form and to use them to drive some yield() type of stack switching in the interpreter loop so the code triggered by the timer can run on the normal application thread.

The esp has an RTC, but I don't know how accurate that ends up being. Fortunately with Wifi NTP usually isn't too far away.

Also, a big one (IMO) is the utility timer.

There definitely is timer support, but it goes through FreeRTOS and I believe is scheduled on "the" app thread, so it may or may not be good enough. For millisec resolution yes, for lower, dunno.

Nope, the LTO fixes didn't work on gcc5. The internal compiler error is fixed in gcc head and scheduled for 5.2. I did not fiddle with options a whole lot to see whether I could get around it.

The always_inline functions that cause all the trouble I believe are https://github.com/espruino/Espruino/blob/master/src/jsvar.h#L305-L321

@Gordon: you have some 6-7 functions in jsvars.[hc] defined to be force-inlined but the function body is in the .c file instead of the .h file. The result is that this can only work with link-time-optimization where the compiler emits intermediate code into the object file and basically the second stage of the compiler runs at link time. While this is cool, given that it's just about 6-7 functions, what do you think about moving these into the .h files so LTO is not required. I have not looked into the code so I don't know whether there's an important reason for these functions to be in the .c file. At this point this is no longer a blocker for the esp8266 but the next port may well run into the same issues.

status update:

1. Gordon merged all the good work @aplikatika and @Kolban did into the main repo, yay!
   
2. I spend time bringing the esp8266 compilation in-line with other targets and went down a rabbit hole when I tried to enable function inlining as is done for other targets: things ceased to compile using gcc 4.8 but worked for Kolban in windows using gcc 5.1
   
3. In the end, by turning link-time optimization on and going through some linker gyrations to deal with iram0/iram1 sections the build worked again for me on linux with gcc 4.8; the link time fun is documented in the Makefile https://github.com/tve/Espruino/blob/master/Makefile#L1480-L1497
   
4. A big issue in all this is that gcc 4.8 uses -std=gnu89 as default and that produces lots of errors, it would be good for Gordon to state which value of -std is intended and which version of gcc needs to be used
   
5. I started to create additional esp board targets to distinguish 512KB/1MB/2B/4MB flash sizes, this becomes important for OTA update and to add a SPIFFS filesystem to store data and programs, this is still only in my private fork and not ready yet
   
6. Kolban points out that it's important to get SPI, I2C and similar other things to work
   

I know that the esp-01 fits the shim, but I'd highly recommend moving to the esp-12 (best the new esp-12f version) because they have 10dB better RF, that makes a huge difference!

thanks for the update!

I reckon it'd be worth pulling the ESP8266 stuff into the main Espruino repo as soon as possible though - it doesn't matter if it's finished or not.

IMHO that would be really, really good to do asap. I'd hate to work on the port and then have to rename and rework a zillion things in order to integrate. Also, it would be really good to normalize the names now and not later.

Adafruit is out of picos, do you know whether they'll be getting more soonish? An alternatives in the US?