Network lib (not?) yielding to timers, main event loop

We're developing an application that requires fast sampling of pins, say every 20ms via setInterval; while "concurrently" we need to do an HTTP PUT every second or so. With some simple demos we've learned that Espruino tends to be a bit choppy and drops readings while performing the queued network requests.

I managed to trace this to two things (and please correct me if I'm wrong):

1. The HTTP request is handled whenever Espruino is idle (good).
   
2. When the HTTP networking code handles a queued request, it never yields along the way to the main event loop; hence, it takes priority over timer intervals and your own processing (bad).
   
   

Please shed some light on this, and correct me if any of my observations are invalid.

We would love to get more perceived "concurrency" while network request are being handled, and we're even willing to throw some dev resources at the problem to improve this for others.

Any feedback is appreciated.
Thx!

Not sure if I understand you correctly:
-there is single queue of events
-there is no pre-emption - the event handler need to run to completion
-then another event is handled from the queue
Let me know if the http event handler completed the task and after this no further (or queued) events is handled (especially those related to setInterval)

Yes, the issue is actually with the TI's CC3000 drivers, which basically just block until they receive a response from the CC3000.

Personally I'd be against calling the idle loop from within the CC3000 drivers (that just sounds like a recipe for pain), but ideally we could re-write the drivers so that they didn't block, or could actually just use a network interface chip that's been properly designed - I've recently added WIZnet support (for wired ethernet) and this behaves a lot more sensibly. The only thing that does still cause delays is the name resolution - but that could be worked around relatively easily.

Otherwise you do have options for working around this delay:

• The Waveform object will read or write analog values at set intervals via Interrupts. This may do what you want right now, but could easily be extended to handle digital signals as well.
  
• Watches are handled on interrupts and are then queued for later execution. Instead of reading the value of a pin, you could listen for the times that it changes state and record those instead.
  

Just to add that I have a personal branch where I'm refactoring the network interface so it's not a mess of #ifdefs for different devices. It should make re-writing the CC3000 drivers a slightly less daunting task.

Thank you Gordon. I'll look into your options; and we may even look into a properly designed network chip (if you have any suggestions, we would be happy to hear them). With your feedback, we have enough to move along. If you don't mind, we may come back here to share our findings and progress.

Absolutely! And if it's not under an NDA, please post up some stuff about your project - it's always good to see Espruino in use!