EMW3165 port

Signs of life on an EMW3165:

Unfortunately, the looks of the above are a tad misleading because console input doesn't work :-D However, this is better than staring at a non-functional device on my desk! I used a WifiMCU and the st-link-2-1 of a nucleo-F411 for flashing. My fork is at https://github.com/tve/Espruino/tree/emw­3165 and instructions for building are at https://github.com/tve/Espruino/blob/emw­3165/README_Building.md#for-emw3165

Just to be clear: at this stage you can't do anything with it, except for helping to move the port forward! The Wifi isn't even hooked up yet either.

I've written a bit about the overall port strategy in the README_Building and it would be nice to discuss this further here. I did some Espruino console hacks (use the WICED stdio for putc) to get to the prompt shown above, but I think it's the wrong direction. Rather than use the WICED platform abstraction I think it's better to reduce the use of WICED to the minimum and make it use Espruino's platform where that's necessary and completely remove the management of non-wifi hardware from WICED. I'm pretty sure this can be accomplished for stuff like usart, i2c, spi, etc., but I don't know what it's gonna take for core processor stuff like clocks, timers, interrupts, etc. At some point this also affects the boot process and thus flash layout. The next step would be to dive into the WICED configuration process and the sources (there's basically no useful documentation) and determine how to strip it back and insert hooks into the Espruino platform stuff where that's necessary.

[Ewwww, the formatting in this forum site sucks. I had pasted the text of the screen shot using the code formatting and it looked great in the preview mode but whenever I saved it the results ended up being illegible. Had to resort to the screen shot.]

1 Attachment

Wow, thanks! This looks really cool. The EMW3165 is something I've been really excited about - more so than the ESP8266 - as it has the potential to be properly realtime and to use all the existing STM32 code.

As you'd suggested, my hope (at least without having looked into WICED too deep) was to use the existing STM32 code files as-is, and to just use WICED for the WiFi.

As far as I could tell from what I'd read, the actual WiFi bolts on over the SDIO peripheral, so you'd hope that WICED could grab control of that without interfering with the existing STM32 code Espruino has.

I guess I'd hoped that most of the timing-specific WiFi stuff was handled by the Broadcom chip, so the actual WICED library didn't need too fine-grained control of the STM32. If it's at all possible that'd definitely be the way to go (rather than using the WICED abstraction) - there's been a lot of work that's gone into making the STM32 stuff work well, so it'd be a huge amount of effort to start from scratch and get something that worked as well.

The STM32 peripherals are actually pretty well segregated - so if (for instance) you don't ask WICED to access TIM1, the existing Espruino code should be able to use that without issues.

I'm not sure I can be much more help without spending some time and looking at the WICED stuff in depth, but if you have any questions, ask away :)

There's someone that said they'd got Espruino running on EMW3162 (iirc?) - something similar to the EMW3165. I'll ask if they can post what they'd done - maybe they had found a way of avoiding FreeRTOS.

Ok, just looked back at my e-mail, it was EMW3288, which was too different to be useful.

However there is an EMW3165 forum which I posted on a while back that might be some use? http://www.emw3165.com/viewtopic.php?f=1­8&t=66

I tried to post the Espruino signs of life text above to the emw3165 forum, but my initial message first sat there for 3 days and now got rejected because of "links to illegal or pirated software", so I think I'm done with that forum.

:( that's not good - wonder why that was...

Hi tve, Gordon, and all other Espruino developers,

My sincerest apologies about the post disapproval on emw3165.com website - I'm the site admin and this has just come to my attention and I'd like to make it right. The site has been getting a high volume of spam (150+ posts per day) over the past week and I must have accidentally included your post in the bulk disapproval action.

I'm really sorry about that, my aim is to a facilitate a prosperous community of developers where everyone feels welcome to contribute. I'll be sure to pay extra attention so that won't happen again. Thanks for your understanding.

Hey, thanks! It's really nice of you to take the time to post here.

I really feel for you about the spam - I've had massive issues with forum spam on other sites I've been trying to run in the past, I've wasted weeks on it. I hope you find a way to sort it out soon!

Gordon, I need a bit of help with the STM32F4 start-up. I'm trying to get the NucleoF411 version of Espruino to work on the emw3165 since it's the same CPU chip I figured that it should work... But it's not. Where it seems to die is in the clock config at targetlibs/stm32f4/lib/system_stm32f4xx.­c line 559:

557     /* Select the main PLL as system clock source */
558     RCC->CFGR &= (uint32_t)((uint32_t)~(RCC_CFGR_SW));
559     RCC->CFGR |= RCC_CFGR_SW_PLL;

Do you have any suggestion? The WICED emw3165 config has the following defines:

 # define HSE_SOURCE              ( RCC_HSE_ON )
 # define AHB_CLOCK_DIVIDER       ( RCC_SYSCLK_Div1 )
 # define APB1_CLOCK_DIVIDER      ( RCC_HCLK_Div2 )
 # define APB2_CLOCK_DIVIDER      ( RCC_HCLK_Div1 )
 # define PLL_SOURCE              ( RCC_PLLSource_HSE )
 # define PLL_M_CONSTANT          ( 26 )
 # define PLL_N_CONSTANT          ( 200 )
 # define PLL_P_CONSTANT          ( 2 )
 # define PPL_Q_CONSTANT          ( 4 )
 # define SYSTEM_CLOCK_SOURCE     ( RCC_SYSCLKSource_PLLCLK )
 # define SYSTICK_CLOCK_SOURCE    ( SysTick_CLKSource_HCLK )

Clock issue resolved. The emw3165 runs off a 26Mhz oscillator and my first attempt at concocting the PLL config register value for that was off. I now have the nucleo build working on the emw3165 :-).
Next step is to try and link the WICED wifi chip interface into Espruino...

Nice, thanks! Sorry for the lack of reply - been a bit busy this weekend (as usual)

I'm having an issue with pin assignments. The package used by the emw3165 doesn't have PB11 brought out. It looks like as a result the pin B11 is assigned to PB12 and so forth. Could that be? I sure hope this is a bug and not intentional...

Yeah, this doesn't look right:

>(new Pin(B10)).getInfo()
={
  "port": "B",
  "num": 10,
  "functions": {
    "I2C2": {
      "type": "SCL",
      "af": 4 },
    "TIM2": {
      "type": "CH3",
      "af": 1 },
    "SPI2": {
      "type": "SCK",
      "af": 5 }
   }
 }
>(new Pin(B11)).getInfo()
={
  "port": "B",
  "num": 12,
  "functions": {  }
 }
>(new Pin(B12)).getInfo()
={
  "port": "B",
  "num": 13,
  "functions": {
    "SPI2": {
      "type": "SCK",
      "af": 5 },
    "TIM1": {
      "type": "CH1N",
      "af": 1 }
   }
 }

The jspininfo.c file look correct, though:

/* PB10 */ { JSH_PORTB, JSH_PIN0+10, JSH_ANALOG_NONE, { JSH_AF4|JSH_I2C2|JSH_I2C_SCL/* 1 Uses */, JSH_AF1|JSH_TIMER2|JSH_TIMER_CH3/* 2 Uses */, JSH_AF5|JSH_SPI2|JSH_SPI_SCK/* 2 Uses */, 0 } },
/* PB12 */ { JSH_PORTB, JSH_PIN0+12, JSH_ANALOG_NONE, { 0, 0, 0, 0 } },
/* PB13 */ { JSH_PORTB, JSH_PIN0+13, JSH_ANALOG_NONE, { JSH_AF5|JSH_SPI2|JSH_SPI_SCK/* 2 Uses */, JSH_AF1|JSH_TIMER1|JSH_TIMER_CH1|JSH_TIM­ER_NEGATED/* 3 Uses */, 0, 0 } },
/* PB14 */ { JSH_PORTB, JSH_PIN0+14, JSH_ANALOG_NONE, { JSH_AF5|JSH_SPI2|JSH_SPI_MISO/* 1 Uses */, JSH_AF1|JSH_TIMER1|JSH_TIMER_CH2|JSH_TIM­ER_NEGATED/* 3 Uses */, 0, 0 } },

Ahh, yes. I've come across that before - it's the code that converts from the pin name to a pin number - usually it does PORTB_OFFSET+NUMBER, but that doesn't work in this case.

Try adding the board to this: https://github.com/espruino/Espruino/blo­b/master/src/jspin.c#L24

There really needs to be a better way of handling it.

I noticed that instruction prefetching from flash is disabled (e.g. https://github.com/espruino/Espruino/blo­b/master/targetlibs/stm32f4/lib/system_s­tm32f4xx.c#L552-L555). Is that on purpose or more of an accident?

I'm looking at WICED, in particular the lower WWD layer, LwIP, and FreeRTOS. I think there are two reasonably clean ways to proceed. One is to strip things to the minimum and remove FreeRTOS. Basically use the scheduling functionality or idle loop in Espruino to call into LwIP and use the raw interface in LwIP so there's only one thread involved. None of the sockets layer stuff, which needs to run in a second thread. I have not looked whether WWD supports this mode of operation, but I suspect it does.
The second approach would be to cleanly integrate with FreeRTOS. At a very minimum this means ensuring that interrupt handlers in Espruino play nice with it, but I suspect that there's more to it.
Overall I think that we'll see the FreeRTOS & LwIP combination more and more, so having a clear and clean strategy would be smart.
Thoughts?

After another tour of most of the pieces involved I'm coming to the conclusion that the best bet is to swallow the FreeRTOS pill. LwIP and WICED WWD can be run without, but it's not great (lots of polling) and then there's still the issue that Espruino doesn't mind disappearing into JS code for arbitrary amounts of time. Using FreeRTOS will ensure that the network keeps running using preemptible tasks.

What seems to make a lot of sense is to integrate FreeRTOS and LwIP from original sources into Espruino, and then integrate WICED WWD using the integration files it provides as a starting point. WICED has LwIP1.4.0rc1 when LwIP1.4.1 is the latest release, and it has FreeRTOS 7 when the current release is 8 (and does away with those horrible name prefixes). I don't think I'm up for this at the moment, though, so I'll probably try to just glue whatever WICED comes with into Espruino for starters...

I have the feeling that I'm coming somewhat full circle with my very first attempts... Espruino is compiled without stdlibc. That's just not gonna fly with FreeRTOS, LwIP, WWD as far as I can tell. Given that I have pull in a libc I'd rather pull in newlib over glibc. The toolchain shipping with WICED already uses newlib. It's not the newest but probably good enough. So I think the easiest path forward is going to be to switch to the WICED toolchain to compile everything...

Tracking @tve's path throught the 'implemenation options landscape', I conclude that putting 'everything' on a single cpu just creates to many dependencies. Yes, it would be great to have a better integration of the communication aspect - something like wifie/ESP8266 - into the application part - JS applicationss - through the Espruino Runtime Environment...

But a long, long time ago evolution has taken a little bit a different approach: cooperation of subsystems that all have their own nPU (nano-PU) - working 'together' by a mPU - and for some faster information interchange bypassing and offloading the mPU - multiple specific DAPU (such as DMAs). First, it was like 'simple' periphery discrete chips, like CTC, SIO, PIO,... and D'M'A Integration.

VLSI and speed - allowed and required it to put it into a single package and shift towards more sophisticated cotrol through software (uCode instead of 'wired' decoding and control, CISC to RISC to hybrid) - and that's what we now have wit the stM hardware. I see kind of a repetition of that over and over again.

I like having an encapsulated communation component/platform - ESP8266 for example - and like-wise application platform (which includes some basic hardware access (w/ limited 'real' time capabilities). On a similar note, running robustly x/y of a plotter directly from Espruino ontop of the controlling app compounded already to 'issues' for me. Smooth operation over the range of given delta-x and delta-y asked already for independent processing/controlling unit that has enough 'free time' to stay within the tolerances of serving the interupts/timing expected by stepper drivers and motors. I'm thinking of Espruino platform as the center and high-level control, and all the other things are like satellites focused on a dedicated task - that doesn't meand they can not be Espruino's as well, but may be it become a bit an over kill.

I'd rather see ESP8266 grow to a point where it becomes as a http(s) service to Espruino by taking on more responsibility, especially such as TSL (and incl. HTTP/1.1). In other words, he code on Espruino to use those capabilities becomes more like a facade or API and the actual implementation is in ESP8266. Espruino has alaready enough on its plate to manage the hardware oriented, hardware-interrupt dictated aspect without 'losses' and at the same time tend optimally to the Javascript side / application.

Enhancing the cooperation by enhancing the software on both ESP8266 and Espruino sides looks most promising to me. ESP8266 and Espruino core should stay what they are, but both cores are complemented with bridge/beach head software optimized towards/for each other. The effort (and resource consumation) to be truthful to such a layerin/API/micro-service approach pays off by robustness in operation due to better testability.

I thought similarly to you when I wrote esp-link. Esp-link includes support for an attached uC to make outbound REST requests and to do pub/sub using MQTT. What I realized is that I then have to concoct a protocol over the serial wire for the attached uC to talk to esp-link and tell it what to do. So here comes SLIP and some RPC-like stuff on top.
Well, it's very easy and quick to come up with a shitty protocol and to write a shitty arduino or espruino library to show that it works on a sunny day. Creating a robust and efficient protocol to make esp-link do REST and MQTT (and do I hear someone clamoring for websockets?) and then writing robust libraries that have good error handling and such is an order of magnitude more work.
Plus now all this has added another level of failures: did the uC have a problem? or did esp-link have a problem? or did the protocol between them mess-up? Who gets to deal with "lost wifi connection" kind of stuff? Etc, etc.
It's when I faced theses questions that I thought: mhhh, maybe for 90%+ of what I do I can do it straight on the esp8266 and I'll be just fine.
And then there's the power stuff. It's already hard enough to run any wifi solution on battery for extended periods, adding more uC's is unlikely to help, I think.
So in the end you either bite the bullet and spend significant effort making two uC's dance or you spend a perhaps comparable effort using interrupts and threads to make your time critical stuff work on one uC. One big advantage of the emw3165 over the esp8266 is that we have full control over the FreeRTOS implementation and the various interrupt routines, and we have a much better processor and I/O devices to play with.
I have to admit that if I could build a system from scratch I'd put LwIP and TLS onto the BCM43362 and present a socket interface across the SDIO bus. Sadly that's not an option as far as I know.

Ugh, I'm running a bit aground... The first "fun" part was that anything in wwd includes the rest of the world's .h files. I've mostly navigated that part, I think. Now I'm stuck in the platform hardware access code. The theory, as Gordon writes so nicely, sounds great: "As far as I could tell from what I'd read, the actual WiFi bolts on over the SDIO peripheral, so you'd hope that WICED could grab control of that without interfering with the existing STM32 code Espruino has." This is in fact a great example. There's a WICED/platform/MCU/STM32F4xx/peripherals­/platform_gpio.c file with the primitives to take control of a gpio pin. Stuff like platform_gpio_output_high is easy to translate or could even be left as-is. But then comes platform_gpio_irq_enable, which ties into FreeRTOS, and on top of that, since there's only one interrupt routine for gpio pins as far as I can tell, well, it just defines that. Looks like I can't really make progress without rolling up my sleeves and writing out all the shims I need, after I understand how Espruino handles all this and what FreeRTOS expects. Sigh.

Monologue continued... I think that what I need to do is grab FreeRTOS from source and integrate it into Espruino. Make that work on the EMW3165. Then continue and deal with LwIP and WWD. Trying to just glue the whole thing together isn't magically going to work...
To integrate FreeRTOS is similar to what we had to do with the esp8266:

• the main espruino loop needs to run in a low priority thread
  
• the timer stuff needs to run in a high priority thread and use FreeRTOS timers
  
• gpio interrupts need to work with FreeRTOS
  
• sleep modes need to work with FreeRTOS
  
• dunno whether something else uses interrupts...
  

dunno whether something else uses interrupts...

...that's why I liked the Superbrain on CP/M... one Z80 for main, a second one Z80 to handle all the IO chips, and the OS was 8..10KBytes (including BIOS) - mass storage address space 8+MB. Worst case, you could manually disassemble it... Parts of it I did to figure out why the two SIOs did just not do what they were supposed to do.

PS: This post is supposed to just break your monologue... which by the way is great to follow... enabling to live in a few seconds through hours of pains of yours... ;-) - and to share your 'sorrows'. If the tooling / toolchains would be a bit more easy to grasp (for me), I could contribute a little bit more then just distract you.

Thanks @allObjects! I think I'm gonna make something actually work on an esp8266 for a change...

I noticed that instruction prefetching from flash is disabled

Yes, its actually only quite a small speed improvement (I think it uses more power too?), and the noise it introduces makes the ADC pretty much unusable!

FreeRTOS

Honestly I find it difficult to say one way or the other as I really haven't had time to delve deep enough, but the more stuff I look at (not just EM3165), the more it seems that Espruino needs to be able to integrate nicely with other people's RTOSes, so any changes that make that easier would be good.

The whole peripheral thing could be a real pain though. If you're having to write your own jshardware.c then a lot of the benefits of having STM32 disappear :(

I'd be tempted to try and get Espruino running under FreeRTOS with no peripheral support, and to then see how much of stm32/jshardware.c can actually be used without it breaking. It should be quite a lot.

It's quite likely FreeRTOS puts the interrupt vector table into RAM, in which case you should just be able to overwrite the relevant IRQ handlers with Espruino ones as you need them...