Avatar for jtq

jtq

Member since Mar 2017 • Last active Mar 2017
  • 2 conversations
  • 6 comments

Most recent activity

  • in Pico / Wifi / Original Espruino
    Avatar for jtq

    Hi @Gordon - thanks for your pointers.

    To answer your questions:

    1. Both my Picos are running v1.91 of the firmware (I believe the latest version as of posting)
    2. Pressing Ctrl-C does not break out of the unresponsive state, and even clicking to disconnect the Web IDE doesn't work (or at least, takes so long to respond that it appears to be unresponsive). The only thing I've found that works is power-cycling the Pico, at which point the Web IDE auto-disconnects. When the Pico is unresponsive the right hand side of the web IDE works as normal, but the left hand side doesn't register any keypresses, and obviously flashing the chip/clicking disconnect does nothing.

    I took on board what you said about implicit redeclarations/copies of variables (dammit - too long writing JS and too many years since I wrote any C...), so I tried replacing the single concatenation-plus-print line with two separate prints instead... and suddenly the sending Pico seems to be pretty consistently completing the transfer without crashing, so perhaps my problem was indeed a memory issue caused by marshalling the data in memory, rather than a problem punting the data through USART as I suspected (now my receiving Pico's crashing instead, but I suspect it's probably a similar RAM-based problem on that end, too - needs further investigating).

    I didn't see any OOM error messages anywhere and the issue definitely wasn't handled "gracefully" in the REPL/Web IDE, so I'll keep investigating with my old (crashy) code, and I'll see if I can produce a minimal test-case for you in the IDE that reliably crashes the Pico, even if it does just turn out to be an OOM problem.

    Thanks for the pointer about E.getErrorFlags() - that and E.getSizeOf() look like a couple of amazingly useful additions to my debugging toolbox, so thanks again for pointing them out to me!

    I'll keep investigating given what you've told me, and I'll report back anything of interest I find in this thread. Thanks again!

  • in Pico / Wifi / Original Espruino
    Avatar for jtq

    ... Right. I think I've finally tracked down where the Serial buffer size is defined in the Espruino source code.

    I'm leaving this step-by-step trail of breadcrumbs here in case it helps anyone else to decode the Espruino source code, and to refresh my memory if I ever need to track down chip details like this again in the future.

    1. First, we know we're looking for something to do with the Serial1 port. Serial1 is an instance of the Serial class, and (as per the API documentation) that's a USART.
    2. So step two is to find the Espruino source code repo (thank god for open source!) and search it for USART.
    3. Hmmm - looks like there are a lot of different source code files, so let's look for one that sounds like it belongs to the chip we're using (in this case, the Espruino Pico). According to the Pico page the chip in the Pico is an STM32F401CDU6 32-bit 84MHz ARM Cortex M4 CPU.
    4. Ok... so scrolling down the list of possible hits in the github repo, "stm32f4xx" looks like a pretty good match for "STM32F401CDU6. Let's look in there.
    5. Hah - this looks like the header file for a C structure that represents a USART in memory. The beginning of the file sets up a struct (basically, "object" in JS-speak) that has a bunch of familiar-sounding fields like USART_BaudRate, USART_StopBits and USART_Parity that pretty obviously map to options/parameters used when setting up the Serial prot in the Espruino JS API.
    6. Jackpot! Towards the bottom of the file we find the definition for a function called USART_SendData: USART_SendData(USART_TypeDef* USARTx, uint16_t Data);. That sounds very much like the bit of code that sends data down a USART, so let's jump to the associated .c file (source code) for this .h (header) file.
    7. Find the USART_SendData function in this file and it's a short one that just seems to set a member variable called DR, belonging to a variable called USARTx, that's a pointer to a variable of type USART_TypeDef that's passed into the function.
    8. Ok, so let's back up and find where USART_SendData() is called from, so we can find where this parameter variable comes from.
    9. The first place the function is actually called (as opposed to being defined in a header file) is targets/stm32/stm32_it.c... and STM32 is a reassuring match for our chip (STM32F401CDU6), so let's look in there.
    10. Search that file for USART_SendData( and we find the following likely-looking code:

    /* If we have other data to send, send it */
        int c = jshGetCharToTransmit(device);
        if (c >= 0) {
          USART_SendData(USART, (uint16_t)c);
        }
    

    So this is basically saying "get a single character from the jshGetCharToTransmit() function, and if it's not 0 then send it out over the USART". That's very good - now we just need to find jshGetCharToTransmit() and try to work out where it gets characters from so we can try to work out how big that buffer actually is.
    11. src/jsdevices.c looks like a pretty good bet, so let's look in there (and search withing the file for jshGetCharToTransmit.
    12. The first hit is what we want. The code is a bit confusing, but if you ignore all the if(DEVICE_IS_UART(...)) stuff then a few lines below that we find unsigned char data = txBuffer[tempTail].data;, and that variable is returned from the function. So txBuffer is our buffer that holds all outgoing data to be sent over a USART (Serialn) connection.
    13. Right at the top of the file we find volatile TxBufferItem txBuffer[TXBUFFERMASK+1];, definine an array of TXBufferItems called txBuffer, of size... TXBUFFERMASK+1.
    14. Sigh... ok, so now let's find out where TXBUFFERMASK is defined. Nowhere in this file. awesome.
    15. At least it's only used in two files. We know it's not src/jsdevices.c because we just looked in that one, so it must be the other one, scripts/build_platform_config.py.
    16. Search in that file and we find (third result) codeOut("#define TXBUFFERMASK "+str(bufferSizeTX-1)+" // (max 255)"). This looks like some Python code that writes C (always fun reading code-that-writes-code), but if I'm reading this right it means that at compile-time TXBUFFERMASK is defined as one smaller than bufferSizeTX
    17. ... and bufferSizeTX is defined a few lines above as:

    if LINUX:
      bufferSizeIO = 256
      bufferSizeTX = 256
      bufferSizeTimer = 16
    else:
      bufferSizeIO = 64 if board.chip["ram"]<20 else 128
      bufferSizeTX = 32 if board.chip["ram"]<20 else 128
      bufferSizeTimer = 4 if board.chip["ram"]<20 else 16
    

    Phew!

    So long story short it looks like the size of the Serial output buffer is defined at compile time based on the RAM size of the chip.

    18. Tracking back up the file we find that board is imported via board = importlib.import_module(boardname);, and boardname is passed in as a command-line parameter to the build. Crap.

    19. Ahah - thank god for comments:

    Reads board information from boards/BOARDNAME.py and uses it to generate a header file

    which describes the available peripherals on the board

    So let's take a look in the boards folder and see if we can spot anything that looks like our chip.

    20. I don't know about you, but I have a good feeling about STM32F401CDISCOVERY.py for an STM32F401CDU6 chip, so let's look in there.

    21. Booyah! I spy a chip object with a ram member with a value of 64. Plugging that into our lookup table above, board.chip["ram"] (64) is certainly not <20, which means it's defined as 128.

    That makes our output buffer for serial data a surprisingly-small 128 unsigned chars big.

    That seems way too small for something that only starts choking on writes of a couple of K or more, so perhaps there's another buffer somewhere, further up the stack that I've missed.

  • in Pico / Wifi / Original Espruino
    Avatar for jtq

    Thanks @oesterle - some great suggestions there, particularly investigating Serial1.pipe(). I've veered away from writing my own solution so far, because this feels like something I'd expect to be handled internally by the Espruino runtime (at least some way to read the current state of the Serial buffer, so you know when it's safe to push more data into the connection)... but if there's nothing then I guess it's a straight choice between either creating a pull request into the Espruino runtime or write my own JS routines to manage it.

    The basic setup is pretty simple - just Serial1.setup(28800, { tx:B6, rx:B7 }) to set up the serial port, and

    var strRepresentation = JSON.stringify(obj);
    Serial1.print(strRepresentation + '\u0004');
    

    to send a data chunk down it. For reference, the data object I'm sending includes a few KB of base64 image and audio data, and going over a couple of KB total is about the point it blows up the connection and crashes the chip.

    Then the same setup line on the receiving side, and

    var buffer = '';
    Serial1.on('data', function(data) {
      buffer += data;  // De-chunk into buffer, and process whole buffer once EOT control character is received
    
      if(data.indexOf('\u0004') !== -1) { // ctrl+D - End of Transmission character
        // Process whole data object - JSON.parse(buffer), etc.
        buffer = '';  // And reset buffer
      }
    });
    

    to receive and process it.

    Edit: Ah- looks like Serial.pipe() won't really do it. It's the sending device that crashes due to an overflowed buffer(?), whereas it looks like Serial.pipe() only takes writeable streams (ie, it's another way to receive data from the USART, not a way to push data into it).

  • in Pico / Wifi / Original Espruino
    Avatar for jtq

    Background: I'm trying to transfer a stringified JSON object between two Picos (using Serial1, tx:B6, rx:B7 on each, shared GND, console shunted to USB, etc). It works for a couple of KB of data, but beyond a certain size of object the transfer appears to simply fail, leaving the sending Pico in an unresponsive state until it's power-cycled.

    I'm guessing that there's some sort of internal buffer on the Serial port for when JS code delivers data faster than it can be transmitted, but I'm having trouble finding any information on how big it is, or strategies for robustly working around this issue (eg, chunking the data and sending a chunk at a time using setTimout/setInterval is an obvious possibility, but how quickly/slowly should I queue each new chunk up, and/or how do I know when the previous chunk has finished transmitting)?

    Also, does anyone have any information on which baud rates the Pico supports for Serial connections? I'm experimenting to ease the problem by selecting higher baud rates to reduce the chance of buffer overrun, but it's a slow process when you have to connect/disconnect and re-flash two different Picos for every test...

    Basically is there anywhere I can look in more detail regarding the USART specs on the Pico? I've even tried digging into the Espruino source code on Github, but I couldn't find any info there either.

  • Avatar for jtq

    Which wires do you have connected? Just:

    3.3v -> B3
    B6 -> B7
    B7 -> B6

    Yes - that's all.

    Do you have ground connected between the two Picos as well?

    Ah - I don't, no. I'll give it a try - thanks!

    Edit: That seems to have fixed it - many thanks! Not knowing much about electronics that was seriously confusing the hell out of me - clearly I need to read up a lot more to get my head around more of these basic concepts. Thanks again!

  • Avatar for jtq

    Complete electronics/embedded newbie here, so apologies if I'm just missing something really obvious...

    I'm trying to set up a system where when I connect two Picos, one realises the connection has occurred and sends some data to the other over Serial1.

    The simple way to achieve this I've found is to have both Picos connected via a Serial1 connection (9600/tx:B6/rx:B7), and then connect the 3.3v pin of Pico A to pin B3 of Pico B, and just use setWatch() on Pico B to detect pin B3 being pulled high when they connect.

    This seems to work fine - every time I connect B's B3 to A's 3.3v the watcher triggers exactly once.

    The problem occurs when I try to print text over Serial1 as a result - suddenly Pico B's setWatch() is continually being triggered, as fast as the debounce setting will permit it.

    If I put a counter in and (for example) only output to Serial1 if(counter < 5), I get the expected number of outputs and no more, so obviously outputting to Serial1 isn't simply triggering the setWatch() event handler endlessly - it's doing it exactly once per write to Serial1... and then the event handler writes to Serial1, re-triggering the setWatch() event handler... etc, etc, etc.

    There's no obvious connection I can see between B3 and Serial1 on B6/B7 - can anyone explain what on earth I'm missing?

    Example code follows - B6/B7 each lead to the other pin on another Pico, and I trigger the behaviour by connecting that other Pico's 3.3v pin to B3 on this Pico:

    // Convenience function so I can tell when it's transmitting/listening
    function setTransmitting(transmitting) {
      if(transmitting) {
        digitalWrite(LED1, 1);  // Red = transmitting
        digitalWrite(LED2, 0);  // Green = listening
      }
      else {
        digitalWrite(LED1, 0);
        digitalWrite(LED2, 1);
      }
    }
    
    // Set everything up (so it can be easily called from onInit or the REPL as necessary)
    function setUp() {
    
      USB.setConsole();
    
      /**** Set up inputs ****/
    
      pinMode(B3, 'input_pulldown');  // Listen for connection (other Pico's 3.3v pin pulls B3 high on connection)
    
      /**** Set up outputs ****/
    
      Serial1.setup(9600, { tx:B6, rx:B7 });  // Serial data connection
      setTransmitting(false);                 // Display indicator
    
      /**** Behaviours ****/
    
      setWatch(function(e) {  // On connection, output data
        setTransmitting(true);
    
        Serial1.print('Hello!');
    
        setTransmitting(false);
      }, B3, { repeat: true, edge: 'rising', debounce:500 });  
    }
    
    E.on('init', function() {
      setUp();
    });
    

    TL;DR: Basically it seems as if sending data over Serial1 resets the setWatch() on pin B3, leading to it triggering the event handler even if B3 was already high before the Serial write occurred. Why would writing to Serial1 re-trigger a watch on pin B3?

    P.S. Apologies for the bad title - I only realised after posting that Serial1 was triggering the watch on B3 exactly once per write (and the event handler was then re-sending to Serial1 leading to an infinite loop), and not continually.

Actions