Bug of Espruino IDE (not supporting UTF-8 comments)

yes, that will be the solution.
Thanks

utf-8 can be broken on many layers, I just tried in WEbIDE with czech letters.
print(E.toArrayBuffer("ěščřýá"));
typing this directly in left side console produce '?' and it really stores characters ?

>print(E.toArrayBuffer("?????ýá"))
new Uint8Array([63, 63, 63, 63, 63, 253, 225]).buffer

putting this to right side editor and uploading is better and produces this, also when typing cursor up it can be sen it is sent via \u unicode encoding

new Uint8Array([196, 155, 197, 161, 196, 141, 197, 153, 253, 225]).buffer
>
>print("\u00c4\u009b\u00c5\u00a1\u00c4\u­008d\u00c5\u0099\u00fd\u00e1");
ěščřýá
=undefined

so now this looks like utf-8 encoded except last two letters ýá which are more common and not specific to czech, correct encoding is maybe prined here https://mothereff.in/utf-8#%C4%9B%C5%A1%­C4%8D%C5%99%C3%BD%C3%A1

so most probably it is broken already when uploading the source code.

however when now tried to upload this print(E.toArrayBuffer("őýá")); the ő looks utf-8 encoded

new Uint8Array([197, 145, 253, 225]).buffer
>
>print(E.toArrayBuffer("\u00c5\u0091\u00­fd\u00e1"));
new Uint8Array([197, 145, 253, 225]).buffer

so maybe 'correct' letters in that web page show because they are not utf-8 encoded and are actually also not shown as utf-8

you maybe right. it occurred to me that perhaps the character encoding of the operating system could also cause this error. utf-8 character encoding seems good, but it is actually saved incorrectly in the source code. under windows, you can set the BOM in the text editor, which, if I know it, saves the block needed for utf-8/16 encoding in the given text file. this option i cant find for ubuntu gedit.

https://www.w3.org/International/questio­ns/qa-byte-order-mark

Do you know what is the maximum length of storable/runnable javascript code in Espruino wifi?

Sun 2021.04.25

'what is the maximum length of storable/runnable javascript code in Espruino wifi?'

Use process.memory() to determine the amount of free JsVars

http://www.espruino.com/Reference#l_proc­ess_memory
JsVars:    https://www.espruino.com/Internals

Although it may be possible to use some fancy techinques to utilize flash in addition to available ROM, typically, Espruino needs as much memory as the code file size being uploaded, in order to unpack and reorganize in memory.

Really rough approximation; take the number of free JsVars and divide by two to account for the need to reorganize during the upload process, then multiply by 16 bytes per JsVar.

I don't have a WiFi handy, but if memory serves, an authentic Espruino Wifi provides around 2200 JsVars.

2200 / 2 * 16 = 17,600 chars rough approximation

Do you know what is the maximum length of storable/runnable javascript code in Espruino wifi?

require("Storage").getFree() -this is space when saving code to flash. as for html you can even upload parts of your pages as separate files to Storage and then use Storage.read("filename") in place of the strings, maybe the utf-8 encoding could be better with that.

i just thought editor minimizing while downloading javascript code, so i can write my code beautifulized with tabulators and newlines. no needed to minimize manually to save flash memory. is it correct?

This solution seems to be better than javascript inline html code. i tried to save the html code to the sd card and read it back to a variable while initializing uc but i don't want my html firmware to be public.

Hi! I'm surprised about the CPU temperature - is anything connected that might be causing a short? The Espruino WiFi does run quite hot when connected to WiFi, but normally I'd expect temperatures in the ~40 degrees C range. Above 50 is getting a bit high.

The whole UTF-8 thing in Espruino is a bit of a pain. It stems from some code Amperka added to better handle encoding Russian characters, but I've had to tweak it a few times and honestly it just seems to make matters worse in most cases I have come across. As @MaBe mentioned, encoding non-ASCII characters in HTML &...; notation is probably the easiest solution (or uploading the files direct to Storage).

Mon 2021.04.26

ref #32: 'so i can write my code beautifulized . . . no needed to minimize manually . . . is it correct?'

I had this same question over four years ago and several provided responses to clear things up.

Originally, I had mistaken an observation (noobie incorrect interpretation) for changes that had just occured with the new WebIDE ver ?? around Nov 2016 circa 1V88 Espruino, and was attempting to resolve.

Just released (then) the WebIDE does a bit of housekeeping before upload along with additional cleanup once Espruino takes over. See @Gordon 's comments:

'and also strips out whitespace and comments between commands, which would have been ignored by Espruino anyway.'

http://forum.espruino.com/conversations/­295641/

'if something happens and data is re-arranged in memory then even that can effect the compression'

http://forum.espruino.com/comments/13488­204/

A quick test would be to upload some heavily commented code both inside and outside functions, then upload. Use dump() to see how comments are removed. Then use process.memory() to determine the amount of free JsVars to compare with the source.

Another useful tool: Many helpful ideas on using trace() to view actual memory contents.

http://forum.espruino.com/comments/15227­580/

I don't know exactly how big my final code will be, but I solved the html processing (including the UTF-8 problem) by reading my html code directly from an sd card. Interestingly, the handling of the SH1106 oled (128x64) is handled very well by Espruino compared to e.g. the Raspberry Pi Pico micropython environment (but maybe I was the only one who was lame).
Question to the software developers, is there a Raspberry Pi Pico version of the interpreter in the pipeline?

Question to the software developers, is there a Raspberry Pi Pico version of the interpreter in the pipeline?

http://forum.espruino.com/conversations/­359042/#comment15855632