Search results for:

Not that it will help you with github, but... I use the IDE named 'QTCreator' to work with the code. Then, I have the ability to easily search through all files in the project and find definitions, etc. Maybe some kind of ide is what you need?

@Gordon,
Thank you.
No worries. Each entry could have it's absolute position and then the sizes of the previous and current lines before the key and data. Just a little math to get to the absolute position for the beginning of each one.

So it could still be done. Awesome.

@Gordon nice!
I wasn't actually attempting to push for that feature, just noting what we could do with it. However, since we will have it, is it based on relative or absolute positions?

If we had the ability to do a relative seek in a file, or the ability to move to an absolute position (forward and backward) in the file, then we could build a relatively unlimited (from the perspective of memory) key value store (or simple db). It's easier if it's relative, though.

Basically, you store in a file(for each entry):
Offset to the beginning of the next entry, offset to the beginning of the previous entry, key, data. (Or, size of this entry, size of previous entry, key, data. It's the same thing when relative)
It could be line delimited, or not.
In any case, you could easily binary search the database for your data, assuming you sort it at insertion time.
Then, when that is done, you only keep the data you want in memory and leave the rest saved to disk.

It might not be as fast as an in memory array, but with a 32GB sd card, you could store and search much more than the memory available.

From what I remember, the fs_kill only deals with the stream handles... unless that has changed. While that would absolutely need to be done, so that streams are handled correctly, the readfile code is a "one shot" open, read, close. Does reset() call wrap_fat_kill and jswrap_fs_kill? Then, once the card is inserted again, you would have to call jsfsInit to get it back into operation.

after code inspection

At least, that is what it looks like from code inspection based on what is in github.
So, here is the test case, it looks like reset will kill all handles and unmount the card when it is pulled out. The second thing is to perform a reset after inserting the card... the reason is that reset also does initialization. Since the system does not auto detect insert and removal, you have to do it twice.

If this works, we would need E.mount to go with E.unmount in order to take full advantage of the hot swap as we are discussing. It looks like it should auto init on read after being unmounted, but I am wondering if that is not working for some reason (as in we are not doing enough to initialize the device, clearly we can initialize on startup).

Unless things have changed, and I didn't look to see if that was the case or not, the readfile function in fs was a simple open, read, close operation. It used to do an fs init call the first time it was executed, or really the first time anything was executed.

From what I know of how it did work, I am wondering if there needs to be an additional flag added for state change of the card.

It would be something like:
On idle, check the card detect pin
If the card was present and now isnt, set the card detected flag to false and the state changed flag to true.
If the card was not present and now is, set the card detected flag to true and the state changed flag to true.

If the state changed is true and the card is not detected, call fs kill and destroy all handles.
If the state changed is true and the card is detected, call fs init so that the filesystem is valid again.

If this is done during idle, you should have the ability to swap out cards without trouble.....it's a guess anyway, from knowledge of a few versions ago.

Well, you could use the file stream api to fetch and write each file one at a time in a loop. It wouldn't matter how big the file was, but it might take a long time.

@Gordon, @rm:

I think it might depend on the eventual use of the library. If the goal is to have something that you could link with some other program and have it execute a string containing JavaScript, it may be possible.

You can look at targets/linux/main.c: line 276. This is the location where the interpreter executes a file of JavaScript. You could take what you need from this file/location and create an interface for running an arbitrary string of JS.
However, all it would do is execute the JS.. you would still need to find a way to get data back out to the running program if you want any feedback from the JS.

I am not saying it is the cleanest way to do it, just that it is a way to do it.

@Loop,

You will want to use the arm-none-eabi-gcc version 4.7.3 (Carlson-Minot Mac OSX Intelx86 2013.05-23 Lite) from Code_Sourcery_2013.5-23. It produces the smallest code. I just took the binaries and placed them on my system and in my path.
Also, I don't know what IDE you use(or if you use one), but I looked high and low and the best one I could find was QT Creator (on osx), all of the others had issues debugging or were incredibly slow. You can find instructions online for how to import projects with existing code. It takes a bit of fiddling, but once you get used to it, the way they setup different configurations is kinds natural... just watch out if you edit the makefile...you have to do it in VIM mode to keep tabs in the correct places.

You will want to install gdu from coreutils.. I got it from macports. It is needed to check the size of your compiled binary to make sure it will fit inside of the device. Linux uses the du command for this, but the osx version doesn't have the correct options, that is why gdu is necessary.

From memory, I think thats all there is...then the code compiles (with my makefile changes) and you can run the target to write it to the device.

@Gordon,
Re: guys on github,
I think there must be a line drawn somewhere. If people begin demanding what they think is best, without talking to the project owner about it, then the direction of the project gets muddled. You and I have had many conversations about these changes, so I thought my original implementation was slotting in with your code.

The difference came in the way it was handled and the manner of the discussion. I won't be shy about it, though, if they did represent espruino, I was not going to contribute anything more to the project. That's the danger of having people who can influence change in your project without your consent. They have the ability to change the perception, right or wrong, of the project in general.

That is what I was working toward. Right now, all I have is a beginning, and only with files.

I can't even express the level of relief I felt when I found out that they didn't represent Gordon and Espruino officially. Sorry for the late update to my previous statement, at the time, I was under the false impression that they did.

Sorry guys.. I tried to push my changes in, but philosophical differences got in the way.
To those of you looking for block reads/writes, I am sure they will provide one that implements the node API eventually (well, some of it). Mine was apparently too different for what they wanted. (Basically, I am saying don't wait on my changes, they wont be arriving.)

I have an alpha version of the api complete and sent it to Gordon. It doesn't support async piping of data yet, but it does do reading and writing on open files in an indexed way. You can specify the number of bytes to read and the position to start reading and writing.

The async pipe code is currently done on my computer, but I need to merge down the latest code and test before I send a pull request.

Gordon,
No worries. I have been looking over their FS API and I am planning to make it the same.
The approach I was going to take with the FILE structure is very similar to how the graphics class works. I will try to mimic the style that you use everywhere else so that the code is uniform. :-)

I am currently working on an implementation of a Stream like interface. Now that I have the ability to build and flash my own Espruino image, I don't think it will take long.
My need is for streaming MP3 encoded sound data over a serial port to a card that will decode and play them. I also need streaming for sending file data over HTTP to browser clients. It seems like the same interface would work for you as well.

There was a moment, when I was trying to flash my Espruino, that I thought I had bricked it. I had flashed the board and the firmware did not work. The lights would not turn on and the serial port would not initialize. Then I tried to set it into bootloader mode, but I found that doing so does not initialize the serial either, so I thought I was out of luck for reflashing my board.

In the end, I held down button 1 and reset while unplugging and plugging in my board. Once the board was plugged in again, I released reset which put the board into BootLoader mode again, but this time the serial port was initialize and I could reflash it with the current good version of the firmware.

Maybe a similar method will re-initialize the usb port and allow you to flash the firmware on your board to get it back into working state (yes, I know that you didn't flash the board initially, but it might not hurt to do so as a means of trying to fix it).

Just an idea.

OK. I finally figured out how everything has to work and I got a new version of the Espruino software on the v1.3 device from a compile on OSX.

Here is what I did:
1) Get the prebuild OSX GNU ARM Tools 4.8 from here: https://launchpad.net/gcc-arm-embedded. I updated my patch to place the binaries at the front, you could also update the makefile as described in other tutorials. How you do that is up to you.

2) Install CoreUtils from macports or something. I issued the command sudo port install coreutils to get it. You will need it for 'gdu' to get the size of the resulting binary. 'du' on OSX does not have a '-b' option to get the size of a file in bytes. Installing coreutils will get you a binary named gdu that does have that option (the name was changed to not conflict with system binaries).

3) The file named scripts/check_size.sh will need to be modified. Simply change du to gdu to get it working, then save the file.

4) -Wl,--gc-sections: There is a variable named MACOSX in the Makefile. This variable is set to '1' if we are building on Darwin. One of the consequences is that -Wl,--gc-sections is not applied if we are cross compiling. This is the source of the problems when compiling on OSX. The spirit seems to be to detect if we are building native on OSX and do something special for it. To Fix this issue, do the following:
remove:

ifeq ($(shell uname),Darwin)
MACOSX=1
endif

from the top and merge it with the ifndef located in the linux section so that it looks like this:

else
BOARD=LINUX
LINUX=1
USE_FILESYSTEM=1
USB=1
USE_GRAPHICS=1
ifeq ($(shell uname),Darwin)
MACOSX=1
else
USE_LCD_SDL=1
# http libs need some tweaks before net can compile
USE_NET=1
endif
endif

Note that I moved USE_LCD_SDL into the else section above. Without it, native builds fail. It happens on OSX and I noticed that it happend on the build server. At least for me, that is because I don't have libSDL installed, but I probably wouldn't be using it when on my OSX box anyway.

5) plug-in your board to the computer and find out the serial port name. I use the WebIDE for that, just hit refresh if it doesn't show up right away. Then update the Makefile and find the location that says something like:
STM32LOADER_FLAGS+=-k -p /dev/ttyPortName

I just search for the word STM32LOADER_FLAGS then replace the /dev/portname with the name of the port where your device is located. Mine was /dev/tty.usbmodemfa121.

6) In the Espruino directory, issue the command ESPRUINO_1V3=1 Release=1 make to build the software for the Espruino boards. After the build completes, you should see something like this:

LD espruino_1v52_espruino_1r3.elf
GEN espruino_1v52_espruino_1r3.lst
GEN espruino_1v52_espruino_1r3.bin
bash scripts/check_size.sh espruino_1v51_espruino_1r3.bin
PASS - size of 214504 is under 215040 bytes

7) Now to flash your board, issue the following command at the command prompt: ESPRUINO_1V3=1 Release=1 make serialflash

Your board will get updated and the next time you connect you will be running on new software.

As it turns out, adding -Wl,--gc-sections to the LDFLAGS, without -lnosys added to the libs, also eliminates the compiler errors. Plus, the size drops down to 214504. Unfortunately, flashing the resulting binary does not work, in either situation.

Gordon,
What command do I need to use to build a usable image for the Espruino 1.3 boards? I read somewhere that you need to merge the bootloader into the image or something?
Also, I took a look at the build log with the automated builds of the binaries. It looks like there is some other option set that you are building, I see 2 sets of builds in the log and one of them seems to build the bootloader. Can you explain that for me, please?

As an additional point of information, I added the following line to the section of the makefile inside else ifdef ESPRUINO_1V3 :

LIBS+=-lnosys

Then I switched to the GNU Arm Tools v4.8 compiler, and now it compiles without error. The resulting size of the bin file is: 228872.
Anyone know if this is correct or why I need to do this?

If I had the version used internally, I am sure this would work fine. According to these instructions , and based on the errors I am getting, there are some stubs needed to make this work.

It is entirely possible that I don't have something configured correctly in my makefile, but I have attempted to follow all of the instructions on the tutorial for building the code. It works fine as a native osx build, but not cross.

All of the gcc-arm-none-eabi versions I have tried from 4.6.2 to 4.8.1 don't work or at least not with the makefile that I have from github. I have added the GNU Arm Tools version 4.7.4 to my list of non-working versions at present.

I can add these additional stubs and make it work, I am just wondering if I need to, or if this is an indication of a code/configuration problem. It should be noted that I have not altered the base code in any way yet. Straight from github it wont build on osx with those compilers or this makefile.

Has anyone succeeded in building a version of the Espruino software as-is from github for their version 1.3 device?
If so, what compiler and version did you use?

mine are all arm-none-eabi, or at least that is what the name of the files say. For instance, I have arm-none-eabi-gcc-4.6.2, arm-none-eabi-gcc-4.7.2 and arm-none-eabi-gcc-4.8.1 in my path.

So, I have been trying to build Espruino on OSX and I am having an issue. When I build it native (for OSX) everything works fine. However, when I build it using the cross tools, I get one of a number of errors.

I have tried 2 different versions of YAGARTO (4.6.2 and 4.7.2), Code Sourcery 2013.11-24 and GNU Arm tools 4.8.
All of them give me a different problem.

YAGARTO and Code Sourcery say:
_sbrk is undefined at link time.
YAGARTO also says _exit is undefined.
The GNU Arm tools say that they cannot find crt0.o.

I am thinking that I must be using tools that are too new.
What version of tools are you using to build the arm binary?

That's the one. I have been looking over the source in prep for doing something like this.

I will have to see how it turns out. I saw an open enhancement somewhere (github?) that suggested opening a file as a stream so that we don't need to read the entire file into memory in order to read the contents of the file. I might have to do that in order to get the data moved around.