DISK_ERR persists following re-insertion of microSD card

This is a follow on from my previous discussion concerning the microSD card.

With all my bits and pieces connected up and working, when I run the following command, assuming a text file called test.txt is already on the microSD card which contains the word hello followed by an empty line:

>var f = require("fs").readFile("test.txt");
="hello\n\n"

If, while my system is running, I remove the microSD card and re-run the above I get the following response:

>var f = require("fs").readFile("test.txt");
ERROR: Unable to read file : DISK_ERR
=undefined

replacing the card and re-running the same command still produces the DISK_ERR.

My questions:

• Is this expected behaviour?
  
• Should I implement error checking to prevent getting this error upon re-inserting the microSD card while the system is live?
  
• If so, any suggestions how?
  
  

Temporary solution
The only way I can get the read to work again is to reset everything.

I've seen similar behavior on the Espruino board. I think something needs to be reset, but the filesystem code we have just doesn't do so yet.

Does a soft reset with reset() fix it? It seems like it should.

As there's no disconnect pin on the Espruino board, it doesn't know if the card has been pulled out, so it doesn't unmount/remount it. Does the error happen if you pull the card out and re-insert it, without doing any file operations when it is removed? I thought that it worked in that case.

I think right now your only solution is to call reset() when you start having issues... Or as you have it compiling, you could fix the underlying problem - probably by forcing an unmount/remount if there was an error when opening a file.

If the error only happens if you read when the card is removed, I'd just explicitly check the value of the card detect pin, and not do anything when it's disconnected.

Unfortunately a soft reset doesn't do the trick @Gordon. This is what happens:

Case 1:

// fresh restart
 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v70 Copyright 2014 G.Williams

>var f = require("fs").readFile("test.txt");
="hello\n\n"

// remove card

>var f = require("fs").readFile("test.txt");
ERROR: Unable to read file : DISK_ERR
=undefined

// re-insert card

>var f = require("fs").readFile("test.txt");
ERROR: Unable to read file : DISK_ERR
=undefined

>reset();
=undefined

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v70 Copyright 2014 G.Williams

>var f = require("fs").readFile("test.txt");
ERROR: Unable to mount SD card : NOT_READY
=undefined

However if I hit the reset button on the board or cycle the power then everything comes back to life.

Case 2:

Does the error happen if you pull the card out and re-insert it, without doing any file operations when it is removed?

Yes, the DISK_ERR also occurs in this case:

>var f = require("fs").readFile("test.txt");
="hello\n\n"

//Removed card and re-inserted without file operation

>var f = require("fs").readFile("test.txt");
ERROR: Unable to read file : DISK_ERR
=undefined

Case 3:

I thought out another situation that could occur:

• Fresh restart.
  
• Card removed and re-inserted.
  
• Attempt to read from card.
  
  

This still works. So it seems a problem arises only once you have already performed a previous card read/write and then remove/re-insert the card, with or without doing any file operations while removed.

If the error only happens if you read when the card is removed, I'd just explicitly check the value of the card detect pin, and not do anything when it's disconnected.

Yes, that is what I thought I'd do. Hardware wise however, as you pointed out, this isn't possible with the Espruino board as the card detect pin is not wired up. Also, it seems from the second case illustrated above, that if you have already performed a file operation, then remove and re-insert the card without any further file operations (blocked by not detecting a card) and then attempt a file operation post re-insertion, you still have a problem.

think something needs to be reset, but the filesystem code we have just doesn't do so yet.

Yes, this is my suspicion too @DrAzzy. It probably has to be looked at in the C code. Not exactly my strength...yet :).

I would have it a guess that upon detecting the card removal (assuming this hardware functionality is available), that the jswrap_fs_kill() function in jswrap_fs.c should be called.

I don't suppose it absolutely must happen in the C code. The main thing is the handle to the FATFS needs to be renewed...I think :)

Unless things have changed, and I didn't look to see if that was the case or not, the readfile function in fs was a simple open, read, close operation. It used to do an fs init call the first time it was executed, or really the first time anything was executed.

From what I know of how it did work, I am wondering if there needs to be an additional flag added for state change of the card.

It would be something like:
On idle, check the card detect pin
If the card was present and now isnt, set the card detected flag to false and the state changed flag to true.
If the card was not present and now is, set the card detected flag to true and the state changed flag to true.

If the state changed is true and the card is not detected, call fs kill and destroy all handles.
If the state changed is true and the card is detected, call fs init so that the filesystem is valid again.

If this is done during idle, you should have the ability to swap out cards without trouble.....it's a guess anyway, from knowledge of a few versions ago.

As there isn't a card detect pin on some boards, we could add an unmount() function that effectively called jswrap_fs_kill.

The problem is that jswrap_fs_kill is actually called from reset(), so if that doesn't solve the problem then there's something else...

A function to reset the fs module would be handy, even if it was manually called.

Just to check, I just tried 'Case 1' here on the Espruino Board and it works fine - so I'm pretty surprised that yours is different as it's using the same code.

I guess it may be possible that you didn't do a make clean before building with the filesystem enabled, and that meant that somehow reset() didn't reinitialise the SD card?

Having just looked into this a bit more, I could detect the specific DISK_ERR and automatically unmount the disk, but this causes some real problems because if you open the same file that you had open before, the filesystem doesn't need to access the disk. That means that you'll get all the way to trying to read the file before it fails.

I'll add an E.unmount() command to explicitly unmount the disk - and if you plan on pulling out the SD card while Espruino is running, you can call it after every filesystem command and it'll mean you can then remove the SD card afterwards.

Ok, if you pull from GitHub you'll now have function called E.unmountSD(). You could call that on a setWatch of the disconnect pin, but if you are developing something where the user could remove the card at any moment I'd suggest you just call it every time you use the SD card.

Not to nitpick, but, why not call it fs.unmount()? That way everything related to the filesystem would be in the same place.

@Gordon thanks for this addition. I cloned the latest from Git, compiled it and am testing it out. This is what I see:

Case 1.1
A small change to case 1. Instead of DISK_ERR errors I'm getting NOT_READY errors

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v70 Copyright 2014 G.Williams

>var f = require("fs").readFile("test.txt");
="hello\n\n"

>E.unmountSD();
=undefined

// removed card

>var f = require("fs").readFile("test.txt");
ERROR: Unable to mount SD card : NOT_READY
=undefined

// re-inserted card

>var f = require("fs").readFile("test.txt");
ERROR: Unable to mount SD card : NOT_READY
=undefined

>reset();
=undefined

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v70 Copyright 2014 G.Williams

>var f = require("fs").readFile("test.txt");
ERROR: Unable to mount SD card : NOT_READY
=undefined

Case 2.1
A slight change to case 2. This time a NOT_READY error instead of a DISK_ERR error.

// fresh restart

>var f = require("fs").readFile("test.txt");
="hello\n\n"

>E.unmountSD();
=undefined

//Removed card and re-inserted without file operation

>var f = require("fs").readFile("test.txt");
ERROR: Unable to mount SD card : NOT_READY
=undefined

Conclusion

Just to check, I just tried 'Case 1' here on the Espruino Board and it works fine - so I'm pretty surprised that yours is different as it's using the same code.

I guess it may be possible that you didn't do a make clean before building with the filesystem enabled, and that meant that somehow reset() didn't reinitialise the SD card?

I did forget to do make clean last time but not this time. For me calling reset() still isn't working

The problem is that jswrap_fs_kill is actually called from reset(), so if that doesn't solve the problem then there's something else...

I think there has to be something else. Using the very latest Github version, calling E.unmountSD() has only managed to change DISK_ERR errors into NOT_READY errors.

From what I remember, the fs_kill only deals with the stream handles... unless that has changed. While that would absolutely need to be done, so that streams are handled correctly, the readfile code is a "one shot" open, read, close. Does reset() call wrap_fat_kill and jswrap_fs_kill? Then, once the card is inserted again, you would have to call jsfsInit to get it back into operation.

after code inspection

At least, that is what it looks like from code inspection based on what is in github.
So, here is the test case, it looks like reset will kill all handles and unmount the card when it is pulled out. The second thing is to perform a reset after inserting the card... the reason is that reset also does initialization. Since the system does not auto detect insert and removal, you have to do it twice.

If this works, we would need E.mount to go with E.unmount in order to take full advantage of the hot swap as we are discussing. It looks like it should auto init on read after being unmounted, but I am wondering if that is not working for some reason (as in we are not doing enough to initialize the device, clearly we can initialize on startup).

I actually tested both of those cases on the Espruino board before committing and they worked fine. I tested again just now and they're still fine, so I'm not sure what's different with your setup.

Are you sure you actually removed the card totally, and that you re-inserted it properly? Could there be a dodgy connection to the add-in board?

Not sure what is going on but there must be something more at play here. I have also just tested all the above cases again, making sure to remove the card completely from the slot, and still no joy :(

Out of desperation I even removed and replaced the whole microSD click board from the mikroBUS slot while powered up and I still got the same errors!

The only difference I can think of is the card detect (CD) pin going low upon card extraction and going high upon re-insertion. I wonder whether that change isn't causing some confusion in the code somewhere?

I'm off visiting a friend for the next few days so my involvement here could be a bit sporadic.

As far as I can see from the code the card detect pin isn't used anywhere, so I doubt that's the issue...

I'm afraid I can't afford to spend any more time on this myself though... If you want an SD card that works, there's always the Espruino board :)