ESP32 based webserver on separate WiFi network

One of my current projects is to get the data from the sensors in my room and make it available over the internet. A cloud provider that I do not control is out of the question, because those services can be shut down at the whim of a third party, but I don't want an Espruino (or anything) exposed to the internet on my real home network.

My plan is thus:
Second router, second wifi network for IoT devices we don't fully trust the security of.
ESP32 on that network, which will serve up json containing data from the sensors
Arduino Pro Mini (8mhz 3.3v) connected to the ESP32's UART2. This will use a 433mhz RF receiver (RXB-14 or RXB-12) and output the received data over serial to the ESP32.
Sensors will broadcast current readings on 433mhz RF.
Since E.setSNTP() doesn't exist on ESP32, will get current time from my webserver on AWS ( drazzy.com/time.shtml ) and use that to set it's internal clock.

ESP32 will thus be able to serve up the current status of my room, as well as data to generate a graph of the past few days. The client pages will be served up from drazzy.com (or could be served from spencekonde.com - since it's personal stuff - or even located on local machine) and will use standard libraries to generate the graphs from it.

The same can also be done in reverse, with the ESP32 telling the Arduino to transmit commands based on received requests - the 433mhz is the "bridge" between the real home network, and the IOT-device one.

The next steps are to do the code that will store and serve the information on the ESP32, and finish the build of the new sensor and control hub that will live on the "real" side of the network.

Likely I will use an ESP32 board with the PSRAM once those are supported for JSVars in order to have more memory to store historical data, so the charts are richer. I'm unsure what, if anything, I'll do for permanent storage. Maybe FRAM. Maybe a uSD card... FRAM prices are way down lately, btw - 1mbit chips for under $7 now!

For the sensor controller (the one on the "real" network) I need to get an Espruino build environment set up for STM32 boards and make a copy of the Espruino Board definition, only for the D-spec part. I think I'll start a new thread about that.

That sounds good - so how will you get that data online? The ESP32 will just POST it to drazzy.com, or drazzy.com will request it direct from your home IP address?

I know you like the 433Mhz stuff but you could take a look at nRF52... The modules are small, cheap, and have radio on them. Also the ESP32 can now grab the BLE advertising data, so with a few of them as receivers you could get good range.

Have you seen these guys too? https://www.silabs.com/products/wireless/multiprotocol-connectivity - there's a port of Espruino to the EFM32 so it should be pretty easy to get it on those chips - and they have all the radio stuff for 433MHz on-chip!

Likely I will use an ESP32 board with the PSRAM once those are supported for JSVars in order to have more memory to store historical data

The esp32 build has 1mb devoted to a flash file system. That should be plenty of space to log to, with no extra hardware. The normal fs module and file objects work as if it is an sd card. If you buffer to ram, and then write 4096k chunks, it should be pretty easy on the flash.

The ESP32 will serve data directly from it's own IP - that's why it needs a separate network. If it was just making outgoing connections, I could use the real network.

I was totally overwhelmed by the amount of learning that would be required to figure out how to store the data on drazzy.com, and I talked to multiple friends who are experienced with setting up database servers, but they all told me different things, so I just threw my hands up in the air and decided to put the logic on the ESP32 because I know how to do that approach, and it's fun to write javascript, but unpleasant to try to figure out how to set up a database server - and then there's security, since I am not qualified to assess whether the database server is secure, I would need to have it isolated on another system, not drazzy.com, or find and hire someone who could assess it as such.

These are all shitty reasons, but I had literally spent over a year with multiple projects blocked because I wasn't able to make any progress without being overwhelmed by options, doubt, and confusion. So I was like - I need to actually do this, and stop letting the obsession with the "right way" prevent me from ever making progress, and use the techniques I know to make a working solution.

I already have the RF code written, so I'm not going to switch to the NRF or EFM boards. I am super busy and struggling to find time to work on projects as is. I have working RF code for arduino and can make a serial<->RF bridge for $3 by plugging an off the shelf receiver and/or transmitter into an off the shelf pro mini and uploading a sketch I've already written to it. Reinventing the wheel is a non-starter - especially since I have over a dozen devices already deployed and in use based on my current method.

Flash filesystem is a possibility - but if I'm actually writing it permanently, I might as well use an SD card so I can easily get the data onto a computer en masse.

:) fair enough - yes, it's quite a steep learning curve if you want to set a whole system up yourself - and it's always hard to know if it is properly secure.

If you're serving up from ESP32 you could look at the TinyDash thing I did. It could do with some work but on the whole if you just want a few graphs it's nice and easy.