Thanks for all the suggestions. I think I've tried them all already.
I've tried all the button combos on the getting started page, and nothing.
After I noticed the tag that says about correct orientation of the charger I made sure it was always the right way round.
I'll try leaving it plugged in again and see if it eventually comes back to life. Maybe it needs more time to get to a state where it can turn on at all. There's just no indication it's doing anything (no charging light etc.), but I'm also not sure there's supposed to be a charging light/animation?
I'd guess the battery died so quickly because something was running still. I think I left it on the clock (screen off) which I assumed was the idle state, but thinking about it that is probably still executing JS. That part I'm not worried about, it was more for context in case it helped.