Is the last selected riding mode saved after the system is turned off?

Yes, the system makes a note of the most recently selected riding mode.
On speed eBikes, however, the mode is set to "Off" when turned on.