Sorry for taking this long to reply.
I've done a clean installation of the latest version (0.95, 64bit) on another PC, and it works as I'd expect it to - the popups show up, and stay on the screen for as long as it's defined in the settings. So, instead of figuring out what went wrong, I decided to reinstall it on this PC, and manually set every option. It took me a while, but everything works fine now.