Update: This problem is solved. Buttons are displaying correctly in IE and Firefox.
With IE, not sure why this is. As far as I can tell it just happened by itself.
To fix it in Firefox, I disabled all my plugins and then re-enabled one by one. The culprit is HTTPS-Everywhere 3.1.4. With this plugin disabled buttons display correctly — so obviously there’s a conflict with this plugin.