It seems that creating a .htaccess file (I did not have one in my wp-admin directory) did not solve the issue.
This is what I’m getting when I hit the shortcode button. I’m not sure what it is.