Modal AMI - Refused to connect

I’m testing an action menu item that will open a modal panel to a secure web application. The POST request works and opens a new tab if I disable the modal option but with the modal option checked it displays:

Any ideas what might be wrong? Are there error logs for stuff like this that might point me in the right direction?