SSL error when uploading version through API

[chris]

Hi,

we would like to upload movies or images through the API directly to Shotgun. We have no problem doing any operations but there is an issue with the upload, giving this error :

Traceback (most recent call last):
File “\tools\shotgun\sgprod.py”, line 104, in
session.upload(“Version”, version[‘id’], img_file, “sg_uploaded_movie”)
File “\tools\python\site-packages\shotgun_api3\shotgun.py”, line 2343, in upload
tag_list, is_thumbnail)
File “\tools\python\site-packages\shotgun_api3\shotgun.py”, line 2379, in _upload_to_storage
self._upload_file_to_storage(path, upload_info[“upload_url”])
File “\hydro\tools\python\site-packages\shotgun_api3\shotgun.py”, line 3794, in _upload_file_to_storage
self._upload_data_to_storage(fd, content_type, file_size, storage_url)
File “\hydro\tools\python\site-packages\shotgun_api3\shotgun.py”, line 3886, in _upload_data_to_storage
result = opener.open(request)
File “C:\Python27\lib\urllib2.py”, line 429, in open
response = self._open(req, data)
File “C:\Python27\lib\urllib2.py”, line 447, in _open
‘_open’, req)
File “C:\Python27\lib\urllib2.py”, line 407, in _call_chain
result = func(*args)
File “C:\Python27\lib\urllib2.py”, line 1241, in https_open
context=self._context)
File “C:\Python27\lib\urllib2.py”, line 1198, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>

We did read through some older messages we could find online, some solutions from Shotgun website are not accessible anymore, we tried the other suggestion we could find or think of such as :

• using Python 2.7.16
• updated to latest shotgun_api3 package
• added the three following lines in our code (which gave an other error written below)

import ssl
import functools
partialcreateHttps = functools.partial(ssl._create_default_https_context, cafile=‘C:\tmp\cert.pem’, capath=‘C:\tmp\certs’)
ssl._create_default_https_context = partialcreateHttps

Traceback (most recent call last):
File “\tools\shotgun\sgprod.py”, line 104, in
session.upload(“Version”, version[‘id’], img_file, “sg_uploaded_movie”)
File “\tools\python\site-packages\shotgun_api3\shotgun.py”, line 2343, in upload
tag_list, is_thumbnail)
File “\tools\python\site-packages\shotgun_api3\shotgun.py”, line 2371, in _upload_to_storage
upload_info = self._get_attachment_upload_info(is_thumbnail, filename, is_multipart_upload)
File “\tools\python\site-packages\shotgun_api3\shotgun.py”, line 2521, in _get_attachment_upload_info
upload_info = self._send_form(url, params)
File “\tools\python\site-packages\shotgun_api3\shotgun.py”, line 3979, in _send_form
resp = opener.open(url, params)
File “C:\Python27\lib\urllib2.py”, line 429, in open
response = self._open(req, data)
File “C:\Python27\lib\urllib2.py”, line 447, in _open
‘_open’, req)
File “C:\Python27\lib\urllib2.py”, line 407, in _call_chain
result = func(*args)
File “C:\Python27\lib\urllib2.py”, line 1241, in https_open
context=self._context)
File “C:\Python27\lib\urllib2.py”, line 1167, in do_open
h = http_class(host, timeout=req.timeout, **http_conn_args)
File “C:\Python27\lib\httplib.py”, line 1247, in init
context = ssl._create_default_https_context()
File “C:\Python27\lib\ssl.py”, line 440, in create_default_context
context.load_verify_locations(cafile, capath, cadata)
IOError: [Errno 2] No such file or directory

Any suggestion?

Thanks
Chris

[chris]

Additionally, the python code used to upload the file :

version = get_version(session, 8792)
img_file = ‘C:\tmp\hdrex.jpg’
session.upload(“Version”, version[‘id’], img_file, “sg_uploaded_movie”)

Where “session” is the shotgun_api3.Shotgun object.

Thanks!

[philip.scadding]

Hi Chris

Welcome to the forums!

The certificate doc you were looking for was moved, you can find it here now.

Hopefully, this will help.
Thanks
Phil

[chris]

Hi Philip,

thanks! That helped a lot!

For anyone that may have the issue in the future, I ended up creating the Shogun object while adding the certificate :

this_path = os.path.abspath(os.path.dirname(shotgun_api3.file))
ca_certs_path = os.path.join(this_path, “lib/httplib2/python2/cacerts.txt”)

session = shotgun_api3.Shotgun(
“https://ourcompany.shotgunstudio.com”,
script_name=“ourscript”,
ca_certs=ca_certs_path,
api_key=“ourAPIkey”
)

not quite sure why it’s not already like this by default though

[frank]

Getting this now as well, of course in the middle of a big turnover, yikes.
I am on MAC though, I thought this was a windows specific issue?!
I’m using Python API v3.0.40

[frank]

Looks like it’s not a windows issue after all. Fixed by pointing connection instance to the certificate as outlined in the docs and above.

Cheers,
frank

[bouchep]

For reference and for those who might stumble on this thread in the future, here is a post from one of our engineers.

CERTIFICATE_VERIFY_FAILED error on Windows

Hi, We’ve received reports of some users experiencing issues uploading and downloading media from Amazon with the Python API in the last few days on Windows. This issue manifests itself by the Python API raising the following error message: shotgun_api3.shotgun.ShotgunFileDownloadError: Failed to open https://xxx.shotgunstudio.com/file_serve/attachment/xyz <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)> It appear…

[frank]

Hi,

the issue has returned for me on OSX now, even though I have updated my certificates from here.
The command I am using to create the connection instance is:
shotgun_api3.Shotgun(self.SERVER_PATH, self.SCRIPT_NAME, self.SCRIPT_KEY, ca_certs=ca_certs_path)

Error is this:
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>

Using Shotgun API 3.0.40

Any ideas? I need to make this work on all operating systems again.

Cheers,
frank

[frank]

Mystery solved, teh dev environment was using Python 3 which caused the issue.

[frank]

Hmm, spoke too soon. If I run the code via my Python IDE it works just fine, if I run it via the AMI it fails with the certificate error. Both are using Pyrhon 2.7. What am I missing?

[EDIT]

It looks like it’s the apple script that handles the event.
If I do this it all works but I get no feedback (I’m donwloading playlist versions):

 on open location this_URL
    do shell script "python2.7 /path/to/python/file.py '" & this_URL & "'"
 end open location

In order to get some sort of simple feedback without having to write a full UI I just want to run the code in a terminal, so I do this, which causes the certificate error for some reason:

on open location this_URL
	tell application "Terminal"
		activate
		do script "python2.7 /path/to/python/file.py '" & this_URL & "'"
	end tell
end open location

Are there any Apple experts here that coudl help out?

[EDIT2]
Looks like I have a dirty python environment and “python2.7” was not pointing to the python version I expected.
Fixed.

Cheers,
frank

[bouchep]

I was just catching up on recent activity in the community and read up on this thread.

This was quite the emotional roller coaster!

More seriously, I’m happy you got it licked.

[tony]

This might be totally out of left field, but I thought it worth mentioning that I’ve had a ton of success using pipenv & pyenv for controlling my python environments in macOS. I build a lot of different versions of SG for various reasons, and being able to specify my environment (and not have to rely on system binaries!) has saved my bacon on more than one occasion.

For your examples, once you have your project set up in pipenv, you could do something like:

do shell script "cd ~/script_folder/; pipenv run python script.py"

or if you want (though this is potentially more fragile but also maybe more fun), look into pipenv-shebang: https://github.com/laktak/pipenv-shebang