I have code that needs some shotgun auth to upload a mov preview to shotgun from our render farm.
From a user authenticated SGTK session, is there a way to either
A). Create temporary credentials that are generated by the HumanUser and able to be used on the farm job for a period of time?
B). A way to get a serialized instance of an authenticated python Shotgun api instance.
I have a third option that I could potentially do, but it feels goofy too.
I could use ShotgunAuthenticator.create_script_user with randomized values and pass those values to the render job. At the end of my job, I would make sure I delete the script user from the site. I haven’t tested this way yet, I don’t know if I can make a script user with credentials to delete itself but not other scripts. Turns out create_script_user does not create a new script user on the site. It creates an instance of an existing script user. Requiring a valid api_key
I have a solution that uses sgtk.authentication.user.serialize_user and shares the session_token (if a SessionUser, api_key/api_user if ScriptUser) with the environment variables in the job and rehydrates a shotgun_api3.Shotgun instance that the job can use while on the farm.
I am having trouble finding documentation on the session_token and how long it is valid for.
As long as it is valid for a few hours this should work for my purposes unless there is something that I don’t know about the session_token.
A snippet of code that I wrote that shows roughly what my solution is. The shared solution does not have error handling and leaves out some stuff that was not required for the example. You should only use this as reference as I do not provide support for the following code.
serialize_to_env.py
import json
from sgtk.authentication.shotgun_authenticator import ShotgunAuthenticator
from sgtk.authentication.user import serialize_user
def serialize_credentials():
authenticator = ShotgunAuthenticator()
user = authenticator.get_user()
if not user:
return {}
data = json.loads(serialize_user(user, use_json=True))
return {data.get("type"): data.get("data")}
credentials = serialize_credentials()
if credentials.get("SessionUser"):
os.environ["SG_TYPE"] = "SessionUser"
os.environ["SG_SESSION_TOKEN"] = credentials.get("SessionUser").get("session_token")
os.environ["SG_HOST"] = credentials.get("SessionUser").get("host")
elif credentials.get("ScriptUser"):
os.environ["SG_TYPE"] = "ScriptUser"
os.environ["SG_API_SCRIPT"] = credentials.get("ScriptUser").get("api_script")
os.environ["SG_API_KEY"] = credentials.get("ScriptUser").get("api_key")
os.environ["SG_HOST"] = credentials.get("ScriptUser").get("host")
Yeah using a user session token is not a good way to go, it will stand a good chance of expiring if the farm job spends any time waiting in the queue. Unfortunately, I don’t have to hand the expiry time for a session token but I would avoid using it.
I guess my question would be, why can’t you use script authentication with a non temporary script key? Why do you need to generate one each time?
A provide a fairly box pipeline for several of our clients, some of them do some config on it after I hand it off, but most do not.
For most of these clients, I do not have access to their shotgun site in order to generate a script key and then build a new release of the pipeline with their keys baked in. Even for the ones that I do have access to, I honestly don’t want to have to manage several script keys for each client. In addition to that, I really don’t want to deal with keys to do actions that the users are able to do themselves. Basically the functionality that I am ideally looking for would be an aws iam assume role style feature where you could generate temporary access keys that a script could use inheriting the permissions of the generating entity (HumanUser for example).
I have thought about storing keys on disk in a standardized location but that isn’t a great solution either. That would require me to ask the client to build X keys, store them on disk named correctly and in the correct place. Even if they were able to do all of that for me, I have personally never been a fan of storing access keys on disk in a location that anyone can access.
If you could look up the time that a session token lasts, that would be great because I could refresh the token just before I serialize it to the job and if the time is several hours, that would be good enough for my use cases. I can just scale up and down the farm as needed so that the entire job will fit within the window of time for a session_token.
I don’t know if temporary tokens or access keys are something that you might consider for the future, but if you would like to have a call about it, I can setup something with the team on my side.
From what I can see in the docs, it sort of appears that the token lasts up to 24 hours?
... These session tokens time out after a certain period of inactivity, usually 24 hours ...
Would love more clarity on that timeout “usually”.
Also in that page they do mention that serialize_user can be used for the render farm.
ShotgunUser objects can be serialized via the serialize_user() and deserialize_user() methods and can be passed between multiple sessions, say for example when a DCC is launched or when a job is sent to the render farm.
Let me know if the docs are not to be believed, but a 24hr timeout sounds exactly like what I would need.
OK, I understand where you’re coming from. I’m not an expert on the authentication side of things, so I’ll see if I can get someone to chime in here.
token variable is dictionary with user login and sessino_token
Next create session user and connect with login and token previously provided:
import sgtk
import shotgun_api3
token = # here you should put token we obtain before
sa = sgtk.authentication.ShotgunAuthenticator()
session_user = sa.create_session_user(token['login'], token['session_token'])
sg = session_user.create_sg_connection()
and now you have access to shotgun, it works for me
I wasn’t aware that you could get user login credentials.
I am hoping that for this use case I won’t have to use raw credentials because they can been seen in the farm job information and won’t ever expire so they could be used forever.
But if this session_token does not end up working, that is a really good thing to know!
Yeah you can obtain session token using current user login.
Token is simple dictionary so for secure reasons it can be good to encrypte it or put it in system envionment variables/in some temporary file where users cannot see them.
Or maybe send it directly to script, without putting them anywhere?
Session token is alive for about 24h, if I understand good. But i think you can refresh creditionals with refresh_credentials() every couple of hours until task is done. If you need them for some longer time.
I’m sorry, I started replying to this the other and had to ask for confirmation about a session token’s lifetime and then got distracted by a million other things.
In the Security section of the Site Preferences in Shotgun, you’ll find the User Session Expiry setting which allows to choose how long a session is valid. By default, I believe the setting is 24 hours. What this actually means is that the session will expire after 24 hours of inactivity . Whether 24 hours is a long enough delay for a render job to start depends on your workflow.
As you may be aware, we continuously release new versions of Shotgun with bugfixes and/or features, usually 3 times a week, for sites hosted on shotgunstudio.com. Normally, this upgrade is seamless and people do not notice. However, sometimes we need to make a change that will invalidate all session tokens and force all our users to log back into Shotgun. We try very hard to avoid these sorts of situations, but this is one of the reasons I’m aware why we say sessions “usually” last X hours.
In the end, your code would roughly look like this:
In DCC
import sgtk
user = sgtk.get_authenticated_user()
# We recommend JSON because it's much easier on the eyes and less finicky then pickle when it comes to reloading between Python 2 and 3.
serialized_user = sgtk.authentication.serialize_user(user, use_json=True)
On the farm
import sgtk
# serialized_user is the value from the In DCC section
user = sgtk.authentication.deserialize_user(serialized_user)
# If you are using the bootstrap manager to start Toolkit do
manager = sgtk.boostrap.ToolkitManager(user)
# The manager takes care of calling sgtk.set_authenticated_user(user), so you
# don't need to call it here.
# If you are using sgtk.sgtk_from_path, do this:
sgtk.set_authenticated_user(user)
sgtk = sgtk.sgtk_from_path(...)
