ShotGrid Dev & API consequences

I may need more :coffee: , but your reply seems to contradict itself.

…there should not be a need for end-user to update any of their current software or the way they were connecting to Shotgun.

That sounds good, no changes needed by us. But then:

…by adding a PAT to your Shotgun user ahead of the migration to Autodesk Identity, you will be able to connect to the server as you would before.

So if we don’t add a PAT to each and every user, they will not be able to continue to access Shotgun via the API with their user credentials like they could before? And how can we begin to generate these tokens for users if we don’t have access to this new Identity management named user system before the date all this goes live in 3 weeks?

1 Like

Sorry, I’ll rephrase:

You need to have a PAT set up for your user first. No PAT, no API access with username/password.

PAT creation and addition to the user account will need to be done by them. You cannot, even as an admin, do that for another user. Admins will likely have to communicate with their users as to the need to go through that process.

Once that is done, users with a PAT will be able to log in using the API as they did before.

For a regular user, it is not clear to them if they are using the API or not. It depends on the tools being used and their workflow.

Still need more coffee ? :slight_smile:

-Patrick

1 Like

As an added note, Identity will not be switched on automatically under your feet.

There will be a transition period, of a few weeks, which will give time to the admins to validate that all of their users have an Identity account tied to their email used in Shotgun.

From Shotgun, as an admin, you will be able to send invites to your users for those that do not have an Identity account. This transition period will allow time for your users to generate their PAT and add them to their account. As an Admin, you will be able to see who has their PAT set up, and those that do not.

You, the Admin, will be able to control the migration process, and you will be the one pulling the trigger to use set Identity as the authentication method on your site.

This is for existing sites. New trial sites, spun up starting from June 7th, will use Identity from the start.

-Patrick

Man, this is going to be a terrible experience for us based on what you’ve described with PATs. We have a large amount of freelancers, and sometimes they may only be booked for a day or two.

If we have a freelancer come in and they set up their PAT, will that still be valid if we deactivate their Identity / Shotgun User and then reactivate it? To add to this will Shotgun user creation / activation be tied to their Autodesk Identity or are we now going to be managing accounts in multiple locations?

Regarding the pain : it depends on what your freelancers are using. If they are just accessing ShotGrid from the Web GUI, Create, RV or Shotgun Desktop, Shotgun Toolkit-based GUI tools, no need of a PAT.

If they are using custom scripts/applications using Rest or Python API, or other third-party apps that do not use the Shotgun Toolkit, they will need a PAT.

There are a few questions in there:

  • The PAT will stick to the Shotgun user account when de-activated.
  • ShotGrid admins still have full control as to who gets on their server. Having a license is required, but not sufficient. You either grant access manually or by import an Excel sheet, or using a script to create your HumanUsers. If there is any auto-provisioning on your site, it is because it is done from your side.
  • Identity user management : everything revolves around the email associated to the Identity user. If it is a personal email, then that person has full control over it (email, first name, last name, deleting the Autodesk Identity account). If it is a company email, things are a bit less clear… it also depends on if SSO is in the mix or not… I do not want to get into that in this post.

-Patrick

Something that I thought was clear to me but now isn’t again…

How does this look in plain simple steps?
At some point in your replies I thought I got it, the PAT is basically a token linked to an Autodesk Identity/Account and it’s then tied to the Shotgun user, PAT is automatically generated and linked between the accounts.

But after a few posts I’m now confused again.

So Plain simple I thought these where the steps:

  1. ShotGrid goes live, we now have 4 weeks to make sure all is well
  2. Admins send their active users a “Autodesk Identity” invite.
  3. Depending on if the user uses their own email to login to Shotgun or a company email they use that email to login/create an Autodesk Account and by clicking the email invite the account is automatically linked between Autodesk Identity and their ShotGrid HumanUser.
  4. This linking process also automatically generated a PAT and linked those between the HumanUser and Autodesk Identity.

This is how I sort of understood it would go from some of your previous posts.
If that’s the case I think maybe people have become confused with the word PAT and maybe started to think users had to go and copy/paste some token into their Shotgun account.

Hi @Ricardo_Musch ,

Our official communication is admittedly lacking in details, our apologies.

About the PAT and why it has been added in the mix :

  • Autodesk Identity requires user to authenticate via a Web browser-like environment (web browsers, Qt-based WebView, etc.). E.g. you cannot just connect to Identity using a straight API calls with your username/password, because your company’s SSO system may be the one doing the actual authentication.
  • This requirement imposes a huge constraint to existing applications, and makes it nearly impossible for command-line and non-graphical apps to connect to ShotGrid. Other solutions may have been possible, but would still have required code changes to client code.
  • The solution adopted uses a PAT, and Identity will accept it as a proof of who you are. In order to prevent misuse and impersonation, it sits securely within your ShotGrid’s account. Presenting your username and password unlocks it and ShotGrid can then use it in the Authentication process. Only the ShotGrid application can do that exchange on your behalf, for added security.
  • As an Autodesk Identity user, you can revoke any PATs that you have created, thus rendering any of them present on a ShotGrid site unusable.
  • Adding a PAT provides an indirection point, allowing us to bridge the old with the new. And that without changing one byte of your current application/script/tool (with the exception of scripts that create new active users).

As for your description, it is nearly spot-on:

  1. Nothing is required from you or your users until ShotGrid goes live on June 7th. Those that want to do so, can already create an Autodesk Identity account with their work email (or more to the point, the one they use in Shotgun). But there is no rush in doing so, and one can wait until their site starts the migration process (see next point).
  2. Starting from that date we will roll out the functionality for Admins to start the migration process. End users will see no immediate changes (other than the Shotgun → ShotGrid part) to their UX until an Admin starts the migration process.
  3. Once that migration process starts, it may last for a while (a number of weeks). There will be a cut-off date later this year, but I do not know what it is. But there should be plenty of time. That period will allow Admins to review what emails are used by their user base, enforce any corporate rules/policies and ensure that the email used is seen as fit. (e.g. should an artist use their personal email, or their work email, etc.)
  4. During the migration process, Admins “link” the HumanUsers’ accounts with the corresponding Autodesk Identity account. For those who already have an Autodesk Identity account tied to that email, that part is done. For those whose email are not tied to an Identity account, the “link” action from the Admin will create a place-holder account and send an invitation to that person. Asking them to complete the registration (setting up a password, etc)
  5. During that migration period, Admins can turn on “hybrid” login mode, where Web users can log in either with their “old” username/password, or use Autodesk Identity to authenticate with ShotGtrid.
  6. During the migration period, API authentication will stay the same.
  7. Starting June 7th, Autodesk Identity account holders will be able to go to https://profile.autodesk.com , in the Security section, and generate a PAT for ShotGrid and copy-paste that PAT into their ShotGrid user profile. Once the PAT is in their account, it will be used for API connections, but that will be transparent to the user.

We realize that things can easily get complicated… While the switch to Identity is a required step in order to fully integrate with the rest of the Autodesk back-office, the idea of bringing PATs into the picture was to minimize the pain of that transition.

Hoping I made a few things clearer, and did not bring in more clouds of confusion in the sky,

-Patrick

2 Likes

Okay this makes sense except that I don’t understand why you still need to generate a manual PAT and copy it over. But I’ not an expert on security in the cloud so will leave that for someone else.

Leaves one more question…

Client Users…

Please tell me they won’t have to go through anything like this… because it’s already a pain to deal with them due to confusion about password and people within the client company forwarding playlist emails…

(Honestly the Client review site needs and overhaul and feature extension… )

1 Like

I know that the manual copy-paste of a PAT seems insecure, but given that it can only be copied to a ShotGrid site where the local user is tied to the same Autodesk Identity account, there are very little security risks if somehow your PAT would leak out. When you enter your PAT in your account, we validate that it indeed a) valid, b) belongs to you and c) is for ShotGrid and not another app.

Having ShotGrid itself generate the PATs for you is seen as a more risky thing : if someone manages to gain privileged access to ShotGrid’s bowels, they could mass-generate PATs for any number of users. That decision was taken by the Identity team, so we had to respect it.

As for Client Users : no changes for the time being…

Your comment about the CRS is duly noted.

-Patrick

3 Likes

Hi all, picking up on these questions, we manage dozens of Shotgun sites, but are not the owners of them, and for most we will need to help or even manage the migration process. Most of them will not require PATs, but we do have a system we’re building for our users so we can enable and disable for the purpose of short-term access. We currently use a script for this purpose, but if I understand correctly, since that will be creating new users, we will need a PAT and to tie the script to a specific Autodesk user. I assume that we can use the same Autodesk user across all of the sites we manage? Does the script user require any additional permissions? Can we tell our clients how billing is handled in this situation under the new rules, where a user is turned on for a few hours and then turned off again?

Also, will this be compatible with particular versions of SGTK / Shotgun Desktop? We have some sites running pretty old versions for compatibility reasons (they don’t want to update anything mid-show).

Thanks,
Den

Hi @dennisserras

You are correct, using a script name/API key pair to activate users will require the script to connect with a user who has a PAT setup in their ShotGrid user settings. The permission used are those of the user, not of the script. So they need to be assigned to a role that can create/modify users.

The same Autodesk Identity user can be used on all of these site, with the same PAT (which will need to be entered manually on the that user’s account on all sites). But you already see a potential issue : that user will be consuming a license.

For sites all under the same contract/subscription, only 1 license is needed to access all the sites. But from what you describe, I get the feeling that each of your client will need to allocate one license to your user.

I am not sure how the setup you describe actually maps onto the new model, so I can’t speak to the billing aspect. (and this thread is about API use, not billing)

One solution : have your script only create inactive users (letting the burden of the activation to a ShotGrid admin on the site) or de-activate users who should no longer be active. This way you can carry on using a script name/API key pair.

-Patrick

Thanks for the information in this thread. It’s very useful.

It seems that there are really two changes going on here: the renaming of Shotgun → ShotGrid, but also the mandatory(?) adoption of Autodesk Identity.

Identity looks like it will cause much more work for customers than any renaming of libraries or API endpoints. (Though I’d still be interested to hear of any other known impacts beyond Identity that you are anticipating, or any other significant changes being deployed at the same time?)

I’m a bit anxious about all the new manual steps for admins (activating users if they were created in an inactive state via scripts; inviting new users to create Identity accounts; auditing the PATs and ensuring end-users registered their Identity accounts with their correct work emails) and for end-users (accepting the invite and creating their own Identity account if they don’t already have one; being sure to use the right work-based email addresses; and linking their PATs to their SG accounts).

At present, Animal Logic’s user onboarding process actually requires zero intervention by admins or end-users. New users get added to ActiveDirectory of course, and our scripts just respond to that and setup their SSO-enabled SG logins immediately. New crew can click through to SG from Okta on day one.

So for us, adding a bunch of manual steps doesn’t sound good for the onboarding process, in terms of speed or reliability.

Also, you’ll notice I mentioned SSO. Could you please clarify the impact of Autodesk Identity on SSO-based logins, if any? (Or direct me to a separate thread for that, if there is one?!)

Thanks,
Justen

2 Likes

Hi everyone!

Just some questions out of curiosity:

  1. Will Shotgun Toolkit also be renamed to ShotGrid Toolkit?
    And what about Shotgun Desktop and Shotgun Create?
  2. Can we expect a big update wave for all the Toolkit parts as soon as the switch happens?
    Guess a lot of License headers and variable names need to change…
  3. Will the Shotgun API get a compatibility layer so you can import shotgun_api3 as well as import shotgrid_api3?
  4. How far do you want to go with that? There is also thinks like ShotgunError exceptions, event types like Shotgun_EntityType_New and environment variables like SHOTGUN_HOME. Will these stay as is or will they get a compatibility layer as well?

Hope there is some further documentation soon! :slight_smile:

Cheers,
Fabian

4 Likes

Hi @justen.marshall

First thing I want to underline is : this integration to Identity and licensing process is something which will happen over a period of time (start June 7th at the earliest) and likely has a hard-deadline of October 1st. I realize that it is a lot to process with the announcement, but there is not really any actions that can be taken yet on the sites or on the licensing side.

I can’t speak to your particular setup at Animal Logic, but the changes to get things working with the new setup may be as “simple” (I know, simple is rarely simple):

  • use an actual HumanUser in your provisioning script, instead of a Script Name/Api Key
  • ensure that that user exists on all of your ShotGrid instances, with the proper permissions (usually as an Admin)
  • ensure that a PAT is generated for that user, and saved in the user’s settings on all your ShotGrid instances

And that should be it (with the usual caveats, and your mileage may vary). You could even do that change right now, as an initial transition step.

When using SSO in the backend of Autodesk Identity, new users are easily created via an invite from ShotGrid (if they do not exist already have one that is). Those users have no extra steps to complete the creation of their Autodesk Identity users (since their email is already considered as being valid). Their initial connection via Autodesk Identity will automatically pull from the SSO backend their first name and last name and set them.

Regarding login with SSO: if using the native Shotgun integration with SSO, a similar process will be needed to setup Autodesk Identity to use your SSO backend. Once configured, and ShotGrid set to use Identity, the experience is: ShotGrid → Identity login page to enter the email → Client SSO for Authentication (based on the domain of the email used in the previous page) → you get into ShotGrid (if you have an active user on that site).

The particulars of the SSO setup/availability/etc. will need go into another thread.

There is one important task which will be required from admins : a sanity check on the email addresses used in Shotgun. For those that are using an automated script, pulling the data from AD or other user management databases, this may already be considered as completed. For sites where there were no rules/guidelines, and things are messy, this will require some work.

At this time, there is no immediate rename of libraries involved in existing code, or change in endpoints you were already using.

-Patrick

1 Like

Hi @Fabian

This thread is about API use and impacts. Other threads will mention the rebranding, which impacts visible aspects of the software. Code is not one of those things.

While the documentation will change over time to use the new names, APIs and endpoint will not immediately change (or more to the point, force you to change). Should libraries/classes/methods/etc. be renamed, this will be communicated in a timely manner. There are usually easy ways to support both old and new names, to not break older code.

So I would suggest that you do not lose sleep over that at this time.

I also hope that there are documentation available soon…

-Patrick

4 Likes

Ok, thanks @patrick-hubert-adsk! Doesn’t sound quite as scary as it did before.

Though, I do wonder about the rationale behind making scripts run as human users. It seems odd from an audit-trail/accountability perspective to muddy the waters between actions taken knowingly by a human, and actions taken by a tool that is just piggybacking on the human’s account to get the access it needs.

Should we be considering making “service accounts” in Autodesk Identity to accommodate these kinds of unattended automated operations? Would it be better than having tools masquerading as a particular human, when no human actually performed the operations in question?

Justen

Hi @justen.marshall

The reason for this is due to the fact that there is some interaction needed with the Autodesk services to invite users. Those actions need to be performed as a user.

While sudo_as_login is something that is okay and accepted within the domain of Shotgun, it is not the case in the Autodesk Identity environment. One cannot assume the identity of someone else. It opens up so many security pitfalls.

The important part here is accountability, wether a script might have been used on the other end of the API call or not.

Using a service account is indeed a possibility, which I do not think we can prevent our clients from using. But it is not encouraged.

As an example here at work, Autodesk does not allow the company’s service accounts to authenticate with SSO, making them unusable with Identity… by design. But it comes down to a corporate decision. Your company may see things another way.

-Patrick

Hello, does this mean that sudo_as_login will be deprecated?
Thanks

Hi Stephane,

I will edit this post with a more detailed explanation, but let me first say : NO sudo_as_login is not being deprecated. The number of things that would break everywhere if we did that is scary.

-Patrick

1 Like

Hey dear-API-users,

This has become a very long thread, and assuming that everyone has gone through all of it is unrealistic. Also, there is the unfortunate situation that I am not a tech writer, which means I can easily trip and make things less clear than they should.

I’ll try to summarize the main points of this thread here:

Regarding the API usage:

  • Our intent has been to minimize the impact of the Integration process on the API users,
  • Script Name / API Key authentication method is not going away. Read further down for more details (re: activation of users)
  • User Name / Password authentication method is not going away, but will require the user to add a Personal Access Token to their ShotGrid user profile (must be done by the user, not an Admin). Read further down for more details (re: Personal Access Tokens).
  • No changes to the Client Review Site authentication for ClientUsers.
  • The addition of the Personal Access Token was to avoid breaking existing tools and pipelines, at the cost of a one-time manipulation of the end-user,
  • If the script worked before the Integration, it will continue working after the Integration (see next point for exceptions) by using a Personal Access Token for Human User authentication.
  • Scripts that are activating Human Users (e.g. flipping the sg_status_list from anything to act) will need to be run as a Human User and not an API User (e.g. not using a Script Name/API Key)
  • sudo_as_login is not going away, but there is a constraint: it can only be used for actions that stay within the ShotGrid site’s itself. It cannot be used to contact other Autodesk services while impersonating someone else. At this time, only user activation is known to be impacted by that constraint.
  • No changes to the library names, class names, etc. If any repos are moved or renamed (of which I am not aware), this would be announced well in advance. The same goes for any deprecations or code-breaking changes.

Regarding the ShotGrid Integration process:

  • Autodesk Identity is only for hosted sites, not local installs. Only sites on the shotgunstudio.com domain will need to undergo the Integration process.
  • It will become available progressively after June 7th, and will need to be completed by October 1st.
  • The ShotGrid Admins will be in full control over the moment at which Identity will be turned on and enforced (prior to October 1st that is).
  • This is a per-site process. So for a company with multiple ShotGrid sites, each can have its own schedule for the migration if needed.
  • New trial sites spun up on June 7th or after will require an Autodesk Identity account for the request. The resulting site will be setup to use Autodesk Identity out of the gate. No Integration process for these. When inviting a user, they will receive a direct link to the site if there is already an Identity account for that email, or will receive an Autodesk Identity Account Creation invite that they need to complete. At the end of the enrolment, they are redirected to the site.
  • There are no special actions needed to be taken now. But there are definitely some things that can be done (listed after)

Possible steps to take now in preparation of the Integration:

  • Review the emails used by the active users on your sites. It is an opportunity for any cleanup/policy enforcement. Autodesk Identity uses your email to log you in, so you need access to that inbox. There is a one-to-one mapping between an Autodesk Identity account and a specific email. Multiple emails implies multiple accounts.
  • Audit/Identify any scripts used that will change HumanUsers to sg_status_list to act. See if you can already modify the script to authenticate as a HumanUser instead of a Script Name/API Key (if not already the case). This will pave the way for an easy transition when it happens.
  • Create, RV and Shotgun Desktop already support Autodesk Identity. For Toolkit using clients, the Web login will be used in a Qt/Graphical context.
  • There is no need to push your users to create Autodesk Identity accounts, as ShotGrid will make it easy for Admins to invite the active users and provide them with an account creation link if needed. That being said, there is nothing wrong with users creating their account now.
  • Creation of Personal Access Tokens on https://profile.autodesk.com will be possible starting June 7th.

I believe that his sums up the main points… I’ll update if needed.

Hoping it helps

-Patrick

6 Likes