Can I make the Shotgun integrations recognise my existing folder structure?

Pipeline Integrations
1

Question

When first starting with the Shotgun Integrations, a common question can be, How can I make Shotgun Toolkit recognize my existing project folder structure?

Sometimes projects can have a long life and you may want to introduce the integrations during the life of that project, rather than starting with a new project.

Answer

The answer is complicated and not a clear yes or no.
First off, it should be said that Shotgun won’t automatically recognize your existing structure if it wasn’t created through Shotgun. However it may be possible to configure Shotgun to use it, but this will depend entirely on how your structure is currently set up.

You need to have set up a project in Shotgun Desktop using the advanced setup wizard, otherwise, the integrations won’t create or make use of folders at all.

The Toolkit folder structure has to conform to rules so that it can be programmatically understood. Toolkit creates folders dynamically based on your Shotgun entities. The default schema that you get when setting up a project for the first time will and must contain a root project folder. From there if you are working with Shots, then the default structure goes as follows:

/sequences/{Sequence}/{Shot}/{Step}/...

This means that you will end up with a static folder called sequences and then inside that, you will get folders dynamically named after your Sequence entities. Inside that, you will get all the Shot folders that are linked to the parent Sequence. After that, you get Step folders for the Step of each Task on the Shot.

This structure can be customized to use different entities, and depths, as well as being able to configure the way the folders are named after the entities. However, there is one main rule you need to keep in mind. The integrations often make use of being able to extract the context from a given path.
For example, given the following path which we shall say conforms to the default structure we mentioned above:

/projects/my-project/sequences/sq10/sq10-s35/lighting/work/maya/scene.v001.ma

You could extract out the Step (lighting), Shot (sq10-s35), and Sequence (sq10). When Toolkit creates these folders, it stores a record of the path to the entity so that it can perform this backwards lookup. Being able to extract the context from the path in an unambiguous way is critical to the operation of the Shotgun integrations.

The take away is, that you can customize the structure to work with different entity types and structures, but the structure should describe the context that the user needs to work in.

Examples of incompatible structures

  • As mentioned above the folder structure must have a project folder in there. So a structure like this wouldn’t work:

    drive_root/sequences/sq10/...

  • Having a flatter structure with no entity folders will mean you will lose the context:

    /projects/my_project/work_files/shot_10_scene.v001.ma

    The schema is the key to working out the context, so even though the path above contains the shot name in the file, Toolkit won’t be able to reliably translate that into a Shot entity so no context can be extracted.

  • You should avoid structures that cause duplicate entity folders for example by moving a step folder to the root.

    /projects/my-project/lighting/sq10/sq10-s35/lighting/work/maya/scene.v001.ma

    Where the schema looks like:

    /projects/{Project}/{Step}/{Sequence}/{Shot}/work/...

    This would run into issues for two reasons.

    1. Since the Step is at the root, it is not constrained by an entity, so it won’t know which steps to consider when creating the folders.
    2. It will mean you would end up with duplicate Sequence and Shot folders within each Step folder.

Conclusion

It may be possible to match your schema to your existing folder structure, providing that your existing folder structure doesn’t break any of the rules. Once you’ve configured the schema correctly, when you run the folder creation, if the schema lines up, it should in fact not create any folders, and instead just register the existing folders.

You can also read more about the folder schema and templates setup here.

1 Like