Table of Contents:
- Table of Contents
- Introduction to distributed configurations
- How distributed configurations work internally
- Distributed vs centralised configurations overview
- Distributed configurations with GitHub
- How to Automate the configuration upload to FPTr with new releases
- CI/CD Automation with GitHub Actions
- Descriptors in a distributed configuration
- Files in a Distributed Configuration
- Tank Commands in a distributed Configuration
- Folder Creation WorkFlows
- Core API Management in Distributed Systems
- Bundle Cache Management
- Troubleshooting Distributed Configurations
- Recommended Studio Best Practices
- Where to Find More Articles
Written by @Felix.Onsare and @sergio.leon
Introduction to distributed configurations
In traditional centralized Toolkit deployments, a shared configuration exists on a studio’s network storage that can be accessed by all users. On the other side, a distributed configuration is stored in the cloud and get cached locally per user. This helps centralized maintenance while allowing local and remote users to use the same project’s configuration and allow easily deployments.
A distributed configuration is uploaded to Flow Production Tracking and automatically downloaded to client machines during bootstrap.
Instead of all users accessing a single shared configuration path:
- Each user receives a locally cached copy.
- Toolkit bootstraps the configuration dynamically.
- The configuration may originate from:
- Git descriptor
- Path descriptor
- Toolkit App Store descriptors
This architecture is manly used with:
- Flow Production Tracking Desktop
- Browser Integration
- ToolkitManager bootstrap workflows
- Cloud-hosted or hybrid deployments
- VPN-sensitive pipelines
Further documentation in:
How distributed configurations work internally
When a user launches Toolkit through Flow Production Tracking Desktop or a Toolkit-enabled integration:
- ToolkitManager authenticates the user.
- ToolkitManager resolves the pipeline configuration.
17.Toolkit downloads the configuration’s descriptor.
18.Toolkit downloads missing engines/apps/frameworks.
19.The configuration is cached locally.
20.Toolkit initializes the tk-core API. - Engines are launched under the resolved context.
Unlike centralized systems, users do not normally interact with a permanent shared configuration directly. When a user bootstraps, this process triggers to update the config, if changes were done in the distributed configuration.
Toolkit caches distributed configurations in locations in the following folder locations depending on your operating system:
- Windows:
- %APPDATA%\Shotgun
- Linux:
- ~/Shotgun
- macOS:
- ~/Library/Caches/Shotgun
Documentation on tk-core environments variables
Distributed vs centralized configurations overview
| Feature | Centralized | Distributed |
|---|---|---|
| Configuration Location | Local or shared in a File system Server | Remote and copied to local user cache |
| Requires Shared Storage | Yes | No |
| Toolkit Updates | Directly on Shared Config | Upstream Descriptor Update |
| Tank Command Usage | Primary Workflow | Mostly Debugging/Admin |
| Offline Capability | Limited | Better |
| Remote Workforce Friendly | Moderate | Strong |
| Local Cache Dependency | Minimal | High |
| Bootstrap Required | Optional | Required |
Distributed configurations with GitHub
It is recommended to work with git in a distributed configuration. Here are the steps.
Create a repo
It is generally recommend storing distributed Toolkit configurations in Git repositories rather than editing the production configurations directly. Git repository provides version control, change history, collaboration workflows, code review capabilities, and the ability to roll back problematic changes. In a distributed configuration setup, Git effectively becomes the source of truth while Toolkit’s bootstrap system downloads the correct version for all users.
Step 1: Create a GitHub repository
On GitHub:
-
Click on New Repository, and name it something meaningful. Samples: tk-config-studio, tk-config-production, tk-config-default2-custom.
-
Choose Private or Public
-
Initialize with a READMe
Step 2: Clone the repository locally
Once created:
[terminal] git clone https://github.com/studio/tk-config-production.git
This downloads the repository to your workstation.
Example:
cd tk-config-production
You can now edit files like the following files, without affecting any production users:
*env/project.yml*
*hooks/tk-multi-publish2*
*templates.yml*
Step 3: Commit changes
Lets assume you modify
*git status #* first review changes
*git add . # ad changes*
*git commit -m "Added custom Nuke publish plugin"*
*git push origin main #* push changes to GitHub
Also rem that a good commit message should describe why the change was made, not just what change.
e.g you can do sth like:
Added support for OCIO v2 publish validation
Step 4: Downloading updates
In case you need to pull the latest changes use this to update your local configuration to match what is exists on GitHub
git pull origin main
For studios with multiple developers, this ensures everyone works from the same configuration.
Create releases (tags) in Github
Distributed Toolkit configurations should not be pointing directly at a moving branch such as main or master. Instead, use immutable version references. This is where Git tags come in.
A tag creates a permanent snapshot of a configuration at a specific point in time. Example: v1.0.0 or v1.1.0. A descriptors can reference these tags directly, ensuring everyone accesses same config.
Creating a Tag from the Command Line:
*git tag v1.2.0 # create a tag*
*git push origin v1.2.0 # push the tag:*
Git will store that version permanently.
Creating a GitHub Release
Navigate to ”Repository –> Releases → Create Release”.
Select:Tag: v1.2.0
Add release notes:
Added custom publish hooks.
Improved Nuke startup performance
Fixed OCIO validation issues
Publish Release.
Step 2: Clone the repository locally
Once created:
[terminal] git clone https://github.com/studio/tk-config-production.git
This downloads the repository to your workstation.
Example:
cd tk-config-production
You can now edit files like the following files, without affecting any production users:
env/project.yml
hooks/tk-multi-publish2
templates.yml
Step 3: Commit changes
Lets assume you modify
*git status #* first review changes
*git add . # ad changes*
*git commit -m "Added custom Nuke publish plugin"*
*git push origin main #* push changes to GitHub
Also rem that a good commit message should describe why the change was made, not just what change.
e.g you can do sth like:
*Added support for OCIO v2 publish validation*
Step 4: Downloading updates
In case you need to pull the latest changes use this to update your local configuration to match what is exists on GitHub
*git pull origin main*
For studios with multiple developers, this ensures everyone works from the same configuration.
Create releases (tags) in Github
Distributed Toolkit configurations should not be pointing directly at a moving branch such as main or master. Instead, use immutable version references. This is where Git tags come in.
A tag creates a permanent snapshot of a configuration at a specific point in time. Example: v1.0.0 or v1.1.0. A descriptors can reference these tags directly, ensuring everyone accesses same config.
Creating a Tag from the Command Line:
*git tag v1.2.0 # create a tag*
*git push origin v1.2.0 # push the tag:*
Git will store that version permanently.
Creating a GitHub Release
Navigate to ”Repository –> Releases → Create Release”.
Select:Tag: v1.2.0
Add release notes:
Added custom publish hooks
Improved Nuke startup performance
Fixed OCIO validation issues
Publish Release.
Referencing the Tag in Toolkit
You could use:
*location:*
*type: git*
*path: https://github.com/studio/tk-config-production.git*
*version: v1.2.0*
When Toolkit bootstraps, it will download the repo, resolves tag v1.2.0, download that exact version, cache it locally, every user receives same configuration code.
How automate the config upload to Flow PTR with new releases
If your configuration uses a Git descriptor, you’re not required to upload the configuration into FPTr after every change.
What happens instead will be:
- GitHub hosts the configuration
- Toolkit reads the descriptor
- Toolkit downloads the tagged release
- Users receive updates automatically
This means, GitHub becomes the deployment platform while Flow PT only stores the descriptor reference.
CI/CD automation with GitHub Actions
You could have a workflow like: Developer → Git Commit → GitHub Pull Request → Automated Tests → Git Tag Created → GitHub Release → Toolkit Descriptor Updated → Users Bootstrap New Version.
GitHub Actions can automatically run validation tests, create tags, publish releases and update descriptor repositories.
Example trigger:
on:
push:
branches:
- main
When changes reach production:
- Tests execute
- A release is generated
- Tag is published
- 4.Toolkit can resolve the new version
Here’s an example step by step workflow using gihub actions. If we want Flow Production Tracking still distributes the uploaded ZIP, we can use git locally with GitHub, and update the configuration with this script:
1. Repository structure
2. Use Github Secrets.
Go to:
GitHub → Repository → Settings → Secrets and variables → Actions
Create these repository secrets:
FPT_BASE_URL
FPT_SCRIPT_NAME
FPT_SCRIPT_KEY
FPT_PIPELINE_CONFIG_ID
3. Requirements.txt
Create requirements.txt file and a deployments script
*scripts/deploy_fpt_config.py*
and you can paste the following code deploy_fpt_config.py
This uses the FPTr python API for uploading a file to an entity( uploaded_config)
But before uploading anything to the web, first validate. So add the file:
scripts/validate_config.py .
You can get the contents of this file from here validate_config.py .
This file is expandable to validate more items like YAML syntax, hooks…
4. Now create:
.github/workflows/deploy-fpt-config.yml
Then paste the following code to it:
deploy-fpt-config.yml
You make your changes:
git checkout main
# Make configuration changes
git add .
git commit -m "Update Nuke publish settings"
git push origin main
Nothing gets deployed yet.
Then when you’re ready:
git tag v1.4.0
git push origin v1.4.0
GitHub sees:
refs/tags/v1.4.0
and starts:
Deploy FPT Distributed Configuration
Github actions then checks out the version and uploads it to the Flow Production Tracking web, from where Toolkit’s bootstrap recognizes the uploaded configuration and downloads it when users start up
Descriptors in a distributed configuration
We just covered the git descriptor. We also have more descriptors supported by the toolkit:
- An app_store descriptor – represents items in the ToolkitApp Store
- A dev descriptor – represents the developer sandbox
- Shotgun descriptor – represents items stored in Flow Production Tracking
- Manual descriptor – gives raw access to the bundle caching structure
The ‘Path descriptor’ represents a location on disk, that would be a centralised configuration.
Further documentation ca be found here.
Files in distributed configurations
How is the configuration’s cache?
Because distributed configs are downloaded and cached locally at startup, a copy of the Pipeline Configuration plus the[/color] tank command ends up on each user’s machine (in local cache paths such as %APPDATA%\Shotgun,~/.shotgun[color=black], etc.) for windows.
This means:
- You do not normally run tank setup_project because the config was already created and delivered via Flow Production Tracking.
*The Toolkit bootstrap system creates or updates the config on first launch, so tank is already present locally. - We discorage using[/color] tank manually.
We write this section for debug purposes only.
cd ~/Library/Caches/Shotgun/Shotgun/_SITE_/pXXXcYYY.basic.shotgun/cfg
./tank
You may notice that within this cache directory, folders follow the pattern:
p<project_id>c<pipeline_config_id>.<plugin_id>
Example:
P355c430.basic.desktop
What Each Component Means?
- p355 → Project ID
- The number after p corresponds to the internal Project entity ID in Flow Production Tracking.
- You can see this in the URL when viewing a project:
- c430 → Pipeline Configuration ID
- The number after c corresponds to the Pipeline Configuration entity ID associated with that project.
- This identifies which configuration definition was bootstrapped.
- You can find it in this URL:
- .basic.desktop → Plugin ID
- This is the plugin_id used during bootstrap (e.g., basic.desktop).
- The plugin ID separates cache instances created by different launchers or integrations.
- Different plugin IDs for the same project/config pair will create separate cache folders.
Why This Matters? This naming convention ensures:
- Multiple projects remain isolated.
- Multiple pipeline configurations per project remain isolated.
- Multiple launch contexts (via different plugin_ids) remain isolated.
- Bootstrap resolution remains deterministic.
Tank commands in distributed configurations
In an older article we saw the Tank command. It is primarily used for automating and streamlining tasks within the studio production pipeline.
One of the most misunderstood aspects of distributed configurations is that the tank command still exists. It could work for certain actions, but it is not recommended to use outside read-only actions. Produced changes won’t align anymore with the remote configuration, and they may even be replace when there are updates on the remote configuration.
Once Toolkit bootstraps and caches a distributed configuration locally, the cached configuration still contains:
- tk-core
- config
- install
- tank command
- cached bundles
This means you could technically execute tank commands from the cached configuration. However, distributed configurations were explicitly designed so that configuration maintenance occurs upstream rather than directly on local caches.
This creates an important operational distinction that;
- In Centralized Configurations: Users intentionally operate directly on the production configuration.
- In Distributed Configurations: Users operate on temporary cached copies.
Note any local modifications may be overwritten when:
- the descriptor version changes
- the cache refreshes
- Toolkit reboots
- the configuration is re-downloaded
Tank commands available in distributed setups
| Command type | Recommendation | Notes |
|---|---|---|
| tank configurations | Safe for diagnostics | Useful for checking available pipeline configurations. |
| tank app_info | Safe for diagnostics | Useful for inspecting installed apps, engines, and frameworks. |
| tank validate | Recommended | Good read-only validation command for checking configuration health. |
| tank folders, tank preview_folders, tank synchronize_folders | Use with care | Useful for folder workflows, but should be run intentionally because they can affect filesystem/path-cache state. |
| tank cache_apps | Recommended for troubleshooting | Can rebuild or pre-cache missing bundles. |
| tank clear_shotgun_menu_cache | Recommended for troubleshooting | Useful when Toolkit menu items are stale or not updating. |
| tank Shot 1122 launch_maya | Use with care | Can help validate launch behavior from a specific context, but should not replace normal bootstrap workflows. |
| tank --debug | Recommended for diagnostics | Useful to collect more detailed troubleshooting output. |
| tank localize | Avoid | Distributed configurations are already local caches; localizing can create unnecessary divergence from the descriptor source. |
| tank move_configuration | Avoid | Moving cached configurations can break descriptor resolution and bootstrap assumptions. |
| tank push_configuration | Avoid | Risky because the cached configuration may not represent the intended production source. |
| Editing cached YAML files | Avoid | Changes to files such as env files, templates.yml, or roots.yml may disappear during the next bootstrap refresh. |
Folder creation workflows
A common misconception is that distributed configurations eliminate the traditional Toolkit filesystem model. The folder creation system remains fundamentally the same. The filesystem process in distributed configurations follows the same logic as centralized setups. What changes is how the configuration is delivered, cached, and initialized but the underlying Toolkit folder schema system remains the same.
It’s important to note that distributed configurations still contain:
- Templates
- Schema definitions
- Folder creation logic
- Filesystem Location entities
- Toolkit hooks
- Dynamic entity mapping
- Filesystem registration
- Context resolution
- Publish path parsing
The only major difference is that the configuration itself is no longer required to live permanently on a shared network path. The filesystem model itself remains one of the foundational parts of the Toolkit.
Core API management in distributed setups
Toolkit Core (tk-core) management changes substantially in distributed systems.
In centralized configurations:
- projects may share a common core
- administrators may manually upgrade core versions
- core swapping is common
In distributed configurations:
- descriptors usually define the core version
- ToolkitManager resolves compatible versions
- tk-core may be bundled dynamically
- local cache consistency becomes important
Core Upgrade Commands
Traditional commands like ‘tank core’ or ‘tank core --version v0.20.25’ are more commonly used in centralized setups. In distributed environments, changing core versions is generally handled upstream through descriptor versioning.
attach_to_core
This command allows configurations to point to an external tk-core installation. Useful for testing, debugging, sandbox upgrades and temporary validation.
Bundle cache management
Distributed configurations rely heavily on bundle caching. Bundles include engines, apps, frameworks or descriptors. Toolkit stores bundles locally.
If bundles are missing or corrupted, this command can rebuild or pre-cache them
tank cache_apps
Descriptor documentation here…
Troubleshooting distributed configurations
Limitations summary
Some important limitations to note when handling distributed configurations are:
1. Local changes are disposable; any changes made inside cached configs may be overwritten.
2. Debugging can become harder, each user may have:
- different caches
- stale bundles
- inconsistent local states
3. Storage root consistency becomes critical. There are differences in:
- capitalization
- UNC paths
- mapped drives
- Linux mount points
may break Toolkit path resolution. Documentation: https://developers.shotgridsoftware.com/9ea34457/
4. Distributed configurations still depend on local disk integrity. Corrupted caches may require:
- deleting bundle cache
- re-bootstrap
- reauthentication
- reinstalling Toolkit Desktop
5. Some legacy tank workflows are less relevant. Commands heavily designed around shared configs are less operationally useful. Examples like share_core, localize or move_configuration.
6. Version drift. Users may temporarily run different cached versions if descriptor refreshes are inconsistent.
Common troubleshooting scenarios
Cannot Resolve Template Data — Usually caused by:
- missing folders
- stale path cache
- unregistered folders
Fix: tank Asset tree01 folders, or: tank synchronize_folders --full.
Reference: Text instead of status icons
Local Storage Errors — Example: Path does not match any defined Local Storage. Cause: mismatch between:
- roots.yml
- Site Preferences → File Management
Reference: https://developers.shotgridsoftware.com/9ea34457/
Menu Items Not Updating — Fix: tank clear_shotgun_menu_cache. This rebuilds Toolkit menu caches.
Missing Bundles — Fix: tank cache_apps, or clear the local bundle cache.
Recommended studio best practices
Recommended
- Manage configs upstream through descriptors
- Use Git-based descriptors where possible
- Version every configuration change
- Automate bootstrap validation
- Use preview_folders before schema deployment
- Keep storage roots consistent across OS platforms
- Standardize mount points
- Maintain backup bundle caches
- Use sandbox descriptors for testing
Avoid
- editing cached distributed configs directly
- manually moving cached configs
- manually editing FilesystemLocation entities
- changing roots.yml without synchronizing caches
Where to find more API articles
We are tagging our articles with adn-articles so they are easy to filter in the community forum. These posts cover API-related topics designed to help you apply Autodesk Platform Services and Flow Production Tracking APIs more effectively in real production workflows with common workflow implementations, troubleshooting guides and research insights.
We hope these resources become valuable to you. Stay tuned!*
.