Shotgrid Desktop unaware of file and system changes

Pipeline Integrations
1

Hello!

While using Shotgrid Desktop, I recognized that it seems to cache pretty hard, so our dependency updates can’t go live, without the manual reloading of a project’s configurations, since it is simply unaware of the file changes happening in the background.

Eg. We’re using rez packages for python dependencies. These rez packages update from time to time. In Shotgrid Desktop if I loaded a project and started a software which is using “rez_pkg 1.0”. If in the background rez_pkg 1.1 arrives, then I relaunch the software from the Desktop launcher, it still only sees rez_pk 1.0. Same goes with system environment variables, no changes are recognized.
Only if I go back to all projects and reload the project, will it update. Which is quite cumbersome for the users to do, anytime something changes.

We’re using a distributed configuration, but the config itself changing is not the issue right here.
Is there a a way to refresh the Desktop launcher, so it becomes aware of the updates that happened in the backround?
Some python or tank commands maybe?

Thanks in advance! :slight_smile:

2

  1. System Environement Variable changes
    To my knowledge it is impossible for running software to update the variables from the system without a restart of said software. If SG Desktop launches your applications (likely) then your applications pick up the Environment that ShotGrid Desktop was launched from.

  2. REZ Package updates
    Not sure how your REZ environment works or is triggered, I use tk-cpenv instead which loads packages at app launch and can be updated without reloading sg desktop/project.

Are your rez updates happening via System Wide Environment Variables?
That woudl explain why they don’t update.

1 Like
3

Thank you for your patience for my late reply.

It’s not really about REZ, it’s about not registering new folders. When a new package appears on the machine, SG launcher is not seeing it until the config is reloaded. Even if I delete a folder, SG still thinks it’s there, and then the software, or python, or REZ fails, becuase it’s not there anymore.
That’s why I would want to refresh/reload it from script, so next time the user starts a program, SG launcher and the REZ in it can use the newer packages.

I hope I could explain it right this time :slight_smile:

4

I’m not sure as I dont have experience with the SG REZ integration.

For CPENV the software will load the package or updates package the moment software is launched since it reads this from the database at boot time.

Without much mnore specifics on what you are seeing and what you mean this is hard to help with.

1 Like
5

I still don’t think it’s REZ related.
The whole configuration, the environment variables, the paths, everything is cached and won’t be updated until I reload the config manually with the Shotgrid desktop.

6

But how are you calling the rez packages and setting their values?

Any link to code you have?
Or a overview of where that happens?

7

My code goes like this. It takes a list of packages and some environment variables and creates a REZ env with ResolvedContext, then in the end starts the DCC in it.

from rez.resolved_context import ResolvedContext
from rez.config import config

rez_parent_variables = rez_info.get("parent_variables", [])           
rez_packages = rez_info.get("packages", [])

config.parent_variables = rez_parent_variables
context = ResolvedContext(rez_packages)
parent_env = os.environ.copy()

launcher_process = context.execute_shell(
    command=cmd,
    parent_environ=parent_env,
    stdin=False,
    block=False,
)
exit_code = launcher_process.wait()

This code runs every time a software is started from the SG Desktop.

8

And where are these rz environment variable stored?

Because any process you launch inherits the system environment variables however since Sg Desktop launches software itself, the software will inherit SG Desktop’s set of environment variables that it got launched with unless these are changed in the running Sg desktop session.

So for example:

On my machine I have an env called REZ_PACKAGES with value package1, package2.
I now launch SG Desktop.
It will inherit these system variables.
Any software launched from SG Desktop will launch with this ENv variable.

Now I change the system env variable to have package1 as value.
For this to affect SG Desktop and its subsequent software launches, you would have to restart SG Desktop so it can pick up the system env.

I use CPENV and that retrieves the packages it needs to load in the before-launch hook so they are always current.

9

It’s actually just PYTHONPATH, and it might be even unneccessary. Without it everything works the same. So it might be irrelevant.

I run the code to create the REZ environment from tk-multi-launchapp.yaml by giving it the python script as a hook_app_launch. I think this should be very similar to the before-launch hook, since it runs every time before launching the software. Yet when REZ collects the the packages it doesn’t register any changes in the package repository. Not when I update a package, not when I delete one. Like the REZ package repository is cached.
I’ve only experienced this in SG Desktop so far.