Change sgtk file and folder creation methods- overwrite tank core?

Pipeline Integrations
1

Hi,

we are looking to use our own python module whenever shotgun is creating folders or files.
Think of it as a custom python os module.

There are tk-apps like tk-shotgun-folders that are configurable with hooks.
However we like to make a change that affect all other tk-apps as well.

The best place to do a modification like that seems to be in
<path_to_sg_config>/install/core/python/tank/util/filesystem.py where the function such as ensure_folder_exists and .touch_file could be overwritten.

Essentially that is a modification to the tank module and with every future tank core update this modification would be undone. So I am really expecting that it is not a recommended thing to do.

Which leads to the questions:
1.) Is there a better way to change file and folder creation that is used by through out all of sgtk ?
(similar to filesystem.py)
2.) Is there an easy way to make this change configurable ?
(Not having to change anything in install/core)

4 Likes
3

Hi @sogrday,

I don’t think you can configure without branching but I’ll check with the TK team to know more.

-David

4 Likes
4

Hey!
The process folder creation core hook will be called when processing the project folder creation and any files that that might entail:

However, the method you have called out, certainly doesn’t call this, I think we use that for any time we need to create say configuration folders, such as the bundle cache or things like that.

If you only need control over the project folder and file creation then the core hook will be what you need, but if you need to control literally everything that we ever create on disk, then you are going to have to start making changes. At that point, I wonder if there isn’t a better approach, that can be taken, maybe even patching things at a lower level like os, but that could also lead to unexpected problems.

5 Likes
5

Hey @philip.scadding

thanks for the prompt reply.

We definitely do not want to modify the python os module, or use it’s namespace.

sgtk.Sgtk.create_filesystem_structure seems like a good example and kind of what I was looking for.

1.) First, however compared to filesystem.ensure_folder_exists that hook is not used much throughout the tk-apps. Instead filesystem.ensure_folder_exists seems to be the favorite call in a lot of tk-apps. Meaning that replacing these call is quite a bit of work and code to replace in your well working number of tk-apps right ?

2.) Secondly, I am wondering where and how you would configure such a core hook.?
For example there is https://github.com/shotgunsoftware/tk-config-default2/blob/master/core/hooks/pick_environment.py but I am looking for the .yml file which contains something like this

...
setting.core.hooks:
   pick_environment: {self}/pick_cnvironment.py:{config}/pick_environment.py
...

Thanks !
Enno

2 Likes
6

Whilst the apps generally require that the folder structure was created with a call like sgtk.Sgtk.create_filesystem_structure, sometimes templates could include folders that aren’t actually in the schema, and so the apps try to ensure that the folders exist.

I would recommend putting a suggestion in on our roadmap page for that logic to be moved over into a hook. But for now if you need to manage all instances of of folder and file creation then you will have to take things over unfortunatly.

Core hooks unlike app hooks don’t require any YAML setup, you just copy them from tk-core over to your config/core/hooks folder. This is because core hook will run regardless of the environment so it doesn’t get configured in the env YAML files…

2 Likes
7

Great thank you,

In that case, could you point me to the file that’s contain the logic of inheriting and running from install/core/hooks to config/core/hooks ?

Thanks @philip.scadding

2 Likes
8

Sure! Here is the place where it decides whether to run the hook from tk-core install or from your config:

There isn’t any inheritance here though, when you take over the hook it just runs your version.

2 Likes