I encountered an issue when I install tk-core via pip, that I cannot bootstrap an SGTK instance via the toolkit manager.
I am running this code:
import tank
sa = tank.authentication.ShotgunAuthenticator()
user = sa.create_script_user(api_script='SOME-USER',
api_key='SOME-KEY',
host='https://SOME-SITE.shotgunstudio.com')
mgr = tank.bootstrap.ToolkitManager(sg_user=user)
mgr.pipeline_configuration = "Primary"
mgr.do_shotgun_config_lookup = True
mgr.plugin_id = "basic.*"
engine = mgr.bootstrap_engine("tk-shell", entity={'type': 'Project', 'id': 123})
and get this error:
File "[...]\test_sgtk.py", line 31, in setUpClass
engine = mgr.bootstrap_engine("tk-shell", entity={'type': 'Project', 'id': 123})
File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 490, in bootstrap_engine
tk = self._bootstrap_sgtk(engine_name, entity)
File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 1098, in _bootstrap_sgtk
config = self._get_updated_configuration(entity, progress_callback)
File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 1055, in _get_updated_configuration
config.update_configuration()
File "[...]\lib\site-packages\tank\bootstrap\cached_configuration.py", line 356, in update_configuration
raise TankBootstrapError(tank.bootstrap.errors.TankBootstrapError: Configuration could not be installed: Cannot execute hook '[...]\lib\hooks\bootstrap.py' - this file does not exist on disk!.
It’d be great if that would work so, because then I could run my tests in a virtual environment. Does anybody know how to fix this?
I just looked this up, found a support ticket with the exact issue… and realized that you were the one that had submitted it. Sadly, not a lot has changed since you submitted that ticket nearly a year ago.
There are quite a few client requests attached to the internal ticket, so I’ll bring it up again with the team and see if we can shift it higher in the backlog.
Having said that, I can offer a bit more info, and a potential workaround to get it to work with newer versions of tk-core:
First off, some details on why it’s breaking: “The ToolkitManager has a dependency on the hooks folder, which is not included when pip installing tk-core, so bootstrapping is now broken when using a pip installed core.”
And a workaround that one client suggested – I can’t vouch that this will work for sure, but it might be worth trying:
# Define some required packages to be included.
DEPENDENT_PACKAGES = [
"shotgun_api3==3.0.40",
"sgtk==0.18.152"
]
# The syntax seen below for the shotgun_api3 package is expected,
# however, if we try to follow the same pattern for tk-core we get
# an sgtk package installed with the version "dev" rather than
# the correct 0.18.152. This is likely because of the get_version
# method found in tk-core's setup.py file. As a result, in order
# to download and install the correctly versioned package, we need to
# use the git+https syntax instead for that package.
DEPENDENT_LINKS = [
"https://github.com/shotgunsoftware/python-api/tarball/v3.0.40#egg=shotgun_api3-3.0.40",
"git+https://github.com/shotgunsoftware/tk-core.git@v0.18.152#egg=sgtk-0.18.152",
]
Let us know if it works!
Hope that helps! I’ve linked this post to the internal bug ticket, so we’ll be notified when it’s resolved, and will notify you here in turn (in addition to on that old ticket).
Thanks for bringing the issue up again with the team!
I tested the client suggestion but it has the same result for me.
I think what causes the issue is that you are not including the hooks folder in the package data of the setup.py (and probably some other code adjustments). Hope that helps a little and crossed fingers that this can be fixed quickly.
Yup, that’s it. These are the details from the internal ticket:
The ToolkitManager has a dependency on the hooks folder, which is not included when pip installing tk-core, so bootstrapping is now broken when using a pip installed core. We should write a test that pip installs Toolkit and bootstraps into a config so we don’t break this in the future
One way to tackle this bug would be to
a) Move the hook base class inside the bootstrap framework (let’s say sgtk.bootstrap.BootstrapHook)
b) have the discovery logic use the hook from the bootstrap module
(Although, no guarantees that the suggested solution here would definitely be the way engineering decides to address the issue.)
Anyway, we’ll keep you posted as there are updates.
So i performed an “archaic” solution, copy the hooks folder from git repo to the path of my python 2.7 (C://Python27/lib) and i get another error:
TankDescriptorIOError: Failed to download into path …\AppData\Roaming\Shotgun\bundle_cache\tmp\8c7ccb3d0c0b48ee9ce23b9bc1916d44: HTTP Error 404: Not Found
Surely this is an error on my project configuration, either way think this is not a real solution but maybe can help.
Hi!
I wanted to share my current workaround for this issue. Launching the ToolkitManager this way only works if you import tank (or sgtk) from a completely checked out tk-core repo.
So do something like:
I haven’t tried this solution yet, but maybe doing a pip install -e git+https://github.com/shotgunsoftware/tk-core.git#egg=sgtk would work.
Because pip will clone the repository in a separate src folder and add add the package path to sys.path using an .egg-link file.
Just tried that and it does work! Good find! Your post will be the accepted solution till Shotgun fixes tk-core!
Thanks for the solution to this long standing issue!
I’m trying to load a toolkit setup via a descriptor path from the renderfarm and I keep running into this error even though the file exists on the machine:
..\p91c1190.basic.\cfg\install\core\hooks\bootstrap.py' - this file does not exist on disk!
Hi @Ricardo_Musch !
I think your issue is slightly different to the one described here. I also remember seeing this error but STGK simply went through it and did not stop bootstrapping the engine. Was a while ago though, don’t know the exact circumstances anymore… Hope you will be able to fix your error!
I came up with a solution for this that is hacky as hell. But basically:
# Install sgtk with pip as indicated in the documentation
# Do all of the steps that SG mentions in the docs but do not run bootstrap_engine. For example:
mgr = sgtk.bootstrap.ToolkitManager(sg_user=AUTH_USER_OBJECT)
mgr.plugin_id = "basic.*"
mgr.pipeline_configuration = "Primary"
mgr.base_configuration = "sgtk:descriptor:app_store?name=tk-config-basic"
for pc in mgr.get_pipeline_configurations(project):
if pc["name"] == "Primary":
pc_object = pc
# get the path for the cached core associated with the Primary pipeline config
core_descriptor = sgtk.descriptor.create_descriptor(AUTH_USER_OBJECT.create_sg_connection(), tank.descriptor.Descriptor.CORE, pc_object.associated_core_descriptor)
cached_core_path = os.path.join(core_descriptor.get_path(), "python")
# now here's the hacky bit. go through sys.modules and delete every reference to sgtk/tank
sg_loaded_modules = list()
for mname in sys.modules.keys():
if mname.startswith('tank') or mname.startswith('sgtk'):
sg_loaded_modules.append(mname)
for mname in sg_loaded_modules:
del sys.modules[mname]
# and delete sgtk
del sgtk
del mgr
# add the cached core location to sys.path, import again, and redo the toolkit manager stuff...
sys.path.insert(0, cached_core_path)
import sgtk
mgr = sgtk.bootstrap.ToolkitManager(sg_user=NEW_AUTH_USER_OBJECT)
mgr.plugin_id = "basic.*"
mgr.pipeline_configuration = "Primary"
mgr.base_configuration = "sgtk:descriptor:app_store?name=tk-config-basic"
# and now, bootstrap the engine
engine = mgr.bootstrap_engine("tk-shell", entity=sg_project)
Hi @nedwilson!
That is very hacky indeed! Thanks for posting your workaround! If I could mark two comments as answers, I would mark yours as well. I leave it as is for now, because I feel the current solution is (ever so slightly) cleaner. However your solution is totally viable when you do not have the possibility to install sgtk in editable mode!
.
Good find! Your post will be the accepted solution till Shotgun fixes 
Hope you will be able to fix your error!
Thanks for posting your workaround! If I could mark two comments as answers, I would mark yours as well. I leave it as is for now, because I feel the current solution is (ever so slightly) cleaner. However your solution is totally viable when you do not have the possibility to install sgtk in editable mode!