# Pip install sgtk breaks ToolkitManager **URL:** \ **Category:** Pipeline Integrations\ **Tags:** python, bug\ **Created:** [August 14, 2020, 9:03am UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695 "2020-08-14T09:03:27Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Fabian](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/fabian/32/3508_2.png) [@Fabian](https://community.shotgridsoftware.com/u/Fabian)\ **Post date:** [August 14, 2020, 9:03am UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/1 "2020-08-14T09:03:27Z")
Hello everyone! I encountered an issue when I install tk-core via pip, that I cannot bootstrap an SGTK instance via the toolkit manager. I am running this code: ``` import tank sa = tank.authentication.ShotgunAuthenticator() user = sa.create_script_user(api_script='SOME-USER', api_key='SOME-KEY', host='https://SOME-SITE.shotgunstudio.com') mgr = tank.bootstrap.ToolkitManager(sg_user=user) mgr.pipeline_configuration = "Primary" mgr.do_shotgun_config_lookup = True mgr.plugin_id = "basic.*" engine = mgr.bootstrap_engine("tk-shell", entity={'type': 'Project', 'id': 123}) ``` and get this error: ``` File "[...]\test_sgtk.py", line 31, in setUpClass engine = mgr.bootstrap_engine("tk-shell", entity={'type': 'Project', 'id': 123}) File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 490, in bootstrap_engine tk = self._bootstrap_sgtk(engine_name, entity) File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 1098, in _bootstrap_sgtk config = self._get_updated_configuration(entity, progress_callback) File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 1055, in _get_updated_configuration config.update_configuration() File "[...]\lib\site-packages\tank\bootstrap\cached_configuration.py", line 356, in update_configuration raise TankBootstrapError(tank.bootstrap.errors.TankBootstrapError: Configuration could not be installed: Cannot execute hook '[...]\lib\hooks\bootstrap.py' - this file does not exist on disk!. ``` It’d be great if that would work so, because then I could run my tests in a virtual environment. Does anybody know how to fix this? Cheers, Fabian ---
**Author:** ![tannaz](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/tannaz/32/32_2.png) [@tannaz](https://community.shotgridsoftware.com/u/tannaz)\ **Post date:** [August 14, 2020, 11:12pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/3 "2020-08-14T23:12:54Z")
Hi Fabian – I just looked this up, found a support ticket with the exact issue… and realized that _you_ were the one that had submitted it. Sadly, not a lot has changed since you submitted that ticket nearly a year ago. 🙈 There are quite a few client requests attached to the internal ticket, so I’ll bring it up again with the team and see if we can shift it higher in the backlog. Having said that, I can offer a bit more info, and a potential workaround to get it to work with newer versions of `tk-core`: First off, some details on why it’s breaking: “The ToolkitManager has a dependency on the hooks folder, which is not included when pip installing tk-core, so bootstrapping is now broken when using a pip installed core.” And a workaround that one client suggested – I can’t vouch that this will work for sure, but it might be worth trying: ```auto # Define some required packages to be included. DEPENDENT_PACKAGES = [ "shotgun_api3==3.0.40", "sgtk==0.18.152" ] # The syntax seen below for the shotgun_api3 package is expected, # however, if we try to follow the same pattern for tk-core we get # an sgtk package installed with the version "dev" rather than # the correct 0.18.152. This is likely because of the get_version # method found in tk-core's setup.py file. As a result, in order # to download and install the correctly versioned package, we need to # use the git+https syntax instead for that package. DEPENDENT_LINKS = [ "https://github.com/shotgunsoftware/python-api/tarball/v3.0.40#egg=shotgun_api3-3.0.40", "git+https://github.com/shotgunsoftware/tk-core.git@v0.18.152#egg=sgtk-0.18.152", ] ``` Let us know if it works! Hope that helps! I’ve linked this post to the internal bug ticket, so we’ll be notified when it’s resolved, and will notify you here in turn (in addition to on that old ticket). ---
**Author:** ![Fabian](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/fabian/32/3508_2.png) [@Fabian](https://community.shotgridsoftware.com/u/Fabian)\ **Post date:** [August 17, 2020, 9:10am UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/4 "2020-08-17T09:10:22Z")
Hi Tannaz! Thanks for bringing the issue up again with the team! I tested the client suggestion but it has the same result for me. I think what causes the issue is that you are not including the hooks folder in the [package data of the setup.py](https://github.com/shotgunsoftware/tk-core/blob/571576d72867e65158d47176619acf0c537ee1d6/setup.py#L81) (and probably some other code adjustments). Hope that helps a little and crossed fingers that this can be fixed quickly. 🙂 ---
**Author:** ![tannaz](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/tannaz/32/32_2.png) [@tannaz](https://community.shotgridsoftware.com/u/tannaz)\ **Post date:** [August 17, 2020, 3:39pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/5 "2020-08-17T15:39:33Z")
Yup, that’s it. These are the details from the internal ticket: > The ToolkitManager has a dependency on the hooks folder, which is not included when pip installing tk-core, so bootstrapping is now broken when using a pip installed core. > **We should write a test that pip installs Toolkit and bootstraps into a config so we don’t break this in the future** > One way to tackle this bug would be to > a) Move the hook base class inside the bootstrap framework (let’s say sgtk.bootstrap.BootstrapHook) > b) have the discovery logic use the hook from the bootstrap module (Although, no guarantees that the suggested solution here would definitely be the way engineering decides to address the issue.) Anyway, we’ll keep you posted as there are updates. 😊 ---
**Author:** ![Dan\_Bradham](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/dan_bradham/32/2479_2.png) [@Dan\_Bradham](https://community.shotgridsoftware.com/u/Dan_Bradham)\ **Post date:** [October 22, 2020, 6:36pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/6 "2020-10-22T18:36:42Z")
Has there been any movement on this issue? I noticed that this is still a problem when I pip install tk-core. ---
**Author:** ![Jaziel](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/jaziel/32/3640_2.png) [@Jaziel](https://community.shotgridsoftware.com/u/Jaziel)\ **Post date:** [December 1, 2020, 8:51pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/7 "2020-12-01T20:51:21Z")
After of rplicate the code i gt the same error ☹ . So i performed an “archaic” solution, copy the hooks folder from git repo to the path of my python 2.7 (C://Python27/lib) and i get another error: > TankDescriptorIOError: Failed to download into path …\AppData\Roaming\Shotgun\bundle\_cache\tmp\8c7ccb3d0c0b48ee9ce23b9bc1916d44: HTTP Error 404: Not Found Surely this is an error on my project configuration, either way think this is not a real solution but maybe can help. ---
**Author:** ![Degraund](https://avatars.discourse-cdn.com/v4/letter/d/3ec8ea/32.png) [@Degraund](https://community.shotgridsoftware.com/u/Degraund)\ **Post date:** [January 15, 2021, 3:16pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/8 "2021-01-15T15:16:28Z")
My solution to the problem ERR\_PC - PC on which the error occurred DONOR\_PC - PC with SG Desktop of required configuration installed 1. Copy ~/.shotgun/bundle\_cache/app\_store from DONER\_PC to ~/.shotgun/bundle\_cache/app\_store on ERR\_PC. 2. Create a link ~/…/hooks to ~/.shotgun/bundle\_cache/app\_store/tk-core/vX.X.X/hooks/ 3. Download [https://github.com/shotgunsoftware/tk-shell.git](https://github.com/shotgunsoftware/tk-shell.git) to ~/.shotgun/bundle\_cache/app\_store/tk-shell/vX.X.X/ ---
**Author:** ![Fabian](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/fabian/32/3508_2.png) [@Fabian](https://community.shotgridsoftware.com/u/Fabian)\ **Post date:** [June 1, 2021, 11:18am UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/9 "2021-06-01T11:18:46Z")
Hi! I wanted to share my current workaround for this issue. Launching the ` ToolkitManager` this way only works if you import `tank` (or `sgtk`) from a completely checked out `tk-core` repo. So do something like: 1. `git clone git@github.com:shotgunsoftware/tk-core.git` 2. And before you `import tank` add a ```python import sys sys.path.append("/path/to/tk-core/python") ``` 3. Than my code from above will run just fine. Pip installing would still be preferable. Hope that helps someone in the future. ---
**Author:** ![Jaziel](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/jaziel/32/3640_2.png) [@Jaziel](https://community.shotgridsoftware.com/u/Jaziel)\ **Post date:** [June 1, 2021, 1:31pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/10 "2021-06-01T13:31:27Z")
I can confirm this, something weird happens with the pip installation, import “by hand” the module works fine to me too. ---
**Author:** ![michael.delaporte](https://avatars.discourse-cdn.com/v4/letter/m/f05b48/32.png) [@michael.delaporte](https://community.shotgridsoftware.com/u/michael.delaporte)\ **Post date:** [June 4, 2021, 8:07am UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/11 "2021-06-04T08:07:15Z")
I haven’t tried this solution yet, but maybe doing a `pip install -e git+https://github.com/shotgunsoftware/tk-core.git#egg=sgtk` would work. Because pip will clone the repository in a separate `src` folder and add add the package path to sys.path using an `.egg-link` file. ---
**Author:** ![Fabian](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/fabian/32/3508_2.png) [@Fabian](https://community.shotgridsoftware.com/u/Fabian)\ **Post date:** [June 4, 2021, 1:06pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/12 "2021-06-04T13:06:11Z")
Hi @michael.delaporte ! Just tried that and it does work! 🥳 Good find! Your post will be the accepted solution till Shotgun fixes `tk-core`! Thanks for the solution to this long standing issue! Cheers, Fabian ---
**Author:** ![Jaziel](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/jaziel/32/3640_2.png) [@Jaziel](https://community.shotgridsoftware.com/u/Jaziel)\ **Post date:** [June 9, 2021, 4:32pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/13 "2021-06-09T16:32:23Z")
> [@michael.delaporte](#): > > pip install -e git+https://github.com/shotgunsoftware/tk-core.git#egg=sgtk best solution so far… thank you! 🙏 ---
**Author:** ![Ricardo\_Musch](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/ricardo_musch/32/405_2.png) [@Ricardo\_Musch](https://community.shotgridsoftware.com/u/Ricardo_Musch)\ **Post date:** [October 20, 2021, 4:57pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/14 "2021-10-20T16:57:32Z")
Is anyone still having issues with this? I’m trying to load a toolkit setup via a descriptor path from the renderfarm and I keep running into this error even though the file exists on the machine: `..\p91c1190.basic.\cfg\install\core\hooks\bootstrap.py' - this file does not exist on disk!` ---
**Author:** ![Fabian](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/fabian/32/3508_2.png) [@Fabian](https://community.shotgridsoftware.com/u/Fabian)\ **Post date:** [October 22, 2021, 5:27pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/15 "2021-10-22T17:27:27Z")
Hi @Ricardo_Musch ! I think your issue is slightly different to the one described here. I also remember seeing this error but STGK simply went through it and did not stop bootstrapping the engine. Was a while ago though, don’t know the exact circumstances anymore… 😬 Hope you will be able to fix your error! Cheers, Fabian ---
**Author:** ![nedwilson](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/nedwilson/32/679_2.png) [@nedwilson](https://community.shotgridsoftware.com/u/nedwilson)\ **Post date:** [April 12, 2022, 8:21pm UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/16 "2022-04-12T20:21:47Z")
I came up with a solution for this that is hacky as hell. But basically: ```auto # Install sgtk with pip as indicated in the documentation # Do all of the steps that SG mentions in the docs but do not run bootstrap_engine. For example: mgr = sgtk.bootstrap.ToolkitManager(sg_user=AUTH_USER_OBJECT) mgr.plugin_id = "basic.*" mgr.pipeline_configuration = "Primary" mgr.base_configuration = "sgtk:descriptor:app_store?name=tk-config-basic" for pc in mgr.get_pipeline_configurations(project): if pc["name"] == "Primary": pc_object = pc # get the path for the cached core associated with the Primary pipeline config core_descriptor = sgtk.descriptor.create_descriptor(AUTH_USER_OBJECT.create_sg_connection(), tank.descriptor.Descriptor.CORE, pc_object.associated_core_descriptor) cached_core_path = os.path.join(core_descriptor.get_path(), "python") # now here's the hacky bit. go through sys.modules and delete every reference to sgtk/tank sg_loaded_modules = list() for mname in sys.modules.keys(): if mname.startswith('tank') or mname.startswith('sgtk'): sg_loaded_modules.append(mname) for mname in sg_loaded_modules: del sys.modules[mname] # and delete sgtk del sgtk del mgr # add the cached core location to sys.path, import again, and redo the toolkit manager stuff... sys.path.insert(0, cached_core_path) import sgtk mgr = sgtk.bootstrap.ToolkitManager(sg_user=NEW_AUTH_USER_OBJECT) mgr.plugin_id = "basic.*" mgr.pipeline_configuration = "Primary" mgr.base_configuration = "sgtk:descriptor:app_store?name=tk-config-basic" # and now, bootstrap the engine engine = mgr.bootstrap_engine("tk-shell", entity=sg_project) ``` ---
**Author:** ![Fabian](https://sea1.discourse-cdn.com/shotgunsoftware/user_avatar/community.shotgridsoftware.com/fabian/32/3508_2.png) [@Fabian](https://community.shotgridsoftware.com/u/Fabian)\ **Post date:** [April 13, 2022, 8:37am UTC](https://community.shotgridsoftware.com/t/pip-install-sgtk-breaks-toolkitmanager/9695/17 "2022-04-13T08:37:48Z")
Hi @nedwilson! That is very hacky indeed! 🙂 Thanks for posting your workaround! If I could mark two comments as answers, I would mark yours as well. I leave it as is for now, because I feel the current solution is (ever so slightly) cleaner. However your solution is totally viable when you do not have the possibility to install sgtk in editable mode! Cheers, Fabian