Tk-houdini engine for Houdini 20

donat · October 28, 2023, 9:14pm

Hi,

We started testing the beta release of Houdini 20.0.485

I almost accidentally found a way to make the tk-houdini engine work in Houdini 20. If anyone else is looking to test this :

With tk-houdini v1.8.6, look into the folder structure of the engine. You’ll find two directories called “python3.9libs” in two separate locations. Duplicate each of those with their content, and rename them “python3.10libs”

In our pipe, with these changes, the SG menu comes up in Houdini. Without these changes, there’s no SG menu at all. I didn’t test this thoroughly but I hope it helps someone.

To the SG team : can you look into this when Houdini 20 is officially released ? Thanks in advance

Donat

johnH · November 8, 2023, 6:57pm

I created an account to say: thank you!

donat · November 8, 2023, 7:53pm

@johnH you’re welcome!

carlos-villavicencio · January 24, 2024, 3:44pm

We recently merged a community contribution to fix the Python 3.10 support for Houdini 20.

This is going to be released soon. Thank you for your patience.

donat · July 28, 2024, 12:24pm

Hi,

Could you please add support for Houdini 20.5 to the tk-houdini engine ?

I did the similar fix as previously (see above) and it seems to work. Albeit for some reason Houdini 20.5.278 crashes when launching it with a compatible HtoA 6.3.3.0 on Windows. But before investigating that bug I’d like to request an official tk-houdini engine

Regards

BTW : I filed a support case requesting this update to tk-houdini

mmoshev · August 5, 2024, 3:22pm

+1 for 20.5
We copied over the python3.11 directory, but it crashes on most machines (with traceback pointing to python execution). So far it seems to work on only one machine on Windows.

donat · August 6, 2024, 9:43am

@mmoshev do you use HtoA ? In my case I only have crashes when trying to start Houdini 20.5 with HtoA. Starting from SG without HtoA is working (only tested on Windows)

My traceback also points to some python dlls

Regards

BTW After contacting the Autodesk support, they gave me this internal ticket reference for the update of the houdini engine to support 20.5: SG-35936 (notice the SG in there… )

mmoshev · August 6, 2024, 9:53am

We do not. Actually we have not installed any plugins for 20.5 yet (and lots of them are not compatible anyway).
I suspect you stack trace might be similar to ours.

donat · August 6, 2024, 10:03am

FYI, our crash log ;

Crash report from e8938948; Unknown App Version 20.5.278 [windows-x86_64-cl19.37]
Uptime 18 seconds
Tue Aug 6 12:00:59 2024
Caught signal 11

Traceback from 34420 ThreadId=0x000053ac
CURRENT THREAD 21420
+0x7ffb71c7eb3c [PyWarnings_Init] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71c877c1 [PyEval_EvalFrameDefault] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71c8b83e [PyEval_EvalFrameDefault] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71b8573d [PyFunction_Vectorcall] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71b850d4 [PyObject_FastCallDictTstate] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71b859c5 [PyObject_Call_Prepend] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71bf64ac [PyType_Ready] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71be9d5f [PyType_Name] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71b852b1 [PyObject_MakeTpCall] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71b85511 [PyObject_Vectorcall] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71c8834d [PyEval_EvalFrameDefault] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x7ffb71c83861 [PyEval_EvalCode] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\python311.dll
+0x132870fd [PY_CompiledCode::evaluateUsingDicts] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libPY.dll
+0x13286f1d [PY_CompiledCode::evaluate] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libPY.dll
+0x13295743 [PYrunPythonStatementsFromFile] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libPY.dll
+0x1566afee [MOT_Director::execPythonRCFiles] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libMOT.dll
+0x156669ff [MOT_Director::MOT_Director] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libMOT.dll
+0x1238e4a6 [OPUI_MainApp::initApplication] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libOPUI.dll
+0x1122d872 [AP_Interface::loadWindowGeometry] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libSI.dll
+0x1122e585 [myWinMain] C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\libSI.dll
+0x1400015a2 C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\houdini.exe
+0x140001a32 C:\Program Files\Side Effects Software\Houdini 20.5.278\bin\houdini.exe
+0x7ffc4bc57614 [BaseThreadInitThunk] C:\Windows\System32\KERNEL32.DLL
+0x7ffc4bfc26a1 [RtlUserThreadStart] C:\Windows\SYSTEM32\ntdll.dll

Will · August 8, 2024, 10:47am

Hey. I had the same issue as above with the same crash log. I tracked the issue down to tank.platform.engine line 1563: if not QtGui.QApplication.instance():. It seems to be getting an instance of the QApplication that causes the crash.

After a bunch of messing around it seems that if you’ve accessed the QApplication before the engine starts, it doesn’t crash.

This is horrendous hack but I’ve added the following to plugin_bootstrap.py at line 85 just before the engine starts:

from PySide2 import QtWidgets
QtWidgets.QApplication.instance()

I don’t know exactly what is going on but, for me, this no longer crashes. It’ll do until ShotGrid release official support.

ETA: This is even weirder. To make things a little neater I tried moving the above code to the start of plugin_bootstrap.py. That crashes instantly. Something is going on around there that makes Houdini particularly fragile when accessing the QApplication.

Dan_Bradham · August 15, 2024, 5:16pm

[DEBUG tk-houdini] Engine init: Instantiating <Sgtk Engine 0x11d98d7d0: tk-houdini, env: project>
[DEBUG tk-houdini] Engine init: Current Context: Project 24-XXX-LUCIDLINK
[DEBUG tk-houdini] <Sgtk Engine 0x11d98d7d0: tk-houdini, env: project>: Initializing...
[DEBUG qt_importer] Requesting Qt4-like interface
[DEBUG qt_importer] Unable to import module 'QtWebKit': No module named 'PySide2.QtWebKitWidgets'
[DEBUG qt_importer] Unable to import module 'QtWebEngineWidgets': module 'PySide2.QtWebEngineWidgets' has no attribute 'QtWebEngineWidgets'
[DEBUG qt_importer] Imported PySide2 as PySide.
[DEBUG qt_importer] Requesting Qt5-like interface
[DEBUG qt_importer] Imported PySide2.
[DEBUG qt_importer] Requesting Qt6-like interface
[DEBUG qt_importer] Qt6 interface not implemented for Qt5
[DEBUG qt_importer] No Qt matching that interface was found.
18304: Fatal error: Segmentation fault

In this debugging message it appears that their bootstrapping code fails to import the QtWeb* modules while attempting to load and rename the PySide2 modules. I don’t have a fix for this at the moment either, just wanted to add the debugging info to this thread for visibility. I think initially the way they built their Qt compatibility up over time made sense. But after years of patching for new releases of Python and Qt / PySide / PySide2 and now PySide6, this entire process is a whirlwind of pain. The best solution would be to use qt.py which is a well supported and more complete compatibility module. It would be painful, but it would probably bring a semblance of sanity to their bootstrapping process and allow them to spend less time focusing on patches for Qt each year.

Dan_Bradham · August 15, 2024, 5:34pm

Here is what ended up working for me. This probably only works with Python 3.7+ but, we are well beyond Python2 at this point aren’t we?!?! I couldn’t tell you why the old __import__ function fails to work in H20.5.

tk-core/python/tank/util/qt_importer.py

diff --git a/python/tank/util/qt_importer.py b/python/tank/util/qt_importer.py
index afec0236..7d27501c 100644
--- a/python/tank/util/qt_importer.py
+++ b/python/tank/util/qt_importer.py
@@ -12,8 +12,10 @@
 Qt version abstraction layer.
 """
 
+import importlib
 import os
 import pkgutil
+import sys
 
 from ..log import LogManager
 
@@ -157,11 +159,11 @@ class QtImporter(object):
         """
         module = None
         try:
-            module = __import__(parent_module_name, globals(), locals(), [module_name])
-            module = getattr(module, module_name)
+            module = importlib.import_module(module_name, parent=parent_module_name)
         except Exception as e:
             logger.debug("Unable to import module '%s': %s", module_name, e)
             pass
+        sys.modules[module_name] = module
         return module
 
     def _import_pyside2(self):
@@ -257,9 +259,8 @@ class QtImporter(object):
 
         QtCore, QtGui = PySide2Patcher.patch(QtCore, QtGui, QtWidgets, PySide2)
         QtNetwork = self._import_module_by_name("PySide2", "QtNetwork")
-        QtWebKit = self._import_module_by_name("PySide2.QtWebKitWidgets", "QtWebKit")
-        QtWebEngineWidgets = self._import_module_by_name(
-            "PySide2.QtWebEngineWidgets", "QtWebEngineWidgets"
+        QtWebKit = self._import_module_by_name("PySide2", "QtWebKit")
+        QtWebEngineWidgets = self._import_module_by_name("PySide2", "QtWebEngineWidgets"
         )
 
         return (

mmoshev · August 21, 2024, 12:18pm

None of the solutions so far fixed it for me on Linux. Will also try on Windows.

Here is the crash dump on Linux. Not exactly the same, but similar, related to evaluating Python.
crash.mmoshev_bottleship.local_827622_log.log (6.8 KB)

donat · August 21, 2024, 1:18pm

FYI,

On the official shotgunsoftware github there’s a branch for H 20.5 support, not yet merged to master

Donat

mmoshev · August 21, 2024, 1:49pm

Thanks donat, I just tried the branch, but still crashes
This time on Windows, I get the console briefly, it says something about “can’t open resource” or similar (too fast to read it), and closes.
So far the QApplication instantiation sounds most plausible.

brett.lambright · August 22, 2024, 3:51pm

Just chiming in here. I have the same problem as @mmoshev.

For Linux I tried the production build (278) and the latest daily build of Houdini 20.5 (Python 3.11). Other plugins like HtoA seemed to load fine but when I launched through the toolkit environment then it would crash.

I can confirm the Houdini 20.5 Python 3.10 version doesn’t crash with the toolkit environment and that’s the same code, so it’s something to do with Python 3.11.

brett.lambright · August 22, 2024, 5:08pm

Seems to be crashing for me on a simple os.path.join call. Pretty weird. When I remove the code where it crashes, it just fails later on a different os.path.join call.

@mmoshev Can you set this environment variable export PYTHONFAULTHANDLER=1 and see where it crashes for you?

Fatal Python error: Segmentation fault

Current thread 0x00007f599efa7380 (most recent call first):
File "", line 83 in join
File ".../shotgun/install/core/python/tank/platform/engine.py", line 2936 in __get_platform_resource_path
File ".../shotgun/install/core/python/tank/platform/engine.py", line 1578 in _ensure_core_fonts_loaded
File ".../shotgun/install/core/python/tank/platform/engine.py", line 199 in __init__
File ".../shotgun/install/core/python/tank/platform/engine.py", line 3202 in _start_engine
File ".../shotgun/install/core/python/tank/platform/engine.py", line 3042 in start_engine
File ".../shotgun/install/app_store/tk-houdini/v1.8.8/python/tk_houdini/bootstrap.py", line 97 in bootstrap_classic
File ".../shotgun/install/app_store/tk-houdini/v1.8.8/classic_startup/python3.11libs/pythonrc.py", line 47 in classic_startup
File ".../shotgun/install/app_store/tk-houdini/v1.8.8/classic_startup/python3.11libs/pythonrc.py", line 65 in

mmoshev · August 22, 2024, 5:56pm

Thanks for suggesting PYTHONFAULTHANDLER.

The error doesn’t look quite the same here.
Seemingly it happens on loading shelf files (or rather destroy_tools, which calls hou.shelves.shelves().get(self._name, None)).
pythonfailure.log (6.9 KB)

eduardoChauca · August 29, 2024, 6:33pm

Hello @donat ,
I was testing in a Linux machine and found the same crash issue, most of the time crashes exist in version 20.5.278 but crashes don’t exist in version 20.5.332. I also tested on Windows getting the same result.

So I recommend using the version 20.5.332 with this new PR support for 20.5.

donat · August 30, 2024, 9:29am

Thanks @eduardoChauca
Will try Houdini 20.5.332 on Windozs as soon as possible

Topic		Replies	Views
Support for Houdini 20 Flow Production Tracking houdini	5	598	January 24, 2024
A new version of tk-houdini has been released! Pipeline Integrations release-notes , release-notes-english	10	829	October 10, 2024
Shotgrid menu doesn't appear in Houdini 20.0.634 Py 3.9 or Py3.10 Pipeline Integrations houdini	3	214	October 24, 2024
Houdini 19.5 ShotGrid Menu Missing Pipeline Integrations	11	1177	February 3, 2023
Novice - how to install latest tk_houdini? Pipeline Integrations fptr-tk	2	228	March 8, 2023

Tk-houdini engine for Houdini 20

Related topics