SG desktop broken pipe error

Pipeline Integrations
1

We sporadically see this behavior where the SG desktop (on Centos 7.9) becomes unresponsive and fails to launch apps. I hesitate to describe it as “frozen” since by all appearances basic UI components continue to work (at the very least you’re able to continue to click the apps as normal). Assuming the user is patient enough, the condition sometimes resolves itself after several minutes of waiting and the user sometimes finds themselves on the receiving end of dozens of simultaneous application launches (stacked up from the multiple button pushes). This state is accompanied by the following error:

Traceback (most recent call last):
  File "/home/vincent.wang/.shotgun/bundle_cache/app_store/tk-desktop/v2.6.0/python/tk_desktop/desktop_engine_site_implementation.py", line 355, in _handle_button_command_triggered
    "trigger_callback", "__commands", six.ensure_str(name)
  File "/home/vincent.wang/.shotgun/bundle_cache/app_store/tk-desktop/v2.6.0/python/tk_desktop/communication_base.py", line 97, in call_no_response
    return self._proxy.call_no_response(name, *args, **kwargs)
  File "/home/vincent.wang/.shotgun/bundle_cache/app_store/tk-desktop/v2.6.0/python/tk_desktop/rpc.py", line 408, in call_no_response
    self._connection.send(pickle.dumps((False, name, args, kwargs)))
  File "/home/vincent.wang/.shotgun/bundle_cache/app_store/tk-desktop/v2.6.0/python/tk_desktop/rpc.py", line 174, in send
    self._conn.send_bytes(payload)
  File "/opt/Shotgrid/Python3/lib/python3.7/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/opt/Shotgrid/Python3/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header + buf)
  File "/opt/Shotgrid/Python3/lib/python3.7/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
BrokenPipeError: [Errno 32] Broken pipe

Since it is exceedingly difficult to reproduce, we’ve not been able to isolate a case for further investigation, but our best guess is the python process that Desktop forks off is stuck waiting on a resource, likely a socket to communicate with Desktop. Also, this condition seems to frequently coincide with a second Desktop (zombie?) python process, although this could be a complete red herring. Long story short, we’re not sure, since the internals of Desktop are enough of a black box to render troubleshooting non-trivial.

Anybody else experienced this issue or anything similar and had success resolving it?
2021-10-12_11-36

2 Likes
2

This is also happening with MacOS use of SG Desktop as of last week.
Have not been able to reproduce “on demand” – but has the same behaviour as Nico described.

No success in resolving it to-date, but would be interested in a solution to troubleshoot / fix.

1 Like
3

Hi @nico.vandenbosch ,
We actually experienced the same issue (comparing the tracebacks).
We’re also not entirely sure what’s causing this, but for us it helped to increase the Windows Open File Limit (default is 512) with: import win32file; win32file._setmaxstdio(5000)
(Linux default is 1024; you can check it by using ulimit -n)

After doing so we saw, that something else in our codebase was producing other python errors, that were not visible before increasing the limit.
I could imagine, that the BrokenPipeError may be caused by a combination of: file limit reached and one or many error(s) occuring during engine initialization.
As this issue is hard to reproduce reliably it may as well not be the solution, but it could be worth a try.

2 Likes
4

Hi there!

Did this issue magically disappear or could you find a solution to work around it? We are facing this issue on a regular manner since a couple of weeks. Launching 3 or 4 apps might be enough to get this error and only reloading the config does ‘fix’ it.

Our open file limit is set to 4096 on Linux CentOS 7.9.

Greets,
Carlo

5

Having the same BrokenPipeError here on Windows 11.

Has anyone found a solve for this?

6

Considering Python2 support has been removed from all frameworks and apps since today, are you perhaps running an old version of Desktop?

7

Apologies for my late response here!

Desktop App v1.9.2 and tk-desktop v2.7.3.

I don’t suspect its related to Py2, have done my best to rid us of any Py2 code in out pipeline here.

8

Do you get the brokenpipe error after loading into the project?

9

Yeah its generally after Desktop app has been open for a while and it occurs when attempting to launch a DCC. No direct feedback that something’s gone wrong besides the app not launching. Have only seen this on Motionbuilder and Maya, haven’t observed on any of our custom apps.

1 Like
10

I’ve generally seen this in situations where our engine_init code takes a little longer to process, and a user clicks to launch an app before engine_init has completed (the apps appear before engine_init hook runs).
It sounds like your experience is different though… perhaps a user clicks on an app twice in quick succession?