Nuke Deadline Render Farm Integration - Bootstrap the ShotGrid Pipeline Toolkit engine

Hello team! I’m working on a little project to attempt to save our artists a few clicks. Currently, before sending a Nuke job to the farm, we have to go to the ShotGrid menu, and convert our ShotGrid write nodes to Nuke Write nodes. In our current setup, we have implemented the method that reveals the “Convert SG Write Nodes to Write Nodes…” menu items, and it’s working great. However, the documentation seems to suggest that with a little elbow grease there’s a way to automatically do this at rendertime. I’m having some trouble getting this set up though. Has anyone had any success with this?

You can bootstrap the api on the rendernodes so the tk-multi-writenode is fully functional there, you can just send the job with a tankwrite node and make sure the rendernode can see the gizmo file in its nuke path, or you can use the tk-multi-writenode app function to convert the writenodes in our script to normal nuke writenodes.

Each of these methods have pros and cons:

1) Bootstrap Toolkit
Adds a little overhead to the render per task, depending on the amount of nodes could slow down the site due to extra api communications.

2) Submit the Nuke script with TankWrites and make sure deadline nodes can access gizmo
This works well except I have found bugs where the writenode doesnt keep the settings on the farm.

3) Convert writenodes, send script, convert back
This works well in small scripts but gets slow in big scripts.
Also information on the writenode settings may get lost.

1 Like

Thanks so much for the quick reply, @Ricardo_Musch! I’m the most interested in option 1. We have option 3 currently working in our setup, and so far we have not experienced any issues, but any place I can save the comp team a few clicks is worth a little overhead in my book! I don’t fully understand what you mean when you say that the number of nodes can slow down the site though. Can you please elaborate on that? Thanks!

@Ricardo_Musch On a personal note, it’s also really nice to meet you. I’m currently working on implementing your getShotgunData tool to help us with our slates!

1 Like

@Ricardo_Musch On a personal note, it’s also really nice to meet you. I’m currently working on implementing your getShotgunData tool to help us with our slates!

The pleasure is all mine!
Oh my! Thats long ago! I hope the code is not so horrible haha.

So if you are bootstrapping on a farm then just be carefull that:

  1. 20, 30, 40… rendernodes are not all bootstrapping at the same time, I can imagine the site/api can becomes a bit sluggish because of the extra traffic (and since rendering usually goes on all/every minute of a day).

  2. Similarly, depending on where your config is stored, if its on a network drive.
    Loading python files from a network drive (NFS/Samba) is slow and causes a lot of overhead due to the small file sizes and the vast amount of them.

At our studio, we’re using option #2 and did experience problems with EXRs written out on the farm with the wrong settings (ex. compression).
Added a checkbox to our Deadline submitter for option #3, however it’s a little hacky because we’re dependent on promoted knobs and apparently the convert feature can’t restore them. so we’re converting and then relying on nuke.Undo. The struggle is real