Shotgun API Tips and Tricks Megapost

Flow Production Tracking
, , , , ,
1

Hey folks,

I wanted to kickstart something here – we/I get a lot of questions from various places about “how to do X” with the API. The API documentation itself is great, but it doesn’t really tell you how to think your way through a particular problem set. If you aren’t familiar with the Shotgun schema or how different entities interrelate, you can have a hard time figuring out how to find the exact information you’re looking for. I know that when I started out, I oftentimes knew what I needed, but had a hard time figuring out how to ask the right question.

If you find yourself in the same boat, you’re in the right place! In this topic, I’m going to post examples of a few things that I’ve had to figure out in the past, in hopes that it will help someone else out there. The examples are specific-to-general, but the thing I like about them is that each one has bits in it that are 100% applicable to other kinds of queries. If you’ve had a particularly gnarly “how can I get at X data from Shotgun” in the past, please feel free to share here with a post of your own! The more, the merrier.

image
image834×1000 311 KB

blowin' up

9 Likes
2

Filtering records for Entities with a combination of Pipeline Step Statuses

Sometimes you might want to find a series of Entities that are, say, final in lighting but CBB in comp, or ‘in review’ in rigging and not ‘CBB’ for something else. This is very easy to do in the UI, but in the API it can be a little brain-bendy.

To start, let’s use a quick one-liner to find out which steps in our Shot pipeline are visible in a specific Project:

p = {'type':'Project','id':42}
[(k, v['name']['value'], v['visible']['value']) for k,v in sg.schema_field_read('Shot', project_entity=p).iteritems() if k.startswith('step_')]

We’ll get a return of something like:

[('step_133', 'Tracking', True),
 ('step_35', 'Layout', True),
 ('step_106', 'Animation', True),
 ('step_2', 'Online', True),
 ('step_0', 'ALL TASKS', True),
 ('step_7', 'Light', True),
 ('step_8', 'Comp', True),
 ('step_6', 'FX', True),
 ('step_135', 'Environment', False),
 ('step_136', 'CFX', False),
 ('step_134', 'Roto', False)]

Armed with which Steps go with which names, we can construct a filter to query. Here is a simple query that defaults to AND-ing components together. If you want to OR something, you’ll need to use a compound filter syntax like this one.

# find Shots whose Animation is In Review or Final,
# and whose Comp is neither Final nor CBB.
filters = [
    ['step_106.Task.sg_status_list', 'in', ['ir', 'fin']],
    ['step_8.Task.sg_status_list', 'not_in', ['fin', 'cbb']]
]

shots = sg.find('Shot', filters, ['entity'])
7 Likes
3

Get a list of all Pipeline Steps that belong to a specific Project

Getting at this information can be counterintuitive; you might think that you’d need to do a schema_field_read() or schema_entity_read(), but they don’t quite provide the information you want. Fortunately, schema_read() not only has the info we need, but it’s also project-context-aware; we just need to filter out the noise, because a full schema read is a lot of data.

In this example, we’re only looking at Assets, so you’d need to do an iteritems() run (similar to the post above) for each Entity type for which you need Pipeline Steps, since they’ll be different for each (Asset, Shot, etc.). Also note that this data that doesn’t change often and is relatively slow to query, so if you’re writing tools, the schema is something you could hang on to and only re-query when you’re confident that your copy is no longer reliable.

# pick a Project
p1 = {'type':'Project', 'id':86}
# pick an Entity type
ent = "Asset"

# using sg.schema_read, grab `name` and `visible` for keys that
# start with `"step_"`.
results = [(v['name']['value'], v['visible']['value']) for k,v in sg.schema_read(p1)['Asset'].iteritems() if k.startswith('step_')]

[('Rig', True),
('Model', True),
('Animation', False),
('Art', True),
('ALL TASKS', True),
('Texture', False)]
7 Likes
4

Creating Notes that have Attachments

It’s a little janky to create Notes with Attachments via the API, but there is a way to do it. Props to @bouchep for the genesis on this one!

In order to create a note with an attachment in the body, the attachment has to already exist so that the Note creation can reference it properly. The issue is, and I’m sure you’ve noticed, that there’s no way to upload something without something already existing to link it to. Very frustrating for Note creation via the API! So one workaround is to keep a project-based dummy Version around to upload attachment/annotations to, so that they exist and have an id that the note creation process can reference. Here’s how I did it:

# create a project-based dummy version to upload attachments to,
# if one doesn't already exist
dummy = sg.find_one('Version', [['code','is','dummyVersion']])
if dummy == None:
    dummy = sg.create('Version', {'description':'dummy version to link files against','code':'dummyVersion','project':{'type':'Project','id':89}})

With this dummy Version in place, we can now proceed:

# find the version you want to attach a note to
version = sg.find_one('Version',[['id','is',6961]])

# upload a new Attachment to the dummy, the result is the id of the new Attachment
att_id = sg.upload('Version', dummy['id'],'magneto.jpg')
attachment = sg.find_one('Attachment', [['id','is',att_id]],['created_at'])

# this create call uses the attachment created just above
sg.create(
        'Note',
        {
            'project':{'id':89,'type':'Project'},
            'created_at':attachment['created_at'],
            'note_links':[version], #note_links expects an array of entity hashes
            'content':'new simultaneous upload test',
            'attachments':[{'type':'Attachment','id':att_id}]
            }
        )

Note in the example above that I’m using the created_at date from the Attachment in the Note. This is something we key on internally when displaying Notes. If the times don’t match, the attachment will look like a separate reply, even though it’s not. If we steal the creation date from the attachment and make that the date in the Note, then the file will look like it’s in the body of the original note (though it’ll say “Attachments” instead of “Annotations”). Note that you can specify creation date at the time of creation, but you can’t modify it after the fact, so be aware!

8 Likes
5

Fix your Statuses!

In my days as a coordinator before joining Shotgun, I was working with a production team for an animated short. Asset production and Layout was largely wrapped, and we were moving towards Animation/early Lighting. In an effort to prep for that work, the director wanted to clean up all the Task statuses that were Asset/Layout specific, and add/adjust for the next stage of the pipe.

Now, my rookie Shotgun self didn’t realize at the time that Task statuses are global, and not bespoke to an Asset, or the Layout work in a Shot. So when I deleted statuses from the list, and renamed others, it affected all the in-flight and finished Tasks. Suffice to say, wiping out the status info for a couple of departments was not well received.

The Event Logs to the rescue! With the script below I was able to remap all the now blank statuses. Hope it helps anyone else who finds themselves in a similar situation!

from datetime import datetime, timedelta

# Find all Task change events in a certain range, in this example between today and yesterday
date_range = [datetime.today(), datetime.today() - timedelta(1)]
event_filters = [
    ['created_at', 'between', date_range],
    ['event_type', 'is', 'Shotgun_Task_Change'],
    ['attribute_name', 'is', 'sg_status_list']
]

events = sg.find('EventLogEntry', event_filters, ['meta'])

# Define status removed, and with what it should be replaced
removed_status = 'pndng'
new_status = 'wtg'

batch_data = list()

# Loop over events, pick out specific events where status field was left empty due to specific status removal
for event in events:
	if event['meta']['new_value'] == None and event['meta']['old_value'] == removed_status:
		print "Id of changed Task: %d, updating status to: %s"%(event['meta']['entity_id'],new_status)
		data = {'sg_status_list': new_status}
		batch_data.append({'request_type':'update','entity_type':'Task', 'entity_id':event['meta']['entity_id'], 'data':data})

# Mass update affected Tasks
updated_tasks = sg.batch(batch_data)
print updated_tasks
12 Likes