Shotgun_api3 CRUD ERRORS - CRUD ERROR #5

Flow Production Tracking
1

Hey gang, I’m struggling to decipher this specific CRUD error. Usually I’m able to see what the issue is but this #5 one in particular is a pain.

  1. Does anyone have reference for all the various CRUD errors and their meaning?
  2. What does API update() CRUD ERROR #5: Update failed for [EntityType.field_name]: Invalid statement. mean?

For context I’m performing a shotgun_api3.Shotgun.batch update to 2x multi-entity data type fields for multiple PublishedFile entity records. I’m trying to remove 3 particular entities from these fields.

I’ve tried both an update with multi_entity_update_modes using the remove mode while specifying the offending entities for removal explicitly and I’ve tried the set mode by supplying the exact value I want in the field, but both methods result in an error.

If I ignore supplying multi_entity_update_modes in my batch update data and just set the field values the “old” way, same thing.

The full traceback output references a specific index in my batch data but I can’t see any issue with the data and when I try batch update the specified index or surrounding items I get no issues.

Need more input if you have it!
AdmirableTemptingFlies-max-1mb

2

Can you share the request and response?

3

The request is something similar to the following with 20000 items:

I’ve tried the 3 examples below:

# Update request with remove mode

batch_data = [
    {'entity_id': 123456,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [
            {'id': 123, 'type': 'CustomEntity99'},
            {'id': 124, 'type': 'CustomEntity99'},
            {'id': 125, 'type': 'CustomEntity99'}],
        'sg_multi_entity_field_2': [
            {'id': 123, 'type': 'CustomEntity99'},
            {'id': 124, 'type': 'CustomEntity99'},
            {'id': 125, 'type': 'CustomEntity99'}],
        },
     'multi_entity_update_modes':{'sg_multi_entity_field_1': 'remove', 'sg_multi_entity_field_2': 'remove'}
     },
    },
    {'entity_id': 123457,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [
            {'id': 123, 'type': 'CustomEntity99'},
            {'id': 124, 'type': 'CustomEntity99'},
            {'id': 125, 'type': 'CustomEntity99'}],
        'sg_multi_entity_field_2': [
            {'id': 123, 'type': 'CustomEntity99'},
            {'id': 124, 'type': 'CustomEntity99'},
            {'id': 125, 'type': 'CustomEntity99'}],
        },
     'multi_entity_update_modes':{'sg_multi_entity_field_1': 'remove', 'sg_multi_entity_field_2': 'remove'}
     },
    },
    {'entity_id': 123458,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [
            {'id': 123, 'type': 'CustomEntity99'},
            {'id': 124, 'type': 'CustomEntity99'},
            {'id': 125, 'type': 'CustomEntity99'}],
        'sg_multi_entity_field_2': [
            {'id': 123, 'type': 'CustomEntity99'},
            {'id': 124, 'type': 'CustomEntity99'},
            {'id': 125, 'type': 'CustomEntity99'}],
        },
     'multi_entity_update_modes':{'sg_multi_entity_field_1': 'remove', 'sg_multi_entity_field_2': 'remove'}
     },
    },
    ...
]

sg.batch(batch_data)

# Update request with set mode

batch_data = [
    {'entity_id': 123456,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [],
        'sg_multi_entity_field_2': [],
        },
     'multi_entity_update_modes':{'sg_multi_entity_field_1': 'set', 'sg_multi_entity_field_2': 'set'}
     },
    },
    {'entity_id': 123457,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [],
        'sg_multi_entity_field_2': [],
        },
     'multi_entity_update_modes':{'sg_multi_entity_field_1': 'set', 'sg_multi_entity_field_2': 'set'}
     },
    },
    {'entity_id': 123458,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [],
        'sg_multi_entity_field_2': [],
        },
     'multi_entity_update_modes':{'sg_multi_entity_field_1': 'set', 'sg_multi_entity_field_2': 'set'}
     },
    },
    ...
]

sg.batch(batch_data)

# Update request with no mode
batch_data = [
    {'entity_id': 123456,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [],
        'sg_multi_entity_field_2': [],
        },
     },
    },
    {'entity_id': 123457,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [],
        'sg_multi_entity_field_2': [],
        },
     },
    },
    {'entity_id': 123458,
     'entity_type': 'PublishedFile',
     'request_type': 'update',
     'data': {
        'sg_multi_entity_field_1': [],
        'sg_multi_entity_field_2': [],
        },
     },
    },
    ...
]

sg.batch(batch_data)
4

I think my batch_data size was too large… If I batch in groups of 500 it seems to be happy now.

It would be good to know the hard limit in terms of number of items or request payload size on the server to better understand the issue though.

5

I think the hrd limit of the web interface is 500 entity creations in one go.
Not sure if that means it’s also the limit of the batch api.