Batching Webhooks

Webhooks
9

Hi all, sharing here as a preview the format we’re implementing for adding support for batched deliveries.

Non-batched-deliveries Webhooks

  • timeout allowance is 6 seconds per delivery
  • i.e. the webhook endpoint must respond to each request within 6 seconds

Batched-deliveries Webhooks

  • timeout allowance is max of 6 seconds, or 1 second per event in the batch
  • throttling limits still apply (1 minute of webhook endpoint response time per minute per shotgun site, across all webhooks)

A word of caution: If you choose to use batched-deliveries, we recommend that you guarantee, with your callback endpoint design, to always respond much faster than 1 second per event. Otherwise you will be at increased risk for timeouts and webhook failure when batches are large.
For webhooks that take on the order of 1 second to respond, there is not any significant benefit in batching, because in that case webhook response time is the main performance factor, not delivery overhead.

Comparison of Webhook Delivery Formats

Non-batched-deliveries Webhook Message Body (always 1 delivery):

{
  "data":{
    "id":"119.110.0",
    "event_log_entry_id":479004,
    "event_type":"Shotgun_Asset_Change",
    "operation":"update",
    "user":{"type":"HumanUser","id":24},
    "entity":{"type":"Asset","id":1419},
    "project":{"type":"Project","id":127},
    "meta":{
      "type":"attribute_change",
      "attribute_name":"code",
      "entity_type":"Asset",
      "entity_id":1419,
      "field_data_type":"text",
      "old_value":"Cypress test asset for Webhooks deliveries",
      "new_value":"Revised test asset for Webhooks deliveries"
    },
    "created_at":"2021-02-22 17:40:23.202136",
    "attribute_name":"code",
    "session_uuid":null,
  },
  "timestamp":"2021-02-22T17:40:27Z"
}

Batched-deliveries Webhook Message Body (may contain 1 to 50 deliveries)

Note that the delivery format for each event is identical, whether batched or unbatched. However, in the batched case, the events are inside an array (even if there is only 1 event in the batch).

{
  "timestamp":"2021-02-22T18:04:40.140Z",
  "data":{
    "deliveries":[
      {
        "id":"170.141.0",
        "event_log_entry_id":480850,
        "event_type":"Shotgun_Asset_Change",
        "operation":"update",
        "user":{"type":"HumanUser","id":24},
        "entity":{"type":"Asset","id":1424},
        "project":{"type":"Project","id":132},
        "meta":{
          "type":"attribute_change",
          "attribute_name":"code",
          "entity_type":"Asset",
          "entity_id":1424,
          "field_data_type":"text",
          "old_value":"Cypress test asset for Webhooks deliveries",
          "new_value":"Revised test asset for Webhooks deliveries"
        },
        "created_at":"2021-02-22 18:04:39.198641",
        "attribute_name":"code",
        "session_uuid":null,
      },
      {
        "id":"170.141.1",
        "event_log_entry_id":480851,
        "event_type":"Shotgun_Asset_Change",
        "operation":"update",
        "user":{"type":"HumanUser","id":24},
        "entity":{"type":"Asset","id":1424},
        "project":{"type":"Project","id":132},
        "meta":{
          "type":"attribute_change",
          "attribute_name":"description",
          "entity_type":"Asset",
          "entity_id":1424,
          "field_data_type":"text",
          "old_value":null,
          "new_value":"Some other *description*"
        },
        "created_at":"2021-02-22 18:04:39.212032",
        "attribute_name":"description",
        "session_uuid":null,
      },
    ]
  }
}