Initial testing feedback

Webhooks
1

I did some quick testing with the BETA Webhooks back in February - but wanted to mirror the information here.

Things I like:

  • The interface is clean and straightforward
  • The replay-ability over missed events, or if the request was not delivered is very nice. Much more transparent than sorting logs of the Shotgun Event Daemon for missed EventLogEntries.
  • The sorting / listing of events which match criteria is also really nice.
  • The secret token / HMAC SHA1 hashing for safety, this is a very necessary feature. Much appreciated.
  • The status display and filtering looks like it’s going to be nice.
  • The Webhooks data payload display is very nice for debugging purposes.

========================

Errors / Suggestions:

  • When creating a webhook I would like the option to set an e-mail or some sort of alert which can report when a webhook is Unstable / Failing.
  • A webhook URL “Test Connection” button
  • On the Webhook creation form, I see that you can set select Projects, or all Projects, but what about excluding from a specific project or few projects. - When a new project is created, I do not want to have to go through and update all existing webhooks to now include the new project.
  • Could be worth expanding this to allow for some conditions to be met before triggering an event.
  • Filters on the Deliveries section could be more extended / customized
  • The “Redeliver Selected” button is not clearly “disabled” for a “Failed” webhook - Suggesting a different color or option or tooltip to indicate why it’s not active. - Currently it just dims the color

  • Oh… Just noticed, for a “Failed” webhook - clicking the “|| Disable” button in the top-right actually re-enabled the webhook.

  • ** This is probably a failure to update the button state, I had not selected a different webhook or reloaded the page between the Webhook being “Active” and it’s transition to “Failed”

Additional note - doing a quick show-and-tell on the functionality of the webhooks with a coworker.

First thing: For past events which I want to re-deliver ‘selected’ - I would like to have a “Select all” checkbox to allow selection of… all “status” - failed deliveries to re-deliver together.

I marked a number of previously delivered events to re-deliver and was ‘quickly’ working with the interface, when we noticed that deliveries were no longer going to the request catcher. - The interface was seemingly non-responsive and did not process events or even put new triggered events on the list.

I think an improvement to the system would be to have a “Pending delivery” status for events which have not been picked up by the Shotgun server to send the request through to the URL yet.

We sat staring at the non-responsive interface and concluded after ~5 minutes that something was either bogged down, or not working on the shotgun side. - There didn’t seem to be anything wrong with the setup, URL, etc.
I just happen to have the request catcher monitor open and after another ~2 minutes a number of webhook POST requests came in altogether.

For our testing just now, it was with ~3 webhooks, handling a simple event.
I am concerned about how well this will scale when there will be hundreds of events happening simultaneously. - What are the throughput limitations?

We’re looking forward to the day when we can rely on this feature as a stable, functional solution to fully replace the Shotgun Event Daemons we have running currently.

Hope this feedback helps improve the workflow!
Cheers,
David

3 Likes
3

Thanks for the feedback David!

1 Like
4

It’s good to see other dabbling with WebHooks. Some great feedback here. We also noticed some sever latency with Webhooks. We have a single broker configured in AWS Lambda that services Actions and Webhook events. I have also noticed that SG’s service in the middle impacts performance. Actions call our lambda directly and don’t suffer from the same bottle neck. Furthermore, Webhooks really need a batching operation that groups Webhooks together based on a time threshold to reduce compute time and cold starts. Hoping more users jump in on these topics and encourage SG to refine Webhooks a bit more.

Romey

1 Like