Written by @sergio.leon
Continuing article: “Events in Flow Production Tracking - Part 1 (ADN Article)”
Webhooks
Webhooks are widely used in web development and allow a cloud service you control, to be notified of events that occur in Flow Production Tracking. Webhooks are automated messages sent from an application (FPTR here) when a specific event occurs. Unlike traditional APIs where you have to constantly “poll” for new data, webhooks proactively “push” data to a designated URL. This real-time communication is crucial for Real-time Automation, Data Synchronization and Event-Driven Architectures.
You will register to a specific type of event and will tell FPTR to what URL (callback URL) to make an HTTP call there when the event is triggered. This call is sending data in its payload about the event and the circumstances it was triggered.
As commented before, webhooks can be written in any programming language.
Listing and creating webhooks in FPTR web Ui are enabled by default for Admin and Manager permission roles. They can make webhooks accessible to the rest of users.
Main documentation and guide can be found in:
Some of the main aspects clarified on this guide are the followings.
How-to:
- How to create a webhook. Available on FPTR website, there is a Webhooks listing page.
- How you can create a webhook from an event listed on the ‘Event Log entries’. Only from those compatible.
- How to test webhooks ( webhook.site )
Management & statuses:
- How to check status from a webhook
- REST API can control webhooks aside from the web UI. Setting up webhooks programmatically, and dynamically using the FPTR Restful API
- 6 seconds to respond to a webhook request. Marked as fail if not.
- Webhook is marked as stable when having between 1 and 99 failures in the last 24 hrs. The webhook will be disabled, and has to be enabled again.
- Delivery logs older than seven days are removed and are not recoverable.
- A webhook consumer service must respond to deliveries in order for the system to consider them successfully delivered. Within six seconds of delivery.
- How to define a group to be notified when webhook fails.
- Make acknowledgements. This means the webhook is successfully received, but the process triggered by the callback may fail for some reason. Acknowledgements allow detailed reporting of these circumstances, handy for debugging processes.
- x_sg_ payload property can be used in the header. It is handy to filter on server side
Performance:
- The Flow Production Tracking Webhook delivery infrastructure is hosted in the AWS US-East-1 (North Virginia) region. Optimization for delivery times may be possible by hosting your webhook endpoint in that general location.
- How to deliver in batch
- The maximum size of a delivery payload is 1 megabyte. Any event triggered in Flow Production Tracking that would result in a payload size larger than 1 megabyte will have its new_value and old_value keys removed, and a warning key added that contains a message explaining what happened, why, and how to retrieve the entire event log entry from Flow Production Tracking.
Security:
- Signed requests: good practice for a higher security, you can set a secret token to be used to sign the payload.
- SSL Validation is possible
- In case you have a firewall installed or security restrictions on your network, be aware to ask It to allow specific IP addresses listed in the guide, so our servers can reach your endpoints for webhook notifications.
Status codes
| Status | Code | Description |
|---|---|---|
| Success | < 400 | The delivery was received and processed successfully. |
| Error | >= 400 | The delivery was received but was not processed successfully. |
| Redirect | 3xx | The delivery was received but should be redirected to another URL. |
Low-code/no-code services for automation with webhooks
For low-code/no-code users, webhooks allow for powerful integrations without writing complex API code, making advanced automation accessible to a wider audience. There are many tools out there, but here is an overview of some popular ones. They are usually not free, but some offer a free plan with limited features.
Low code
- Node-Red (free, open sourced): Requires some developer knowledge to use but enables the creation of workflows using a visual editor, combining various hardware devices, APIs, and online services.
- Postman Flows: Designed primarily to send further requests using RESTful APIs, allowing users to build, test, and debug API workflows with minimal code.
No code
- IFTTT: Integrates with many cloud applications and platforms (Slack, Instagram, LinkedIn, Domotic, project management tools, etc.), allowing users to create automated workflows through simple conditional statements.
- Power Automate: Enables business logic automation such as sending emails, adding records to Excel, and managing files in OneDrive through an intuitive interface.
- Zapier: Connects various web apps to automate tasks without coding, offering extensive integrations and easy-to-use automation workflows.
- Make (formerly Integromat): Facilitates workflow automation between apps and services with a visual editor, offering advanced features like error handling and scheduling.
- Bubble: Allows users to build web applications with a visual programming language, enabling complex workflows and integrations without code.
- Softr: Empowers users to create web applications, portals, and internal tools from Airtable data without coding.
- OutSystems: Provides a platform to visually create graphs and applications with built-in automation and integration capabilities.
About the Event Framework
The framework is a daemon process that runs on a server and monitors the Flow Production Tracking event stream. When events are found, the daemon check the events out and react accordingly based on a series of registered plugins. Each plugin will be process according to the events they are following. This framework makes sure to separates well the monitoring of events with the logic of the processes they may trigger.
Pipeline engineers, IT, TDs or developers can focus on implementing the business logic in plugins that can be easily added to the framework. Your trigger Scripts can run independent of Flow Production Tracking. Alos your scripts can run from anywhere that has network access to the Flow Production Tracking server.
Remember the event framework will monitor the Event log entries we see in the ‘Event Log Page’. The history of events that are happening in a specific context.
The Event Framework (Daemon) documentation can be found in:
- Documentation: https://help.autodesk.com/view/SGDEV/ENU/?guid=SGD_Event_Driven_Triggers_Event_Framework_html
- GitHub Repo: https://github.com/shotgunsoftware/shotgunEvents
- GitHub doc: https://github.com/shotgunsoftware/shotgunEvents/wiki
- API: https://help.autodesk.com/view/SGDEV/ENU/?guid=SGD_tr_event_daemon_api_html
We recommend taking a look to the guides on that site. Some of the aspects and features these will cover are:
How-to:
- Installation
- You will need:
- Python v3.9+
- Flow Production Tracking Python API
- Network access to your Flow Production Tracking server
- Run as Administrator
- You will need:
- Configuration
- Don’t forget to setup shotgunEventDaemon.conf settings
- Plugin managements:
- See plugin implementation and samples. There is a suite of samples in ‘shotgunEvents/src/examplePlugins’. A good starting sample is LogArgs. In the doc it shows how to create a Note when a field is deleted.
- Automatic deactivate any crashing plugins.
- Automatic reload of plugins when they change on disk.
- Starting from the last processed event id on daemon startup.
- Remembering the last processed event id and any backlog.
- Each event is always processed in the same predictable order so that if any plugins or callbacks are co-dependent, you can safely organize their processing
- Each plugin within a plugin path is then processed in ascending alphabetical order
- The framework stores a backlog of unprocessed events for each plugin and remembers the last event each plugin was provided
- The newly fixed plugin gets to process all events, including those that have occurred in the past between the failure and the fix.
You can see the Event framework daemon main loop in:
Be sure to enable the “Generate Events” option for any script that uses script-based authentication. Without this selection, your script won’t create events. This feature was originally introduced to help control and minimize the number of events produced by script processes.