Toolkit's Git-descriptor behavior change

Pipeline Integrations
, ,
1

Hi everyone,

We’re considering optimizing our git descriptor, but this will introduce a subtle behavior change when Toolkit tries to get the latest version of a git descriptor. But first, a few definitions:

  • most recent tag: This is the tag that was the most recently created. The actual value of the tag is irrelevant.
  • greatest tag: The greatest tag number. The creation time is irrelevant.

[EDIT: we’d like to clarify that the use of a version-less git descriptor is possible only for a Pipeline Configuration and not for an individual Toolkit application, engine or framework.]

First, the TL;DR version: We’re considering a change where Toolkit would consider the latest tag of a repository to be the latest version instead of using the most recent tag. Objections?

Now…

We’re looking into making some optimizations to the git descriptor in Toolkit. One of those is around the enumeration of tags from a git repo. Our current approach, which is to consider the most recent tag as the latest version, has two pitfalls. First, it requires a complete clone of the repository so we can get the creation date of each tags, which slows down the process of retrieving the latest version. Second, the most recent tag is not necessarily the latest version. Imagine a situation where v5.7.0 is the most recent tag of your configuration, but you make a bugfix for a project that is on v5.6 branch. If you were to release v5.6.x after you’ve released v5.7.0, Toolkit would today consider the latest version to be v5.6.x and not v5.7.0, because v5.6.x is the more recent tag.

Updating the latest version detection logic would do two things

  • make sure that the greatest tag number is the latest and not the most recent tag
  • sorting tags by the version string does not require a complete clone of the repository, which means getting the latest version is a LOT faster.

Does this subtle behavior change, i.e. the latest version becoming the greatest tag instead of the most recent tag, sound like something that may be an issue for any of you?

If you are using semantic versioning, you will be fine. However, if you’ve been creating tags over time like

v1.3.11.bob_fix
v1.3.11.frank_fix
v1.3.11.dom_fix
or just
second_version
third_version
fourth_version

then you’ll be impacted.

Does those two last examples sound like an issue for anyone? If it’s an issue, how feasible would it be for you to switch to semantic versioning?

Thanks for reading this far!

JF

8 Likes
2

We won’t be impacted by such a change, but I don’t find it wrong to enforce a versioning standard in any case. It brings multiple benefits that won’t hurt.
It sounds to me like the change would be beneficial, not only due to speed, but seems to make sense that the highest tag should be installed if you do not specify anything.

5 Likes
3

Only here to say that, we are not affected by this change and like to see it implemented. :slight_smile:

4 Likes
6

Absolutely support this.

4 Likes
9

This can be awesome. Today, in each commit or in the Github version, I need to change the actual version in app_location.yml, so that it is no longer needed.

Does this feature already have a date to be released?

Thanks

1 Like
10

This would not impact us but do I now understand that if I don’t supply an implicit version Shotgun will just choose the latest release anyway? :slight_smile:

1 Like
11

Actually, we do not support nor will we support version-less descriptors for applications and engines, only for a pipeline configuration like in this example. You’ll need to keep managing the version number in app_locations.yml.

Version-less descriptors are only supported on the pipeline configuration entity. Supporting it on apps, engines and frameworks would raise several issues, the most important being:

  • Reproducing the same pipeline over time would be difficult, as there would not be a central point of truth for which versions to use for a configuration.
  • Because Toolkit wouldn’t know the specific version of an app to use, a roundtrip to the Git, Github or AppStore to get the latest version of any given bundle would slow down considerably Toolkit’s startup time. And let’s be honest, you don’t want it to be any slower!

I hope this helps clarifying the scope of this feature.

3 Likes
12

Ok @jfboismenu, now I understand, you’re right. Thank you!

Yeah, slower is always a bad option, just to illustrate this situation, I think you’ve been through this:

if pipe is slower:
    raise ArtistCryingWarning("Hey dude...I clicked on Maya and it still isn't open, what's going on here ??")

:smile:

5 Likes
14

I think that every pipeline TD has come across this particular bit of code countless times :grimacing:

2 Likes