Template key regex filter with | or?

Pipeline Integrations
1

I am trying to apply a filter_by: regex to one of our keys in our templates to accommodate the following:

  1. VDC3310
  2. 167_VAL_1560

I’ve tried many different systems involving a|b switches and (a)? optional blocks, but the filter system seems to reject both of those.

I have figured out a regex that does work without those optional blocks, but I’d really like to know if and how I can include those OR switches and optional groups for future reference in case we have more non-conforming shot names to deal with.

Thanks.

Edit: I’ve already read both these articles, which imply that any valid regex should work, but that simply does not seem to be the case for filter_by in the templates.


https://developer.shotgunsoftware.com/tk-core/core.html

2 Likes
2

Hi @rthompson,

I’ll check with our Reg-Expert :wink: to see if they can help with this. Are you able to share with us the filter/s you believe should work that are not?

-David

2 Likes
3

Gladly.

I tested all of these (and some others I can’t recall) using the website https://www.regextester.com/
As well as a Python test script, just to be sure they were valid.
They all should have worked according to those tests, but none of them did…

filter_by: '\b([0-9a-zA-Z]+(_\w{1,3}_\d{1,4})?)'
filter_by: '(^[0-9a-zA-Z]+(_\w{1,3}_\d{1,4})?)'
filter_by: '\b([0-9a-zA-Z]+)(_\w{1,3}_\d{1,4})?'
filter_by: '(^[0-9a-zA-Z]+)(_\w{1,3}_\d{1,4})?'
filter_by: '^[0-9a-zA-Z]+(_\w{1,3}_\d{1,4})*'
filter_by: '(^\d+_\w+_\d+)|(^[0-9a-zA-Z-]+)'
filter_by: '^\d+_\w+_\d+|^[0-9a-zA-Z-]+'
filter_by: '^(^\d{1,3}_\w{1,3}_\d{1,4}|[0-9a-zA-Z-]+)'
2 Likes
4

Hi @rthompson,

I’m not sure what the problem was with the regexes that you tried, but I can show you one that worked for me:

'[0-9]{0,3}_?[A-Z]{3}_?[0-9]{4}'

First, to talk through the regex:

  • [0-9]{0,3} : 0-3 digit characters. (I’m sure you could replace the [0-9] with \d.) Allowing 0 of these makes it optional by nature.
  • _?: optional underscore
  • [A-Z]{3}: exactly 3 uppercase letters
  • _?: second optional underscore
  • [0-9]{4}: exactly 4 digit characters

How I tested it:

  • I created the following template key in templates.yml, using the above regex:
    Shot:
        type: str
        filter_by: '[0-9]{0,3}_?[A-Z]{3}_?[0-9]{4}'
  • I created Shots called VDC331 and 167_VAL_1560 on my test project.
  • I used tank shell to get an interactive Python session with an existing Toolkit API handle called tk.
>>> template_obj = tk.templates['maya_shot_work']

# test 1: 167_VAL_1560
>>> fields = {'Shot': '167_VAL_1560', 'Step': 'comp', 'name': 'test', 'version': 3, 'Sequence': 'sq200'}
>>> template_obj.apply_fields(fields)
'/sgtk/projects/goats/sequences/sq200/167_VAL_1560/comp/work/maya/test.v003.ma'

# test 2: VDC3310
>>> fields = {'Shot': 'VDC3310', 'Step': 'comp', 'name': 'test', 'version': 3, 'Sequence': 'sq200'}
>>> template_obj.apply_fields(fields)
'/sgtk/projects/goats/sequences/sq200/VDC3310/comp/work/maya/test.v003.ma'

# test 3: s009, which should *not* match the filter regex:
>>> fields = {'Shot': 's009', 'Step': 'comp', 'name': 'test', 'version': 3, 'Sequence': 'sq200'}
>>> template_obj.apply_fields(fields)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/sgtk/software/shotgun/goats/install/core/python/tank/template.py", line 260, in apply_fields
    return self._apply_fields(fields, platform=platform)
  File "/sgtk/software/shotgun/goats/install/core/python/tank/template.py", line 604, in _apply_fields
    fields, ignore_types, platform, skip_defaults=skip_defaults
  File "/sgtk/software/shotgun/goats/install/core/python/tank/template.py", line 317, in _apply_fields
    value, ignore_type=ignore_type
  File "/sgtk/software/shotgun/goats/install/core/python/tank/templatekey.py", line 229, in str_from_value
    raise TankError(self._last_error)
TankError: <Sgtk StringKey Shot> Illegal value 's009' does not fit filter_by '[0-9]{0,3}_?[A-Z]{3}_?[0-9]{4}'

So, using the ? operator for optional characters does indeed work – it correctly matched the two Shots we expected it to match, and errored on the one that didn’t. I’m not sure what the specific issue was with the regexes that you tested, but maybe start with mine and then make modifications to make it match your exact use case?

Let me know if you’re still stuck, and we can keep digging.

1 Like