Houdini Intergration - Publish HDA?

Pipeline Integrations
1

I haven’t really looking into it, but shotgun doesn’t seem to be able to publish HDA and load it back in ? Cheers

1 Like
4

Out of the box the Shotgun integrations do not handle publishing or loading of HDA files.
You would need to take over the publish hooks and configure the loader to be able to import HDA files.
It is possible, and I’ve seen someone else attempt to set this up, but I don’t have any specific details I can share other than:

The pipeline tutorial covers writing your own publish plugin and configuring the loader to import a PublishedFile type, so that might be of use if you are looking to add this your self.

Best
Phil

4 Likes
6

Hello Sun. Yes Philip is right in that you will first need to take over the publishing processes. Is this something you have done yet? If so, I do have an example I can share with you.

2 Likes
7

Please, share a step to step tutorial, I have not done anything like that. And I am not a programmer, I can do some python, but not at a super pro level.

1 Like
8

I think it will be better for you to go through the pipeline tutorial first. Once you have your custom configuration set up and looking something like you see here:

then it will be easier to add more custom code to include publishing and loading HDAs. I can go through this with you as you progress until you have the final pipeline.

They even have a good explanation of publishing in the section titled “Custom Shader Publish”. It explains how Shotgun collects items to be displayed in the Publish window.

By the way, how well do you know Houdini’s Digital Assets?

7 Likes
10

Hi Andrewa

Thanks, I am going through the pipeline tutorial, everything seems to make sense. I am a Houdini TD, so I am very familiar with HDA. I probably need guidance from you, regarding to the next step, after the “Custom Shader Publish” tutorial, for some code example for Houdini integration, just to get me started. Basically I need to take over the Houdini loading and publishing default behaviour by the look of it.

I had a brief thought of how I imaged workflow would work in Houdini. Maya/Houdini Model .ABC publishing (Model Step) -> Load .abc in Houdini for Asset LookDev, publish as Object level HDA (lookdev/Surfacing Step)- > Load in Houdini Scene for scene layout (layout Step), then publish as Layout HDA (Include multiple nested Asset HDA, created at the lookDev step) - > Load layout HDA in a new Houdini scene for Lighting and final rendering.

4 Likes
11

Hi Sun,
When you have created your own config let me know and I’ll try to create a step by step for you.

Yes we have a similar approach going on here. As long as you are publishing versioned HDAs it should work just fine.

3 Likes
12

OK, Andrewa.

I have managed to create my own config, took over the hooks and pluggin. Can you help me with step by step HDA implementation ? Cheers

2 Likes
13

Hi Sun,
Nice to hear. Sorry, I am still putting together something for you. Seems as though I’m writing a book here! I may just send the steps in chunks. I’m not sure how you have it set up on your end so you may need to make adjustments from how you see my directory setup.

3 Likes
14

I suppose I should start with the tk-multi-publish2.yml and the tk-houdini.yml files. You should see them somewhere around config->env->includes->settings. This is where you can add custom code you will need.

In the tk-houdini.yml file you should be specifying where to go when publishing. First add the tk-multi-publish2.yml to the includes list. This is how I have it.

includes:
- ../app_locations.yml
- ../engine_locations.yml
- ./tk-houdini-alembicnode.yml
- ./tk-houdini-mantranode.yml
- ./tk-multi-loader2.yml
- ./tk-multi-publish2.yml
- ./tk-multi-screeningroom.yml
- ./tk-multi-shotgunpanel.yml
- ./tk-multi-snapshot.yml
- ./tk-multi-workfiles2.yml

All the files you add to the include list should exist in the “settings” folder along with the tk-houdini.yml. Now to define how Houdini handles operations while in the Asset Step. It is during this Step you will be publishing the Digital Assets. It should look something like the following:

# asset_step
settings.tk-houdini.asset_step:
  apps:
    tk-multi-about:
      location: "@apps.tk-multi-about.location"
    tk-multi-breakdown:
      location: "@apps.tk-multi-breakdown.location"
    tk-multi-loader2: "@settings.tk-multi-loader2.houdini"
    tk-multi-publish2: "@settings.tk-multi-publish2.houdini.asset_step"
    tk-multi-screeningroom: "@settings.tk-multi-screeningroom.rv"
    tk-multi-shotgunpanel: "@settings.tk-multi-shotgunpanel.houdini"
    tk-multi-snapshot: "@settings.tk-multi-snapshot.houdini.asset_step"
    tk-multi-workfiles2: "@settings.tk-multi-workfiles2.houdini.asset_step"
    tk-houdini-alembicnode: "@settings.tk-houdini-alembicnode.asset_step"
    tk-houdini-mantranode: "@settings.tk-houdini-mantranode.asset_step"
  menu_favourites:
  - {app_instance: tk-multi-workfiles2, name: File Open...}
  - {app_instance: tk-multi-snapshot, name: Snapshot...}
  - {app_instance: tk-multi-workfiles2, name: File Save...}
  - {app_instance: tk-multi-publish2, name: Publish...}
  location: '@engines.tk-houdini.location'

I’ll let you decide which one’s you want to use, but each one of these should be in the “settings” directory. The important one’s being:

tk-multi-loader2: "@settings.tk-multi-loader2.houdini"
tk-multi-publish2: "@settings.tk-multi-publish2.houdini.asset_step"

Now to the tk-multi-publish2.yml file.
You will need to add the lookup for tk-multi-publish2.houdini.asset_step. You can start with something like this:

# ---- Houdini
# asset step
settings.tk-multi-publish2.houdini.asset_step:
  collector: "{self}/collector.py:app_customization/publish/tk-houdini/houdini_collector"
  collector_settings:
      Work Template: houdini_asset_work

This defines which collector script to use. Then you need to add the publish app locations:

  publish_plugins:
  - name: Publish to Shotgun
    hook: "app_customization/publish/publish_file"
    settings: {}
  - name: Upload for review
    hook: "{self}/upload_version.py"
    settings: {}
  - name: Begin file versioning
    hook: "{engine}/tk-multi-publish2/basic/start_version_control.py"
    settings: {}
  - name: Publish to Shotgun
    hook: "app_customization/publish/publish_file:app_customization/publish/tk-houdini/publish_session"
    settings:
        Publish Template: houdini_asset_publish
  - name: Publish to Shotgun
    hook: "app_customization/publish/publish_file:app_customization/publish/tk-houdini/publish_session_hdas"
    settings:
        Publish Template: houdini_asset_publish_hda
  help_url: *help_url
  location: "@apps.tk-multi-publish2.location"

You should notice the new file I created called “publish_session_hdas”. I have this one located in the config->hooks->app_customization directory. Your directory may be named something else. This is where we put all our custom hooks. In this directory we have a sub-folder called “publish”. For each application we publish using Shotgun we have a an isolated folder. So in the end I have all my Houdini publish scripts in:
config–>hooks–>app_customization–>publish–>tk-houdini.
These scripts should be in your custom project config directory. If not then copy the from the tk-houdini and tk-publish2 directory. Usually located in the default shotgun install->app_store directory.

I’ll stop here for a minute so you can catch up. Let me know if you have any questions,

Next I’ll talk about the collector script.

8 Likes
15

houdini_collector.py

In this file you should see a method called process_current_session. In there I gather all the digital assets by adding:

# HDAs
hda_nodes = list()
obj_network = hou.node("/obj")
all_nodes = obj_network.children()
for node in all_nodes:
    cur_type = node.type().definition()
    if cur_type is not None:
        hda_path = cur_type.libraryFilePath()
        hda_nodes.append(node)
if len(hda_nodes) != 0:
    self.collect_hdas(item, hda_nodes)

You decide how you want to collect them. I put an extra check in my version to make sure the hda name matches the current file name. Since I create versioned hda files, it makes it easier to only publish the one digital asset you need. That’s something you can decide.

This now points to a new collect_hdas method. This method will add a new item to the collection process. Using Shotgun’s create_item method you can go ahead and do this

    def collect_hdas(self, parent_item, hda_nodes):
        """
        Creates items for HDAs to be exported
        :param parent_item: Parent Item instance
        :param hda_nodes: list of items to export
        :return:
        """
        # print "\nSorting HDA imformation for: %s\n" % hda_nodes
        # print "parent_item = %s" % parent_item
        print("\nCOLLECTING HDA ITEMS")
        geo_items = list()
        for scene_geo in hda_nodes:
            geo_item = parent_item.create_item(
                "houdini.session.hda.hdaitem",
                "Digital Asset",
                scene_geo.name()
            )
            # get the icon path to display for this item
            icon_path = os.path.join(icon_dir, "generic", "files_32.png")
            geo_item.properties['hda_node'] = scene_geo
            print("\nSetting hda_node to: %s" % scene_geo)
            definition = scene_geo.type().definition()
            geo_item.properties['hda_location'] = definition.libraryFilePath()

            geo_item.set_icon_from_path(icon_path)
            geo_items.append(geo_item)
        return geo_items

For each item you create, you will eventually see it in the Shotgun Publish window with a checkmark beside it. Take note that I create a new item type called:

houdini.session.hda.hdaitem

This is the item type you will be looking for later. You can see I use a lot of print statements. You can either get your feedback this way or with the logger. The print statement just sends it to the Houdini Console of Python Tab if you have one open. I usually open a new Python Tab with every change I make so I can compare the results.
You can also create custom item properties. I have included the digital asset’s location so that it can be used in the publish process to copied over. For each item you can also attach an icon.

After this, the new publish_session_hdas.py script will be looking for the houdini.session.hda.hdaitem in its filter.

5 Likes
16

Publish Session

Back in the tk-multi-publish.yml file we added a line to look for the publish_session_hdas script during the publish process. This script is just a copy of the publish_process script and renamed. Inside the script I have also renamed every instance of the class name HoudiniSessionPublishPlugin to HoudiniSessionHDAPublishPlugin

To start, we need to specific what publish item this script is looking for to process. This is done with the item_filters method.

    @property
    def item_filters(self):
        """
        List of item types that this plugin is interested in.

        Only items matching entries in this list will be presented to the
        accept() method. Strings can contain glob patters such as *, for example
        ["maya.*", "file.maya"]
        """
        return ["houdini.session.hda.hdaitem"]

If you stopped here you should see your collected digital assets in Shotgun’s publish window as items with a check mark beside them.

Templates
Taking a side step for a second I wanted to mention the templates.yml file. This file is located in the config–>core directory. Here is where you can define how and where your hda file will be saved. In mine I have set it up like the following:

    # HDAs - Houdini Digital Assets
    houdini_asset_publish_hda:
        definition: '@asset_root/publish/houdini/otls/{Asset}_{Step}_{task_name}_v{version}.hda'
        root_name: primary

The publishing process will look to this definition for naming. By default Shotgun doesn’t provide the task_name. I had to go through a couple of hoops to provide this value. For the simplicity of this example we can hard code this value for now. I can explain how to change this later perhaps.

I hope all this is making sense. Before I continue I’d like to know if this is helping. Are you getting the results I hope you are getting?

7 Likes
17

Just to finish this off (sorry for the delay), the final process is the publishing. In the publish method you only need to copy the hda to the publish folder you specify in your Templates.

        publish_path = item.properties["path"]
        publish_template = item.properties['publish_template']
        publish_fields = publish_template.get_fields(publish_path)
        # ensure the publish folder exists:
        publish_folder = os.path.dirname(publish_path)
        self.parent.ensure_folder_exists(publish_folder)

Now that these are set you can continue. Insert the following code to the publish method and you should have it complete.

        # -----------------  PUBLISH HDA  ----------------- #
        hda_node = item.properties['hda_node']
        definition = hda_node.type().definition()
        hda_name = definition.nodeTypeName()
        definition.copyToHDAFile(publish_path, hda_name, hda_name)

Hopefully this should give you the complete picture on how to publish an HDA.

7 Likes