Copy project data to new project

[mgarward]

Hi, we are starting our new project, which is being created starting with many assets from the last project. We started a new Project in SG, copying templates and pages etc, but I also want to be able to migrate entity data, including task status, notes, versions etc. Is there any way to do that, or should i keep going with my current effort to copy and create records using a python script?

[shaynad]

Hi @mgarward , welcome to the community!

So as I think you’ve discovered, duplicating, creating a project based on a template, or changing a Projects configuration can include:

1. Project Navigation Pages: Controls which pages appear in the main navigation bar for the Project.
2. Record Detail Pages: Choose the layout used to display information about each type of entity you track, such as Shots, Assets, etc.
3. Custom Pages: Includes any page layouts you create in a Project to organize information.
4. Tracking Settings: Settings used to determine which entities should appear in a Project, the fields, steps, and statuses used on each entity, as well as how you want to navigate each entity. You can find Tracking Settings under Project Actions > Tracking.

But, as you discovered, it doesn’t copy the actual data/records within the project. The more manual approach for users not savvy with the API would be to export/import on entity landing pages from the old project to the new one.

That said, your current path in creating a python script to help automate this sounds awesome, and if you create something you like and would like to promote it here in our community, that would be awesome too!

If you need any pointers along the way, we’d be happy to help.

[mgarward]

I’m still surprised by this. We can’t be the only project that rolls-over assets from one project to the next. How do other people deal with this situation?

[shaynad]

Hi @mgarward,

Ah, I see what you mean. If your goal is to ultimately share assets across projects, like having an Asset Library you can pull from, there are a couple of options worth exploring in this post from Tommy:

Creating an Asset or Elements Project/Database in shotgun and accessing through other projects

Hi @marvinmetzner, great question. We don’t really have proper “Asset Library” support in Shotgun. (Yet!) But there are a couple of ways people tend to go about this, and what you describe is probably the most common approach. The caveat is that you cannot link Elements from one Project into another, so you would need to make a copy of the Element in question if you wanted to use it in a different Project. This might be a little more complicated if you have linked entities attached to it that y…

We also have a couple more posts and related topics:

Creating an Asset or Elements Project/Database in shotgun and accessing through other projects

The way this works in our pipeline is: we have a lib project, to which all reusable assets are added in other projects, the loader app is configured to load assets from the lib project In order to configure tk-multi-loader2, add the following under “entities:” - caption: Library assets entity_type: Asset filters: - [project, is, {type: Project, id: 359}] hierarchy: [project, sg_asset_type, code] - caption: Library shots entity_type: Shot filters: - [project, is, {type: P…

Creating an Asset or Elements Project/Database in shotgun and accessing through other projects

I built an element library using Shotgun back in 2013 for a collection of practical elements (a few thousand.) Been working fine for six years. I made Version entities for the element proxy. Artists enjoy searching use the nice animated thumbnails. They chose to put them in about a dozen categories (aka Sequence entity). They can make tags to help find them too. Drag-n-drop works nicely into Nuke. I made a submission dialog in RV to allow creating new elements, along with metadata (eg. stan…

Custom non project entity with published Files without project

Thanks for the added details, Christian. We have a short how-to guide for setting up a Project as an Asset library of sorts that leverages the Toolkit loaders/publishers: I hope that’s helpful!

Admittedly, this can be handled better. If you’d like to share ideas with our product team or request this as a feature, submit it directly to our product team over on our public roadmap:

How do I submit an idea or feature request?

If you’ve got an idea or need for a feature, you can submit it directly to our product team over on our public roadmap . When you submit an idea, your feedback is delivered to our product team and the product owner of that area of Shotgun will review and determine next steps. The best ideas are the ones that explain why this would be useful in your workflow and how you would use it. This helps the product team to understand if there is a need to be filled. While the roadmap is the best place …

[mgarward]

Thanks for all those links, but I don’t think those solutions are going to meet our needs. I think the problem is that it isn’t a pure library that’s shared, it’s more like we finish one project, and start the next using the old one as a starting point. I don’t want to lose the “time capsule” info of what we did, but I might be removing huge chunks of data that we won’t need on the new project, up updating task templates etc.

I thought I remembered years ago being able to clone a Project as a starting point but I don’t see evidence of that so I must be mis-remembering. I think I’m going to forge ahead with my script so we end up with one “current” library of data starting as a copy of the last one.

[shaynad]

@mgarward, sorry that didn’t hit the spot. While I’ve heard of people creating scripts to automate (like you’re doing) the copying of data, I’d love to hear more from our community on this

Definitely keep us posted with how your script works out. We’d be happy to help guide you with any questions that come up as you develop it further.

[davidma]

@mgarward, we have been working through this problem ourselves.
At this point, I think the most effective solution is to use the “Template” project field links which inform the project your new project was created from.

I recommend maintaining a template project with the “latest” project-entities you want to populate new projects. Past projects as the basis for new projects tends to come with a lot of garbage. (Custom pages / detail views specific to the project / users)

Then setting up an automated trigger (Webhook / Event Daemon) to pick up on the events for new projects, and migrate the Entity Type records you’re looking to transfer. - Possibly filtered by your criteria.

One lesson learned: we initially stumbled on the common problem with CSV export/import when the Entity records have entity and multi-entity links to their own Entity Type which may or may not already exist (pretty common if you’ve set up a schematic Entity hierarchy structure) - Our way around is to create without setting entity/mutli-entity fields, then update those fields as a 2nd pass.

[Ricardo_Musch]

Just a suggestion, but maybe use a episode or season based project?
Because it seems all the work is for the same project which spans multiple episodes/seasons?

That way it’s all in one project but you can branch it out.