Use google drive as primary storage

Hi,

I’m trying to setup a project and use google drive as primary storage using Drive File stream.
But doing so in the Project Wizard is issuing a Warning in the console when accessing the storage configuration:
[WARNING] Storage root primary could not be mapped to a SG local storage
And the “continue” does nothing.

Any Idea on how to use Google drive as primary storage?

3 Likes

Hi @michael.delaporte

Currently the way you would need to set this up, is have the synced google folder in the same location on everyone’s machine. The Shotgun storage roots should then point to that folder.

The error you are getting is because the path you are trying to use isn’t defined in your site preferences in Shotgun, you can read more about that here.

Hi Michael,

Just a heed of caution as I have tried this in the past as well (who doesn’t want “unlimited” shared storage with a online/offline function right?)

Well two things:

  1. Drive File Stream Team Drives have a theoretical limit of about 100.000 inodes (items/folders/files). I have not yet reached this limit with my project backups but you may well hit it fast if you work in a team.

  2. A year or 2 back when I wanted to use this as an awesome way to have synced storage without needing massive local storage… at least on Windows… Performance of cached content was appalling compared to anything stored outside the G drive.

Considering that the G drive cache (where it stored any offline available files) was on my SSD raid and I tested performance on that same raid but outside of the G drive I attributed this performance hit to the Filesystem driver that G drive uses to create a “fake” drive.

I have not tested this recently so this performance hiccup could have disappeared or greatly improved. It’s worth testing as I noticed quite a slowdown in Nuke!

2 Likes

I think the limit is now 400,000.

Maybe using a configuration with multiple project roots is a way to further safeguard against hitting the limit.

1 Like

Ah yes, they upped it.

Still 400.000 is easily hit by any multi-person project if you render exr’s/file sequences.

I finally fixed there was a typo in the storage name.
I encountered an other issue:
I was on a computer configured in french. And Google Drive File Stream name its “shared drives” “drives partagés”. The issue is that Shotgun Toolkit complain about the accent in the folder name.
I switched in english to fix it, but maybe it can be fixed on the toolkit side.
Now its all working fine :slight_smile:

1 Like

Regarding the Inodes and upload limits, Google recommend creating multiple shared drives.
I use it only for development, but I imagine that for production we could have one shared drive per project, or if not suffisant, have separate shared drives for renders or per episodes or sequences.
The limit of shared drives displayed in the Google Drive page is 1000. But you can still create more and access them by url.

Additionally I modified our toolkit configuration / schema so the work and publish folders are at the root of the project and only published files are stored in Google Drive. We’re new to Shotgun so I’m not sure if that is bad practice, but it seems sensible because lots of users have limited upload bandwidth and we only want to send published files - not every working file or snapshot they create locally…

2 Likes

The way I handled this is by creating symlinks inside the project root to the actual Google Drive location. That way the toolkit configuration can be homogenous and only the symlinks need to be created differently in case the location of the Google Drive is different. In fact this means it doesn’t really depend on Google Drive at all, could be swapped to Dropbox or a local network share behind the symlinks.

Furthermore on Windows I used subst to map the entire project root to a consistent virtual Drive location “S:\” that contains the symlinks.

It sounds complicated but only a few steps to set up and then everyone has the same virtual storage location regardless of the physical drive path or location of cloud storage.

2 Likes

Hello everyone,

I have successfully setup Shotgun with Google Drive and used symlinks as @jrapczak suggested. Thanks a lot for that tipp.

There is a recurring issue with duplicates thou. Sometimes shotgun doesn’t seem to detect the existing project folder and no files can be visible. When a new file is created, shotgun creates a new project folder for that single file. Then there are suddenly two projects on the Google Drive. For example, one is called “project_name” and the second one “project_name (1)”.

Does anyone know how to resolve this?

Never even heard of subst and its 36 years old… :zipper_mouth_face:

Hello everyone,

I have successfully setup Shotgun with Google Drive and used symlinks as @jrapczak suggested. Thanks a lot for that tipp.

There is a recurring issue with duplicates thou. Sometimes shotgun doesn’t seem to detect the existing project folder and no files can be visible. When a new file is created, shotgun creates a new project folder for that single file. Then there are suddenly two projects on the Google Drive. For example, one is called “project_name” and the second one “project_name (1)”.

Does anyone know how to resolve this?

Sorry for the delayed response but you might want to make sure Google Drive is set to always keep the files locally instead of keeping them in the cloud until used.

Thanks for your answers!

I managed to solve it by using Microsoft OneDrive instead. Doesn’t create any weird duplication problems like Google Drive.