Exporting data from Shotgun

Our studio uses shotgun for external assetsmanagement and feedback across 4 mobile titles. We do monthly KPIs on vendor performance based on version interations on each task. This data is manually copied over to an Excel doc, which we use to have monthly talks on training or improvements. Is there a way to export these notes on versions and keep the tasks the versions are related to? Maybe in a .csv format?

1 Like

Hey @Andrew_Knight—welcome to the forum!

I split this out to its own topic for @jonjones to reply and so it’s not stuck in that big thread.

1 Like

Thank you, @johnny.duguid!

@Andrew_Knight, what information precisely are you wanting to export from Shotgun? There are a few ways to approach this. The first and simplest would be to set up a custom page showing off the fields you want to export, then clicking on the “More” button and then “Export CSV.” You can also do that via a URL:

https://{yoursite}.shotgunstudio.com/reports/csv/{page_no}

The second option would be using a Production Insights widget to graph and format the Versions data you want, and exporting to CSV from that widget. More info here: https://www.shotgunsoftware.com/blog/shotgun-76-introduces-production-insights/

There’s also information in the Event Log that could be exported and pivoted off of to generate information, but it depends on what you’re looking for. :slight_smile: Happy to follow up!

1 Like

Great thanks for the information, I’ll look into these suggestions. We are using keywords in notes attached to versions for tracking on iterations.

I wrote a short linked in article on how the proccess I use. Looking for suggestions on an easier automated proccess :slight_smile:

https://www.linkedin.com/pulse/effectively-tracking-external-feedback-data-game-andrew-knight

3 Likes

I read your article, and I’m impressed! Using keywords as the basis of that is really clever. Did you start from a master list you worked on with your art leads, or did that evolve over time?

If it’s keyword-based, you could create a custom page in Shotgun populated with Versions, using a page filter searching for keywords (and possibly date ranges? different vendors? different projects?) to populate the page with, including whatever other custom fields you’re using to provide context. From there, you could either manually export, or set up a periodic automated export into a spreadsheet you can work from. Using that exported spreadsheet as the source data for another sheet of calculations, graphs, and pivot tables could be one way to go.

Of course, I’m happy to talk about this in more detail offline if you’d like to go into more specifics.

BTW, are you attending SIGGRAPH Asia, by any chance? I’ll be there in Brisbane next week Mon-Weds.

1 Like

Yeah that would be great! I loved the dashboards you had setup that you showed in the presentation. I haven’t dealt too much with custom dashboards, so any help on that when you have a moment would be great.

The keyword list evolved after taking with the leads about common themes that come up in past feedback, as well as the sign-off gates of specific assets. It evolves as certain keywords are addressed with more tutorials or documentation, where it doesn’t seem significant enough of an issue to track. Sometimes new keywords are added as a new procedures or feature is introduced to the game. It really helps narrow down where a vendor is struggling when you deal with thousands of effort days of work across multiple studios.

Unfortunately i wont be at SIGGRAPH due to workload at the studio. I usually really only have enough time to attend XDS in Vancouver. :wink:

3 Likes

Thank you very much! I’ve just reached out to you on LinkedIn – let’s connect there and set up a time to talk.

Clever! That’s a smart way of approaching the problem, especially at that scale. I especially appreciated the categorization and graphing of feedback types and trends over time – inspiring food for thought!

2 Likes

This is such a great feature ( I actually asked for it many years ago) I wonder why it’s not written up anywhere in the documentation?

Now I know why. It was never finished unfortunately… Pity as it would have been such an amazing way to get data from sg to automatically show up and refresh in Excel… For anyone playing with it some of the limitations are

  • Linked fields are unsupported
  • Only the first view/tab on a page is supported to be exported

If you are okay with those limits then it does work to automatically load SG Data into Excel this way!!

2 Likes

Is that url no longer supported?

The https://{yoursite}.shotgunstudio.com/reports/csv/{page_no} format URL?
Works for me
You may need to set your Legacy Login username and passphrase in Account settings

Using legacy login did the job, thanks a lot :slight_smile:

Hi Patrick,

Our studio would like me to do the same thing. Automatically load SG Data into an Excel spreadsheet that sits on Google Drive. I am pretty new to this stuff and would like to know how did you go about this? Would you just manually hit the Export All to Excel (CSV) or is there a way how to automatically keep syncing the Shotgrid webpage with the same CSV file sitting on the server?

Can I possibly use python to call the https://{yoursite}.shotgunstudio.com/reports/csv/{page_no} and download the file for me? Currenlty, I get only an empty header downloaded using this option without any Shotgrid data.

1 Like

@FilSus or anyone else looking to automate this in Python using reports/csv/{page_no}

First, note the following limitations I’ve seen:

  1. If a field is being used for grouping on a page, it will not be pulled through this url
  2. If a field is a query field, it will not be pulled through this url (and you’ll need a different solution)

But for basic pages that don’t rely on the above, you can do the following.

import csv
from dotenv import load_dotenv
import os
import requests

# pulls USER and PASS from a .env file that sits at the same folder as this file. These creds are the legacy login credentials from "Account Settings -> Legacy Login and Personal Access Token"
load_dotenv()
username = os.environ.get('USER')
password = os.environ.get('PASS')

if __name__ == '__main__':
    # Fill in with your own shotgrid url and page id
    shotgridUrl = ''
    pageId = ''

    # setup session with basic auth for making request
    session = requests.session()
    auth = requests.auth.HTTPBasicAuth(username, password)

    # make a request to the url using our basic auth session
    response = session.get(f'{shotgridUrl}/reports/csv/{pageId}', auth=auth)

    # do wonky stuff with the response since it isn't json
    decodedResponse = response.content.decode('utf-8-sig')
    outputData = []
    lines = decodedResponse.split('\n')
    for line in lines:
        row = line.split('",','')
        for i,v in enumerate(row):
            datum = v.replace('"','')
            row[i] = datum
        outputData.append(row)

    # write data to a csv
    with open('my_report.csv', 'w', newline='') as f:
        writer = csv.writer(f, quoting=csv.QUOTE_ALL)
        writer.writerows(outputData)
2 Likes

Hi there!

Sorry to ping an old post, but this is exactly what I was looking for… though do you know if this still only works with legacy logins, or is there a way now to use an Autodesk account?

My company stopped supporting legacy logins recently. I haven’t figured out how to implement this in a way that works with Autodesk only, and I can’t find a workaround online. I only just barely know what I’m doing (sorta kinda maybe), so it’s possibly just outside my skills, but I’m thinking I could also just be out of luck.

Figured I’d ask before giving up!

Thanks!

I’m not clear how your company could prevent you from setting a legacy login in your SG account settings

1 Like

+1 for this. I don’t know how to add a legacy login to our site.

It’s a per-user setting (https://yourshotgridsite/page/account_settings)

From what I’ve been told, the ability to log in with a legacy account is an http request. My company has restricted our Shotgrid access to https only, effectively disabling the legacy logins.

Even when I’m already logged in with Autodesk, going to the reports URL here brings up a browser http login field. I can enter my credentials, but that leads to an http access denied page.

I haven’t been able to figure out a way to access this url without this happening… I’m not sure if using legacy logins through https is something I can enable myself or not.

I highly doubt legacy logins work via http.

ShotGrid, before Autodesk ID, used ssl/https urls for about everything.