Shotgrid Python api CRUD method 'Shotgun.update' is too slow

Dear community members,

Can you please help me out to improve or explain the performance of updating four attributes on one entity; for instance status, status_changes, start_date, and true_date on a task consumes 0.5 sec? Which is a bottle-neck for the performance.

Best regards,
TJ

You should probably look at the network latency using e.g. traceroute. This might show that a certain hop on the route is slow.

Also how are you updating these?
Do you have code to show us?

products = get_all_ians(sg)
update_data = {‘sg_status_list’: ‘ip’, ‘sg_change_request_count’: 3000, ‘start_date’: ‘2023-03-18’, ‘sg_true_date’: ‘2023-03-20’}
tt =
ttt =
for product in products:
print(product[“code”])
tasks = get_all_tasks_from_ian(sg, product[“code”])
start0 = time.time()
i = 1
for task in tasks:
start = time.time()
sg.update(“Task”, task[“id”], update_data)
print(u"set data task: {} (took {} s)“.format(str(i)+task[“content”],
time.time() - start))
i+=1
tt.append(time.time() - start)
print(u"set data on all tasks (took {} s)”.format(
time.time() - start0))
ttt.append(time.time() - start0)
print(len(tt))
print(min(tt))
print(max(tt))
print(sum(tt))
print()
print(len(ttt))
print(min(ttt))
print(max(ttt))
print(sum(ttt))

average of 10 repetations

no.of tasks = 1836
minimum time took for a task = 0.29140734672546387
maxtime took for task = 0.5197522640228271
total time for 1836 tasks = 625.6770815849304

num. of products = 162
minimum time took for task = 0.0
maxtime took for task = 12.030255556106567
total time for 162 products = 625.6931428909302

Hi Ricardo,
products = get_all_ians(sg)
update_data = {‘sg_status_list’: ‘ip’, ‘sg_change_request_count’: 3000, ‘start_date’: ‘2023-03-18’, ‘sg_true_date’: ‘2023-03-20’}
tt =
ttt =
for product in products:
print(product[“code”])
tasks = get_all_tasks_from_ian(sg, product[“code”])
start0 = time.time()
i = 1
for task in tasks:
start = time.time()
sg.update(“Task”, task[“id”], update_data)
print(u"set data task: {} (took {} s)“.format(str(i)+task[“content”],
time.time() - start))
i+=1
tt.append(time.time() - start)
print(u"set data on all tasks (took {} s)”.format(
time.time() - start0))
ttt.append(time.time() - start0)

print(len(tt))
print(min(tt))
print(max(tt))
print(sum(tt))
print()
print(len(ttt))
print(min(ttt))
print(max(ttt))
print(sum(ttt))

set data on all tasks

tasks = 1836

min time per task = 0.26454901695251465
max time per task = 0.5756251811981201
total time = 611.5521297454834

of products = 162

tasks per product = 34

min time per product = 0.0
max time per product = 12.550530195236206
total time = 611.575187921524

It might be interesting to try this update as a single batch instead
https://developer.shotgridsoftware.com/python-api/reference.html?highlight=update#shotgun_api3.shotgun.Shotgun.batch

2 Likes

Yeah batch will come into play.

Also not sure what you’re using to test, if you are using the ShotGrid Python console to test your code then that will likely add a bit of latency (it gets slower over time).

1 Like