It might have been discussed before, but we are wondering if having entities with a huge number of data could directly affect the site loading performace.
We are currently using two custom entities with more than 30000 data in each, and the number keeps increasing every time we run our in-house tools (i.e. publisher).
Does it slow down the entire site even in other projects where we don’t use those entities? Could the performance be better if we disable them from Site Preferences?
Thanks for asking, and thanks for joining the community!
The very short answer to your question is “yes and no”
The longer answer is more in-depth: yes, having entities with a huge number of records can affect performance, but it depends on some things (more on that later). The qualifier on this “yes” answer is that 30k records isn’t really “a lot” by any means. I think you don’t need to worry about performance degradation due to # of records until you’re getting into places where you have several hundred thousand or more records. Even then, if you’re judicious in your use of filters and the amount of information you’re displaying per-page, you should still be okay. The limitations of Shotgun are really only the limitations of the database on the back-end that stores all your data and performs queries.
Having said that, of course, please let us know if you’re experiencing unexplained slowness, especially if you have a specific url or activity where that slowness is manifesting.
Lastly, your activity on your SG site is generally bounded by a project (with the obvious exception of Non-Project Entities), so records that belong to other projects aren’t calculated and won’t factor in to site responsiveness.
