We are kinda moving in the same direction, but atm we do not publish files for each version that will be rendered. One reason for this is the lack of automatic cleanup.
A publish is meant to be like a checkpoint, a fixed state that you can load later, and won’t change.
This is usually for downstream use, e.g. by other departments. That is why you want a unique file, which will not be touched anymore.
It warrants an entire separate discussion how to handle the versioning of files. For instance, do publishes follow the version line of scenes, or have their own? There are arguments for both.
One of the tradeoffs is exactly accumulating a lot of data on disk, vs. having guaranteed fixed points in the work process.
Shotgun’s default implementation would register those published files as dependencies even though they aren’t actually needed to rebuild your render output.
But the point is that they were needed to produce the render, not that you need them to view it.
I’ve just made sure that we don’t create duplicate published files as I see no need for that, only one file can live in that location on disk, dependencies upstream and downstream are added as we go.
I’m probably missing some points. The default publish plugins do check if there are conflicting publishes (and maybe do not let you publish unless you raise the version number, don’t remember).
It makes sense to update an existing publish, if you want to prevent a lot of data on disk.
This does invalidate other artifacts (e.g. Versions) that depended on that publish, since it might not represent the same state anymore. In practice it might not matter, though.