Usually the approach would be that you download a version of sgtk, and store it somewhere centrally that the farm nodes can find, and then use that to bootstrap the project config.
The bootstrap process will download everything required by the config including the core API.
The initial sgtk that you download will not need to be updated that often since it will only be used to start the bootstrap process, after that the process will swap out the core API for the one required by the config.
It used to be the case that you could pip install sgtk, but unfortunately this is currently not working.