Deploying a small toolchain

Pipelines
,
#1

We have a small set of python/pyqt scripts that we would like to deploy a handful of artists - probably about 5 to 10. There’s only 2 of us working on the scripts and we don’t have a tremendous amount of tools specific experience, (our backgrounds are mostly in render) and certainly limited knowledge when it comes to distribution.

We’re in a unique position where we have no legacy code so we’ve chosen to use Python 3.6 and PyQt5. I anticipate us occasionally having to run a local server to load some html / Javascript specifically due to same origin policy.

Our biggest hurdle at this moment is figuring out a deployment method. We’d like to do this in the least invasive way as to not disrupt the engine team or IT. Right now we have users run a batch file that runs the python installer exe and then proceeds to uses pip to install the pyqt5 and perforce modules. We then distribute the scripts in a extremely rudimentary way with perforce.

I don’t know the nuances of deploying tools so I’m curious if this is a stable and sustainable approach. If not I was hoping for some guidance or suggestions on how to approach this pipeline.

1 Like
#2

Some Q’s first:

  1. Sounds like you have no DCC tools to worry about? No hooks to Maya or Max?
  2. Any chance this will be going out of house? Particularly to people with different IT setupts?
  3. Are the users technical at all?

In general getting things to people once is easy… Making sure that everybody is in sync over time, with the same stuff so that you don’t get irreproducible bugs for one user’s wierd state is hard.

I personally shy away from using P4 for distribution, because it conflates your source control as a developer and the user’s need for distribution – you don’t want to be discouraged from checkpointing your own work by the fear of pushing half-done stuff to the users. You can work around this by maintaining separate depots and copying only the built, tested products into the distribution repo.

Some related threads:

#3

Hey Theodox,

Thanks for the reply. I’ll have to read through the linked posts, apologies for not searching first. To answer your questions:

  1. No Max/Maya hooks. The tools are Painter/Designer exclusively
  2. It’s possible yeah, there would be a few Qt apps we’d possibly want to make available externally.
  3. Honestly, users are not terribly technical.

The staging issue makes total sense to me and something I hadn’t thought about. A two stage system sounds like it would be the easiest and require the least support from IT/Engineering as something like a git server would.

Our Python install batch file grabs the specific build of the network, so everyone should be on the same version unless they manually downloaded and installed for whatever reason. From the same batch we wait for the install to complete and ask pip to install the modules we need. However I suppose that does not force them to be on the latest module version, I’m not sure how often this happens.

The pyc files are also something I had not taken into account, if I understand correctly - even if we update the script, check it into perforce, and have artists sync - they could still be out of date unless the pyc files are deleted?

Again apologies for my limited understanding I will have to re-read the posts above a few times.

#4

The usual problem with pycs is that Python looks for them first. If you have moved things around – especially if you have turned a single-file module into a multi-file package – leftover pycs can be used in preference to the new .py files, which creates a lot of confusion. Your distribution method should clear old files reliably or you’ll probably run into hard-to-repro bugs.

If the tools are simple, you can use something like py2exe or cz_freeze to make executables and distribute them as single items.

Pip can be a false friend in these cases, because it’s possible for pip to change the behavior of an app thats already installed when making an update required by a different app. Unless you’re really low on disk space you want a solution that maximizes the independence of different apps from each other.

2 Likes