Strategy session: perforce triggers

Pipelines
#1

So, I’m setting out to do some perforce triggers for content validation and I have an interesting little problem to contemplate.

The actual perforce server is hosted offsite, behind a lot of security. Getting RDP or other access to it is a bureaucratic hassle. Unfortunately, perforce triggers run locally on the server (and on windows, they run on the default System account) – which means that it’s pain in the ass to manage and test.

So there’s two parts to the puzzle:

  1. How to set up the system so I can iterate on it without spending all my time begging for remote access to the server from the security Nazis. Options:
    a) It’s been suggested that the triggers have their own dedicated p4 client on the server machine and that they force-sync a directory of trigger executables before they fire: that would make sure that I can update the trigger behavior by updating files in the perforce server itself. Bad part: testing and developing on a different environment and only finding out what works after altering the server state
    b) Perhaps a web app running on my own machine? The trigger is a dumb HTTP call that just passes the trigger info to the web app which does the real work. Good part: I can see the tool running and debug it locally, and the servers side of things never changes. Bad part: I’ve got to have my own server up all the time.
    c) ?? Any bright ideas?

the second part:
2) what language to do it in?
a) I’d want to do it in python except that commits me to maintaining a python environment on the server machine – with all the usual crap about paths, etc – through the remote connection. Especially annoying because the perforce .pyd wants to be located near they python binaries, so making it all self contained is harder.
b) I could see publishing each trigger as a click-once windows exe. I think it’s possible for those to update themselves on startup . However the version I’m familiar with requires user confirmation, which won’t work remotely.
c) (getting kind of crazy here) Jython with the whole trigger environment in a single JAR, run by Java Web Start. I’m pretty sure the server has Java on it.
d) …?

Discuss.

#2

One useful lead: http://www.portablepython.com/wiki/PortablePython2.7.3.2. Should let me create a self-contained python-in-a-box I can rely on. Hopefully.

#3

We check out trigger scripts into perforce, and those are then synced by the server automatically as you suggest. It works well, and you can control access with the regular Perforce protect file. All of our newer scripts are in Python, and some of our older ones are in Perl (ug!). Set up your triggers table on a small directory and use that for testing before rolling it out wider. You can also test locally, but you will notice some small differences in how perforce responds depending on where the script is running. This isn’t usually an issue.

Remember that by using an HTTP call as indicated, you are introducing another point of failure. If your webserver(or equiv) goes down, no one will be able to check in.

I strongly recommend adding an override to your scripts. Our system uses special comments in the changelist description. You could add the keyword DISABLE_CHECKS (or whatever) to suppress the triggers when there are technical difficulties, or when you need to submit oddball items. Likewise flags like FORCE_FAIL and VERBOSE_DEBUG are handy.

1 Like
#4

So, the working plan for the time being it to include a copy of Portable Python in the depot and to write triggers against that. The trigger themselves will be python scripts and the server machine will have a ‘triggers’ perforce client which includes the python and the trigger scripts. Debug logging, etc will be shot up to our logging server (so as not to fill the p4 server with log files) using a stripped down version of the sql reporting tool that runs in all our maya scripts.

The nice thing about portable python is that it is install-free – no registry and no MSI required. It’s a bit of a pain because way too many other python packages – including the P4 api – only look for Python via the registry, which means a lot of them are hard to install (the perforce installer in particular is very finicky - I had to install python 2.7 to a spare machine, install perforce, and copy the files back manually. Blecch). However it works fine once it’s configured and since i’ll be running the triggers with a bat in the same directory there are no issues with Path, PythonPath, etc.

I’d be happier if there were a Portable Python implementation for mac (our build servers run on Macs) so the same code could run on the build servers - it’s not a real necessity, since the triggers are for our (windows) artists not the server, but it would still be nice to have as standardized way to push cross-platform tools. My experience of just using the default python install on OSX is uninspiring - it seems almost random what version, libraries and so on you get if you just open a shell and type “python”.

I don’t think that this corner of the toolset will grow large enough to worry about things like conflicting packages, dueling upgrades, or similar Python hassles - if needed I could wrap the whole thing in virtualEnv on a per-tool basis, but that seems ridiculous.

#5

[EDIT - This is similar to what someone else posted… Didn’t read the thread before posting :)]

This is something I’ve done before that had worked pretty well…

1.) Setup a Pre-Commit trigger that looks for a some sort of token (like an MD5 hash) in a changelist description. If the token exists, let the use submit. If not, reject and inform the user. The nice thing about this is you can mask the trigger for specific directories.

2.) Write a tool that wraps the P4 submit functionality and inserts your token at the end of the changelist.

For whatever operation you need to do, this forces people to submit through your custom tool. That way you can do all your validation and things locally, and when you’re done, call your wrapped submit method.

The nice thing about this is that you can write multiple things that use the same trigger, but perform different operations.

Hope this helps!
-J