Best way to create a non-blocking zeromq server in Max or Maya?

Coding
#1

Hi there,

so i’ll finally try and wrap my head around doing some zmq work. Step 1: Recompiling pyzmq/libzmq to work with Max 2015’s python (compiled with a different VS version) has been accomplished (whee).

So now i am trying to wrap my head around the problem i am trying to solve and what the best approach would be using zmq.
Basically i am trying to replace COM/OLE interfaces in some of our tools. We have several standalone tools that use COM/OLE Interface to remote control max and do changes to the scene. So as a simple
example imagine working in max and then having a standalone tool that allows you to load/unload xrefs.

Now the way this works in OLE is that the OLE registration happens intransparent to the user. Once you fire commands they are executed in max. I did a basic preliminary test with the sample for Lazy Pirate
Req/Rep Patterns and this works fine. So now i am wondering how to best turn this into a non blocking zmq server listening in the background when normally using max.

Any pointers would be greatly appreciated.

Cheers,
Thorsten

#2

I have no experience with ZMQ, and my Max knowledge is getting pretty stale, but I implemented a WCF interface in Maya a while back with similar goals, so the issues are possibly similar. I’m going to guess that there is not an easy way to simply expose all of the Max/Python API. You will have to create a set of your own RPC-like commands, hook them up to the zmq interface, and figure out a way of relatively cleanly returning values/data from those call. Since the client at the other end probably isn’t going to understand Max native data (such as mesh objects, or UVs), you’re going to have to figure out how to package these in a platform neutral format if you want to pass those as well. There’s a variety of choices here such a XML, JSON, or protobuf encapsulation layer, or even raw binary data. If you want strongly typed data at the other end, then the both the client and server will need code to do the conversion to those types. For me, WCF provided the ability to serialize semi-arbitrary data over it’s supported streams, but that data is all .NET/CLR types, so I couldn’t avoid the same conversion issues.

Since you probably want to make calls to this interface and get data back, you will need to create a system that caches the outgoing commands and reattaches the resulting data that comes back in a different message, possibly in an out-of-order sequence from the way they were sent. I haven’t played with zerorpc, but that might solve some of these issues.

If you just want to go cheap-n-cheesy on the API, you could just expose Python’s eval command over the interface and do all of the conversion and massaging of the resulting strings and errors on the client. If this interface is exposed to the internet (versus your intranet), I would definitely recommend adding some sort of authorization security so people don’t start executing random bits of Python code on your machines.

Good luck!

#3

Heya,

thanks a lot for the input! The problem is a tad different though. For now i only need to trigger Max commands. I don’t need any data back except for some kind of “k,thanks,did it”. We already do expose a set of special functions via mxs, so simply giving a means to execute mxs would prolly be sufficient.

The focus of my question is rather what is the best way to make a non-blocking zeromq server within a host application. So that max always silently listens, is not blocked and does not permanently consume resources in a loop. With OLE this is built in. With zmq it seems it isnt, but it is probably very well capable and i am looking for design patterns.

Security will not be an issue for now (even though security is a BIG issue in the biz we’re in) as this will typically be limited on a single machine, or worst case the intranet which is rather closed down.

Cheers,
Thorsten

#4

Well, when Max is busy performing a function, it of course will be blocking. zeromq should be an event driven system. In other words, once you create the interface, it shouldn’t consume significant resources, and will effectively be idle until it receives a message. The message will call the registered callback. Did you take a look at zerorpc? It seems like they’ve done much of this for you. Maya has commands for dealing with asynchronous Python threads. You can have the thread wait until Maya becomes available before executing Maya specific code. Doesn’t Max have an equivalent?

#5

Hey guys,

Sorry to dig this subject, but I’m also struggling to make zeromq non blocking. I guess I should make a separated thread, but it doesn’t seem to be well supported by Max (a simple print in a thread will crash it, although it “seems” to work if the thread does nothing to max). So I really wonder what’s the way to go to make that listening loop non-blocking ?

I assume there should be a way as for example, PyQt supposedly run in some sort of loop, and doesn’t block Max nonetheless.

In last resort I was thinking about making a BackgroundWorker in C# who plays the role of the server and simply relays the request to the python side in max, but that seems quite hacky :slight_smile:

Cheers !

#6

This may be more trouble than it’s worth. In maya, at least, only the main thread can touch gui or scene objects so the zeromq server could receive and respond to requests in a non-blocking way but the actual changes to Maya would take place only through evalDeferred() or executeInMainThreadWithResult(): basically they’d be waiting for a maya idle signal before actually happening. So you have to choose between a server that blocks or a server that accepts requests, initiates actions, and then sends a callback ping at some future date when the work is really complete. I no longer remember if this is the case with Max as well but I’d be surprised if it wasn’t