Could we improve the speed of PyMEL?

Coding
#1

I got a crazy idea, but please hear me out:
When people talk about the pros and cons of PyMEL what comes up are things like:

-PyMEL is quicker to write
-PyMEL is pythonic
-PyMEL is object oriented
…but certain PyMEL things are simply SLOWER

PyMEL is slow because certain operations (like pm.ls(selection=True, flatten=True) results in PyMEL having to create a lot of PyMEL objects (PyNodes) like pymel.core.general.MeshEdge(dolansEdge).

I’m curious though if this object creation part somehow can be done with OpenMaya instead? Or even better: a compiled C++ plugin? You would identify the slower parts of PyMEL and write OpenMaya substitutes for them (which ofc, need to yield objects compatible to the rest of the PyMEL lib).

…or, I could simply mix OpenMaya and PyMEL and use OO where necessary.

Thoughts?

#2

I’ve looked at doing this in the past, and even going so far as to write some cython shims to see if maybe that could speed stuff up, and honestly the gains were completely not worth the effort from what I remember.

To expand on this a bit, the biggest speed impact on building a new PyNode object is not actually the nature of the object itself, but the fact that you need to walk a fairly complex decision tree to identify the best “most-derived” PyNode that matches the corresponding object.

Which would still exist in some form or another no matter how objects are defined under the hood. Mind you, PyNodes are already a wrapper around the OpenMaya objects, so not a lot of a speedup there.

If you’re really hoping to speed things up, what you probably want to be looking at is how to either offload that entire search process to C/C++, or some kind of complex caching mechanism (that has to survive arbitrary object lifetimes, scene changes, references and all that messy stuff without dramatically slowing down the rest of the system)

So my final advice would be. Use mel (maya.cmds) or OpenMaya where you don’t want to deal with the PyMel overhead, and use PyMel where the overhead isn’t an issue.

2 Likes
#3

I never had a case where cmds couldn’t do what pymel did.
Learn how maya handle longnames and how to check conflicting names, and you won’t need pymel.

#4

PyMel most certainly can do things that aren’t exposed through mel (maya.cmds), just by virtue of it exposing methods from the underlying API objects. That isn’t to say that you can’t perform similar operations by leveraging the various mel commands to get similar results. But PyMel is absolutely a superset of the exposed mel commands.

However that extra functionality comes at a cost of complexity. PyMel does a fairly amazingly job of abstracting that complexity away from the user, vs say OpenMaya where that complexity is exposed directly to the coder. But that abstraction comes at a performance cost.

Each layer has its positives and negatives, and which you choose can come down to a variety of competing concerns. So ultimately there is no 100% answer to all the things when it comes to this.

#5

Maybe I’ve been too categoric :slight_smile:
I also often write my own simpler class to represent objects inside maya and the few attributes I want to use.

1 Like
#6

Totally valid approach.
I’ve done this a lot, even leaving some comments behind like “Yeah PyMel has a class like this, but its to heavy and all I want is x, y, z,”.

#7

Another con to PyMEL is the long initialization time. IMO that is reason enough to avoid PyMEL for pipeline setup, asset build procedures, and common art tools. Up to you and your fellow tech artists to determine if it is still worth the cost. In my experience it’s not uncommon for studios to avoid PyMEL entirely and encourage greater use of OpenMaya.

#8

My whack at this is Nodule, which lets you write pymel-style property accessors but is not otherwise an attempt at wrapping all of Maya. In my tests it’s typically 5% faster than the equivalent Pymel, and does not have the startup hit.

A smarter person might be able to push that a few percentage points farther but probably not too much farther – there’s not actually much going on in the code beyond some syntax sugar and it’s reasonably lean, which suggests that the real speed bump here is the Python/C++ boundary and not anything happening in Python proper. That means that even doing in the API is unlikely to be significantly faster, since OpenMaya still has to cross the line. I know that when I did Minq it turned out that the API was actually perceptibly slower than carefully written cmds.

In any case, if all you want is

 position = my_object.translate
 my_object.stringAttr = 'new_string'
 connect (this_object.rotateX, that_object.rotateY)

You can get there without too much hassle; my implementation is a single file of about 800 lines.

2 Likes
#9

Seems as good a time as any to mention my crack at this.

In a nutshell, a thin wrapper around OpenMaya with PyMel-like syntax (better, I think, including dict-like access to attributes and optional snake_case), 140x faster than PyMel on average (see reproducible performance tests and graphs toward the bottom). I made it for a commercial (yet unreleased) project involving real-time physics simulation in Maya and still use it for anything involving Maya.

Have been meaning to “announce” it once it was “finished”, but hey, when is software ever finished? xD

It’s open source, happy to hear your thoughts and continue improving it. :+1:

Also one of my pet peeves, and something I specifically addressed, and benchmarked.

Indeed. I solved this in two ways; (1) objects are neglibly small to start with and (2) objects are reused based on their API ID.

A side-effect of that reuse, is that you can store Python data along with a Maya node.

a_node = cmdx.selection()[0]
a_node.data["My Data"] = "Indeed"

for node in cmdx.ls():
  if "My Data" in node.data:
    print("This node had custom data: %s" % node.name())

Attribute lookups is another bottleneck, with both cmds, PyMEL and OpenMaya, and those are reused as well. Which is what makes it suitable for use at real-time in rigs and callbacks, listening on thousands to tense of thousands of attribute change signals and doing something with it.

Oh, and it’s a single-file too. Easy to distribute and share, either copy/paste the contents into the script editor, or paste the file somewhere you can import it.

8 Likes
#10

@marcuso …i like it! a single file, simple and wraps API, whats not to like?
Will give it a go, thanks for sharing!:slight_smile:

#11

Is API 2.0 still under development? I’ve been under the impression that it’s not, though I may be wrong.

#12

API 2.0 is not unlike Viewport 2.0; it does a few things better than 1.0, but it doesn’t do everything 1.0 does and it isn’t making any obvious strides towards being complete.

It does gain features on occasion; for example the reason cmdx requires a minimum of Maya 2015 SP2 is because a feature was added that made cmdx possible in the first place (i.e. MObjectHandle) and 2016 added a few new iterators to compete with cmds.ls in terms of performance, and so forth.

For the things 2.0 doesn’t do, cmdx uses 1.0 (e.g. for creating NURBS curves). It isn’t an attempt to stick with particular any API, but rather to use whichever one is quickest, which in most cases is 2.0 (and sometimes even cmds!).

There’s things being added still that could make cmdx faster still, and it’s interesting to stay up to date with the release notes. They typically have a section in there for OpenMaya. Soon 2015 will be a thing of the past, at which point we can bump up the minimum to 2016, and reap those benefits for all.

4 Likes