Easiest way to run higher OpenGL operations within Maya?

#1

It’s possible to draw simple shapes and tools on screen using the various UI and Draw managers provided by maya, using platform-independent calls to Maya’s own drawing api, and the GLSL shader allows a normal vertex->fragment pipeline to be scripted.

However I’ve been unable to find a good way to set up raw OpenGL pipelines to shade specific objects.
MPxHardwareShader defines a render() method which looks ideal - the example even shows raw GL commands being called. However as far as I can tell that method isn’t even called when connected up to a shading group.

I know there is also MPxShaderOverride which seems much more widely used in the examples. It seems to interface more closely with the ogsfx/glsl scripting system - however it does also define a draw() method, which from the sound of it seems very similar to the render() method on the hardware node. I haven’t been able to trigger this either though.

I’d be interested if anyone has any experience setting up pockets of fully custom rendering like this.
Thanks

#2

SO I’ve been on a semi-deep dive through this stuff, and while I don’t have a grasp on the entire system that Maya uses to draw stuff yet, I’m on the way.

In short, the stuff above (including individual nodes defining draw() methods) is old news, only working in legacy viewport (which isn’t even accessible anymore).

Viewport2.0 relies on Overrides (like MPxDrawOverride and MPxShaderOverride), and on registering those overrides to act on a specific kind of node - this is done through MDrawRegistry, during the same initializePlugin() function as you normally use to register nodes.

You do still need to register your own nodes for these (for example, if you want to draw a custom locator, you still need to define a new MPxNode for it), but only as specific targets for the drawing overrides.

That’s all well and good, but I still had a time trying to get the drawOverrides to do anything - I neglected a small point in the docs: drawOverrides finally call a “Drawing callback”, passing an MFrameContext and an MUserData container, to actually draw the node. In the c++ docs it’s given as the GeometryDrawOverrideCb , but in both c++ and python it can just be a normal function.

To pair this orphan function up to the override class, there is one last tricky point, in the override constructor - here I show in python, but c++ is the same:

def myDrawCallback(
    frameCtx:omr.MFrameContext,
    drawData:om.MUserData):
    """do your fancy raw GL drawing here"""


class MyDrawOverride(omr.MPxDrawOverride):
    def __init__(self, obj:om.MObject):
        # pass this object and the function above
        # to the superclass' constructor
        omr.MPxDrawOverride.__init__(
            self, 
            obj, 
            myDrawCallback, 
            True)

Personally I think the docs should place more emphasis on this mechanism, as I didn’t find it obvious to figure out.

The DrawDB classification string used to register the override is also important, but the various Footprint locator plugin examples are quite good at explaining that. I highly recommend looking through all of them in the devkit if you’re interested in the drawing stuff here.

A lot of this stuff is probably obvious, and explained in the Viewport2.0 docs and whitepaper - I’m finding that I understand much more of that paper now, where previously I didn’t know where to start.

I’m still not sure of how actual hardware shader material overrides work, or stuff like writing to and accessing viewport render targets - those are next for me.

Hopefully this can help someone who was as lost as I was.

2 Likes