How to convert maya uv area to it's coverage space in textures(pixels) with python?(Create a mask)

Hi,
Does anyone know to convert maya selected object’s uv to it’s coverage space in textures?
I want to write a script that allows users to select one or multiple objects and with one click to create a mask of the objects in it’s textures.
Using arnold there is a “Render Selection To Texture” Utilities, but that requires an image file connected to the shader, and even with a pure white image textures connected to the color input, but there is no way to set the background to black, thus can’t create a mask out of it.

With maya’s own cmds.convertSolidTx() I can create a black background but grey “white area”.

cmds.convertSolidTx('White.outColor' ,'testGeo', backgroundColor=[0,0,0], backgroundMode='color', fileFormat='tif', fileImageName='test_mask.tif', force=True, fillTextureSeams=True, resolutionX=1024, resolutionY=1024)

The shader is an aiFlat material like below


The connected file node is a pure white texture.
But the probelems are:

  1. with or without the image texture the output image “test_mask.tif” suppose white color are grey color;
  2. If without image texture connected to flat shader’s input color, it doesn’t matter what the color is set to in the flat shader, the output image is still grey.

The setting doesn’t make sense.
And I’m wondering if this is something maya can do natively or is it something that needs library like numpy to do the conversion.

In Maya natively, you can try setting a face color attribute and do a render to texture

Otherwise if you have some python knowledge you can do that with PIL in python:

  • create a face set for each matte you wantb
  • get each face vertex uv as a 2d vector in a dictionary
  • get each face vertex order
  • in PIL, on a square canvas, draw a polygon with each uv value in order of face vertex. The color of the polygon will depend on the face set

You might hav some trouble getting the exact aliasing of your mattes on the edges, if so you might have to extend the edges of the polygon based on the alpha (you can search how nuke’s gizmos do that and reproduce it in PIL).

Thanks for replying!

Using arnold Render to Texture, I can’t set the backgroud to black, maybe there’s some setting I missed?

Using PIL ImageDraw It seems to need to sort that list of vertex in order, either clockwise or anticlockwise along the edge, or else the shape will end up with random hole. For simple shape, chatGPT’s algorithm to sort the order is pretty simple, but if the UV shell shape is a concave hull, than that’s where I’m stuck.

Rather than dig deep into figure out how to sort complex shape outline vertex order, I’m going back to maya to find a native way, such a simple function, I just can’t believe there’s no native way to do it.

You need to select the shader component that you will bake.
For example color (Lambert), or base, or baseColor (StandartSurface).
At the same time, it is important to understand that it is not necessary to assign a shader to the geometry you are baking!
Just refer to the component of any shader with which you want to bake the texture.
For example, you could create a Lambert and assign the color value to white.
And set the background color of the baked texture to black.
For example:

cmds.convertSolidTx("Lambert2.color", "MyGeoShape",
                    backgroundColor = [0, 0, 0],
                    fileFormat = "tif",
                    resolutionX = 1024,
                    resolutionY = 1024)

If you want to create a mask only for the selected polygons, then you need to set the componentRange = True flag.

There is detailed documentation for this command with examples:
Maya command (Python): convertSolidTx

Below is a small demo.
The first function converts the selected polycomponents into Shapes.
Next we create a shader whose color component we will bake.
The next function creates a Solid texture for each PolyShape from our list prepared in the previous step.
The last function creates a Solid texture for the selected polyfaces of any, but only one PolyShape.

from maya import cmds
import os

# Get Mesh Shapes for delected polygonal components and polygonal shapes:
def selected_to_mesh_shapes():
    sel = cmds.ls(selection = 1, long = 1)
    if sel:
        sel_filter = cmds.filterExpand(selectionMask = (12, 31, 32, 34, 35, 70), ex = 0, fullPath =1)
        if sel_filter:
            mesh_shapes = set()
            sel = cmds.filterExpand(selectionMask = 12, ex = 0, fullPath =1)
            if sel:
                mesh_shapes.update(set(sel))
            sel = cmds.filterExpand(selectionMask = (31, 32, 34, 35, 70), ex = 0, fullPath =1)
            if sel:
                mesh_shapes.update(set(cmds.listRelatives(sel, parent=1, shapes=1, fullPath=1)))
            if len(mesh_shapes):
                return list(mesh_shapes)
            else:
                cmds.confirmDialog(title = "Incorrect select",
                                 message = "Select not contains mesh_shapes.\n" +
                                           "Select polygonal mesh(es) or any components on the same mesh.")
                cmds.error("Incorrect select...")
        else:
            cmds.confirmDialog(title = "Incorrect select",
                             message = "Select does not contain polygonal elements.\n" +
                                       "Select polygonal mesh(es) or any components on the same mesh.")
            cmds.error("Incorrect select...")
    else:
        cmds.confirmDialog(title = "Nothing selected",
                         message = "Nothing selected.\n" +
                                   "Select polygonal mesh(es) or any components on the same mesh.")
        cmds.error("Nothing selected...")


# create temp shader for mask:
def create_temp_shader_for_create_mask():
    shader_name = 'FOR_MASK_MATERIAL'
    material_name = cmds.shadingNode('lambert',
                                     asShader = True,
                                     name = shader_name,
                                     skipSelect = True)
    cmds.setAttr('{}.color'.format(material_name), 1.0,1.0,1.0, type = 'float3')
    return material_name


# create mask texture files for selected meshes and meshes for selected polycomponents
def create_masks_for_selected_shapes(file_format = "tif"):
    all_selected_to_shapes = selected_to_mesh_shapes()
    for item in all_selected_to_shapes:
        txt_name = item.replace("|", "_")
        txt_file_name = "mask_for_{}.{}".format(txt_name, file_format)
        solid_txt = cmds.convertSolidTx("{}.color".format(material_name), item,
                                        name = txt_name,
                                        antiAlias = True,
                                        backgroundColor = [0, 0, 0],
                                        fileFormat = file_format,
                                        fileImageName = os.path.join(save_path, txt_file_name),
                                        resolutionX = 1024,
                                        resolutionY = 1024,
                                        force = True)
    return solid_txt


# Or create mask texture files for selected polygons on one mesh:
def create_masks_for_selected_polygons(file_format = "tif"):
    m_shapes = selected_to_mesh_shapes()
    if m_shapes:
        if len(m_shapes) !=1:
            cmds.confirmDialog(title = "Incorrect select",
                             message = "Select contains items for more than one mesh.\n" +
                                       "Select one polygonal mesh or any components on the same mesh.")
            cmds.error("Incorrect select...")
        else:
            polygons = cmds.filterExpand(selectionMask = (34), ex = 0, fullPath = True)
            if polygons:
                txt_name = "mask_for_custom_selected"
                txt_file_name = "{}.{}".format(txt_name, file_format)
                solid_txt = cmds.convertSolidTx("{}.color".format(material_name), polygons,
                                                name = txt_name,
                                                antiAlias = True,
                                                backgroundColor = [0, 0, 0],
                                                fileFormat = file_format,
                                                fileImageName = os.path.join(save_path, txt_file_name),
                                                resolutionX = 1024,
                                                resolutionY = 1024,
                                                componentRange = True,
                                                force = True)
    return solid_txt

# Set patch for saving solid textures:
save_path = r"C:\temp"

# create temp shader for mask:
material_name = create_temp_shader_for_create_mask()

# selected polygonal component and/or polymeshes
solid_txt = create_masks_for_selected_shapes()


# Or selected polygonal faces on one polymesh
solid_txt = create_masks_for_selected_polygons()

This work! Thanks!

First to specify the shader color should do it your way,

'shaderName.color'

Not

'ShaderName.outColor'

Otherwise it will create the image with grey color rather then white.