Invert selection of polygons

Hi there,
I have a problem I can’t resolve:
I have a list all the polygons selected in my viewport on 1 object and I would like to have an list of the non-selected polygons.

For the moment I use the mel.eval(invertSelection;) function then get the new selection.
But it can be really slow depending on the number of polygon.

Is there a way to get the list easily ?

I don’t do a lot of Maya (and especially mel programming) but Maybe you can get a list of all the polygons and just create a new one out of All - Selected?
I don’t know if that’s a PITA in mel or not though.

I can’t do that (or at least it does not work the way I do it)
The list of all the polygon for a cube is like this:

pCube1.f[0:5]

So I can’t use all - selected :frowning_face:

In MEL you can do:

whatIs invertSelection;

It will return e.g.:

// Result: Script found in: C:/Program Files/Autodesk/Maya2022/scripts/startup/invertSelection.mel // 

Check that MEL script for the implementation of the invertSelection command. Potentially that helps.

from maya import cmds
import re

def invert_components(components):
    
    components_match_all = set()
    for path in components:
        if "." not in path or not "[" in path:
            # Not a component
            continue
        
        # Replace all component indices with * so we can query them with ls
        path_component_wildcard = re.sub("(\[)[0-9:]+(\])", r"\1*\2", path)
        components_match_all.add(path_component_wildcard)
        
    before = set(cmds.ls(components, long=True, flatten=True))
    all = set(cmds.ls(list(components_match_all), long=True, flatten=True))
    
    return list(all - before)

selection = cmds.ls(sl=1)
inverted = invert_components(selection)
cmds.select(inverted, noExpand=True)

Otherwise try this quick implementation - it flattens the list so for high-res meshes this could be relatively slow. But maybe it’s fast enough for you.

2 Likes

Well I meant like get “all” polygons and then “selected” polygons into lists/arrays and then if you have to, iterate it somehow you just check if the number isn’t in the selected and put it in a new list of “invert” (or like the invert one Roy said above, which is much the same idea)

Well I meant like get “all” polygons and then “selected” polygons into lists/arrays and then if you have to, iterate it somehow you just check if the number isn’t in the selected and put it in a new list of “invert” (or like the invert one Roy said above, which is much the same idea)

I think originally @yann.b was saying that he wasn’t sure how to “flatten” those lists and thus was unable to subtract one from another as python sets. The trick is cmds.ls(components, flatten=True) I suppose.


Here’s another way that doesn’t require the flattened results. Haven’t tested whether this is faster or not.

from maya import cmds
import maya.api.OpenMaya as om
import re

def invert_components(components):
    
    components_match_all = set()
    for path in components:
        if "." not in path or not "[" in path:
            # Not a component
            continue
        
        # Replace all component indices with * so we can query them with ls
        path_component_wildcard = re.sub("(\[)[0-9:]+(\])", r"\1*\2", path)
        components_match_all.add(path_component_wildcard)
        
    original = om.MSelectionList()
    for comp in components:
        original.add(comp)
        
    all = om.MSelectionList()
    for comp in components_match_all:
        all.add(comp)
        
    all.merge(original, om.MSelectionList.kRemoveFromList)
    return all.getSelectionStrings()

selection = cmds.ls(sl=1)
inverted = invert_components(selection)
print(inverted)
cmds.select(inverted, noExpand=True)

If only polygons are selected and only on one mesh, and you need to invert this selection, then everything is simple.


import maya.cmds as cmds

#For example, let's create a polygon cube:
mesh_name = (cmds.polyCube(w=1, h=1,d=1,sx=4, sy=4, sz=4, cuv=4, ch=0))[0]

#Let's select several polygons on this cube:
cmds.select(mesh_name+'.f[2]', mesh_name+'.f[12:14]', mesh_name+'.f[16]')

#Let's invert this selection:
poly_invert = cmds.select(mesh_name + '.f[0:]', tgl=1)

# '.f[0:]' - means the range that includes all the faces of the "mesh_name" mesh.
# Flag "toggle(tgl)" - Indicates that those items on the given list which are on the active list should be removed from the # # active list and those items on the given list which are not on the active list should be added to the active list .

If besides faces, other components (edges, vertices) may be selected, and possibly components are selected on different meshes. If for such a selection set we want to invert the selection only for polygonal faces, then using the “polyListComponentConversion” command can help us with this.
https://help.autodesk.com/cloudhelp/2023/ENU/Maya-Tech-Docs/CommandsPython/select.html


import maya.cmds as cmds

# Getting a selection list:
select_list = cmds.ls(sl=1)

# We get from this list a new list containing only the selected faces:
from_face = cmds.polyListComponentConversion(select_list, ff=1, tf=1)

# We get from this list a new list containing only meshes on which faces are selected:
from_face_objs = cmds.polyListComponentConversion(from_face)


# Using a loop, we invert the faces on all meshes with selected faces:
for i in range(len(from_face_objs)):
    invert_select = cmds.select((from_face_objs[i]+'.f[0:]'),tgl=1)

Good luck!

yes, I didn’t know how to flatten the list. It was as simple as adding “flatten = True”. I feel silly not knowing this :sweat_smile:

Thank you very much :grinning_face_with_smiling_eyes:

1 Like

A good investment is a set of functions which extract the index numbers from a Maya-style selection list:

test
import re
brackets = re.compile("(\[)(.*)(\])")

def extract_indices (maya_selection):
    comps = (brackets.search(c) for c in maya_selection)
    non_empty_string_comps  = (cmp[2] for cmp in comps if cmp)
    for eachcomp in non_empty_string_comps:
        s, _, e = eachcomp.partition(":")
        if (s and e):
            for item in range(int(s), int(e) + 1):
                yield item
        else:
            yield(int(s))
            
def generate_components(object, comp, indices):
    for i in indices:
        yield object + "." + comp + "[" + str(i) + "]"

indices = extract_indices(cmds.ls(sl=True))
# [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41]

recap = generate_components("pCylinder", "v", indices)
# ['pCylinder.v[0]', 'pCylinder.v[1]', 'pCylinder.v[2]', 'pCylinder.v[3]', 'pCylinder.v[4]', 'pCylinder.v[5]', 'pCylinder.v[6]', 'pCylinder.v[7]', 'pCylinder.v[8]', 'pCylinder.v[9]', 'pCylinder.v[10]', 'pCylinder.v[11]', 'pCylinder.v[12]', 'pCylinder.v[13]', 'pCylinder.v[14]', 'pCylinder.v[15]', 'pCylinder.v[16]', 'pCylinder.v[17]', 'pCylinder.v[18]', 'pCylinder.v[19]', 'pCylinder.v[40]', 'pCylinder.v[2]', 'pCylinder.v[3]', 'pCylinder.v[4]', 'pCylinder.v[7]', 'pCylinder.v[8]', 'pCylinder.v[10]', 'pCylinder.v[11]']
                

these two offer bidirectional conversion between Maya component strings (as you can see, what kind of component doesn’t matter) and integer indices. You can always use the builtin for inversion selections, but using index lists is good for things like intersecting two different selection, seeing if one component is in an list, or other set-like operations.

Another useful function is one that takes an array of flattened indices, and turns it into range pairs.
So like [1, 2, 3, 4, 17, 18, 19] would get turned into [(1, 4), (17, 19)]
When using this to build a selection list with many components, the actual cmds.select can easily run 10x faster.

from itertools import groupby

def list_to_ranges(iterable):
    iterable = sorted(set(iterable))
    for key, group in groupby(enumerate(iterable), lambda x: x[1] - x[0]):
        group = list(group)
        yield (group[0][1], group[-1][1])

def generate_components(object, comp, indices):
    for pair in list_to_ranges(indices):
        if pair[1] == pair[0]:
            yield '{0}.{1}[{2}]'.format(object, comp, pair[0])
        else:
            yield '{0}.{1}[{2}:{3}]'.format(object, comp, pair[0], pair[1])

indices = [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41]
recap = generate_components("pCylinder", "v", indices)
print(list(recap))
# ['pCylinder.v[20:39]', 'pCylinder.v[41]']

2 Likes

Unfortunately I don’t have maya on my pc to test this and I am doing it from memory/docs, but hopefully it can get you going.

import maya.cmds as cmds

#this returns list of currently selected faces with each individual face listed.
selected_faces = set(cmds.ls(sl=True, fl=True))
all_faces = //So this part depends on your need to support faces selected on multiple meshes and I could not really verify code but it could be something like this: all_faces = set(cmds.ls("mesh_name.f[0:*]",fl=True))
inverted_faces = all_faces - selected_faces //now subtract contents of one set from the other.

Apologies again for not being able to test code, but hopefully it provides some interesting techniques. I hope that Autodesk goes back to providing a free version of Maya for people who just want to learn or mess around.

Another useful function is one that takes an array of flattened indices, and turns it into range pairs.
So like [1, 2, 3, 4, 17, 18, 19] would get turned into [(1, 4), (17, 19)]
When using this to build a selection list with many components, the actual cmds.select can easily run 10x faster.

I didn’t know it could run faster when pairing. It might be useful in the future.
Thanks !