Maya API Vertex Colors

[bob.w]

View in #maya on Slack

@jeroen_heemskerk: Anyone knows what I’m doing wrong here?
meshFn = self.meshFn

vertexCount = om.MIntArray()
vertexList = om.MIntArray()
sourceColors = om.MColorArray()

meshFn.getVertices(vertexCount, vertexList)
meshFn.getColors(sourceColors, self.colorSet)
meshFn.clearColors(self.colorSet)

i = 0
for face in range(vertexCount.length()):
for vert in range(vertexCount[face]):
print face, vertexList[i], sourceColors[i][0]
i += 1

meshFn.setColors(sourceColors, self.colorSet)
meshFn.assignColors(vertexList, self.colorSet)
meshFn.setCurrentColorSetName(self.colorSet)
As a test I have a cube, and 2 faces have a white vertex color, while the rest is black.
If I read the color data it looks all fine. I can see exactly which vertexFace is white and which is black
but when I reapply them, even without editing the colors, they aren’t applied correctly?

@bob.w: @jeroen_heemskerk By applied incorrectly, what exactly do you mean? Is the color bleeding into other faces?
If so, you probably want to be applying the colors based on the vertexFace, not the vertex

@jeroen_heemskerk: hmm, yes I guess you could say its bleeding to other faces. But isnt om.MFnMesh.getColors() and setColors() by default already using vertexFaces?
when I getColors() on a cube, it gives me 24 colors… so Im assuming those are vertexFaces

@bob.w: Sounds right. Its been awhile since I’ve used get/set colors through the API.

@jeroen_heemskerk: its just really weird that when I getColors() and apply the same mColorArray that I got, I end up with a different result…

@bob.w: So the docs for assignColors states that it needs to be a set of ids for each vertex-face. That might be where things are going haywire, as you’re just giving it the vertex list

Assigns colors to all of the mesh's face-vertices. The colorIds
sequence must contain an entry for every vertex of every face, in
face order, meaning that the entries for all the vertices of face 0
come first, followed by the entries for the vertices of face 1, etc.

Though I could easily be mis-remembering how getVertices actually works.

@jeroen_heemskerk: getVertices gives you 2 lists, the first is the count per polygon [4,4,4,4,4,4] for a cube. The second one is a list of vertex ID
So yeah, it doesnt give a list of the vertexFace order, but aint that just [0,1,2,3] per polygon?

@bob.w: For a cube, should be. Things with tri’s or n-gons would be different. Also if the normal is flipped the order would be flipped as well

@jeroen_heemskerk: hmm, yeah that makes sense from a render pov, but I wonder if that applies as well for when you just query some attributes. You would expect it to go by nr instead?
But you are right tho, getVertices gives you object-relative vertex indices, instead of face-relative indices
im going to try the other one
Nope, that didnt do it

@bob.w: Well that is annoying

@jeroen_heemskerk: trololol, found it…

@bob.w: oh? do tell.

@jeroen_heemskerk: assignColors() need an array of Ids for each vertexFace
so basically, [0,1,2,3,4,5,6,7,8,9…]
up to the total amount of faceVerts

@bob.w: Ah, so one single array

[Faranell]

For those curious. This was the code in questioning.
I wrote my first custom maya api command, which will transfer vertexColors from one channel to the other, or from one colorSet to the other. Multiple channels/sets can be set.

This isn’t heavily tested yet, so if you want to use it use with caution

If anyone sees something that I could improve let me know! I’m always happy for some feedback

import sys
import maya.OpenMaya as om
import maya.OpenMayaMPx as omMPx

commandName = "transferColors"

kObjectNameFlag = "-n"
kObjectNameLongFlag = "-name"

kObjectRedFlag = "-r"
kObjectRedLongFlag = "-colorR"
kObjectGreenFlag = "-g"
kObjectGreenLongFlag = "-colorG"
kObjectBlueFlag = "-b"
kObjectBlueLongFlag = "-colorB"
kObjectAlphaFlag = "-a"
kObjectAlphaLongFlag = "-alpha"
kObjectColorSet1Flag = "-s1"
kObjectColorSet1LongFlag = "-set1"
kObjectColorSet2Flag = "-s2"
kObjectColorSet2LongFlag = "-set2"

kHelpFlag = "-h"
kHelpLongFlag = "-help"
helpMessage = """This command is used to transfer vertex colors per channel and from one colorSet to the other.\n"""


class pluginCommand(omMPx.MPxCommand):
	"""This command is used to transfer vertex colors per channel and from one colorSet to the other.
	If no colorSet is specified the current one will be used.
	-r, -g, -b, -a = [index colorSet (0/1), index vertexColor (0/1/2/3)]
	set1 = colorSet
	set2 = colorSet"""
	def __init__(self):
		self.meshFn = None
		self.sourceColors = None
		self.polygons = []
		self.vertices = []

		self.r = None
		self.g = None
		self.b = None
		self.a = None
		self.set1 = None
		self.set2 = None

		omMPx.MPxCommand.__init__(self)

	def argumentParser(self, argList):
		syntax = self.syntax()
		parsedArguments = om.MArgDatabase(syntax, argList)

		# Get selection
		selection = om.MSelectionList()
		parsedArguments.getObjects(selection)
		component = om.MObject()
		mDagPath = om.MDagPath()
		selection.getDagPath(0, mDagPath, component)
		self.meshFn = om.MFnMesh(mDagPath)

		# Parse all flags
		if parsedArguments.isFlagSet(kHelpFlag):
			self.setResult(helpMessage)

		if parsedArguments.isFlagSet(kObjectRedFlag):
			idx1 = parsedArguments.flagArgumentInt(kObjectRedFlag, 0)
			idx2 = parsedArguments.flagArgumentInt(kObjectRedFlag, 1)
			if idx1 > 1 or idx2 > 3:
				return False
			self.r = [idx1, idx2]

		if parsedArguments.isFlagSet(kObjectGreenFlag):
			idx1 = parsedArguments.flagArgumentInt(kObjectGreenFlag, 0)
			idx2 = parsedArguments.flagArgumentInt(kObjectGreenFlag, 1)
			if idx1 > 1 or idx2 > 3:
				return False
			self.g = [idx1, idx2]

		if parsedArguments.isFlagSet(kObjectBlueFlag):
			idx1 = parsedArguments.flagArgumentInt(kObjectBlueFlag, 0)
			idx2 = parsedArguments.flagArgumentInt(kObjectBlueFlag, 1)
			if idx1 > 1 or idx2 > 3:
				return False
			self.b = [idx1, idx2]

		if parsedArguments.isFlagSet(kObjectAlphaFlag):
			idx1 = parsedArguments.flagArgumentInt(kObjectAlphaFlag, 0)
			idx2 = parsedArguments.flagArgumentInt(kObjectAlphaFlag, 1)
			if idx1 > 1 or idx2 > 3:
				return False
			self.a = [idx1, idx2]

		if parsedArguments.isFlagSet(kObjectColorSet1Flag):
			self.set1 = parsedArguments.flagArgumentString(kObjectColorSet1Flag, 0)

		if parsedArguments.isFlagSet(kObjectColorSet2Flag):
			self.set2 = parsedArguments.flagArgumentString(kObjectColorSet2Flag, 0)

		# If no colorSet is defined use current
		if not self.set1:
			self.set1 = self.meshFn.currentColorSetName()
		if not self.set2:
			self.set2 = self.meshFn.currentColorSetName()

		if not self.set1 and not self.set2:
			sys.stderr.write("Object has no colorSets.\n")
			return False

		# Get components
		if not component.isNull():
			if component.hasFn(om.MFn.kMeshPolygonComponent):
				self.polygons = om.MIntArray()
				om.MItMeshPolygon(mDagPath, component)
				face_itr = om.MItMeshPolygon(mDagPath, component)
				while not face_itr.isDone():
					self.polygons.append(face_itr.index())
					face_itr.next()

			elif component.hasFn(om.MFn.kMeshEdgeComponent):
				om.MItMeshEdge(mDagPath, component)
				edge_itr = om.MItMeshEdge(mDagPath, component)
				while not edge_itr.isDone():
					self.vertices.append(edge_itr.index(0))
					self.vertices.append(edge_itr.index(1))
					edge_itr.next()

			elif component.hasFn(om.MFn.kMeshVertComponent):
				self.vertices = om.MIntArray()
				om.MItMeshVertex(mDagPath, component)
				vert_itr = om.MItMeshVertex(mDagPath, component)
				while not vert_itr.isDone():
					self.vertices.append(vert_itr.index())
					vert_itr.next()
		return True

	def isUndoable(self):
		return True

	def doIt(self, argList):
		if self.argumentParser(argList) and (self.r or self.g or self.b or self.a):
			# get current colors and store it for undo
			self.sourceColors = om.MColorArray()
			self.meshFn.getFaceVertexColors(self.sourceColors, self.set1)

			self.redoIt()
		else:
			sys.stderr.write("No flag specified or setup incorrectly for '%s' command.\n" % commandName)

	def redoIt(self):
		meshFn = self.meshFn

		vertexCount = om.MIntArray()
		vertexList = om.MIntArray()
		meshFn.getVertices(vertexCount, vertexList)

		# Set2 colors
		sourceColors2 = om.MColorArray()
		meshFn.getFaceVertexColors(sourceColors2, self.set2, om.MColor(0, 0, 0, 1))

		# Set1 colors
		targetColors = om.MColorArray()
		meshFn.getFaceVertexColors(targetColors, self.set1, om.MColor(0, 0, 0, 1))

		# inner function to transfer colors from set2 to set1
		def transferColors(i):
			if self.r:
				if self.r[0] == 0:
					targetColors[i].r = self.sourceColors[i][self.r[1]]
				elif self.r[0] == 1:
					targetColors[i].r = sourceColors2[i][self.r[1]]
			if self.g:
				if self.g[0] == 0:
					targetColors[i].g = self.sourceColors[i][self.g[1]]
				elif self.g[0] == 1:
					targetColors[i].g = sourceColors2[i][self.g[1]]
			if self.b:
				if self.b[0] == 0:
					targetColors[i].b = self.sourceColors[i][self.b[1]]
				elif self.b[0] == 1:
					targetColors[i].b = sourceColors2[i][self.b[1]]
			if self.a:
				if self.a[0] == 0:
					targetColors[i].a = self.sourceColors[i][self.a[1]]
				elif self.a[0] == 1:
					targetColors[i].a = sourceColors2[i][self.a[1]]

		i = 0  # faceVertex index
		colorIds = om.MIntArray()
		for polygon in range(vertexCount.length()):
			for faceVert in range(vertexCount[polygon]):
				# Check if any components are selected
				if self.polygons or self.vertices:
					# Only transfer selected components
					if polygon in self.polygons or vertexList[i] in self.vertices:
						transferColors(i)
				else:
					transferColors(i)
				# Add faceVertex to colorIds
				colorIds.append(i)
				i += 1

		# Apply colors to all faceVertices
		meshFn.setColors(targetColors, self.set1)
		meshFn.assignColors(colorIds, self.set1)
		meshFn.setCurrentColorSetName(self.set1)

	def undoIt(self):
		meshFn = self.meshFn
		vertexCount = om.MIntArray()
		vertexList = om.MIntArray()

		meshFn.getVertices(vertexCount, vertexList)

		colorIds = om.MIntArray()
		for i in range(vertexList.length()):
			colorIds.append(i)

		# Apply old colors (pre doIt)
		meshFn.setColors(self.sourceColors, self.set1)
		meshFn.assignColors(colorIds, self.set1)


def syntaxCreator():
	syntax = om.MSyntax()

	# Add objects/selection to the syntax
	syntax.useSelectionAsDefault(True)
	syntax.setObjectType(om.MSyntax.kSelectionList)

	# Add all needed flags
	syntax.addFlag(kHelpFlag, kHelpLongFlag)
	syntax.addFlag(kObjectRedFlag, kObjectRedLongFlag, om.MSyntax.kUnsigned, om.MSyntax.kUnsigned)  # 2x om.MSynstax.kUnsigned to create a float2 parameter
	syntax.addFlag(kObjectGreenFlag, kObjectGreenLongFlag, om.MSyntax.kUnsigned, om.MSyntax.kUnsigned)
	syntax.addFlag(kObjectBlueFlag, kObjectBlueLongFlag, om.MSyntax.kUnsigned, om.MSyntax.kUnsigned)
	syntax.addFlag(kObjectAlphaFlag, kObjectAlphaLongFlag, om.MSyntax.kUnsigned, om.MSyntax.kUnsigned)
	syntax.addFlag(kObjectColorSet1Flag, kObjectColorSet1LongFlag, om.MSyntax.kString)
	syntax.addFlag(kObjectColorSet2Flag, kObjectColorSet2LongFlag, om.MSyntax.kString)
	return syntax


def cmdCreator():
	return omMPx.asMPxPtr(pluginCommand())


def initializePlugin(mobject):
	mplugin = omMPx.MFnPlugin(mobject, "Jeroen Heemskerk", "1.0")
	try:
		mplugin.registerCommand(commandName, cmdCreator, syntaxCreator)
	except:
		sys.stderr.write("Failed to register command: %s.\n" % commandName)


def uninitializePlugin(mobject):
	mplugin = omMPx.MFnPlugin(mobject)
	try:
		mplugin.deregisterCommand(commandName)
	except:
		sys.stderr.write("Failed to de-register command: %s.\n" % commandName)