[Maxscript] of UVs and math

Coding
, ,
#1

Hey all, I’ve got a mesh here that’s been UV unwrapped for an engine where its normals are unaffected by any UV flipping or odd orientations. Like most renderers the target renderer will create odd shading if the UVs even look at it funny.

I’m pretty confident that there has to be some way to compare a mesh triangle to its corresponding UV triangle to identify any oddities, but beyond identifying flipped UVs by comparing the signs on the area of the two triangles, my math skills are lacking

However I have determined that the problems aren’t confined to just flipped UVs - well, what 3ds Max considers an inverted UV anyway. It turns out that faces that are flipped both horizontally and vertically are effectively rotated 180 degrees and thus aren’t considered to be inverted, despite still adversely affecting the shading of the model

I’ve already come up with a solution on how to maintain propper texture mapping regardless of which direction(s) the UVs need to be unflipped, but I simplty have no way of determining what that specidic need is. Hell, it doesn’t even need to be completely automated, just being able to deterimine when the UV face is flipped in the correct direction would be enough - even if I have to manually test all 16k triangles lol

I’ve even tried creating a test texture made up of tiled 8px "F"s that would make the direction of UVs apparrent. Unfortunately that requires a lot of trial and error to determine which direction those Fs should face - even if they’re not flipped.

I appreciate any help you can give, and will be willing to provide any additional assets/information that you may need. Thank you.

#2

I don’t clearly understand the problem but I guess you want to check “matching” geo-face and corresponding map-face normals…
there is a specified normal and direction made by face edges. My idea is to check and find all faces where geo-specified and edge-made normals have different signs than map-specified (z-dir) and map-edge-made normal.
Here is the code:

fn findSuspiciousFaces mesh channel:1 = if meshop.getMapSupport mesh channel do
(
	fn _sign x = if x < 0.0 then -1 else 1  

	local m0 = z_axis -- default map face dir
	
	flipped = #{}
	flipped.count = mesh.numfaces
	
	for f=1 to mesh.numfaces do
	(
		vv = getface mesh f
		v1 = getvert mesh vv[1]
		v2 = getvert mesh vv[2]
		v3 = getvert mesh vv[3]
		
		n0 = getfacenormal mesh f
		n1 = cross (v2 - v1) (v3 - v1)
		
		tt = meshop.getmapface mesh channel f
		t1 = meshop.getmapvert mesh channel tt[1]
		t2 = meshop.getmapvert mesh channel tt[2]
		t3 = meshop.getmapvert mesh channel tt[3]
		
		m1 = cross (t2 - t1) (t3 - t1)
		
		if _sign (dot n0 n1) != _sign (dot m0 m1) do append flipped f
	)
	flipped
)
/*
findSuspiciousFaces $.mesh channel:1
*/

give it a try. maybe it will help or give you an idea.

#3

First, thanks a lot for the edge-based detection, I’ve had some issues with simply comparing the signs of the area of the two triangles. As far as the problem - I’m not certain of it either. On the surface it’s easy to simplify it down to flipped UVs with a normal map, but the issue isn’t confined to just what 3ds Max considers an inverted UV face. For example, for one UV island, simply “unflipping” the flipped UV faces resolves the abrupt dark shading of those flipped UVs. But when I repeat the process for a different island, the result is a smooth gradient of shading, which tells me that there may be something wrong with the UVs that 3ds Max doesn’t identify as inverted.

Another potential method I thought of assisting me would to use a neutral normal map, and figure out how to calculate the final rendered face normal, influenced by the UV face and the normal map, and compare that to the basic face normal. Theoretically, on faces with no UV issues, the two normals should be equal, and any that have said issues, the face normal can be used to determine how it must be altered.

At any rate, thanks again for the code, hopefully it will give me the info I need to get to the bottom of this!