It looks like what you’re doing is a 120deg rotation around the
(1,1,1) vector, which I don’t think I’ve ever seen used as a translation from Z-up to Y-up. I don’t see anything wrong with it, it’s just not what I expect. I deal with 3dsMax to Maya and back, where everything is just a 90 degree rotation around the X axis.
So only because I haven’t seen this before, I have to ask if this is how you are supposed to be doing it? The way to know for sure is to export a mesh from Maya to your engine and see if the X axis is pointing in the same direction relative to the mesh in both cases.
Or if you’re going to be the one creating that exporter, take care and make sure that if I imported something from the engine into Maya, exported that to another software, and then exported from that software to the engine, then I would get what I expect in the engine.
Yes, the translation between Z-up and Y-up can be handled by a matrix multiplication.
That said, down in your matrix mult section you’re handling vectors and quaternions separately, and I wouldn’t do that here. There’s definitely a time and place for keeping them separate, but it’s just so much easier for this lower level stuff to combine everything into one matrix.
Or if you’re feeling really fancy, you could create a new class that keeps tran/rot/scale separate, but allows you to multiply with it like a matrix. I wrote something like that at work for python scripting in our rig builder.
The rotation of 120deg around
(1,1,1) is the same as the quaternion
(0.5, 0.5, 0.5, 0.5). I took a random unit quaternion and multiplied it by that
(0.5, 0.5, 0.5, 0.5), and the output wasn’t the same as your brute force manipulation. It wasn’t even a rearrangement of the same numbers, all the numbers in my output were completely different from the input.
I don’t have a real intuition with quaternion components, so I could easily be wrong here, but I would definitely double-check that your brute force method is doing what you expect.
Another thing that may be biting you is you may be applying the ZtoY matrix in the wrong places.
One very simple way to handle this is to just put everything under a parent object, and rotate that.
However, when I had to deal with this, our modelers didn’t want rotation values on their meshes, but I still had to handle skeleton hierarchies, and I couldn’t add extra objects. So here are some rules I found so that hierarchies work as I expect, but meshes come in oriented to the current world up.
These rules are not the end-all, they’re just what’s worked best for me in my studio.
- If the current object is a mesh, apply the ZtoY matrix to the vertices, and multiply the inverse of the ZtoY matrix with the object transform.
- If the current object is at the root of the scene, multiply the ZtoY matrix to its transform. This happens whether or not step 1 happened.
- If the current object is a child of a non-root mesh, compensate for the mesh’s ZtoY rotation in its parent space.
So if there’s a mesh at the root of the scene, you move all the verts, and counter-rotate (step 1). Then you apply the ZtoY matrix because it’s at the root (step 2). The
counter_rotation * rotation cancels out and means that meshes at the root of the scene come in standing up without any rotation values, but any other meshes stay oriented to however the artist set up the scene.
That also means that you don’t mess with skeleton hierarchies. Riggers don’t care what the up vector is for the scene (or, at least mine don’t ), they care more for what the first, second, and third Euler axes are, and will set up their joints with that in mind to avoid gimbal lock.
In this section, I talk about a ZtoY matrix, but you can use whatever rotation matrix you need (be it 120 around (1,1,1), or 90 around (1, 0, 0), or whatever else) because it’s just multiplied into the hierarchy.
Another thing is that step 3 above can be generalized to work with any ruleset. It’s kind of complicated to explain. If you want to know, just ask