Thanks for the interesting and useful article.
I noticed small mistake in the beginning of simplification steps from (4) to (5) in Appendix - middle summand of y component Ry=b(-x^2 - y^2 + z^2) - 2x(bx - ay) + 2cyz should have inverted sign: Ry=b(-x^2 - y^2 + z^2) + 2x(bx - ay) + 2cyz (or Ry=b(-x^2 - y^2 + z^2) - 2x(ay - bx) + 2cyz). Considering this fix, the rest steps are correct.
Thanks for reporting this! I’ve updated the post with the correction.
Sorry for not replying until now!
I don’t think this is the correct fix, but it’s really hard to give advice without a complete working example. I’m also not familiar with how Unity encodes normal maps except for the code posted in an earlier comment by @invadererik in 2013, which may well have changed since then, so you’re probably better off asking for help in the Unity forums. Still if you have a repro, I could take a quick look.
I’ve found a curious artifact of reoriented normal mapping when used on extreme angles. My test case is the normal of a hemisphere with a detail normal of a lot of smaller hemispheres, and where the blended normals are getting pushed to z<0 the cross vector to the normal and z is getting flipped.
Though, I should probably mention my use case was for triplanar mapping, so that first hemisphere normal is actually the world normals of a sphere mesh.
The z clamping proposed back in Sep '13 doesn’t really solve the issue since the normals are already inverted.
Hey Ben,
The method isn’t really designed for object/world normals. In your case, since you’re (triplanar) blending between normal maps, could you not do this in tangent space with PD blending and then transform by the mesh tangent frame after? Or is the goal to blend both base and detail normal maps together?
where the blended normals are getting pushed to z<0 the cross vector to the normal and z is getting flipped.
I’m not sure I 100% follow; there aren’t any cross products in the method, so presumably this is something else that you’re doing afterwards. I’d be happy to take a look at the full thing if you have a minimal Unity demo.
It was my bug. I was blending the normals in the wrong order. As for triplanar mapping, RNM is a perfect use case for it. Similar to how the GPU Gems 3 triplanar mapping is the “UDN” normal map blending with some smart swizzling, you can do the same thing with RNM. I swizzle the vertex normals to match the normal map, apply the blend, then swizzle back.
Ahh, that makes sense. I’m glad to hear that it’s all working now.
Thanks also for posting your results – looking good!