glm::quat_cast behaviour with rotation-scale matrix

[MikeGameDev]

If the input to glm::quat_cast is a rotation-scale matrix, the returned quaternion seems to be incorrect. glm::quat_cast only seems to work with pure rotation matrices (matrices that are orthonormalized). Am I misunderstanding the expected behaviour of this function or is this a bug? See the sample below for a full demonstration of what I mean:

	// Create sample translation, rotation and scale matrices
	glm::mat4 translationMatrix = glm::translate(glm::vec3(1, 2, 3)); 
	glm::quat qX = glm::angleAxis(glm::radians(45.0f), glm::vec3(1.0f, 0.0f, 0.0f));
	glm::quat qY = glm::angleAxis(glm::radians(45.0f), glm::vec3(0.0f, 1.0f, 0.0f));
	glm::quat qZ = glm::angleAxis(glm::radians(45.0f), glm::vec3(0.0f, 0.0f, 1.0f));
	glm::quat qXYZ = qZ * qY * qX;
	glm::mat4 rotationMatrix = glm::mat4_cast(qXYZ);
	glm::mat4 scaleMatrix = glm::scale(glm::vec3(0.01f));

	// Compound transforms into one homogeneous matrix
	glm::mat4 m0 = translationMatrix * rotationMatrix * scaleMatrix;

	// Attempt to re-obtain rotation quaternion from homogeneous 4x4 transform
	glm::quat q_from_4x4_with_scale = glm::quat_cast(m0);

	// Attempt to re-create m0 using newly obtained quaternion
	glm::mat4 m1 = translationMatrix * glm::mat4_cast(q_from_4x4_with_scale) * scaleMatrix;

	// m0 and m1 are not the same matrix
	std::cout << glm::to_string(m0) << "\n\n\n";
	std::cout << glm::to_string(m1) << "\n\n\n";

	// If the input to glm::quat_cast is orthonormalized, it works as expected:
	glm::mat3 r;
	r[0] = glm::normalize(m0[0]);
	r[1] = glm::normalize(m0[1]);
	r[2] = glm::normalize(m0[2]);

	glm::quat q_from_4x4_without_scale = glm::quat_cast(r);
	glm::mat4 m2 = translationMatrix * glm::mat4_cast(q_from_4x4_without_scale) * scaleMatrix;
	
	// m0 and m2 are the same matrix
	std::cout << glm::to_string(m2) << "\n\n\n";

[Groovounet]

Hi,
My expectation is that quat_cast should work only on pure rotation matrices. I clarified the API documentation in master branch for GLM 0.9.9 release.

Thanks for contributing,
Christophe

[Groovounet]

This issue was actually resolved in GLM 0.9.9.0.

Thanks,
Christophe