Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EXR Metadata Rotation Bug #722

Closed
adrianmeyerart opened this issue Oct 11, 2019 · 31 comments
Closed

EXR Metadata Rotation Bug #722

adrianmeyerart opened this issue Oct 11, 2019 · 31 comments
Assignees
@buddhi1980
Labels
BUG

Comments

@adrianmeyerart
Copy link
Contributor

Hi,
working on using the new additional passes and the metadata.
Found out that there is a bug in the rotation Metadata written into the EXRs.

To get it right I had to do the following:
rotate_x = 90 - float(metadata("camera_rotation.y"))
rotate_y = float(metadata("camera_rotation.z"))
rotate_z = float(metadata("camera_rotation.x"))

Might be cleaner if you would do this directly and write the correct values into the exr, so not everybody has to figure this out by itself.

Position Metadata is correct!
Let me know when this is done so I will undo the correction in my code.

Thanks
@buddhi1980 buddhi1980 self-assigned this Oct 11, 2019
@buddhi1980 buddhi1980 added the BUG label Oct 11, 2019
buddhi1980 added a commit that referenced this issue Oct 12, 2019
@buddhi1980
Fixed issue #722. Correceted rotation metadata (swapped axes and incr…
3fd4372 
…eased precision of cooirdinates)
@buddhi1980
Copy link
Owner

Corrected. Check if rotation axes are correct now.

@adrianmeyerart
Copy link
Contributor Author

Thanks, I'll wait for the next Travis CI Build then right?

@adrianmeyerart
Copy link
Contributor Author

adrianmeyerart commented Oct 12, 2019 •
edited
Loading

Unfortunately I have discovered another thing...

  1. The "stereo_eye_distance" meta value should be multiplied by 2.
    It seems to be a distance from the stereo center, but not the actual distance between the eyes.
    My recreated stereo looked correct when using value x 2.

  2. Somehow the camera rotation values in the metadata are not exactly correct.
    After the corrections I made (that you have now included) the camera lines up pretty well.
    But not extact, (the position seems to exact though I think).
    There is some kind of rotation offset, its just about 1° of error, but this is a big problem for what I want to achieve.
    Might also be something with Rotation Orders etc.
    I have tried all possible rotation orders, but never got it right.
    (So the best I got was corrections mentioned above and using these values with Rotation Order RX/RY/RZ )
    Could you maybe check on this and see if there might be a little error in how you calculate the rotation values?
    Or the other way around, I could alos try to not use the rotation values but line up the camera using main_camera_target values and main_camera_top_xyz.
    But I think it might be cleaner and also less error-prone if the correct rotation values come from MDB.

Thanks so much!

@adrianmeyerart
Copy link
Contributor Author

PS.
When this works it will be very easy with the tool I build to combine mandelbulber renderings with other normal 3D Objects/Renderings.
Like real Volumetric Clouds etc.

@adrianmeyerart
Copy link
Contributor Author

... Probably it would also be great to export the metadata with a bit more precision.
Example:

EXR meta:
exr/camera.x : 0.544565
Mandelbulber value:
main_camera_x : 0,5445646302125389

@buddhi1980
Copy link
Owner

I have already increased accuracy of coordinates in this commit 3fd4372

@adrianmeyerart
Copy link
Contributor Author

adrianmeyerart commented Oct 14, 2019 •
edited
Loading

I have already increased accuracy of coordinates in this commit 3fd4372

Nice! Unfortunately this won't fix the rotation problem. This is more than precision problem...

PS.
adding main_camera_top_xyz to meta would be nice anyways. even though as I mentioned rotation values should be fixed internally

@buddhi1980
Copy link
Owner

The problem is probably in the sequence of rotations. How they are defined in Houdini?

@buddhi1980
Copy link
Owner

You have to remember that in Mandelbulber axes are swapped. y is depth and z is vertical. The reason of this is to be complain with fractal coordinates (like in M-Set). XY plane is defined like for flat fractal, and z is extension for third dimension. Default view is from side of the 3D fractal.

Rotation matrix is defined in following way:

	// preparing rotation matrix
	mRot.RotateZ(viewAngle.x); // yaw
	mRot.RotateX(viewAngle.y); // pitch
	mRot.RotateY(viewAngle.z); // roll

void CRotationMatrix::RotateX(double angle)
{
	if (angle != 0.0)
	{
		CMatrix33 rot;
		const double s = sin(angle);
		const double c = cos(angle);
		rot.m11 = 1.0;
		rot.m22 = c;
		rot.m33 = c;
		rot.m23 = -s;
		rot.m32 = s;
		matrix = matrix * rot;
		zero = false;
	}
}

void CRotationMatrix::RotateY(double angle)
{
	if (angle != 0.0)
	{
		CMatrix33 rot;
		const double s = sin(angle);
		const double c = cos(angle);
		rot.m22 = 1.0;
		rot.m33 = c;
		rot.m11 = c;
		rot.m31 = -s;
		rot.m13 = s;
		matrix = matrix * rot;
		zero = false;
	}
}

void CRotationMatrix::RotateZ(double angle)
{
	if (angle != 0.0)
	{
		CMatrix33 rot;
		const double s = sin(angle);
		const double c = cos(angle);
		rot.m33 = 1.0;
		rot.m11 = c;
		rot.m22 = c;
		rot.m12 = -s;
		rot.m21 = s;
		matrix = matrix * rot;
		zero = false;
	}
}

@adrianmeyerart
Copy link
Contributor Author

The problem is probably in the sequence of rotations. How they are defined in Houdini?

Might be yes...
In Houdini you can choose the Rotation Order (All possible ones).
But the default is: RX/RY/RZ
Transform Order: Scale/Rotation/Translate

Coordinate System:
hou_coord
Y-Up
Z-Forward

@adrianmeyerart
Copy link
Contributor Author

adrianmeyerart commented Oct 15, 2019 •
edited
Loading

But I am not converting the coordinate system.
I just take the values raw as they are (despite the rotation corrections I mentioned).
So more precisely I use the raw position values from the EXR word position image layer and directly convert each pixel into s 3D Point.
Also I directly import the raw Camera position values from Metadata.
Now the fractal Pointcloud scene has an unchanged orientation as in Mandelbulber.
So Y stays Y, Z stays Z etc.
Just the scene does not line up with Houdini’s default orientation, but that does not matter.
But to achieve that also for the camera orientation Y stays Y, Z stays Z and X stays X I had to play around and found that I had to do the rotation corrections mentioned above.
Now the scenes in Houdini and Mandelbulber line up identical.
Despite that little rotation mismatch that remains somehow and causes problems.

Sorry, that was complicated description :-P
Hope you know what I mean.

@buddhi1980
Copy link
Owner

buddhi1980 commented Oct 15, 2019 via email

@adrianmeyerart
Copy link
Contributor Author

I don’t convert degrees to radians at all.
The rotation values in the metadata are degrees, exactly what I need in Houdini.
So no conversion. Just raw values.

@adrianmeyerart
Copy link
Contributor Author

adrianmeyerart commented Oct 24, 2019 •
edited
Loading

Rotation matrix is defined in following way:

	// preparing rotation matrix
	mRot.RotateZ(viewAngle.x); // yaw
	mRot.RotateX(viewAngle.y); // pitch
	mRot.RotateY(viewAngle.z); // roll

void CRotationMatrix::RotateX(double angle)
{
	if (angle != 0.0)
	{
		CMatrix33 rot;
		const double s = sin(angle);
		const double c = cos(angle);
		rot.m11 = 1.0;
		rot.m22 = c;
		rot.m33 = c;
		rot.m23 = -s;
		rot.m32 = s;
		matrix = matrix * rot;
		zero = false;
	}
}

void CRotationMatrix::RotateY(double angle)
{
	if (angle != 0.0)
	{
		CMatrix33 rot;
		const double s = sin(angle);
		const double c = cos(angle);
		rot.m22 = 1.0;
		rot.m33 = c;
		rot.m11 = c;
		rot.m31 = -s;
		rot.m13 = s;
		matrix = matrix * rot;
		zero = false;
	}
}

void CRotationMatrix::RotateZ(double angle)
{
	if (angle != 0.0)
	{
		CMatrix33 rot;
		const double s = sin(angle);
		const double c = cos(angle);
		rot.m33 = 1.0;
		rot.m11 = c;
		rot.m22 = c;
		rot.m12 = -s;
		rot.m21 = s;
		matrix = matrix * rot;
		zero = false;
	}
}

Hey,
I don't quite get this... Could you send me the link git link so I can see it in context?
What is "viewAngle.x" ? Is it "main_camera_top_x" ?

Also I noticed that I can add Yaw/Pitch/Rotation to Keyframes.
If I convert this to Flight I have Rotaion Keyframes for every frame.
Would this be exactly the same as the Rotation values that you put in the EXR metadata?

@buddhi1980
Copy link
Owner

In this file there is a code for rotation matrices:
https://github.com/buddhi1980/mandelbulber2/blob/master/mandelbulber2/src/algebra.cpp

Here is some theory what is rotation matrix: https://en.wikipedia.org/wiki/Rotation_matrix

viewAngle.x, viewAngle.y, viewAngle.z are components of rotation stored in vector structure.
viewAngle.x is yaw
viewAngle.y is pitch
viewAngle.z is roll

main_camera_top is a vector which is perpendicular to the front of the camera (to the line between camera and target) and directed towards top of the camera. E.g. if you change roll angle of the camera, this vector rotates.

Here is preparation of rotation matrix:
https://github.com/buddhi1980/mandelbulber2/blob/master/mandelbulber2/opencl/engines/fast_engine.cl#L109

		// main rotation matrix
		matrix33 rot;
		rot.m1 = (float3){1.0f, 0.0f, 0.0f};
		rot.m2 = (float3){0.0f, 1.0f, 0.0f};
		rot.m3 = (float3){0.0f, 0.0f, 1.0f};

		rot = RotateZ(rot, consts->params.viewAngle.x);
		rot = RotateX(rot, consts->params.viewAngle.y);
		rot = RotateY(rot, consts->params.viewAngle.z);

Here us usage of rotation matrix
https://github.com/buddhi1980/mandelbulber2/blob/master/mandelbulber2/opencl/engines/fast_engine.cl#L158

		float3 viewVectorNotRotated = CalculateViewVector(normalizedScreenPoint, consts->params.fov);
		float3 viewVector = Matrix33MulFloat3(rot, viewVectorNotRotated);

viewVector is a direction of ray starting from the camera. In the center of the screen is {0, 1, 0} (towards y direction which is depth - default view is from side of 3D fractal)

	viewVector.x = normalizedPoint.x * fov;
	viewVector.y = 1.0f;
	viewVector.z = normalizedPoint.y * fov;
	viewVector = normalize(viewVector);

@zebastian
Copy link
Collaborator

@buddhi1980 can we close this?

@buddhi1980
Copy link
Owner

It is still open topic

@adrianmeyerart
Copy link
Contributor Author

Yes still open...

@adrianmeyerart
Copy link
Contributor Author

Hey guys,
back working on the mandelbulber / houdini / nuke project.

I spend some time to try and resolve this issue.
I got closer, but not fully there. I hope we can fix it together.
I made a little video where I explain what I did, guess thats much easier to understand.
https://vimeo.com/397787998/299692b8c3
(sry something went a bit wrong with recording, some wird artefacts)
Hope you find the time to check it out.

Thanks!!

Here the VEX code used:

Values:

from = {0.536337, -0.678756, 6.65716}
to = {-0.0423664, -1.13841,6.52942}
top = {0.0755473, 0.177564, -0.981205}
trans_order = 0 // XFORM_SRT
rot_order = 0   // XFORM_XYZ

// mandelbulber cam data
vector from = chv("from"); //exr meta camera (pos)
vector to = chv("to"); //exr meta target
vector top = chv("top"); //camera_top (from .fract file flightanimation)

// calculate vectors
vector forward = normalize(from - to);
vector right = cross(normalize(top), forward);
vector up = cross(forward, right); 

// prepare xform matrix
vector4 right_c = set(right.x, right.y, right.z, 0);
vector4 up_c = set(up.x, up.y, up.z, 0);
vector4 forward_c = set(forward.x, forward.y, forward.z, 0);
vector4 from_c = set(from.x, from.y, from.z, 1);

// set xform matrix
matrix m = set(right_c, up_c, forward_c, from_c);
printf("\n%s", m);

// extract rotation angles from matrix
int trans_order = chi("trans_order"); // XFORM_SRT
int rot_order = chi("rot_order");     // XFORM_XYZ
vector pivot = {0,0,0};
vector pivot_rotate = pivot;
vector t,r,s,shears;
// cracktransform returns rotation angles in degrees
cracktransform(trans_order, rot_order, pivot, pivot_rotate, m, t, r, s, shears);

// export rotation angles
v@rotation_angles = r;
printf("\n%s", v@rotation_angles);

@mclarekin
Copy link
Collaborator

mclarekin commented Mar 17, 2020 via email

@mclarekin
Copy link
Collaborator

mclarekin commented Mar 17, 2020 via email

@adrianmeyerart
Copy link
Contributor Author

hey guys,
so I think we're finally kind of there.
first off, the very strange fudge factor of 2.25° Y was something I oversaw.
in my test scene, there was somehow a value of 2.25 in the "Sweet Spot Angle Horizontal".
I never noticed that parameter before and still don't exactly know what it does, seems like kind of a post rotation that does not get included in the actual camera transform data?
Sorry that I oversaw this.

anyways, just got rid of that now and also tested with simpler scenes such as Menger Sponge.
with my self caluclated xform matrix based on camera, camera_target and camera_top now everything works perfectly as expected.

now I again tested all of this manually by grabbing the values from Mandelbulber Flight Anim Keyframe Spreadsheet. So that I can automate that, would be awesome again if you could include "camera_top" in EXR Metadata!

for the actual rotation values you calculate and export to EXR Metadata, somehow this does not work properly. Also tried to directly grab the values from Mandelbulbers Navigation Panel "Yaw, Pitch, Roll", they seem to be the same values initially with the exr Metadata (makes sense).
But also when I manually apply these rotation value sin Houdini it does not give correct results.

so I actually have the feeling that your Mandelbulber Rotation values are incorrect.
I guess you dont really use them internally as you also use a cam matrix from camera, camera_target and camera_top.
so probably the mistake did never appear.

Though, as I mentioned you recently did some modification on the Rotation Metadata based on my suggestion:

rotate_x = 90 - float(metadata("camera_rotation.y"))
rotate_y = float(metadata("camera_rotation.z"))
rotate_z = float(metadata("camera_rotation.x"))

but somehow something weird happened there and trhe results are very wrong.
(with the intial values without your correction and mine applied it was close to the correct result)

So I would suggest, (unless you find that your rotation values are actually completely incorrect and do a complete fix for this) that you revert the rotation metadata to the point before the corrections (i.e. values like they appear in Mandelbulber Navigation Tab).

but that just for cleanliness, as I mentioned, as soon as I have camera_top in EXR metadata I can get it properly working now.

Thanks a lot!
Adrian

@adrianmeyerart
Copy link
Contributor Author

BTW.
it came to my mind that it would also be an option that you directly write the full cam transform matrix to the metadata.
16 float values opposed to 9 (camera, camera_top, camera_target).
Guess thats not really a big overhead and then other people wouldn't have to calculate it on their own.

Might be even nicer. But to test it out, it would be great if you could first export all of that, and than I can try what works best. Afterwards you could strip it down again.

thx

@buddhi1980
Copy link
Owner

In next days I will add top vector to the metadata. From Camera, Target and Top vector you can calculate rotation matrix:

    vec3 ww = normalize(target - camera);
    vec3 uu = normalize(cross(ww, top);
    vec3 vv = normalize(cross(uu, ww));

where uu, vv, ww are rows or rotation matrix

@buddhi1980
Copy link
Owner

Example of this calculation of rotation matrix is used in my shader in ShaderToy: https://www.shadertoy.com/view/WlSSWW

@adrianmeyerart
Copy link
Contributor Author

alright thanks. let me know

buddhi1980 added a commit that referenced this issue Mar 26, 2020
@buddhi1980
Added camera top vector to image metadata (issue #722)
566ae83
@buddhi1980
Copy link
Owner

buddhi1980 commented Mar 26, 2020 •
edited
Loading

In last commit I have added camera top vector to the image metadata

@adrianmeyerart
Copy link
Contributor Author

thx, will try it out once the CI build is ready

@adrianmeyerart
Copy link
Contributor Author

works!
new tool completed and pull request send.

PS.
exr/rotation metadata is incorrect and I don't use it anymore.

@buddhi1980
Copy link
Owner

Can I close this issue?

@adrianmeyerart
Copy link
Contributor Author

yep!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@buddhi1980 buddhi1980

Labels
BUG
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@zebastian @buddhi1980 @mclarekin @adrianmeyerart