Skip to content

Unexpected Y-axis translation when using ortho() and resetMatrix() #891

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SableRaf opened this issue Dec 17, 2024 · 7 comments · Fixed by #954
Open

Unexpected Y-axis translation when using ortho() and resetMatrix() #891

SableRaf opened this issue Dec 17, 2024 · 7 comments · Fixed by #954
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@SableRaf
Copy link
Collaborator

Note

This issue is a repost of processing/processing#6175 by @usuallyannoyed, updated and edited for tone and clarity.

Description

The ortho() function does not behave as expected.

Expected Behavior

ortho(0, width, 0, height) with a cleared modelview matrix (resetMatrix) should allow users to draw within the specified bounds (e.g., 0 to width on the x-axis and 0 to height on the y-axis).

Current Behavior

Instead, the drawing area behaves incorrectly, requiring adjustments (e.g., translating -height on the y-axis) to achieve the expected results. This behavior makes it unintuitive for users who expect the specified bounds to align with the visible viewport.

Steps to Reproduce

void setup() {
  size(500, 500, P3D);
}

void draw() {
  ortho(0, width, 0, height);
  resetMatrix(); // clear modelview matrix
  fill(255);
  
  // Uncomment the line below to 'fix' the behavior
  // translate(0, -height); 
  
  beginShape(TRIANGLES);
  vertex(640, 640);
  vertex(0, 640);
  vertex(0, 0);
  endShape();
}

Observed Result

The triangle only becomes visible when compensating for the unexpected offset by translating the y-axis.

Expected Result

The triangle should appear within the specified ortho bounds without additional transformations.

Environment

  • Operating System: macOS 13.4
  • Processing Version: 4.3.1

Possible Causes / Solutions

The issue appears to stem from the following block of code in the Processing source:

// The minus sign is needed to invert the Y axis.
projection.set(x, 0, 0, tx,
0, -y, 0, ty,
0, 0, z, tz,
0, 0, 0, 1);

Despite the comment saying that The minus sign is needed to invert the Y axis., this implementation does not fully invert the y-axis as intended. Instead, it flips the y-axis around zero, resulting in all the y-coordinates being offset into negative space.

Suggested Fix

To invert the y-axis properly in normalized device coordinates (NDC), pre-multiply the projection matrix with a -1 scale for the y-axis. This can be achieved by modifying the projection matrix calculation as follows:

// Add a pre-multiplied -1 y scale to flip the y axis in NDC.
    projection.set(x,  0, 0, tx,
                   0, -y, 0, -ty,
                   0,  0, z, tz,
                   0,  0, 0,  1);

This approach ensures the y-axis behaves as expected without requiring additional transformations.

Additional comments

This behavior stems from Processing's choice to have the Y-axis increase downward, consistent with traditional computer graphics. While reasonable, the implementation relies on ad-hoc adjustments throughout the code. Centralizing it at a lower level (NDC or viewport) and exposing it as an optional flag would provide more clarity and flexibility.

@SableRaf SableRaf added the bug Something isn't working label Dec 17, 2024
@Stefterv Stefterv added the help wanted Extra attention is needed label Jan 14, 2025
@SableRaf
Copy link
Collaborator Author

SableRaf commented Apr 25, 2025

Reopening this as the original fix was reverted in c83f44c

The suggested fix (implemented in #954) fixed this particular issue but caused unexpected behavior in almost every shader example.

@SableRaf SableRaf reopened this Apr 25, 2025
@IIITM-Jay
Copy link

IIITM-Jay commented May 7, 2025

Hi @SableRaf , while giving some thoughts to this issue, I have few ideas and approaches to share and discuss here first.

1. Using Shader Uniform

To ensure that shaders handle the Y-axis orientation correctly, we could declare a uniform boolean and then applying in vertex shader, followed by setting from Processing, something like-

uniform bool uFlipY;


if (uFlipY) {
    pos.y = -pos.y;
}
gl_Position = pos;

// In `PShader`
shader.set("uFlipY", true);

2. Using Internal Flag for Y-Axis Inversion

We could declare a boolean something like - boolean flipYAxis = false; // Default: no flip and then using in ortho() to behave accordingly:

float yScale = flipYAxis ? -y : y;
float yTranslate = flipYAxis ? -ty : ty;

projection.set(x,  0, 0, tx,
               0, yScale, 0, yTranslate,
               0, 0, z, tz,
               0, 0, 0, 1);

and then a user can simply use like-

g.flipYAxis = true;
ortho(0, width, 0, height);
resetMatrix();

3. Adding a whole new method to handle this

We could add a new method next to existing ortho(...) method -

public void orthoTopDown(float left, float right, float bottom, float top) {
  float x = 2.0f / (right - left);
  float y = -2.0f / (top - bottom);     // Flipping y here (as compared to existing method)
  float z = -2.0f / (far - near);
  float tx, ty, tz = (all as-is with no change)
 

  projection.set(x, 0, 0, tx,
                 0, y, 0, ty,
                 0, 0, z, tz,
                 0, 0, 0, 1);

  updateProjmodelview();
}

Let me know your views on above.

@IIITM-Jay
Copy link

Hi @SableRaf and @Stefterv , after giving some more time and observations to this, I think we should use both the Approach 2 which is of using internal flag and Approach 1 using Shader Uniform. Because, ortho() only affects the projection matrix — not the programmable shader pipeline. When we modify the Y-axis via the projection matrix (Approach 2), we're flipping coordinates at the CPU/OpenGL matrix level. But if we use shaders (which bypass fixed-function pipeline behaviors), then gl_Position is computed in the shader, not by ortho(). As for custom vertex shader, we're explicitly telling the GPU how to calculate the final screen-space position (gl_Position) of each vertex. So ortho() flips won't affect what the vertex shader does unless we replicate the logic.

In my opinion, for 2D/3D sketches with default P3D renderer, Only Approach 2 (flipYAxis in ortho()) will suffice, but for Custom shaders with PShader, we must also use Approach 1 (pass uFlipY to shader) for full support.

What you folks think of 🤔 ...

@IIITM-Jay
Copy link

Hi @SableRaf and @Stefterv, If you don't mind, may I request for suggestions and guidance on above to proceed ahead.

Thanks once again!

@IIITM-Jay
Copy link

Hi @SableRaf let me know among the methods that I have discussed above, which one would be the best suitable approach to fix this or some other concerns I should be taking care of and keep in mind to check upon before working on this. I would love to discuss feedback as well as other solutions that you think of if any.

@IIITM-Jay
Copy link

Hi @SableRaf and @Stefterv am I thinking in right direction as discussed in above comments:

I think we should use both the Approach 2 which is of using internal flag and Approach 1 using Shader Uniform. Because, ortho() only affects the projection matrix — not the programmable shader pipeline. When we modify the Y-axis via the projection matrix (Approach 2), we're flipping coordinates at the CPU/OpenGL matrix level.

I would be more than happy to raise a PR if you folks think that I am approaching in a correct way. Will revise the PR as per the feedback and changes as required.
I can understand the life of maintainers and the busy schedule of them and how much requests to handle ... just like the one here by me! 😅 Pardon me for this.

Thanks!

@Stefterv
Copy link
Collaborator

Hi @IIITM-Jay, thanks so much for your thorough work and patience!

After merging the earlier PR, we realized that—even though the current behavior is technically incorrect—many things have already been built on top of it. We’d like to fix it properly in the future, but don’t currently have the bandwidth to approach this with the care it needs.

Really appreciate your effort here, and we’ll definitely revisit your suggestions when we’re ready to address it fully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants