Labels move after placement

[daotyl000]

After placing and confirming a label, once I pan to shift my view the label would move to a different location. Sometimes slightly to still be ok but sometimes it will shift it very far away.

Link to a screen recording:
https://youtu.be/y0hdn6gk67I

[misaugstad]

is this just happening on that pano, or are you seeing that elsewhere?

[misaugstad]

also what browser are you using?

[daotyl000]

I have seen it on several panos but it was never as bad as this time, I was on Safari.

[misaugstad]

Interested to hear if others have this issue, particularly on Safari

[jonfroehlich]

We need more investigation of this Issue... because if Sidewalk doesn't work well in Safari, we should let our users know (and encourage them to use an alternative browser).

So, for example, one heavy user of the Newberg deployment always had slightly misplaced labels, which made it hard to validate their work. I wonder if this was due to using Safari (@misaugstad, can you check what browser that user was using?)

[nch0w]

I'm not experiencing this issue on Safari, but I do notice that the label lags behind the panorama when panning.

[jonfroehlich]

Does the label seem to properly stick to the tagged location or does it move when you pan?

…

On Tue, Jul 9, 2019 at 11:33 AM Neil Chowdhury ***@***.***> wrote: I'm not experiencing this issue on Safari, but I do notice that the label lags behind the panorama when panning. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1764?email_source=notifications&email_token=AAML55LJVUTM5FF7NJ7BOX3P6TKXLA5CNFSM4H4JBL2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZREHCQ#issuecomment-509756298>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAML55LC2IGHNPYRCBBL4UDP6TKXLANCNFSM4H4JBL2A> .

-- Jon Froehlich Associate Professor Paul G. Allen School of Computer Science & Engineering University of Washington http://makeabilitylab.io @jonfroehlich <https://twitter.com/jonfroehlich> - Twitter Help make sidewalks more accessible: http://projectsidewalk.io

[nch0w]

It properly sticks to the tagged location.

[timvng]

I don't have this issue on Safari. Although when I move the cursor onto the label, this happened.

Now I can't delete the label anymore. I'm not sure if this has been mentioned before.

[jonfroehlich]

I don't think this has been mentioned before either. Can you make a new Issue with a screen recording (movie)? Jon

…

On Tue, Jul 9, 2019 at 12:19 PM Tim Nguyen ***@***.***> wrote: I don't have this issue on Safari. Although when I move the cursor onto the label, this happened. [image: Screen Shot 2019-07-09 at 12 08 46 PM] <https://user-images.githubusercontent.com/51970755/60916387-0dd3be80-a243-11e9-82eb-5bd2caa4fd7b.png> Now I can't delete the label anymore. I'm not sure if this has been mentioned before. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1764?email_source=notifications&email_token=AAML55IFAAVN7D7Y3JZAY6TP6TQC5A5CNFSM4H4JBL2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZRIFHQ#issuecomment-509772446>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAML55IZUJIBICEXFWPULFDP6TQC5ANCNFSM4H4JBL2A> .

-- Jon Froehlich Associate Professor Paul G. Allen School of Computer Science & Engineering University of Washington http://makeabilitylab.io @jonfroehlich <https://twitter.com/jonfroehlich> - Twitter Help make sidewalks more accessible: http://projectsidewalk.io

[misaugstad]

The rest of this thread above is a bit messy, but ultimately we are aware that labels will move right after you place them! This is worse when you are more zoomed in. The issue comes from the calculatePointPov and povToPixel3DOffset functions in UtilitiesPanomarker.js. These are pulled from the code for Panomarkers, and the math that they have there is pulled from this stackoverflow post. The person who wrote up the Panomarker code made a writeup for how the math works out here.

Unfortunately, there is clearly some error(s) in there! To be fair, part of the calculations include approximations. For example, Google documents the available field of view at different zoom levels, but they've noticed that these zoom levels are just not at all accurate in practice, so they've written up an approximation. But maybe things have changed since this code was written in 2014, and maybe we could improve upon it?

Here's the gist of the issue:

We start out with the pov (heading, pitch, zoom) of the GSV window and the pixel location on the canvas where the label was placed (canvasX, canvasY). We then use the calculatePointPov function mentioned above to figure, which figures out where the heading and pitch of GSV would have to be for the input label to be centered on the canvas.

Later on, we take that heading and pitch and use it to figure out where the label should now show up on the canvas after we've panned. To do that, we call the getCanvasCoordinate function (but all the math for that is in the povToPixel3DOffset function under the hood).

This all sounds fine! But one thing that should happen is that when you haven't panned at all, if you call calculatePointPov and pass the result into getCanvasCoordinate, you should get the same thing that you input to calculatePointPov. But unfortunately this is often not the case. Here is an example.

let currPov = {heading: 14.625, pitch: -15.625, zoom: 3};
let canvasX = 426;
let canvasY = 214;
let labelPov = util.panomarker.calculatePointPov(currPov, canvasX, canvasY, svl.CANVAS_WIDTH, svl.CANVAS_HEIGHT);
// result: {heading: 16.665, pitch: -13.966, zoom: 3}

// labelPov is the heading/pitch that GSV should be at to have the label centered on the canvas.
// Now let's use this and the exact same input POV to find canvasX/Y. It should be the same as our input X/Y...
util.panomarker.getCanvasCoordinate(labelPov, currPov, svl.CANVAS_WIDTH, svl.CANVAS_HEIGHT, svl.LABEL_ICON_RADIUS);
// result: {x: 411, y: 198}

In this example, we are off in the X direction by 15 pixels, and the Y direction by 16 pixels!

[jonfroehlich]

This is great @misaugstad but I was expecting the kicker or reveal but you left us hanging. You state:

This all sounds fine! But one thing that should happen is that when you haven't panned at all, if you call calculatePointPov and pass the result into getCanvasCoordinate, you should get the same thing that you input to calculatePointPov. But unfortunately this is often not the case.

But then don't offer a suspected reason or possible solution... unless that's what you were trying to cover by "part of the calculations include approximations"

[misaugstad]

This is great @misaugstad but I was expecting the kicker or reveal but you left us hanging

Sorry about that! That's partially because I needed to head off to the gym yesterday, and partially because I don't have a solution yet! My main goal of posting here was to actually document the problem with an easy to test example, with the issue narrowed down as much as possible.

I haven't actually dug into the math yet, or the writeup of the math that I linked to. It's possible that the math is all theoretically sound as long as the FOV approximations are correct... In that case, we could do something like we did for approximating label lat/lng (PR #2434). For that, I ran some linear regressions to find predictors of the distance from the panorama, giving us a decent estimate of lat/lng. Since we know what the output canvas x/y values should be (in the case where we haven't panned), we might be able to come up with a better approximation than we currently have.

But again, I haven't dug into the math enough yet :) The main update is that I've narrowed down the problem to a few lines of code, and cleaned up the surrounding code so that what we're looking at here makes a lot more sense and should be easier to debug when we try to do so again!

[jonfroehlich]

Gotcha. Great progress on a thing that has long-plagued us! Thanks for the update @misaugstad!