follow_* and walkers set_position through StepWalker #5038

mjmadsen · 2016-09-01T04:56:25Z

I do not feel this is ready to merge. But I want to get testers.

Goals it to drive all set position api calls through StepWalker. That includes centralizing noise and humanizing to position to one location. Outputting position updates from one location.

Repeat, I'm not done. Do not merge.

mention-bot · 2016-09-01T04:56:25Z

@mjmadsen, thanks for your PR! By analyzing the annotation information on this pull request, we identified @douglascamata, @tstumm and @leanhdaovn to be potential reviewers

mjmadsen · 2016-09-01T05:30:48Z

Not entirely sure why the CI is crapping out. But I'm still not finished :)

th3w4y · 2016-09-01T08:21:00Z

@mjmadsen

is it .get_position() or .position() ?

I see some inconsistencies between:

https://github.com/mjmadsen/PokemonGo-Bot/blob/8a57a2656d8d8636901512fc07258d2b4d1efd6e/pokemongo_bot/cell_workers/follow_path.py#L146

and

https://github.com/mjmadsen/PokemonGo-Bot/blob/8a57a2656d8d8636901512fc07258d2b4d1efd6e/pokemongo_bot/cell_workers/follow_spiral.py#L70

th3w4y · 2016-09-01T08:36:41Z

pokemongo_bot/walkers/polyline_walker.py

+            cLng,
+            cAlt
+        )
+


@mjmadsen This is not going to change the bot position anymore...

since you replaced .set_position() with just an object initiation.
But there is nothing calling .step() on the new step_walker object

th3w4y · 2016-09-01T09:23:32Z

@mjmadsen I created a PR towards your fork https://github.com/mjmadsen/PokemonGo-Bot/pull/1

modified the PolylineWalker to not override the super .step()
corrected some issues:

Not entirely sure why the CI is crapping out. But I'm still not finished :)

no longer round lat, lng, alt in Polyline class
fixed some issues that would arise from this changes regarding the PolylineObjecthandler cache
mjmadsen@858f661

PolylineWalker no longer overrides super .step() method

mjmadsen · 2016-09-01T15:44:09Z

@th3w4y Thank you!

mjmadsen · 2016-09-01T15:54:54Z

pokemongo_bot/cell_workers/follow_spiral.py

-
-        if dist <= 1 or (self.bot.config.walk_min > 0 and step_walker == None):
-            if self.ptr + self.direction >= len(self.points) or self.ptr + self.direction <= -1:
-                self.direction *= -1


Am I breaking anything by removing this? It didn't seem to be applied last night when I was looking.

@mjmadsen it is breaking indeed is returning always the first point only!...

you still need something from the if dist <=1

if dist <= 1 or step_walker == None: if self.ptr + self.direction >= len(self.points) or self.ptr + self.direction <= -1: self.direction *= -1 if len(self.points) != 1: self.ptr += self.direction else: self.ptr = 0

Bah, I was too sleepy when I change all of this! I'll try to put the corrections in this afternoon.

mjmadsen · 2016-09-02T19:33:32Z

@solderzzc PolyWalker seems to not add in the human variance changes still.

th3w4y · 2016-09-02T19:33:58Z

@mjmadsen what do you mean?

in this commit it should... since it does not override the StepWalker.step() anymore...

mjmadsen · 2016-09-02T19:34:49Z

Should we make polywalker the default for movetofort? This honestly seems to be superior.

It could use some tweaks (as it walks down the middle of the road).

th3w4y · 2016-09-02T19:49:10Z

@mjmadsen you don't see any jitter to it from the StepWalker? from the random_lat_long_delta()?

mjmadsen · 2016-09-02T19:49:45Z

Very little compared to the other walkers (follow_*)

th3w4y · 2016-09-02T19:51:28Z

@mjmadsen i though the random_lat_long_delta() would be enough...
(that is why i removed the rounding from the .get_pos() hmmm...)

the random_lat_long_delta() should add something between [-2, +2] m variance..

mjmadsen · 2016-09-02T19:52:37Z

Those random deltas are the only thing added if our noise is disabled right? The variance seemed quiet high with the followers and the polylinewalker seems to be very minimal.

th3w4y · 2016-09-02T20:04:11Z

@mjmadsen wow i think i know why the follow ones have a bigger jitter it has to do with the magnitude self.magnitude way of calculation.. the higher the distance to walk the higher the jitter maybe...

I have to do the math...

mjmadsen · 2016-09-02T20:14:28Z

Lol. I'll probably work on a few other tasks before returning to this. It's a bit of a minefield.

mjmadsen · 2016-09-02T20:50:11Z

follow_path is also affected.

Or it's that spiral and the like give such a distant target, our step variance spikes. Would seem that logic needs toe be altered.

th3w4y · 2016-09-02T21:04:25Z

This looks odd to me the method we calculate now....

made a small test with speed 3 trying to reproduce what step walker does now...

In [105]: def current_calculation(orig, dest):
     ...:     distance = haversine.haversine(orig, dest)*1000
     ...:     speed = 3
     ...:     steps = int(distance/speed)
     ...:     if steps == 0:  # set steps to 1 to prevent division by zero
     ...:         steps = 1    
     ...:     dLat = (dest[0] - orig[0])/steps
     ...:     dLon = (dest[1] - orig[1])/steps
     ...:     self_magnitude = sqrt(dLat**2+dLon**2)
     ...:     totalDLat = (dest[0] - orig[0])
     ...:     totalDLon = (dest[1] - orig[1])
     ...:     magnitude = sqrt(totalDLat**2+totalDLon**2)
     ...:     unitLat = totalDLat / magnitude
     ...:     unitLon = totalDLon / magnitude
     ...:     scaledDLat = unitLat * self_magnitude
     ...:     scaledDLon = unitLon * self_magnitude
     ...:     cLat = orig[0] + scaledDLat
     ...:     cLng = orig[1] + scaledDLng
     ...:     return haversine.haversine(orig, dest)*1000, steps, haversine.haversine(orig,(cLat, cLng))*1000
     ...:     
     ...: 

In [106]:

15 cm 1 step 2.6m walked

In [106]: current_calculation((8.000001, 8.000001),(8.000002, 8.000002))
Out[106]: (0.15649005443906103, 1, 2.653969095391703)

1.56 m 1 step 2.87m walked in 1sec

In [107]: current_calculation((8.00001, 8.00001),(8.00002, 8.00002))
Out[107]: (1.5649005174894743, 1, 2.8753466491876645)

15.65m 5 steps 3.46m walked in 1sec

In [108]: current_calculation((8.0001, 8.0001),(8.0002, 8.0002))
Out[108]: (15.649002609436945, 5, 3.4607670521667617)

156m 52 steps 3.40m walked in 1sec

In [109]: current_calculation((8.001, 8.001),(8.002, 8.002))
Out[109]: (156.4897695005808, 52, 3.406428240494767)

1.5km 521 steps 3.4m walked in 1sec

In [110]: current_calculation((8.01, 8.01),(8.02, 8.02))
Out[110]: (1564.8720111329158, 521, 3.40380763031615)

In [111]:

julienlavergne · 2016-09-03T02:49:02Z

I have some big problems with this merge:

Polyline worker totally broken:
- The polyline step must be paused before step and unpaused after. Otherwise the polyline worker is jumping directly to the destination when calling step.
- Polyline pol_alt now default to a random value when altitude is zero. I would prefer it stays zero if that is the altitude returned. zero is not an error case.
- There is way too much variance in the polyline step. gps does not jump 5 meters in every direction every seconds.

th3w4y · 2016-09-03T06:41:41Z

Polyline worker totally broken:
-The polyline step must be paused before step and unpaused after. Otherwise the polyline worker is jumping directly to the destination when calling step.

yes it jumps directly to destination.. but the destination is the .get_pos() of the cached polyline not final destination so i don't see an issue...

-Polyline pol_alt now default to a random value when altitude is zero. I would prefer it stays zero if that is the altitude returned. zero is not an error case.

~~that should be only if there is no cached polyline...~~
in the comparison .get_alt() or uniform(self.bot.config.alt_min, self.bot.config.alt_max)
That is a valid issue!
will fix as @mjmadsen suggested with self.api.actual_alt th3w4y#1

-There is way too much variance in the polyline step. gps does not jump 5 meters in every direction every seconds.

I agree that gps does not jump 5m every second but you kinda have exact opposite behaviour then @mjmadsen who is complaining of too little variance..
The variance comes now only from the random_lat_long_delta() in the StepWalker

mjmadsen · 2016-09-03T06:46:11Z

#5132 I have a few alterations to this. The first portion is not implemented yet.

julienlavergne · 2016-09-03T06:46:20Z

get_pos() return the final destination if Walker has never been paused.

random_lat_long_delta is too big. When I look at my GPS position reported by my phone versus what the bot set in StepWalker, there is a very big difference.
After some tests, I settled on dividing by 5 whatever number is returned by random_lat_long_delta. That is what gave me the results closest to reality.

th3w4y · 2016-09-03T06:53:37Z

@mjmadsen @anakin5 #5135 will fallback to previous bot position...

self.actual_alt = self.bot.position[-1]
self.pol_alt = self.polyline.get_alt() or self.actual_alt

th3w4y · 2016-09-03T06:55:38Z

@anakin5

how big is the distance till final destination? it should return final destination only if enough time passes...

get_pos() return the final destination if Walker has never been paused.

th3w4y · 2016-09-03T07:01:30Z

@anakin5 ~~You are actually right... i think i found the issue...~~
I cannot reproduce...

In [4]: a = Polyline((0,0),(0.1,0.1),3)

In [5]: a
Out[5]: <pokemongo_bot.walkers.polyline_generator.Polyline at 0x108031a50>

In [6]: a.points
Out[6]: [(0, 0), (0.1, 0.1)]

In [7]: a.get_pos()
Out[7]: (0.00028574742477962823, 0.00028574742477962823)

In [8]: a.get_pos()
Out[8]: (0.000317644389002829, 0.000317644389002829)

In [9]: a.get_pos()
Out[9]: (0.0003394669497900716, 0.0003394669497900716)

In [10]: a.get_last_pos()
Out[10]: (0.0003394669497900716, 0.0003394669497900716)

In [11]: a.get_last_pos()
Out[11]: (0.0003394669497900716, 0.0003394669497900716)

In [12]:

do you have an example of from... and final destination?

julienlavergne · 2016-09-03T07:41:22Z

I think the problem comes from the fact that step_walker.step return True at the first step of the PolylineWalker. That is because the first point of the Polyline is the current position, so the distance to step is zero, which translate to "we are arrived at destination".
The StepWalker.step should not return True until final destination is reached, as it was the case before.

julienlavergne · 2016-09-03T07:45:30Z

But my opinion is that the real evil there is in the fact that we do not reuse walkers. There is no concept of final destination and next point in the walker, we are expected to create a new object at each step.
Without these concepts, there is no way for the walker to know that we are arrived. It would be nice to keep the same walker object during the whole trip.

th3w4y · 2016-09-03T07:53:11Z

what are you combining the PolylineWalker with?

I think the problem comes from the fact that step_walker.step return True at the first step of the PolylineWalker. That is because the first point of the Polyline is the current position, so the distance to step is zero, which translate to "we are arrived at destination".
The StepWalker.step should not return True until final destination is reached, as it was the case before.

julienlavergne · 2016-09-03T07:57:08Z

It return True whatever is the task I am using it with. Be it MoveToFort or CampFort. Both of them relying on the fact that it should return True only when arrived.

th3w4y · 2016-09-03T08:03:56Z

@anakin5 i had to override step() again... i didn't want to but hey... 130d630

th3w4y · 2016-09-03T08:23:36Z

The change was big... some issues slipped through...

Looks like I was removed as an admin over this :(

I wouldn't have merged if I was told by @solderzzc

th3w4y · 2016-09-03T09:04:17Z

@anakin5

What do you consider a normal range for variation?

get_pos() return the final destination if Walker has never been paused.
random_lat_long_delta is too big. When I look at my GPS position reported by my phone versus what the bot set in StepWalker, there is a very big difference.
After some tests, I settled on dividing by 5 whatever number is returned by random_lat_long_delta. That is what gave me the results closest to reality.

mjmadsen · 2016-09-03T09:10:54Z

Oh I see. random_lat_long_delta is supposed to make us look more human with wandering. What would be consider as a better range? Currently supposed to be -9.1 to 9.1m variance. Even assuming those comments are correct, we could move 9.1 x 9.1 in a single step (which is a lot)

The gps noise bit is the one that is supposed to alter our coords to match phone's inaccuracies. @anakin5 I assume you have this bit disabled too?

th3w4y · 2016-09-03T09:18:34Z

@mjmadsen @anakin5

this was the rookie mistache causing everything....

8f7f22c

mjmadsen · 2016-09-03T09:25:05Z

Doh! At least you found it 👍

mjmadsen added 8 commits August 31, 2016 23:21

Always use StepWalker

6614d5c

Removed unreachable code

65f04e8

Update follow_spiral.py

030adc3

Update follow_spiral.py

e1a7812

Force to StepWalker

57ad6ca

Remove API call

58a7390

Remove API call

0a8c00d

Remove API call

72dd71b

mjmadsen added 3 commits August 31, 2016 23:58

Update follow_cluster.py

46c2db3

Force to StepWalker

8e4b114

Update follow_spiral.py

7a88f5c

mjmadsen added 3 commits September 1, 2016 00:31

Update follow_cluster.py

8ee38ed

Update follow_spiral.py

4c3e0a5

Merge branch 'dev' into mjmadsen-walker-test

8a57a26

mjmadsen added Help to Test Second Review Required labels Sep 1, 2016

th3w4y reviewed Sep 1, 2016
View reviewed changes

th3w4y and others added 6 commits September 1, 2016 12:00

PolylineWalker no longer overrides the super .step() method

3942a3b

PolylineObjectHandler cache invalidate only when absolute offset > 8m

858f661

no longer round results in Polyline class

b54a4ef

update polyline_generator tests to reflect non-rounding

3e52993

Merge pull request #1 from th3w4y/mjmadsen-walker-test

88f1c93

PolylineWalker no longer overrides super .step() method

Merge branch 'dev' into mjmadsen-walker-test

aa6177a

mjmadsen reviewed Sep 1, 2016
View reviewed changes

Revert spiral changes

670fae6

mjmadsen merged commit c9a6f08 into PokemonGoF:dev Sep 2, 2016

mjmadsen mentioned this pull request Sep 3, 2016

Reduce lat_long_delta variance range #5140

Closed

follow_* and walkers set_position through StepWalker #5038

follow_* and walkers set_position through StepWalker #5038

Conversation

mjmadsen commented Sep 1, 2016

Repeat, I'm not done. Do not merge.

mention-bot commented Sep 1, 2016

mjmadsen commented Sep 1, 2016

th3w4y commented Sep 1, 2016

th3w4y Sep 1, 2016 • edited Loading

Choose a reason for hiding this comment

th3w4y commented Sep 1, 2016 • edited Loading

mjmadsen commented Sep 1, 2016

mjmadsen Sep 1, 2016

Choose a reason for hiding this comment

th3w4y Sep 1, 2016 • edited Loading

Choose a reason for hiding this comment

mjmadsen Sep 1, 2016

Choose a reason for hiding this comment

mjmadsen commented Sep 2, 2016

th3w4y commented Sep 2, 2016 • edited Loading

mjmadsen commented Sep 2, 2016 • edited Loading

th3w4y commented Sep 2, 2016 • edited Loading

mjmadsen commented Sep 2, 2016

th3w4y commented Sep 2, 2016 • edited Loading

mjmadsen commented Sep 2, 2016

th3w4y commented Sep 2, 2016 • edited Loading

mjmadsen commented Sep 2, 2016

mjmadsen commented Sep 2, 2016 • edited Loading

th3w4y commented Sep 2, 2016

julienlavergne commented Sep 3, 2016

th3w4y commented Sep 3, 2016

mjmadsen commented Sep 3, 2016 • edited Loading

julienlavergne commented Sep 3, 2016 • edited Loading

th3w4y commented Sep 3, 2016 • edited Loading

th3w4y commented Sep 3, 2016

th3w4y commented Sep 3, 2016 • edited Loading

julienlavergne commented Sep 3, 2016

julienlavergne commented Sep 3, 2016

th3w4y commented Sep 3, 2016

julienlavergne commented Sep 3, 2016

th3w4y commented Sep 3, 2016

th3w4y commented Sep 3, 2016 • edited Loading

th3w4y commented Sep 3, 2016 • edited Loading

mjmadsen commented Sep 3, 2016 • edited Loading

th3w4y commented Sep 3, 2016

mjmadsen commented Sep 3, 2016

th3w4y Sep 1, 2016 •

edited

Loading

th3w4y commented Sep 1, 2016 •

edited

Loading

th3w4y Sep 1, 2016 •

edited

Loading

th3w4y commented Sep 2, 2016 •

edited

Loading

mjmadsen commented Sep 2, 2016 •

edited

Loading

th3w4y commented Sep 2, 2016 •

edited

Loading

th3w4y commented Sep 2, 2016 •

edited

Loading

th3w4y commented Sep 2, 2016 •

edited

Loading

mjmadsen commented Sep 2, 2016 •

edited

Loading

mjmadsen commented Sep 3, 2016 •

edited

Loading

julienlavergne commented Sep 3, 2016 •

edited

Loading

th3w4y commented Sep 3, 2016 •

edited

Loading

th3w4y commented Sep 3, 2016 •

edited

Loading

th3w4y commented Sep 3, 2016 •

edited

Loading

th3w4y commented Sep 3, 2016 •

edited

Loading

mjmadsen commented Sep 3, 2016 •

edited

Loading