-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ghosts feature #56
Comments
Merged the changes (3rd approach) into standalone.py in https://github.com/oldnapalm/zwift-offline/tree/ghost How to use:
Issues:
If someone wants to test this I can create a Windows release. |
Thanks for this @oldnapalm
If you can do a Windows release I will test it.
Cheers
Mark
…On Wed, 20 May 2020 at 17:33, oldnapalm ***@***.***> wrote:
Merged the changes (3rd approach) into standalone.py in
https://github.com/oldnapalm/zwift-offline/tree/ghost
How to use:
- Create a file called "save_ghosts.txt" inside storage folder
- When rider stops, a ghost file will be saved in
storage/player_id/ghosts
- Copy a ghost file to storage/player_id and rename it to ghost.bin,
on next ride it will be loaded
- You must ensure that the ghost file is from the same map and route
you are going to ride
Issues:
- random spawn position (although never very far)
- must do a full stop to save the ghost
- rely on TCP connection to clear ghost data (in case you restart
Zwift without restarting zoffline)
If someone wants to test this I can create a Windows release.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQHWLVQSWHIMZQ3XXZDRSQA4LANCNFSM4NEHTAOQ>
.
|
Here it goes Edit: updated release, put ghost files in zoffline_1.0.49821_ghosts_test5.zip In this test version you don't need to create the file enable_ghosts.txt Remember to backup your files, mainly if you are updating from previous Zwift version, the database will be updated to fix segment timing issue. |
Thanks for the file, I will test it over the weekend.
Cheers
Mark
…On Wed, 20 May 2020 at 18:35, oldnapalm ***@***.***> wrote:
Here it goes
zoffline_1.0.49821_ghosts.zip
<https://github.com/zoffline/zwift-offline/files/4658430/zoffline_1.0.49821_ghosts.zip>
In this test test version you don't need to create the file save_ghosts.txt
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQCQOBBVTBTTROQTP4DRSQIGXANCNFSM4NEHTAOQ>
.
|
Before testing please check my previous comment again, maybe I will update the release before the weekend. |
I have done some preliminary tests and found every time I stop during my
workout a new bin file is created. So during the 5 minute workout if I stop
twice there will be 2 bin files created for the one ride.
Cheers
Mark
…On Thu, 21 May 2020 at 22:24, oldnapalm ***@***.***> wrote:
Before testing please check my previous comment again, maybe I will update
the release before the weekend.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQH52BRCSATUJ5PPFCDRSWLZLANCNFSM4NEHTAOQ>
.
|
Hi, thanks for the feedback. Yes, that's expected because it's the way I found to know when to save the ghost (save when the rider stops). Need to find a better way to determine when to save the ghost. For now ghosts must be always moving. I found another issue, if you change your weight, the W/Kg info for the ghost will be wrong, but it's just cosmetic, won't affect the ghost "performance". Except that it's working fine for me. |
Its all looking great for me also, the only small issue I have found is
that the ghost starts before you start peddling so in theory it can get a
head start.
Apart from that everything is great, thank so much for your effort, just
let me know if there is anything else you need from me.
Cheers
Mark
…On Sat, 23 May 2020 at 15:52, oldnapalm ***@***.***> wrote:
Hi, thanks for the feedback.
Yes, that's expected because it's the way I found to know when to save the
ghost (save when the rider stops). Need to find a better way to determine
when to save the ghost. For now ghosts must be always moving.
I found another issue, if you change your weight, the W/Kg info for the
ghost will be wrong, but it's just cosmetic, won't affect the ghost
"performance".
Except that it's working fine for me.
https://youtu.be/b94Xwn387TM
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQCSM7IOQENORQWYJKLRS7PLVANCNFSM4NEHTAOQ>
.
|
Are you using the first release? I updated to make the ghost start when you reach it's spawn location. Another option is to give the ghost a few seconds to start moving, so you don't get too ahead. About the save issue, probably going to make it save the ghost when the activity is saved (client calls Observed another issue where the ghost disappears and reappears. Happened arriving a hairpin turn, I'm guessing it's because the 3 seconds update frequency. |
I am using version zoffline_1.0.49821_ghosts.zip that you uploaded to
this thread.
Yes saving the ghost when the activity is saved would be perfect.
I never saw the ghost disappear and reappear, but will continue testing anyway.
Cheers
Mark
…On Sat, 23 May 2020 at 17:17, oldnapalm ***@***.***> wrote:
Are you using the first release? I updated to make the ghost start when you reach it's spawn location. Another option is to give the ghost a few seconds to start moving, so you don't get too ahead.
About the save issue, probably going to make it save the ghost when the activity is saved (client calls /api/profiles/<int:player_id>/activities/<string:activity_id> with upload-to-strava argument.
Observed another issue where the ghost disappears and reappears. Happened arriving a hairpin turn, I'm guessing it's because the 3 seconds update frequency.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
The client only calls This is a new test version Changes:
Other options I thought about:
The disappear/reappear issue was in a test with 4 ghosts, it may also be related to draft between ghosts. If the ghost drafts and gets ahead of its recorded roadTime, at some point it will be too off and need to be relocated. Edit: it's probably related to drafting, removed 2 other ghosts and the problematic one started to behave normally. Another question: when you stop during an activity, do you want the stopped time to be recorded or not? Right now only moving time is being saved to ghost file. |
Thanks for this update, I will do some further testing and let you know how
I get on.
…On Sat, 23 May 2020 at 20:53, oldnapalm ***@***.***> wrote:
The client only calls
/api/profiles/<int:player_id>/activities/<string:activity_id> with
upload-to-strava argument after 5 km, so for now we will save ghosts on
/api/users/logout. It will save when you exit Zwift, no matter if you
click "save activity" or the trash bin.
This is a new test version
zoffline_1.0.49821_ghosts_test6.zip
<https://github.com/zoffline/zwift-offline/files/4672420/zoffline_1.0.49821_ghosts_test6.zip>
Changes:
- save ghost on Zwift exit
- ghost will start when you pass its spawn position plus 15 seconds
- if multiple ghosts, they will start when you pass the one in the back
- if you spawn ahead of the ghost, it will start once you start
pedaling
Other options I thought about:
- ghost starts when you pass its spawn position (you can have an
advantage if you pass it fast and the ghost takes time to gain speed)
- with multiple ghosts, they start when you pass the one in the front
The disappear/reappear issue was in a test with 4 ghosts, it may also be
related to draft between ghosts. If the ghost drafts and gets ahead of its
recorded roadTime, at some point it will be too off and need to be
relocated.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQAYXWLZCBCJBM3RNFLRTASS3ANCNFSM4NEHTAOQ>
.
|
Ok, thanks for the testing and feedbacks. I have 2 questions:
|
These are good questions, I believe as these are supposed to be a race if
you stop, the time should continue and if you do stop the ghost should
continue, that would he make it a real representation of a race.
If I usually stop it is because my legs are aching, if you are on the road
outside racing and you stop it's tough luck.
What's your thoughts?
…On Sun, 24 May 2020 at 13:20, oldnapalm ***@***.***> wrote:
Ok, thanks for the testing and feedbacks.
I have 2 questions:
- when you stop during an activity, do you want the stopped time to be
recorded or not? Are these stops expected? Right now only moving time is
being saved to ghost file.
- if you are riding with a ghost and you stop, do you want the ghost
to stop too, or continue moving?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQDXQBO77WLJA544ENTRTEGKFANCNFSM4NEHTAOQ>
.
|
I don't know, because I don't race, my workouts are usually under 1 hour and I rarely stop. My wife said the ghosts should stop when you stop (she also doesn't use to stop, only in "emergencies"). Maybe it can be an option. She also thinks the stopped time should be disregarded when recording ghosts. What do you think about recording the stopped time? |
Yes I think the stop time, should be disregarded.
…On Sun, 24 May 2020 at 15:36, oldnapalm ***@***.***> wrote:
I don't know, because I don't race, my workouts are usually under 1 hour
and I rarely stop.
My wife said the ghosts should stop when you stop (she also doesn't use to
stop, only in "emergencies"). Maybe it can be an option. She also thinks
the stopped time should be disregarded when recording ghosts.
What do you think about recording the stopped time?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQAAYL3HXCR4LMC4ZQ3RTEWH7ANCNFSM4NEHTAOQ>
.
|
I agree with you, to simulate a race the ghost can't stop if you stop. So I'm keeping the test6 behavior, stop time is disregarded when saving ghost, and ghost don't stop when you stop. |
Perfect
…On Sun, 24 May 2020 at 17:57, oldnapalm ***@***.***> wrote:
I agree with you, to simulate a race the ghost can't stop if you stop.
So I'm keeping the test6 behavior, stop time is disregarded when saving ghost, and ghost don't stop when you stop.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
This is a new test release with minor changes:
zoffline_1.0.49821_ghosts_test7.zip Added the changes in #57 Please let me know if you find any issues. Thanks. |
Great work, thanks for your effort, it is much appreciated.
…On Sun, 24 May 2020 at 22:22, oldnapalm ***@***.***> wrote:
This is a new test release with minor changes:
- Ghost will be saved on activity save (must have at least 5 km, like
upload to Strava).
- Removed the 15 seconds in ghost delay. If you spawn too close to the
ghost it will have an advantage because it starts at full speed. If you
spawn behind you can go slow until the ghost spawns to have a fair dispute
(like if you spawn ahead, you can wait for the ghost to get closer).
- Fixed a delay bug in roads that start in reverse direction
(decreasing roadTime).
zoffline_1.0.49821_ghosts_test7.zip
<https://github.com/zoffline/zwift-offline/files/4674414/zoffline_1.0.49821_ghosts_test7.zip>
Added the changes in #57
<#57>
Please let me know if you find any issues. Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQCQOJ3AQHRJTGTXI5DRTGFYDANCNFSM4NEHTAOQ>
.
|
Glad you enjoyed it. Added a release here Cheers |
This is the bug I mentioned before It happens when there are various ghosts. My guess is Zwift calculates draft between nearby players (ghosts in this case), at some point roadTime is too ahead of value received from server, the rider makes a loop, roadTime gets behind the value from server, then the rider disappears and reappears a little ahead, or takes a shortcut to reach roadTime from server. |
Yes I have seen this bug a couple of times now, I just assumed it was a
consequence of drafting
…On Wed, 27 May 2020 at 18:01, oldnapalm ***@***.***> wrote:
This is the bug I mentioned before
https://youtu.be/ABB4ayd8p9g
It happens when there are various ghosts. My guess is Zwift calculates
draft between nearby players (ghosts in this case), at some point roadTime
is too ahead of value received from server, the rider makes a loop,
roadTime gets behind the value from server, then the rider disappears and
reappears a little ahead, or takes a shortcut to reach roadTime from server.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC52DQGVTHBOGUSFYS5ZTZDRTVBOXANCNFSM4NEHTAOQ>
.
|
The ghost feature has been merged into zoffline. I'm closing this issue. Feel free to reopen it if you find anything else. Cheers |
Caused the bug mentioned here #56 (comment)
The supposed drafting bug actually happened because of this misplaced increment 62a4161 When a ghost finished, others with higher id would get their id decremented. |
Biggest difference in roadTime between two states near start line at Volcano Circuit was ~7700 (difference before last point before start and first point after), so 4000 should do the work. Anyway, it gives a lot of fun to ride with ghosts so thanks again for you work :) I just saw edit in your previous comment, will try to look at this commit and run this version tomorrow. |
Sorry still not had a chance to do any testing my return home has been delayed.
As soon as I get back I will get right onto it
…On Mon, 15 Jun 2020 at 19:57, oldnapalm ***@***.***> wrote:
Did some tests in the desert
https://youtu.be/nYzocir3klw
Would need to test on the other roads but 4000 looks good to me, what do you think?
Maybe a better approach would be storing the roadTime/distance ratio for each road.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
It's a good idea but I couldn't think of a way to implement that. Would have to look for 2 adjacent points where one is > and other is < than start line. Also need to know when we pass the start line, and there's the roadTime "reset" situation (from 1005000 to 5000 and vice versa). Will think about it. Test version with commit oldnapalm@6a8e2c2 |
Don't worry, we are all doing this for fun ;) |
@msobecki think this should work, check when pass the start line instead of when get close oldnapalm@77cef21 Need to test if will work in all cases. If you want the ghosts to spawn further back, just remove this |
Great job @oldnapalm, I'll try to check it tomorrow, looks promising. thanks |
Don't know exactly, but it's more than once a second. You can change update_freq to 2 or 1. I believe there are no other negative consequences, just the bin files bigger. Files saved with different update_freq won't work. If you will run from source, please use this branch, I made a few changes since the last blob https://github.com/oldnapalm/zwift-offline/blob/start-line/standalone.py Another issue with the start line is when there are 2 or more routes with the same spawn point and different start lines (like jungle circuit and road to sky) you can't have both in the csv because we don't know which route you are going to ride when you spawn. |
@msobecki if that happens again please check Zwift log for UDP errors or timeout
Not sure if it's the cause but I think we should disregard incoming packets if can't decode as ClientToServer Line 237 in 0e7f45d
|
Thanks @oldnapalm Thanks again, BR. |
Maybe you didn't allow python.exe in Windows firewall?
Just adding the ports in this line solves the problem or need something else? Thanks for that. Line 12 in 0e7f45d
Here's a bundle with the latest changes And the updated csv (road to sky doesn't work, need to figure a way to fix) https://github.com/oldnapalm/zwift-offline/blob/start-line/start_lines.csv Added a few more start lines (untested). Removed jungle and desert (breaks ghost spawn for alpe and titans grove). |
Another update, need to store the spawn direction in the csv because there are routes in both directions. Fixes desert and titans grove. Road to sky is still broken if jungle circuit start line is present. Other maps untested, just added the start lines, but should work. https://github.com/oldnapalm/zwift-offline/blob/start-line/start_lines.csv |
Update Use spawn direction only when start line is different for each direction https://github.com/oldnapalm/zwift-offline/blob/start-line/start_lines.csv |
I think I found a bug! But on the other hand I'm just scratching the surface with this code so sharing thoughts here to see if I'm completely off base :) Context: I've been working through an attempt to set up so that ghosts pick up at the start of the Alpe du Zwift climb, which has been tricky. Finally got it to work, but one of the things that gave trouble turned out to be the 'isForward' comparison when intaking the csv. As it's written, it's comparing the text strings of the isForward status, with the isForward indicator in the csv. Problem is, one is capitalized, other is not (based on the capitalization in existing rows of the csv) - so it was handling most fine because they were blank (so falling into the 'or not' condition), but those that were populated were trying to compare 'TRUE' to 'True' (for example), and thus thinking there was no matching starting line defined. One super simple fix would be to just match cases in the csv file, but really that seems like leaving a vector for future errors if someone enters true/false in the wrong case at some future point. It seems like a better way would be to make a quick/simple boolean check like:
And then in the csv row check, update it to:
Basically just forces case and comparing to a list of strings that would evaluate to true, otherwise evaluate to false, creating a super-quick/rough string to boolean converter. With the boolean/case check fixed, all worked :) To get road to sky + Alpe du Zwift working, I removed:
With this result (success!): As I'm looking through this, I'm wondering if it wouldn't be pretty straightforward to do more of a segment-based ghost system... Going to do some experimentation, if I come up with any ideas solid enough to share I'll do so here. |
Yes, the current code expects the csv to contain "True" or "False" (case sensitive). Your booleans function could be
then the check could be But in this specific case you don't need to check because the routes starting on this road are in the same direction, so you can leave it blank (falls in Only Desert Flats/Titans Grove require this check because there are routes starting on the same road, going in both directions, with different starting points (if the start line is the same for both directions, like Downtown and UCI courses, it's also not necessary). I think it would be simple to do a segment based ghost system. I tried a road based system in my early tests, but then I found that saving a ghost for the entire activity would be more useful to me. Tomorrow I can send you the code if you want to take a look. |
Of course I'd love to take a look 😄 Aha! I see why I called it a bug: user error of course :) |
As you can see the csv reading is very basic, things can go wrong is the file is not perfectly formatted (e.g. I'm attaching the code I mentioned yesterday, but looking at it now I don't think it will help with your idea, it's very preliminar (e.g. used MAP_OVERRIDE instead of course for the first directory level). It's the second approach mentioned in the opening of this issue. |
Looks like the issue mentioned in #56 (comment) is actually caused by a second UDP connection before the ghosts are loaded
Hello @scouseman @msobecki @defiancecp how are you? Any of you are still using this or have many ghosts for a route? I had an issue when I reached 13 ghosts for the same route, looks like it exceeds the UDP datagram size limit.
I tried to fix it by sending multiple messages with a maximum of 10 ghosts per message. It seems to be working for this specific case, but I would like to do some more tests, please let me know if you can help. Thanks. |
Hi @oldnapalm But of course, again, thanks for your work and commitment 👍 |
Looks like Excel converts True/False to uppercase. Thanks @defiancecp #56 (comment)
After a few months of use and some bugs fixed, I changed the ghosts handling a bit, now it saves the files organized in Also added a checkbox in the launcher window to easily enable/disable the feature. |
I'm trying to implement ghosts feature in zoffline, it was requested by @scouseman in #35 (comment)
PlayerState structure is defined in https://github.com/Ogadai/zwift-mobile-api/blob/master/src/zwiftMessages.proto
The main issue I found (for now) is that rider position is not determined by x, y, altitude and heading, the only fields that seem to influence are roadTime and speed.
Found in https://github.com/wiedmann/zwift-line-monitor that it's an integer value from 5000 to 1005000, verified it by reading values sent from client to zoffline, but couldn't find a relation between roadTime and distance.
Values from a 5 km ride in Fuego Flats
https://github.com/oldnapalm/zwift-offline/blob/ghost/flats.csv
This is what I've tried so far
https://github.com/oldnapalm/zwift-offline/blob/ghost/udp.py
Any help or ideas are welcome.
Edit: this test is with a roadTime increment of 38 per meter. Around km 0.3 the ghost does some strange maneuvers, looks like it's when roadTime value becomes too off. It happens again around km 1.3
https://youtu.be/tdMKQu5L0tQ
Maybe the only way is storing roadTime values (one per meter?) for each segment?
Edit: maybe a better approach would be, instead of using FIT or GPX files for ghosts, making the UDP server store one full PlayerState record each 5 seconds during activities and save a file in player_id/ghosts/world_id/road_id directory. When the player gets on that road again (in the same direction) the server sends back the ghost. It would work only for new activities, but easier to implement.
Edit: here is my first try on the second approach
https://github.com/oldnapalm/zwift-offline/blob/ghost/udp2.py
Problems:
Maybe it's a better idea to store the ghost for the entire activity, not per road.
Edit: this is the 3rd approach, my favorite so far. It saves ghosts in
storage/player_id/ghosts
and loads fromstorage/player_id/ghosts/load
https://github.com/oldnapalm/zwift-offline/blob/ghost/udp3.py
The text was updated successfully, but these errors were encountered: