-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Locations.fit files #762
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for taking this on.
Please add a test case. See testo.d/garmin_fit.test. We would prefer a fit file that is sufficient to test your changes but isn't too large. Without a test case maintenance is problematic and things tend to get broken with future changes.
garmin_fit.cc
Outdated
auto* lappt = new Waypoint; | ||
lappt->latitude = GPS_Math_Semi_To_Deg(lat); | ||
lappt->longitude = GPS_Math_Semi_To_Deg(lon); | ||
lappt->shortname = name; | ||
lappt->description = description; | ||
waypt_add(lappt); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't have anything to do with a lap, does it? How about using locpt instead of lappt?
it seems you may have altitude data from kFieldLocAltitude you aren't setting.
garmin_fit.cc
Outdated
if (f.type == 7) { | ||
QString val = fit_getstring(f.size); | ||
switch (def.global_id) { | ||
case kIdDeviceSettings: // device settings message | ||
case kIdLocations: | ||
switch (f.id) { | ||
case kFieldGlobalUtcOffset: | ||
case kFieldLocationName: | ||
if (global_opts.debug_level >= 7) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps it is better to teach fit_read_field to handle f.type 7 data and return a QVariant. Then the handling of kIdLocations can be handled in the def.global_id switch block kIdLocations case below along with latitude, longitude, altitude.
garmin_fit.cc
Outdated
GarminFitFormat::fit_getstring(int size) | ||
{ | ||
if (fit_data.len < size) { | ||
throw ReaderException("record truncated: expecting " + std::to_string(size) + " bytes, but only got " + std::to_string(fit_data.len) + "."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please correct the indentation here.
Also please delete the trailing white space on two other lines.
This can be done manually or with "astyle --options=astylerc garmin_fit.cc"
@fabzzap Thanks for this. Other users have complained of the same shortcomings in our fit handling, but you are the first to come up with a fix. Can you try the attached patch to your commit?b6ab6ca This resolves all the review issues and a few more with debug messages(qPrintable, appropriate message), except for a test case. Hopefully you can supply a test case. I will squash this PR when I merge, so you can just add commits with doing a forced push. Thanks again. |
I applied the patch to my tree. It seems to work well, Thank you |
@fabzzap This looks great. Can you supply a fit file that includes the kIdLocations entries, including kFieldLocationName and/or kFieldLocationDescription ? I would like to add this to our regression test suite. If so can you also tell me the model of the device that generated the fit file? Thanks |
Another user contributed a fit file for regression test. I will add a test after this PR merges. Thanks for fixing this issue. |
Anyway, I pushed a reference Locations.fit (taken from a Garmin Edge Explore, manually edited to remove sensitive locations), and a test case to this branch |
Wow, your commit just missed the merge. I understand the file can contain sensitive information. Your file gives slightly more coverage (description), but it is nice to have fit files straight from the device in case we discover oddities in the future. I think we are good with the file from the other user. Thanks for your efforts. |
This fixed #719 |
commit 40429dd Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Dec 6 16:37:53 2021 -0700 fix dotnet time conversion (#784) commit 54b0136 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Dec 6 07:13:44 2021 -0700 Kill CarteSurTable format. (#783) Apparently it hasn't ever been used. Our sample has coordinates in UTM, but we have been interpretting them as degrees/100000. "Le pic du midi" referred to in the sample is The Pic du Midi de Bigorre. commit 1ed5895 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Dec 5 15:30:05 2021 -0700 harden gbfgetcstr, gbfgetpstr. (#782) Hangs were detected with afl and the gdb reader and gbfgetcstr. One such hang reads a fuzzed 32 bit url count with FREAD_i32, and then proceeds to fetch that many strings with FREAD_CSTR_AS_QSTR. This results in fetches from beyond EOF that are undetected. gbfgetcstr_old is modified to throw a fatal error if reading is attempted beyond EOF. A similar failure was not detected with gbfgetpstr, but it is also modified to throw a fatal error if reading is attempted beyond EOF. commit 85502cb Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Dec 3 07:16:02 2021 -0700 hopefully the real fix for garmin_xt tool issues. (#781) Fix LGTM detected "Comparison between i of type uint8_t and Count of wider type int." This latest find by LGTM is one in a long series of issue detections by various tools. However, close analysis reveals the previous solutions to these deteced issues didn't find or fix the real root issues: commit 233f3c8, 10/22/2013, Pad internal buffer to appease -fsanitize=address. While the fix appeased the sanitizer, it didn't fix the root problem. The root problem was a bug in the original translation of format_garmin_xt_decrypt_trk_blk when converting to zero-based array indexing. commit dcf0dd8, 7/8/2015, A bunch of busy work to satisfy hyperactive warnings in newer GCC builds. Again, the fix didn't fix the root problem. The root problem was a bug in the original translation of format_garmin_xt_proc_strk. This bug resulted in a mismatch between our test output and that of the original project. In our output the timestamp of the last two points in a track was identical. This fix results in our output matching the reference file of the original project (with allowances for precision). commit 727df52 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 17:17:40 2021 -0700 retire cet (#777) * teach mkshort to split utf8 on grapheme boundaries. and retire some cet functions. * add the grapheme test. * wean gbfile from cet. * restore cet.cc, cet.h in prepare for deprecation. * retire cet.cc, cet.h * drop attempted support of \r line endings in gbfgetutf16str. * add test of surrogate pairs for gbfgetutf16str. * fix test commit 2371d11 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 13:11:33 2021 -0700 add lgtm configuration file (#780) commit 1682055 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 12:40:38 2021 -0700 move Qt6 CI builds to 6.2.2. (#779) commit d377ef6 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 06:26:08 2021 -0700 convert arcdist, polygon, and xcsv to use gpsbabel::TextStream instead of gbfgetstr (#778) * read style files with a textstream. * use gpsbabel::textstream instead of gbfile. commit f2da18d Merge: 41cdf06 2341445 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Tue Nov 30 15:46:26 2021 -0700 Merge pull request #776 from tsteven4/nits3 miscellaneous minor fixes III commit 41cdf06 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Nov 29 07:29:41 2021 -0700 move Qt 5.12 CI builds to 5.12.12 (#774) commit 2341445 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 28 07:44:28 2021 -0700 remove support for non standard entity replacements. This was disabled in d47e364 August 29, 2005. commit dd56f42 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sat Nov 27 23:37:59 2021 -0700 fix lgtm detected ill-defined for loop. commit 59a325d Merge: 56c86e2 0769479 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sat Nov 27 17:45:30 2021 -0700 Merge pull request #772 from tsteven4/lgtm miscellaneous minor fixes commit 0769479 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 26 14:42:42 2021 -0700 sequester temporary style files in TMPDIR commit 3f38cef Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Thu Nov 25 10:09:04 2021 -0700 prefer rte_waypt_ct() over waypoint_list.count(). this provides some insulation from the underlying WaypointList container. commit bdd7865 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Thu Nov 25 08:37:57 2021 -0800 sort SOURCES and HEADERS in build. this gives a reasonable initial project view in Xcode. this gives a resaonable order to compilation. commit dd735b6 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 11:11:15 2021 -0800 minor correction to resample doc. commit f7d3d60 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 10:30:58 2021 -0800 further minimize xcsv_get_char_from_constant_table commit f483fc3 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 09:51:14 2021 -0800 use list initialization to construct xcsv char map table. commit d62f479 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 08:52:22 2021 -0800 more nits. commit b2957b5 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Tue Nov 23 16:09:14 2021 -0800 limit variable scope, found by LGTM. Poor global variable name 'ops'. Prefer longer, descriptive names for globals (eg. kMyGlobalConstant, not foo). commit 56c86e2 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Tue Nov 23 09:40:30 2021 -0700 fix parsing of nmea hms fields. (#770) commit fa20976 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 21 14:04:55 2021 -0700 Introduce resampling filter and nvectors. (#763) * Introduce resampling filter and nvectors. * use ellipsoid model with nvectors. the code had been forcing a spherical earth model to match the existing grtcirc.cc calculations. commit 1a422a6 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 21 14:04:03 2021 -0700 improve string handling, fix 9 year old FIXME. (#769) commit e09f7fe Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 21 13:34:23 2021 -0700 support generation of xcode project files with qmake. (#760) commit ff76e25 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sat Nov 20 08:05:01 2021 -0700 update fit locations test with improved sample (#767) * update fit test with improved sample. * reference file mode correction. commit 67ad13d Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 19 06:48:53 2021 -0700 Pick up fit locations test case (#766) * Add support for Locations.fit files * patch by tsteven4 to address review comments * Add a test case for Locations.fit files from Garmin devices Co-authored-by: Fabrizio Gennari <fabrizio.ge@tiscali.it> commit 368ad39 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Thu Nov 18 16:40:49 2021 -0700 add test for fit Locations(Lctns) files. (#765) * add test for fit Locations(Lctns) files. * fix new bug with change in Qt6 behavior for fit. commit 871efe3 Author: fabzzap <fabrizio.ge@tiscali.it> Date: Thu Nov 18 22:07:59 2021 +0100 Add support for Locations.fit files (#762) * Add support for Locations.fit files * patch by tsteven4 to address review comments commit 2504d97 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 17 08:52:08 2021 -0700 Eliminiate redundant route_head data member (#764) eliminate redundant route_head data member rte_waypt_ct, replacing it with a member function rte_waypt_ct() that gets the count from the waypoint_list container. commit 0beebdd Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Nov 8 05:22:30 2021 -0700 kill xmap, xmap2006. (#758) * kill xmap, xmap2006. * dprecate obsolete readme. commit 6593293 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 5 15:13:05 2021 -0600 kill xmapwpt. (#757) note is was wounded previously, see #754. commit 7b6235c Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 5 13:01:00 2021 -0600 add fedora 35 to CI. (#756) commit c3bc9c8 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 5 11:17:45 2021 -0600 serial port names (#755) * try qserialportinfo again. * update docker images with qt serial port. * enhance windows device text to match device manager on windows.
…aggeo, mapsend, nm4, pcx, pocketfms, skyforce, stmsdf, stmwpp, tiger, vpl, vidaone, vitosmt, vitovtt, wfff, and yahoo. (#785) * Remove compegps, cst, g7to2in, gopal, gpsutil, jtr, maggeo, mapsend, nm4, pcx, pocketfms, skyforce, stmsdf, stmwpp, tiger, vpl, wfff, and yahoo. * Missing reference files from above. * Change classic-3 test to read gzipped NMEA instead of deprecated formats. * Drop gpsutil from tomtom test as it tested nothing unique. * Remove gpsman test. It's a unicsv test and this tests nothing of value. * Eliminate use of gpsutil in gpx test; it covered no unique paths. * Update other GPSBabel.pro :-( * Update Cmaeklists enough for successful Mac compile * Cleanup internal doc references to newly retired formats. * Scrub (and commit) the remaining doc changes for what we just deleted. * Squashed commit of the following: commit 40429dd Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Dec 6 16:37:53 2021 -0700 fix dotnet time conversion (#784) commit 54b0136 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Dec 6 07:13:44 2021 -0700 Kill CarteSurTable format. (#783) Apparently it hasn't ever been used. Our sample has coordinates in UTM, but we have been interpretting them as degrees/100000. "Le pic du midi" referred to in the sample is The Pic du Midi de Bigorre. commit 1ed5895 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Dec 5 15:30:05 2021 -0700 harden gbfgetcstr, gbfgetpstr. (#782) Hangs were detected with afl and the gdb reader and gbfgetcstr. One such hang reads a fuzzed 32 bit url count with FREAD_i32, and then proceeds to fetch that many strings with FREAD_CSTR_AS_QSTR. This results in fetches from beyond EOF that are undetected. gbfgetcstr_old is modified to throw a fatal error if reading is attempted beyond EOF. A similar failure was not detected with gbfgetpstr, but it is also modified to throw a fatal error if reading is attempted beyond EOF. commit 85502cb Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Dec 3 07:16:02 2021 -0700 hopefully the real fix for garmin_xt tool issues. (#781) Fix LGTM detected "Comparison between i of type uint8_t and Count of wider type int." This latest find by LGTM is one in a long series of issue detections by various tools. However, close analysis reveals the previous solutions to these deteced issues didn't find or fix the real root issues: commit 233f3c8, 10/22/2013, Pad internal buffer to appease -fsanitize=address. While the fix appeased the sanitizer, it didn't fix the root problem. The root problem was a bug in the original translation of format_garmin_xt_decrypt_trk_blk when converting to zero-based array indexing. commit dcf0dd8, 7/8/2015, A bunch of busy work to satisfy hyperactive warnings in newer GCC builds. Again, the fix didn't fix the root problem. The root problem was a bug in the original translation of format_garmin_xt_proc_strk. This bug resulted in a mismatch between our test output and that of the original project. In our output the timestamp of the last two points in a track was identical. This fix results in our output matching the reference file of the original project (with allowances for precision). commit 727df52 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 17:17:40 2021 -0700 retire cet (#777) * teach mkshort to split utf8 on grapheme boundaries. and retire some cet functions. * add the grapheme test. * wean gbfile from cet. * restore cet.cc, cet.h in prepare for deprecation. * retire cet.cc, cet.h * drop attempted support of \r line endings in gbfgetutf16str. * add test of surrogate pairs for gbfgetutf16str. * fix test commit 2371d11 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 13:11:33 2021 -0700 add lgtm configuration file (#780) commit 1682055 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 12:40:38 2021 -0700 move Qt6 CI builds to 6.2.2. (#779) commit d377ef6 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Dec 1 06:26:08 2021 -0700 convert arcdist, polygon, and xcsv to use gpsbabel::TextStream instead of gbfgetstr (#778) * read style files with a textstream. * use gpsbabel::textstream instead of gbfile. commit f2da18d Merge: 41cdf06 2341445 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Tue Nov 30 15:46:26 2021 -0700 Merge pull request #776 from tsteven4/nits3 miscellaneous minor fixes III commit 41cdf06 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Nov 29 07:29:41 2021 -0700 move Qt 5.12 CI builds to 5.12.12 (#774) commit 2341445 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 28 07:44:28 2021 -0700 remove support for non standard entity replacements. This was disabled in d47e364 August 29, 2005. commit dd56f42 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sat Nov 27 23:37:59 2021 -0700 fix lgtm detected ill-defined for loop. commit 59a325d Merge: 56c86e2 0769479 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sat Nov 27 17:45:30 2021 -0700 Merge pull request #772 from tsteven4/lgtm miscellaneous minor fixes commit 0769479 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 26 14:42:42 2021 -0700 sequester temporary style files in TMPDIR commit 3f38cef Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Thu Nov 25 10:09:04 2021 -0700 prefer rte_waypt_ct() over waypoint_list.count(). this provides some insulation from the underlying WaypointList container. commit bdd7865 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Thu Nov 25 08:37:57 2021 -0800 sort SOURCES and HEADERS in build. this gives a reasonable initial project view in Xcode. this gives a resaonable order to compilation. commit dd735b6 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 11:11:15 2021 -0800 minor correction to resample doc. commit f7d3d60 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 10:30:58 2021 -0800 further minimize xcsv_get_char_from_constant_table commit f483fc3 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 09:51:14 2021 -0800 use list initialization to construct xcsv char map table. commit d62f479 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 24 08:52:22 2021 -0800 more nits. commit b2957b5 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Tue Nov 23 16:09:14 2021 -0800 limit variable scope, found by LGTM. Poor global variable name 'ops'. Prefer longer, descriptive names for globals (eg. kMyGlobalConstant, not foo). commit 56c86e2 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Tue Nov 23 09:40:30 2021 -0700 fix parsing of nmea hms fields. (#770) commit fa20976 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 21 14:04:55 2021 -0700 Introduce resampling filter and nvectors. (#763) * Introduce resampling filter and nvectors. * use ellipsoid model with nvectors. the code had been forcing a spherical earth model to match the existing grtcirc.cc calculations. commit 1a422a6 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 21 14:04:03 2021 -0700 improve string handling, fix 9 year old FIXME. (#769) commit e09f7fe Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sun Nov 21 13:34:23 2021 -0700 support generation of xcode project files with qmake. (#760) commit ff76e25 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Sat Nov 20 08:05:01 2021 -0700 update fit locations test with improved sample (#767) * update fit test with improved sample. * reference file mode correction. commit 67ad13d Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 19 06:48:53 2021 -0700 Pick up fit locations test case (#766) * Add support for Locations.fit files * patch by tsteven4 to address review comments * Add a test case for Locations.fit files from Garmin devices Co-authored-by: Fabrizio Gennari <fabrizio.ge@tiscali.it> commit 368ad39 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Thu Nov 18 16:40:49 2021 -0700 add test for fit Locations(Lctns) files. (#765) * add test for fit Locations(Lctns) files. * fix new bug with change in Qt6 behavior for fit. commit 871efe3 Author: fabzzap <fabrizio.ge@tiscali.it> Date: Thu Nov 18 22:07:59 2021 +0100 Add support for Locations.fit files (#762) * Add support for Locations.fit files * patch by tsteven4 to address review comments commit 2504d97 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Wed Nov 17 08:52:08 2021 -0700 Eliminiate redundant route_head data member (#764) eliminate redundant route_head data member rte_waypt_ct, replacing it with a member function rte_waypt_ct() that gets the count from the waypoint_list container. commit 0beebdd Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Mon Nov 8 05:22:30 2021 -0700 kill xmap, xmap2006. (#758) * kill xmap, xmap2006. * dprecate obsolete readme. commit 6593293 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 5 15:13:05 2021 -0600 kill xmapwpt. (#757) note is was wounded previously, see #754. commit 7b6235c Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 5 13:01:00 2021 -0600 add fedora 35 to CI. (#756) commit c3bc9c8 Author: tsteven4 <13596209+tsteven4@users.noreply.github.com> Date: Fri Nov 5 11:17:45 2021 -0600 serial port names (#755) * try qserialportinfo again. * update docker images with qt serial port. * enhance windows device text to match device manager on windows. * gbfile test can't depend on PCX any longer.\n Might be a hole in coverage. * Remove vitosmt and vitovtt. * Remove jogmap vidaone yahoo * Remove gpsman. * Remove more crumbs of g7towin. * Run automated cleanup on the list of files in commit message to clean up last pieces. * keep gpsutil.h in GPSBabel.pro Co-authored-by: Robert Lipe <robertlipe@gmail.com>
See issue #719
6 out of 9 fields are supported. The others are unknown, except the location icon field, which is not especially interesting