Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameter for max speed threshold #230

Merged
merged 1 commit into from
Dec 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,7 @@ PHONE_LOCATIONS:
MINIMUM_DAYS_TO_DETECT_HOME_CHANGES: 3
CLUSTERING_ALGORITHM: DBSCAN # DBSCAN, OPTICS
RADIUS_FOR_HOME: 100
THRESHOLD_MAX_SPEED: 250 #km/h; set to 0 to disable
SRC_SCRIPT: src/features/phone_locations/doryab/main.py

BARNETT:
Expand Down
1 change: 1 addition & 0 deletions docs/change-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- Add parameter to exclude rows with implausible step count values from Fitbit steps intraday feature computation
- Update tests for Fitbit steps intraday features
- Fix bug of None/NULL values in parsed Fitbit heartrate summary JSON data
- Add parameter to PHONE_LOCATIONS DORYAB provider to exclude rows of locations data with implausible speed values from feature computation

## v1.9.4

Expand Down
1 change: 1 addition & 0 deletions docs/features/phone-locations.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ Parameters description for `[PHONE_LOCATIONS][PROVIDERS][DORYAB]`:
|`[MINIMUM_DAYS_TO_DETECT_HOME_CHANGES]` | The minimum number of consecutive days a new home location candidate has to repeat before it is considered the participant's new home. This parameter will be used only when `[INFER_HOME_LOCATION_STRATEGY]` is set to `SUN_LI_VEGA_STRATEGY`.
| `[CLUSTERING_ALGORITHM]` | The original Doryab et al. implementation uses `DBSCAN`, `OPTICS` is also available with similar (but not identical) clustering results and lower memory consumption.
| `[RADIUS_FOR_HOME]` | All location coordinates within this distance (meters) from the home location coordinates are considered a homestay (see `timeathome` feature).
| `[THRESHOLD_MAX_SPEED]` | Any rows of locations data with calculated speed greater than this threshold value in km/hr will be dropped prior to feature computation. Set to 0 to disable and retain all rows.


Features description for `[PHONE_LOCATIONS][PROVIDERS][DORYAB]`:
Expand Down
6 changes: 6 additions & 0 deletions src/features/phone_locations/doryab/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ def doryab_features(sensor_data_files, time_segment, provider, filter_data_by_se
cluster_on = provider["CLUSTER_ON"]
clustering_algorithm = provider["CLUSTERING_ALGORITHM"]
radius_from_home = provider["RADIUS_FOR_HOME"]
threshold_max_speed = provider["THRESHOLD_MAX_SPEED"]

if provider["MINUTES_DATA_USED"]:
requested_features.append("minutesdataused")
Expand All @@ -136,10 +137,15 @@ def doryab_features(sensor_data_files, time_segment, provider, filter_data_by_se
# the subset of requested features this function can compute
features_to_compute = list(set(requested_features) & set(base_features_names))

# if not disabled (threshold_max_speed=0), drop any rows of data where speed is greater than the specified value in km/h prior to feature computation
if threshold_max_speed > 0:
location_data = location_data.drop(location_data[location_data.speed > threshold_max_speed].index)

location_data = apply_cluster_strategy(location_data, time_segment, clustering_algorithm, dbscan_eps, dbscan_minsamples, cluster_on, filter_data_by_segment)

if location_data.empty:
return pd.DataFrame(columns=["local_segment"] + features_to_compute)

location_features = pd.DataFrame()

# update distance after chunk_episodes() function
Expand Down
1 change: 1 addition & 0 deletions tests/settings/mtz_event_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ PHONE_LOCATIONS:
MINIMUM_DAYS_TO_DETECT_HOME_CHANGES: 3
CLUSTERING_ALGORITHM: DBSCAN # DBSCAN, OPTICS
RADIUS_FOR_HOME: 100
THRESHOLD_MAX_SPEED: 0 #km/h; set to 0 to disable
SRC_SCRIPT: src/features/phone_locations/doryab/main.py

BARNETT:
Expand Down
1 change: 1 addition & 0 deletions tests/settings/mtz_frequency_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ PHONE_LOCATIONS:
MINIMUM_DAYS_TO_DETECT_HOME_CHANGES: 3
CLUSTERING_ALGORITHM: DBSCAN # DBSCAN, OPTICS
RADIUS_FOR_HOME: 100
THRESHOLD_MAX_SPEED: 0 #km/h; set to 0 to disable
SRC_SCRIPT: src/features/phone_locations/doryab/main.py

BARNETT:
Expand Down
1 change: 1 addition & 0 deletions tests/settings/mtz_periodic_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ PHONE_LOCATIONS:
MINIMUM_DAYS_TO_DETECT_HOME_CHANGES: 3
CLUSTERING_ALGORITHM: DBSCAN # DBSCAN, OPTICS
RADIUS_FOR_HOME: 100
THRESHOLD_MAX_SPEED: 0 #km/h; set to 0 to disable
SRC_SCRIPT: src/features/phone_locations/doryab/main.py

BARNETT:
Expand Down
1 change: 1 addition & 0 deletions tests/settings/stz_event_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ PHONE_LOCATIONS:
MINIMUM_DAYS_TO_DETECT_HOME_CHANGES: 3
CLUSTERING_ALGORITHM: DBSCAN # DBSCAN, OPTICS
RADIUS_FOR_HOME: 100
THRESHOLD_MAX_SPEED: 0 #km/h; set to 0 to disable
SRC_SCRIPT: src/features/phone_locations/doryab/main.py

BARNETT:
Expand Down
1 change: 1 addition & 0 deletions tests/settings/stz_frequency_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ PHONE_LOCATIONS:
MINIMUM_DAYS_TO_DETECT_HOME_CHANGES: 3
CLUSTERING_ALGORITHM: DBSCAN # DBSCAN, OPTICS
RADIUS_FOR_HOME: 100
THRESHOLD_MAX_SPEED: 0 #km/h; set to 0 to disable
SRC_SCRIPT: src/features/phone_locations/doryab/main.py

BARNETT:
Expand Down
1 change: 1 addition & 0 deletions tests/settings/stz_periodic_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ PHONE_LOCATIONS:
MINIMUM_DAYS_TO_DETECT_HOME_CHANGES: 3
CLUSTERING_ALGORITHM: DBSCAN # DBSCAN, OPTICS
RADIUS_FOR_HOME: 100
THRESHOLD_MAX_SPEED: 0 #km/h; set to 0 to disable
SRC_SCRIPT: src/features/phone_locations/doryab/main.py

BARNETT:
Expand Down
3 changes: 3 additions & 0 deletions tools/config.schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -751,6 +751,9 @@ properties:
RADIUS_FOR_HOME:
type: integer
exclusiveMinimum: 0
THRESHOLD_MAX_SPEED:
type: integer
minimum: 0

BARNETT:
allOf:
Expand Down
Loading