Remove AS uplink storage #6317

cvetkovski98 · 2023-06-08T15:46:40Z

Summary

The AS Uplink Storage is used only for the LoRa Geolocation Services integration. This PR moves its functionality to the GLS association data, adds appropriate methods to the association registry for cleaning association when a device or application link is removed and an appropriate migration for the removed functionality.

This PR also removes the automatic determination of the window size based on the frame payload of the GLS integration and limits the frame size in the [1, 16] range. This can be set through the Console.

Closes #6215

Changes

Deprecate and remove the uplink storage registry
Refactor the multi-frame queries in the GLS integration to use the association data
Add Clear methods to the application registry store.

Testing

UT
Manual

Regressions

...

Notes for Reviewers

...

Checklist

Scope: The referenced issue is addressed, there are no unrelated changes.
Compatibility: The changes are backwards compatible with existing API, storage, configuration and CLI, according to the compatibility commitments in README.md for the chosen target branch.
Documentation: Relevant documentation is added or updated.
Changelog: Significant features, behavior changes, deprecations and fixes are added to CHANGELOG.md.
Commits: Commit messages follow guidelines in CONTRIBUTING.md, there are no fixup commits left.

adriansmares

This is going in the right direction and looks correct so far. My only change to the design is to optimize a bit storage, as saving the field names has no benefit since only the AS has to be able to encode and decode the data.

cmd/ttn-lw-stack/commands/as_db.go

pkg/applicationserver/applicationserver.go

pkg/applicationserver/io/packages/loragls/v3/data.go

pkg/applicationserver/io/packages/redis/registry.go

cmd/ttn-lw-stack/commands/as_db.go

pkg/applicationserver/applicationserver.go

pkg/applicationserver/config.go

pkg/applicationserver/io/packages/loragls/v3/package.go

pkg/applicationserver/io/packages/loragls/v3/structs.go

pkg/applicationserver/io/packages/redis/registry.go

pkg/applicationserver/io/packages/redis/registry_test.go

adriansmares

We should add a changelog entry for these changes. I would say that the migration is the main point to put an emphasis on, and the deprecated configuration options.

pkg/applicationserver/config.go

pkg/applicationserver/io/packages/loragls/v3/package.go

cmd/ttn-lw-stack/commands/as_db.go

adriansmares

LGTM. @johanstokking could you also take a look ?

CHANGELOG.md

johanstokking

So this is only unit tested; are you unable to do a local integration test by purging data as well as seeing if GLS still works?

cmd/ttn-lw-stack/commands/as_db.go

pkg/applicationserver/applicationserver.go

CHANGELOG.md

cmd/ttn-lw-stack/commands/as_db.go

cvetkovski98 linked an issue Jun 8, 2023 that may be closed by this pull request

Remove Application Server uplink storage #6215

Closed

12 tasks

cvetkovski98 changed the title ~~Feature/remove as uplink storage~~ Remove AS uplink storage Jun 8, 2023

github-actions bot added c/application server This is related to the Application Server compat/config This could affect Configuration compatibility compat/db This could affect Database compatibility labels Jun 8, 2023

adriansmares assigned cvetkovski98 Jun 9, 2023

adriansmares added this to the v3.26.2 milestone Jun 9, 2023

adriansmares reviewed Jun 9, 2023

View reviewed changes

cvetkovski98 force-pushed the feature/remove-as-uplink-storage branch 6 times, most recently from 1158ea8 to faf1a09 Compare June 15, 2023 14:04

cvetkovski98 marked this pull request as ready for review June 15, 2023 14:14

cvetkovski98 requested review from johanstokking and KrishnaIyer as code owners June 15, 2023 14:14

cvetkovski98 requested a review from adriansmares June 15, 2023 14:14

adriansmares reviewed Jun 15, 2023

View reviewed changes

cvetkovski98 force-pushed the feature/remove-as-uplink-storage branch 2 times, most recently from f2dd469 to 7af55c9 Compare June 16, 2023 09:58

cvetkovski98 requested a review from adriansmares June 16, 2023 10:10

cvetkovski98 force-pushed the feature/remove-as-uplink-storage branch 2 times, most recently from 3c2aefb to d8e504b Compare June 16, 2023 10:36

adriansmares reviewed Jun 16, 2023

View reviewed changes

pkg/applicationserver/config.go Outdated Show resolved Hide resolved

pkg/applicationserver/io/packages/loragls/v3/package.go Outdated Show resolved Hide resolved

cmd/ttn-lw-stack/commands/as_db.go Outdated Show resolved Hide resolved

cvetkovski98 force-pushed the feature/remove-as-uplink-storage branch from d8e504b to 3311ac4 Compare June 16, 2023 14:36

cvetkovski98 requested a review from adriansmares June 16, 2023 14:46

adriansmares approved these changes Jun 16, 2023

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

CHANGELOG.md Outdated Show resolved Hide resolved

johanstokking reviewed Jun 17, 2023

View reviewed changes

cmd/ttn-lw-stack/commands/as_db.go Outdated Show resolved Hide resolved

pkg/applicationserver/applicationserver.go Outdated Show resolved Hide resolved

CHANGELOG.md Outdated Show resolved Hide resolved

cmd/ttn-lw-stack/commands/as_db.go Outdated Show resolved Hide resolved

cvetkovski98 force-pushed the feature/remove-as-uplink-storage branch from 3311ac4 to bc87da8 Compare June 26, 2023 10:14

cvetkovski98 added 25 commits July 3, 2023 13:46

as,cli: Deprecate the uplink storage registry

530472f

cli: Add cleanup uplink registry command

93dbc2b

as: Move application uplink storage to GLS package association

5fa3f03

as: Add clear methods to application package registry

a5cb706

as: Use gob to serialize recent md in gls pkg data

4bacfb9

as: Cleanup associations on device and app link removal

47578a4

as: Add gls package description

49b1434

as: Add gls package data tests

7bad117

as: Add association registry mock

852c6e4

as: Add association registry clear methods tests

8696b6d

as: Ensure device registry invokes association clear

fc5d60f

as: Add redis pkg description

1261ee3

as: Use updated package data after uplink push

7f75ef7

as: Use locked watch when clearing association registry

aea5c98

as: Improve association registry clear tests

e600580

as: Move GLS struct definitions to api package

c9c6492

as: Remove location source field

df95b56

cli: Rename uplink registry cleanup command to purge

4fbf3c2

all: Update CHANGELOG.md

1567d0c

as: Encode gob bytes as base64

73f40ac

as: Handle possible nil association in GLS integration

c05e224

as: Fix GLS package data merging

00b835a

as: Add constraint to the GLS multi frame query window size

c198ba1

console: Remove checkbox for automatic window size in GLS integration

b731b4e

as: Remove stored recent metadata when using single frame GLS query

57e9872

cvetkovski98 force-pushed the feature/remove-as-uplink-storage branch from a44d2fd to 57e9872 Compare July 3, 2023 11:50

johanstokking approved these changes Jul 4, 2023

View reviewed changes

kschiffer approved these changes Jul 4, 2023

View reviewed changes

cvetkovski98 merged commit 2acf07e into v3.27 Jul 4, 2023

cvetkovski98 deleted the feature/remove-as-uplink-storage branch July 4, 2023 11:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove AS uplink storage #6317

Remove AS uplink storage #6317

cvetkovski98 commented Jun 8, 2023 •

edited

Loading

adriansmares left a comment

adriansmares left a comment

adriansmares left a comment

johanstokking left a comment

Remove AS uplink storage #6317

Remove AS uplink storage #6317

Conversation

cvetkovski98 commented Jun 8, 2023 • edited Loading

Summary

Changes

Testing

Regressions

Notes for Reviewers

Checklist

adriansmares left a comment

Choose a reason for hiding this comment

adriansmares left a comment

Choose a reason for hiding this comment

adriansmares left a comment

Choose a reason for hiding this comment

johanstokking left a comment

Choose a reason for hiding this comment

cvetkovski98 commented Jun 8, 2023 •

edited

Loading