Implement FSSSignalDiscovered in EDDN plugin #1586

chennin · 2022-06-12T20:11:28Z

Re:

This version 1 of extending the EDMC EDDN plugin to send FSSSignalDiscovered. Open for discussion. It does send OK, but of course schema validation fails as the schema isn't deployed anywhere; and beta is not currently up.

FSSSignalDiscovered are batched until a non-FSSSignalDiscovered event is encountered, which is possibly naive. Is it OK?
Is line 1386 not paranoid enough? It re-checks SystemAddress but only if SystemAddress is in the non-FSSSignalDiscovered that triggered batch sending
The event is dropped if we don't know system/starpos at the time. Can we get that info? Is it a good idea to save the messages til we get it?
SystemAddress is removed from the signals and re-added to the batched message. So, this will actually fail the schema validation as currently defined. Discuss?

Example of a generated message:

{
  "$schemaRef": "https://eddn.edcd.io/schemas/fsssignaldiscovered/1",
  "message": {
    "event": "FSSSignalDiscovered",
    "signals": [
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "Howe Landing"
      },
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "Bartolomeu de Gusmao Vision",
        "IsStation": true
      },
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "Camelot Fortification Group"
      },
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "ACA-588 Thomas-class Hauler"
      },
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "Primary Tourism Agency"
      },
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "Gunter Orbital"
      },
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "Inferno Heights"
      },
      {
        "timestamp": "2022-06-12T02:25:25Z",
        "event": "FSSSignalDiscovered",
        "SignalName": "$MULTIPLAYER_SCENARIO42_TITLE;"
      }
    ]
  },
  "horizons": true,
  "odyssey": true,
  "StarSystem": "Acani",
  "SystemAddress": 2621867788651,
  "StarPos": [
    189.90625,
    -173.96875,
    9.96875
  ]
}

merge Develop in

plugins/eddn.py

Athanasius · 2022-06-12T21:36:03Z

plugins/eddn.py

+            logger.warning("USSType is $USS_Type_MissionTarget;, dropping")
+            return 'Dropping $USS_Type_MissionTarget;'
+        # Can check SystemAddress here, but we'll remove it from this signal list, to be added to the batch
+        if this.systemaddress is None or this.systemaddress != entry['SystemAddress']:


Now this is a case for having the augmentations be optional (but that needs discussing over on EDCD/EDDN#154 .

My thinking here is that it's better to have all the FSSSignalDiscovered events for a system, even if quirks of them mean we can't always have more than the SystemAddress for them.

Athanasius · 2022-06-12T21:42:38Z

Whilst the proposed method of "queue events until a different event is seen" might work out OK, it is going to result in often seeing two batches of events per system.

Also, have you investigated to be sure that any following event will definitely be an appropriate one, with system information ? On a cursory check I can see that it can at least be any of: Location, FSDJump and FSSDiscoveryScan. Given the somewhat asynchronous nature of FSSSignalDiscovered events I'd want to be sure there are no other variants.

I am more inclined to push for Listeners just accepting that we send the FSSSignalDiscovered events as-is (in batches), without augmentations.

chennin · 2022-06-12T22:25:11Z

For following events I also get "SupercruiseExit", "SupercruiseEntry", "ScanBaryCentre", "Scan", "CodexEntry" and more with SystemAddress, but I do also get "bad" ones like Music and ReceiveText [1].

I think USSes spawning at any time means that FSSSignalDiscovered can appear anywhere in the journal. So it sounds like we should send without augmentation if we don't want to lose signals. This does also mean that some batches would have a size of 1.

Or, idea: wait to send the batch until we have enough information, which may mean waiting.. but could also lose data. e.g. [2]

[1]

Journal.220506155830.01.log:{ "timestamp":"2022-05-06T21:32:30Z", "event":"FSSSignalDiscovered", "SystemAddress":669880100121, "SignalName":"SATANIST J1G-GHN", "IsStation":true }
Journal.220506155830.01.log-{ "timestamp":"2022-05-06T21:32:30Z", "event":"Music", "MusicTrack":"Unknown_Exploration" }

Journal.210923103403.01.log:{ "timestamp":"2021-09-23T14:52:51Z", "event":"FSSSignalDiscovered", "SystemAddress":1733119939282, "SignalName":"$USS_DegradedEmissions;", "SignalName_Localised":"Unidentified signal source", "USSType":"$USS_Type_Salvage;", "USSType_Localised":"Degraded emissions", "SpawningState":"$FactionState_None;", "SpawningState_Localised":"None", "SpawningFaction":"$faction_none;", "SpawningFaction_Localised":"None", "ThreatLevel":0, "TimeRemaining":1107.540527 }
Journal.210923103403.01.log-{ "timestamp":"2021-09-23T14:52:54Z", "event":"ReceiveText", "From":"UMBRELLACORP", "Message":"to define started you have to define ownership of territory", "Channel":"starsystem" }
Journal.210923103403.01.log:{ "timestamp":"2021-09-23T14:52:57Z", "event":"FSSSignalDiscovered", "SystemAddress":1733119939282, "SignalName":"$USS_DegradedEmissions;", "SignalName_Localised":"Unidentified signal source", "USSType":"$USS_Type_Salvage;", "USSType_Localised":"Degraded emissions", "SpawningState":"$FactionState_None;", "SpawningState_Localised":"None", "SpawningFaction":"$faction_none;", "SpawningFaction_Localised":"None", "ThreatLevel":0, "TimeRemaining":1026.007202 }
Journal.210923103403.01.log-{ "timestamp":"2021-09-23T14:53:04Z", "event":"ReceiveText", "From":"Julian McCoy", "Message":"Oh don't you start.", "Channel":"starsystem" }

[2]

{ "timestamp":"2021-09-23T14:53:10Z", "event":"FSSSignalDiscovered", "SystemAddress":1733119939282, "SignalName":"$USS_DegradedEmissions;", "SignalName_Localised":"Unidentified signal source", "USSType":"$USS_Type_Salvage;", "USSType_Localised":"Degraded emissions", "SpawningState":"$FactionState_None;", "SpawningState_Localised":"None", "SpawningFaction":"$faction_none;", "SpawningFaction_Localised":"None", "ThreatLevel":0, "TimeRemaining":975.480469 }
{ "timestamp":"2021-09-23T14:53:11Z", "event":"ReceiveText", "From":"Julian McCoy", "Message":"Have fun Wolfe.", "Channel":"starsystem" }
{ "timestamp":"2021-09-23T14:54:04Z", "event":"Music", "MusicTrack":"Supercruise" }
{ "timestamp":"2021-09-23T14:54:06Z", "event":"ReceiveText", "From":"Julian McCoy", "Message":"It was /supposed/ to be Veliaze in 2849.", "Channel":"starsystem" }
{ "timestamp":"2021-09-23T14:54:23Z", "event":"Shutdown" }

Athanasius · 2022-06-13T12:39:59Z

For following events I also get "SupercruiseExit", "SupercruiseEntry", "ScanBaryCentre", "Scan", "CodexEntry" and more with SystemAddress, but I do also get "bad" ones like Music and ReceiveText [1].

I think USSes spawning at any time means that FSSSignalDiscovered can appear anywhere in the journal. So it sounds like we should send without augmentation if we don't want to lose signals. This does also mean that some batches would have a size of 1.

Or, idea: wait to send the batch until we have enough information, which may mean waiting.. but could also lose data. e.g. [2]

It's not a matter of "until we have enough information". Once we're in a system then the FSDJump, CarrierJump or Location event will have given that augmentation information.

The only concern with newly popped USS is how often they occur and how often we'd end up sending a batch with only a single signal in it.

We don't need to batch perfectly, just enough to not spam so many messages through EDDN, as we would if sending literally each event in its own message (and there's this thing with them preceding the arrival in system).

Athanasius · 2022-06-13T16:43:23Z

So, this code will definitely need to both be tracking "prior events" location and checking if the new non-FSSSignalDiscovered event is a "Location" one.

In Odyssey when you login in space you get the signals and then the Location event.

In Horizons when you login in space you get the Location event and then the signals.

Athanasius · 2022-06-13T21:27:25Z

plugins/eddn.py

@@ -1657,6 +1745,18 @@ def journal_entry(  # noqa: C901, CCR001
        #     drop those events, or if the schema allows, send without those
        #     augmentations.

+        elif event_name == 'fsssignaldiscovered':


Note to self: Will need to check if we have any signals queued up when a new Location/FSDJump/CarrierJump comes in, cross-check SystemAddress, send.

Keep in mind that in Horizons the Location/FSDJump comes before the new system's signals, in Odyssey it's after.

Athanasius · 2022-06-15T12:13:46Z

Thanks for the initial work, I've cloned your branch and am now beating it into shape, see #1590 . Closing this.

chennin added 5 commits June 12, 2022 15:21

Merge pull request #3 from EDCD/develop

e257dad

merge Develop in

Implement FSSSignalDiscovered sender v1

df9f0bf

fix flake8 errors

1b37991

fix more flake8 errors

4dfcd91

remove unnecessarily committed debug messages

4bf2965

Athanasius reviewed Jun 12, 2022

View reviewed changes

plugins/eddn.py Outdated Show resolved Hide resolved

Athanasius reviewed Jun 12, 2022

View reviewed changes

Athanasius mentioned this pull request Jun 12, 2022

FSSSignalDiscovered event support EDCD/EDDN#154

Merged

Change this log to debug level

afda168

Athanasius reviewed Jun 13, 2022

View reviewed changes

Athanasius mentioned this pull request Jun 15, 2022

EDDN: Implement fsssignaldiscovered/1 schema #1590

Merged

Athanasius closed this Jun 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement FSSSignalDiscovered in EDDN plugin #1586

Implement FSSSignalDiscovered in EDDN plugin #1586

chennin commented Jun 12, 2022 •

edited

Loading

Athanasius Jun 12, 2022

Athanasius commented Jun 12, 2022

chennin commented Jun 12, 2022 •

edited

Loading

Athanasius commented Jun 13, 2022

Athanasius commented Jun 13, 2022

Athanasius Jun 13, 2022

Athanasius commented Jun 15, 2022

Implement FSSSignalDiscovered in EDDN plugin #1586

Implement FSSSignalDiscovered in EDDN plugin #1586

Conversation

chennin commented Jun 12, 2022 • edited Loading

Athanasius Jun 12, 2022

Choose a reason for hiding this comment

Athanasius commented Jun 12, 2022

chennin commented Jun 12, 2022 • edited Loading

Athanasius commented Jun 13, 2022

Athanasius commented Jun 13, 2022

Athanasius Jun 13, 2022

Choose a reason for hiding this comment

Athanasius commented Jun 15, 2022

chennin commented Jun 12, 2022 •

edited

Loading

chennin commented Jun 12, 2022 •

edited

Loading