Feat/vip list overhaul

[cemathey]

tl;dr this overhauls the VIP system to be very similar to how blacklists work and handles keeping CRCON in synch with the connected game servers.

VIP Lists

Just like a blacklist; a server can have 1 or more VIP list and players can have 0 or more VIP records.

A record can be active or inactive, and a record can either have an expiration (expires_at), or a null value which will never expire.

A VIP list has a sync method which is either IGNORE_UNKNOWN (the default) or REMOVE_UNKNOWN to handle players with VIP on the game server that aren't on any VIP list. Since some communities use BattleMetrics or other external tooling to manage VIP; we don't want to override those unless they specifically choose to.

If ANY list on the server is set to IGNORE_UNKNOWN then all unknown VIPs on the game server are ignored regardless of the other lists settings because of how we synchronize lists with the game server.

There is a vip_lists service which runs for each game server that listens (just like the blacklists) for redis pub/sub commands when lists or records are added/edited/deleted.

There is now a new cron task that runs every 5-7 minutes (randomized since it will run once per container; but records are global across CRCON, it's probably fine if they run consecutively but this should avoid any potential conflicts) that will inactivate any expired records.

The old expiring VIP service and configuration has been removed because it's redundant.

Old Style VIP Lists

Old style VIP lists can still be imported using the convert_old_style_vip_records endpoint which will parse them and add them to the specified VIP list.

The endpoints/internal code for uploading/downloading lists have been removed; they're no longer necessary and there are endpoints for adding/removing VIP list records in bulk; lists and their records can be exported through the API.

TODO: the UI needs to be updated to allow importing old style VIP lists

Game Server VIP vs. VIP Lists

Unlike blacklists; VIP is meaningless unless it is active on the game server; the game server and a list should never be out of synch since they are synched on CRCON startup, when records are created/edited/deleted and every 5-7 minutes.

But some communities use external (non CRCON) tools to manage or supplement their VIP such as BattleMetrics, so we still need to indicate if a player has VIP on the game server, but also indicate and distinguish when they have VIP from a VIP list.

/api/get_vip_ids will still pull VIP status from the game server but will augment it with the VIP records and return the list with an expires_at field for each player; if they have VIP on the game server but no VIP records it is null or it contains their highest expiration date from all of their VIP records (null or a string timestamp 2025-02-22T19:25:33.186398+00:00)

TODO: the UI needs to be updated to account for this; and the API needs to be updated/tweaked to reflect both of these values

API Endpoint Changes

Modified Endpoints

1. add_vip no longer accepts an expiration parameter (this should now be handled with the VIP list endpoints) and will validate player IDs (steam/win store formats)
2. get_players_by_appearance now accepts is_vip for searching for players with an active and non-expired VIP list record
3. get_players_by_appearance no longer returns the vip_expiration key/value pair

New Endpoints

1. add_vip_list_record
2. bulk_add_vip_list_records
3. bulk_edit_vip_list_records
4. bulk_delete_vip_list_records
5. bulk_add_vips
6. bulk_remove_vips
7. convert_old_style_vip_records
8. create_vip_list
9. delete_vip_list
10. delete_vip_list_record
11. edit_vip_list
12. edit_vip_list_record
13. extend_vip_duration
14. get_active_vip_records
15. get_all_vip_records_for_server
16. get_inactive_vip_records
17. get_player_vip_records
18. get_vip_list
19. get_vip_list_records
20. get_vip_lists
21. get_vip_lists_for_server
22. get_vip_record
23. get_vip_status_for_player_ids
24. inactivate_expired_vip_records
25. revoke_all_vip
26. synchronize_with_game_server

Removed Endpoints

1. describe_expired_vip_config
2. download_vips
3. get_expired_vip_config
4. set_expired_vip_config
5. upload_vips
6. upload_vips_result
7. validate_expired_vip_config

Permission Changes

New Permissions

1. can_view_vip_lists
2. can_create_vip_lists
3. can_change_vip_lists
4. can_delete_vip_lists
5. can_change_vip_list_records
6. can_add_vip_list_records
7. can_change_vip_list_records
8. can_delete_vip_lists_records

Removed Permissions

1. can_change_expired_vip_config
2. can_view_expired_vip_config

Database Changes

The database migration that creates the new tables (vip_list and vip_list_record) creates two lists (IDs of 0 and 1 respectively) for a generic default list; and a Seed VIP list. The list ID of 0 cannot be deleted, people can choose to delete the Seed VIP list.

The player_vip table (previously stored VIP expiration dates for players with VIP on the game server) is deleted and records from it are converted to vip list records and added to the default (ID of 0) VIP list.

Since player_vip stored entries per game server; records are merged and one record is created per player using the highest expiration date they had amongst all the servers.

The PlayerID model (steam_id_64 table) now has email and discord_id columns which are used by the VIP list endpoints to allow communities to better manage their VIP records; as well as support communities that use Discord with their external tools.

1. steam_id_64 table (PlayerID model)
   1. email column added
   2. discord_id column added
   3. vips renamed to vip_lists
   4. vip property removed

Type Changes

PlayerProfileType

1. vips removed
2. vip_lists added
3. is_vip added
4. email added
5. discord_id added.
6. is_vip added (True/False for if the player has VIP (according to VIP list records) on the server the profile was fetched for)

VipIdType

This was renamed to VipIdWithExpirationType

1. expiration renamed to expires_at

Seed VIP

1. SeedVIPUserConfig adds vip_list_id property

Seed VIP uses the new list system and defers game server modifications to the VIP list handler.

The default configuration uses a list ID of 1 (which is created during the database migration).

Since VIP is so important; if the configured list does not exist a new one will be created and their config updated.

Players who earn VIP will either have their existing record on the Seed VIP list updated to their new expiration timestamp, or a new record created if they don't have one.

Upgrade Steps

supervisord.conf

If your supervisord.conf has local modifications:

Remove:

[program:expiring_vips]
command=/code/manage.py expiring_vips
environment=LOGGING_FILENAME=expiring_vips_%(ENV_SERVER_NUMBER)s.log
startretries=10
autorestart=true

Add:

[program:vip_lists]
command=/code/manage.py vip_lists
environment=LOGGING_FILENAME=vip_lists_%(ENV_SERVER_NUMBER)s.log
startretries=100
startsecs=10
autostart=true

crontab

If your crontab has local modifications, add the following job: */5 * * * * /code/manage.py inactivate_expired_vip_records

Update

Then perform a standard upgrade

Sample Data

get_player_profile.json
new_endpoints.json