Skip to content

Conversation

@Keymasterer44
Copy link
Collaborator

This PR adds a new DataExtension file at NeoModLoader.General.Game.extensions with the classes DataExtension, SerializedCustomData, ICustomData, and BasicCustomData<TDataClass>, including documentation for all public members. DataExtension contains two extension methods applied to BaseSystemData which can be used to serialize/deserialize any object supporting our ICustomData interface to and from the custom_data_string Dictionary offered by BaseSystemData via the SerializedCustomData wrapping class by NML, storing mod ID and data version alongside the custom data to make support for changes in data format easy to detect. The extension methods can accomplish this by using ICustomData.Serialize() and ICustomData.Deserialize(), which are expected to be able to both export and import the stored custom data to/from a JObject stored within the SerializedCustomData.
Additionally, there's a BasicCustomData<TDataClass> for enabling modders to get a quick basic grasp on how to interface with the system. However, actually using this example implementation in distributed mods is discouraged, as the default system for serialization/deserialization completely lacks support for versioning, making it incredibly difficult to adapt to changes in data storage with it (much more so than with a well-written custom ICustomData implementation).
All a mod has to do to leverage the feature is to create its own ICustomData implementation and to use the relevant extension methods from DataExtension.

@Keymasterer44 Keymasterer44 added documentation Improvements or additions to documentation enhancement New feature or request labels Apr 20, 2025
@Keymasterer44 Keymasterer44 requested a review from inmny April 20, 2025 20:43
@Keymasterer44
Copy link
Collaborator Author

The following code was used to test the feature:

Data Classes:

class ExampleData {
  public class NestedExampleData {
    public int nestedTest;
  }
  public string test;
  public NestedExampleData nesting = new NestedExampleData();
}

Runtime Code:

LogInfo("Testing custom data thing");
ActorData testingData = new ActorData();
BasicCustomData<ExampleData> testingCustomData = new BasicCustomData<ExampleData> {
  Data = {
    test = "test",
    nesting = {
      nestedTest = 6
    }
  }
};
testingData.Set("custom_data_test", testingCustomData);
if (!testingData.TryGet("custom_data_test", out BasicCustomData<ExampleData> deserializedCustomData)) {
  LogError("Failed to deserialize custom data for unclear reason!");
  return;
}
LogInfo($"Deserialized custom data: {deserializedCustomData.Data.test}, {deserializedCustomData.Data.nesting.nestedTest}");

When running the code, both the test field and the nestedTest fields retained their original values between saving and loading.

Copy link
Contributor

@inmny inmny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to cache the data object with ConditionalWeakTable to support read/write without (de)serialize. In addition, it also supports change value without "Set" manually.
It can be implemented by serializing only in BaseSystemData.save and check cached deserialized value first in TryGet

Example code:

public abstract class BasicCustomDataBase
{
    public abstract Type Type {get;}
    public abstarct string Key {get;}
    protected object wrapped_data;
}
public class BasicCustomData<T> : BasicCustomDataBase
{
     public override Type Type => typeof(T);
     public override string Key => typeof(T).FullName;
     public readonly T Data;
     public BasicCustomData(T data)
     {
          wrapped_data = data;
          Data = data;
     }
}
private static ConditionalWeakTable<BaseSystemData, Dictionary<Type, BasicCustomDataBase>> _cache_table = new();
private static void BaseSystemData_save_prefix(BaseSystemData __instance)
{
     if (!_cache_table.TryGetValue(__instance, out var data_dict) return;
     foreach (var item in data_dict)
     {
           __instance.set(item.Value.Key, Serialize(item.Value.wrapped_data, item.Key));
     }
}
public static TData Get<TData>(this BaseSystemData data)
{
     if (_cache_table.TryGetValue(data, out var data_dict) 
     {
           if (data_dict.TryGetValue(typeof(TData), out var t_data)
           {
                   return t_data.wrapped_data as TData;
           }
           goto DESERIALIZE_DATA;
     }
     else
     {
           data_dict = new();
           _cache_table.Add(data, data_dict);
     }
     DESERIALIZE_DATA:
     data.get(typeof(TData).FullName, out string data_str);
     if (string.IsNullOrEmpty(data_str)) return null;
     var deserialized_data = Deserialize<TData>(data_str);
     data_dict[typeof(TData)] = new BasicCustomData(deserialized_data);
     return deserialized_data;
}

@Keymasterer44
Copy link
Collaborator Author

@inmny Honestly, I am not sure if getting rid of the manual set is a good idea. Personally, I'd argue that there can be a lot of benefits in stability to the more "transactional" behaviour of the base game system, as it prevents issues like ending up with half changed data if an exception occurs halfway through a set of changes to a data entry.
This is more of a side point, but the system also remains more easily understandable to people who are used to the base games primitive storages if we keep the usage (Get/Set) as close as possible to the base game design (get/set).

@inmny inmny merged commit c976039 into master Apr 23, 2025
1 of 2 checks passed
@Keymasterer44 Keymasterer44 deleted the keymasterer-add-custom-data-storage branch September 11, 2025 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants