Feature: property-morphable abstract data

[bentleyo]

This PR adds a new feature where abstract data classes can dynamically choose which concrete variant of the class to use based on the payload. This feature works with validation, collections and nested payloads.

While the existing abstract class functionality is good, I find it to be limiting for our use case as it's only available when casting. I've maintained this existing functionality without any breaking changes.

An example use case is having Vehicle as an abstract data class with type (to determine which concrete to use) and a common max_passengers property used by every class that extends Vehicle. Each concrete can then add its own additional properties e.g.:

"vehicles": [
    { "type": "horse", "max_passengers": 1, "name": "Bill" },
    { "type": "car", "max_passengers": 5, "transmission": "manual" },
]

This is a feature that's been super useful for us (we have it working on v3 using a trait, but that's no longer possible) and it's a feature that I think would be really great to have built-in.

To use the feature you implement the new PropertyMorphableData contract and associated static morph method. This method is supplied with an array of properties and should return a class string.

E.g.

abstract class VehicleData extends Data implements PropertyMorphableData
{
    public function __construct(
        #[In('car', 'horse')]
        public string $type,
    ) {
    }

    public static function morph(array $properties): ?string
    {
        return match ($properties['type'] ?? null) {
            'car' => CarData::class,
            'horse' => HorseData::class,
            default => null,
        };
    }
}

[bentleyo]

The reason the return value for morph is nullable is because we need to know which type of class we're building when generating validation rules. However, at this point not all required properties may be available or supplied.

[Sagmedjo]

Really looking forward to this feature. Is it going to be in an upcoming release?

[rubenvanassche]

Hi @bentleyo,

This is a really cool PR! Would love to merge it but have a few question/suggestions. Once that has been worked out I think this one can be merged.

There are some points I couldn't add to the review:

• What would happen if we transform an morphed class? Of course it will just be converted into an array/json/... but since we also have support for including/excluding properties will that still work? Especially if a property exists in one class but not in another, again the same with nested structures. Tbh, I have no idea how the code would react we haven't taught about this when building the partials system. Some tests here would be really useful.
• At the moment there aren't docs on this new feature, which I'm sure a lot of people want to use. Would be cool to also include this!

Would love to merge this one! As (probably) my child will be born next week I will be on parental leave for four weeks so could be I'm not directly approving it 😄

[rubenvanassche]

I'm not completely sure why we first run the pipeline here with the abstract class and then run it again with the inherited class? That first run seems to be completely unnecessary right?

[bentleyo]

@rubenvanassche this might just be from me misunderstanding how or why there are multiple payloads 😄

In order to know which concrete class to create we need to collect the properties from the payloads and pass them into the morph method. I've made the assumption these properties could come from $payloads[0], $payloads[1] etc. or a combination thereof. It also made sense to me to have the set of properties run through the pipeline so that they're cast / have default values etc.

Does that make sense or do you think I should change the approach there?

[rubenvanassche]

Let's rewrite this to:

Suggested change

	$payload = $path->isRoot() ? $fullPayload : Arr::get($fullPayload, $path->get(), []);
	$class = $class::morph($payload) ?? $class;
	$dataClass = $this->dataConfig->getDataClass($class);
	$morphedClass = $class::morph(
	$path->isRoot() ? $fullPayload : Arr::get($fullPayload, $path->get(), [])
	);
	$dataClass = $this->dataConfig->getDataClass($morphedClass ?? $class);

[bentleyo]

I've updated this 👍

[rubenvanassche]

Let's try to inline these classes as much as possible, the Fakes directory is already sooo big with a lot of specific classes. My goal is trying to eliminate 90% of them one day so that the definitions of the classes are closer to the tests and so that we don't have to come up with more class names 😄.

Feel free to use anonymous classes or inline classes (just prefix the names with Test).

If a class is being used a lot it might be a fit to put in Fakes but most of the cases it isn't. Except for classes with class attributes because they aren't parsed when written inline if I'm correct.

[bentleyo]

@rubenvanassche I've had a go at inlining the classes. Can you have a look and let me know what you think?

Unfortunately it's a bit difficult to work with the abstract classes anonymously as I can't create an instance of them (e.g. new abstract class () wouldn't work). I also wanted to avoid duplicating all the classes I had in CreationTest.php many times (especially as the classes would need unique names), so for now I've created a describe section and put the class definitions at the top. Are you happy with that as a solution?

[bentleyo]

@rubenvanassche thank you for reviewing! I'll respond over the next couple of days. This is a feature I'm willing to put effort into getting right 😊

How exciting! Congratulations 🎉 Enjoy the next few weeks with your family!

[bentleyo]

@rubenvanassche I'm happy to put together some documentation for this feature, but before I do I wanted to point out some potential awkwardness with my current approach and see if you had any thoughts. Sorry for the long message 😅

Potential Problem 1:

At the moment the code depends on the user adding validation rules on their abstract class to ensure that a concrete class can be determined. For example, in ValidationTest.php I have:

abstract class TestValidationAbstractPropertyMorphableData extends Data implements PropertyMorphableData
{
    public function __construct(
        #[In('a', 'b')]
        public string $variant,
    ) {
    }

    public static function morph(array $properties): ?string
    {
        return match ($properties['variant'] ?? null) {
            'a' => TestValidationPropertyMorphableDataA::class,
            'b' => TestValidationPropertyMorphableDataB::class,
            default => null,
        };
    }
}

It is the In validation that ensures a concrete can be determined. If you provided { "variant": "c" } as data it would pass validation, but then immediately fail on creation because it wouldn't be able to determine which concrete class to use.

Potential Problem 2:

The morph method gets called from two places. On validation when the properties are usually uncast (e.g. from a request) or from creation when the properties have been cast (see my other comment #921 (comment)). This means that if you're checking a cast property, e.g. an enum, that you'd need to handle the situation it's a string OR an enum instance.

Do you have any thoughts on those potential problems?

The reason I implemented it like this is because our particular situation uses two properties to determine the concrete class to use. E.g. platform_type and module_type which are both enums.

One potential solution to the above problems is to make it so the abstract classes only have one property used to determine the concrete class and have it always be a string e.g. public string $type;

This would allow us to avoid the overhead of casting properties before they go to morph on creation and allow us to add a validation error if the property cannot be used to determine a concrete class.

I'm happy to implement those changes if you think that's the best path forwards.

Update: see comment with PR below

[bentleyo]

@rubenvanassche I have created a PR on my repo that resolves the two problems I identified in #921 (comment) while retaining the ability for us to use multiple properties to determine the morphed class and supporting casts 🎉

Can you let me know if you would be happy with this approach?
bentleyo#1