Deserialization & Type system limitation

[andrewdavidmackenzie]

Example happens in Mandelbrot.

Generate_pixels in fact generates a row of pixels as array/array/number
E.g. [[0,0], [1, 0], [2, 0] … [x, 0]]

This is fed into pixel to point sub-flow that accepts array/number
E.g. [0,0]

Semantically this should be deserialized, and many jobs possibly launched to handle the queue of inputs…

But subroutes are used on that array/number input, so the connection is formed from the source to the destination, not using the type declared on the sub-flow input.

The problem is that deserialization is currently left to run-time, and not taken into account at compile time, so the subroute is applied incorrectly.,

[andrewdavidmackenzie]

Test files to reproduce it are attached.
test.toml generates array/array/number and sends it to the input of sub_flow.toml.
E.g. [[0,10], [1, 9]… [10,0]]

That sub flow has the input types declared as array/number, expecting it to be deserialized.
E.g.
[0,10]
[1,9]
…
[10,0]

It then takes the first element using “/0”, what out wants to do is take the first number of each tuple of coordinates
E.g.
0
1
…
10

So it can add each tuple

But as it is not deerialized, “/0” picks the first element of array/array/number
E.g.
[0,10]

Not the single number if was expecting.

[andrewdavidmackenzie]

Proposal
Re-write Connection to build up a chain of operations and subroutes, before collapsing, so it is aware of the deserialization that will be done at runtime.

It indicates in the connection what deserialization will be needed at run-time, and the runtime only implements it as instructed when directed to do so?

Need to keep general deserialization at runtime to cover generic type cases, where only the runtime knows the types involved?

See if can get existing code running using this change.

Then add a test like above that fails.

Then implement the deserialization in runtime as si trusted by the compiled and manifest and see if can get test passing.

Then fix any errors introduced

Then go back and look at Mandelbrot, see if we can get many jobs queued up for the prows of pixels.

[andrewdavidmackenzie]

test_toml_.txt

test_sub_toml_.txt