General metadata system #2
Comments
|
I see two possible approaches. One copies Attributes from C# and Rust. I'm going to describe the Attributes approach now, and think about the other over lunch. AttributesIn both C# and Rust you can put attributes on most language items. E.g. in Rust you can put #[doc = "..."] on a type or module, and you can put #[serde(default)] on a field. Both Rust and C# have a set of built-in attributes (e.g. Attributes are structured: each attribute defines what schema it follows. E.g. the
Eventually other clients could create their own libraries with structured attributes. For example, users might create a library with attributes for shapeways.com 3D printing service. Like Ideally we'd allow users to put attributes on basically anything. Because KCL is such a simple language, it doesn't have many language constructs -- only functions and expressions. In Rust you cannot currently put an attribute on an expression (see their open RFC for this) -- but you can put them almost anywhere. If you want to put an attribute on an expression, you can just move it into its own function, though, so it's not hard to work around. Hopefully because KCL is simpler, the KCL compiler would support attributes almost anywhere. |
|
tagging @Irev-Dev to read this when he wakes up |
DatabasesOn reflection, our language is really outputting:
Does this kinda sound like a database to anyone else? SemanticsWe could "compile" KCL source code into a database:
If we do this, KCL basically becomes a domain-specific language (DSL) for inserting into a database. This has some big advantages:
SyntaxSee below |
|
I haven't used attributes much because I've used rust so little, so they don't click with me quiet as much if I had more exposure, would you mind outlining the advantages of attributes over something like a metadata param with key-value pairs? |
|
We totally could use a metadata param, we'd just need to put it in every single function that can create an object. So it would be a somewhat inconvenient user experience. On the other hand, we could make metadata an implicit first parameter to every function... |
|
Yeah cool, sorry taking me a bit to wrap my head around things, so if we could use params, but would likely start looking noisy. Maybe a good rubric is that modeling and 3d-geo is a first class citizen, meta-data is an important thing to layer in, so modeling data goes in params, medata is layered on with attributes. One thing I wasn't sure about either from your the example
I had a very tangential idea with attributes, after writing it, it became obvious it should probably be in it's own issue #3. |
|
For background, I think having structured notes could be useful. You can see in my example I imagine different notes having different intended audiences, so the manufacturers could filter only notes relevant to them, or the designers could filter only notes relevant to them. The way Rust does it,
The former is nice because you can do it all on one line, and it reuses the very general, flexible attribute syntax. The latter is nice because you can write really long comments that span multiple lines, without needing to enable word wrap in your IDE. |
Keyword arguments(an alternative to attribute syntax) MotivationIf types like Solid3d and Line are database tables, then to the user, they would probably be structs with
The problem with structs is that you have to initialize all their fields. This would be really inconvenient for users -- you might not care about adding a note to every value, or a surface finish, or whatever. So we should define default values for many (perhaps all) of these fields. If users don't set the value explicitly, we use the default value. Unfortunately, there's an awkward tension between optional arguments and positional arguments. Because the compiler needs to know which fields are being omitted! Say you have a function call like Python has this problem, but solves it by using keyword arguments (kwargs) How do kwargs workFor example, the def open(file, mode='r', buffering=- 1, encoding=None, errors=None, newline=None, closefd=True, opener=None)Here # You have to specify all positional arguments. There's only one here.
# Default values are used for all keyword arguments.
f1 = open("log.txt")
# You can override the defaults for keyword arguments.
f2 = open("log.txt", encoding="UTF-16")I really like this, because it allows us to:
So, returning to the problem above: how do you invoke the When you define a function with kwargs, you specify the default values. Improving ergonomicsShorthandI think it'll be pretty common for users to define values and then pass them into keyword arguments like Delegating kwargsI think we'll need a way for users to define their own functions and "delegate" all the kwargs to the inner functions they're calling. For example, say you're dealing with a lot of really big cubes throughout your code. You want to avoid repeating yourself all over the codebase, so you define a function for the big cube. Later on, you realize that each cube needs to have a different note on it. So you add a kwarg for notes, with a default value. You reuse the standard library's So far so good. But as your program evolves, you'll probably want to add more and more properties. This gets unwieldy pretty quickly: You can omit the type annotations to make this nicer. KCL can infer the types of all your kwargs here, because it knows the underlying types of the stdlib function But this is still unwieldy. You'll probably want, at some point, to let users of So, we should support a syntax like Python's special This seems much nicer, but I worry that it's hard to compose. What if you have two different shapes? Say you want to put a sphere on top of a cube. Now if you delegate metadata, you've got to send the same metadata to each shape... that's not good... This locks you into using the same set of metadata for both cube and sphere. We'd need some syntax for overriding that metadata -- unpacking it into a struct, updating some fields, then packing it back together and passing it into the other function calls. Even worse, how do you delegate kwargs to different underlying types? The kind of kwargs that are needed for the |
|
On the other hand, a function's keyword arguments are basically equivalent to structs. The set of keyword args accepted by a function is equivalent to a struct, with each individual keyword arg corresponds to a field of that struct (the field is optional and has a default value). So maybe instead of adding kwargs, we should add struct types. Then each function can explicitly declare a metadata struct, with whatever fields make sense for them. If users omit a field, it's set to a default value. This way, we can add new fields in new versions of KCL without breaking users -- we just define a default value. I like this because users will probably want struct types anyway, and it simplifies how functions work (which simplifies the type system). I can't help but be a little peturbed that no other static, functional languages use keyword args... maybe there's a reason why. @greg-kcio what do you think? |
|
I like the pythonic style of default args and kwargs (disclaimer: I'm biased bc Python is my daily driver). When using default arguments in Python, you must declare required arguments before optional arguments: # this is good
def cube(length: Distance, note: Note=Note.default(), material: Material=Material.default()):
pass
# valid calls:
cube_01 = cube(Distance(10, 'mm'))
cube_02 = cube(Distance(10, 'mm'), Note("This is Cube 02")) # kwarg without key, but in order
cube_03 = cube(Distance(10, 'mm'), material=Material("PETG")) # omit the first kwarg and specify the second
cube_04 = cube(Distance(10, 'mm'), Note("This is Cube 02"), Material("PETG")) # kwargs ordered, without keys
cube_05 = cube(Distance(10, 'mm'), note=Note("This is Cube 02"), Material("PETG")) # when ordered, kwarg keys do not need to be specified
cube_06 = cube(Distance(10, 'mm'), Note("This is Cube 02"), material=Material("PETG"))
cube_07 = cube(Distance(10, 'mm'), material=Material("PETG"), note=Note("This is Cube 02")) # kwargs can be out of order when you specify the key
cube_08 = cube(material=Material("PETG"), note=Note("This is Cube 02"), length=Distance(10, 'mm')) # required args can also be out of order when you specify the key
kwargs = {material=Material("PETG"), note=Note("This is Cube 02"), length=Distance(10, 'mm')}
cube_09 = cube(**kwargs) # this works too and all the same rules apply
# invalid calls:
cube_10 = cube(note=Note("This is Cube 02"), material=Material("PETG"), Distance(10, 'mm')) # illegal even though we can reason about the args
cube_11 = cube(Distance(10, 'mm'), Material("PETG"), Note("This is Cube 02")) # kwargs out of order with no keys
# this is illegal because there is a required arg declared after the optional arg
def bad_cube(length: Distance, note: Note=Note.default(), material: Material):
passPersonally I prefer leaving no ambiguity for type requirements and would like typed args and kwargs. Python has built-in types that support this (nominally only, there is no runtime type checking this way): |
|
But... if we have structs, then I don't see a need for keyword args anymore. You just have a struct with a field for each kwarg, and if you want them to be optional, you just use Option instead of String. Then your function unwraps the optional with its desired default value. This solves the composition problem I outlined above. Instead of declaring **kwargs and delegating them to both |
|
So, I discovered that OCaml has "label args" (docs) which are basically just like keyword args. You can define any argument with a label. You can also define optional parameters. Declaring an optional T parameter is just syntactic sugar for a required Option parameter. They don't have something like
OCaml solves this by saying "No, you can't declare the function as supporting any metadata that the underlying cube supports". I guess that's OK. You just declare the kwargs you want, and if the function signature becomes very long, that's OK. Reading the OCaml docs for kwargs definitely reduces my worries about including them in the language. I know you can accomplish the same things using structs, but kwargs seems like a better DX. |
|
It appears ECMA-335 / ECMA-334 attributes are something which are non-extensible by users. This is problematic for users with unforeseen needs. As proposed by @Irev-Dev , key-value pairs (objects) would be much more ideal, and flexible, and type-safe if users are able to define their own types. Sure, there can be a set of standardized keys and values, but there must be a way to extend this by the users themselves. Some will want attributes to propagate and others not. The only solution to this is to either have syntax to explicitly propagate or not. The easiest thing is have the user repeat the application of an attribute. The best thing is making it easy to combine attributes. Named arguments are like @adamchalmers said, essentially structures, but in their defense a bit more user-friendly. I'm sure this particular aspect could be bikeshedded a lot, but we're all aware structures are more widespread than kwargs 🙂 |
|
I agree, I think attributes are the wrong way to solve this. These metadata values need to be first-class concepts in the language, so they have to become return values or arguments to functions. So far in the language design, I've found places where I'd like keyword arguments (here), and other places where they'd help introduce backwards-compatible API changes. On the other hand, I haven't really found places where users would need to design their own structs. I think because this is going to be such a limited, single-purpose language, structs from the stdlib might be enough, i.e. there may be no need for users to define their own types. If that's so, then avoiding structs would really simplify the language implementation. It frees us from thinking about
So, for now, keyword arguments seem much simpler to implement and design. I suggest we:
|
It's hard to comment without seeing some potential examples. That would further help everyone else understand the direction of the language. I too rarely, if ever, use a structure that includes things other than measurements and positions. So I completely agree. I don't think metadata values have to be returned by anything, but user defined function with user defined keyword arguments to generate particular values sound super dang useful (maybe this is what you meant: "metadata [derived] values")! |
|
OK, I feel good about this discussion. Keyword args, no **kwargs syntax, and maybe we'll get to structs down the road. Thanks! |
|
Apologies for being late to this, I think javascript has some pretty good patterns, with spread operator, rest operator and destructuring from your example @adamchalmers If I were to mix in some js syntax here What happens here is all args are collected in Any decent js dev would immediately refactor to the destructuring in the function param definition, definitely cleaner This to me seems like all the benefits of kwarg, with the one exception of ordered params are not part of this, but I actually like this, ordered params are kinda hostile to a new reader To hammer this home imagine you're a mechanical engineer and you are very familiar with modelling concepts, every modelling software you've used takes a sketch or similar and extrudes it by some scalar, you're reading someone's example KCL and you see the line |
|
There are a few programming languages where only keyword arguments are allowed, or at least, the default is keyword arguments. E.g. Swift. // Declare a function, looks normal.
func greet(person: String) -> String {
let greeting = "Hello, " + person + "!"
return greeting
}
// Use a function: note the parameter name on each argument!
print(greet(person: "Anna"))
print(greet(person: "Brian"))You can also use different labels for the argument (passed by caller) and the parameter (input within the function). func greet(person: String, from hometown: String) -> String {
return "Hello \(person)! Glad you could visit from \(hometown)."
}
print(greet(person: "Bill", from: "Cupertino"))In this example there's one value, its argument label is You can opt into positional arguments (i.e. just a parameter with no argument label, see docs) and also give default values for arguments. I will say that all these features will complicate our compiler, but they definitely will provide a nice flexible UX. I agree that this will make function calls much more readable for new programmers. The |
All KCL objects (lines, paths, points, corners, solids, etc) should support metadata. This way users can:
These metadata need to support both statically-typed metadata (which we typecheck), and flexible data (for external devices or clients, whose schema we won't understand).
Right now the fantasy docs describe a
Materialtype, which all 3D objects take as a parameter when they're constructed. In my opinion this could be a special case of the more general and powerful metadata system I described above.The text was updated successfully, but these errors were encountered: