CQGI: displaying cqparts components

[fragmuffin]

displaying objects created in cqparts using cqparts.cqgi at the moment involves extracting the cadquery.Workplane instance from the Component, and passing it to show_object.

from cqparts_misc.basic.primatives import Cube
cube = Cube(size=10)
show_object(cube.local_obj)

Problem
Displaying a cqparts.Component with editable parameters currently requires the workaround described in cqparts/cqparts#95 (comment)

from cqparts_misc.basic.primatives import Cube
g_size = 10
cube = Cube(size=g_size)
show_object(cube.local_obj)

Solution?
The display of a cqparts component would be much more intuitive with cadquery.cqgi if the parameters to be modified are picked up from the cqparts.Component instance itself.

[fragmuffin]

I'm not implying that cadquery.cqgi should natively support cqparts.
But a way to communicate with cadquery.cqgi to indicate which values can be modified, and what their default values are would be awesome.

For example, a list of default parameter values can be extracted from any cqparts.Component with:

>>> from cqparts_misc.basic.primatives import Box
>>> {n: p.default for (n, p) in Box.class_params(hidden=False).items()}
{'height': 1.0, 'length': 1.0, 'width': 1.0}

How could we intuitively use that dict to inform cadquery.cqgi what's editable?

[dcowden]

@fragmuffin you are definitely right, there must be a flexible way to inform cqgi of the parmameters and their types.

One of the reasons I do not like this approach:

@cadquery.part
def cube(size=10, width=true):
    ...

Is because it is very limiting. You cannot necessarily infer the types correctly here, and ( as you point out) if you have a more complex library ( like cqparts, or custom code), it requires you to expose a method vs a class. Plus, there's no way to communicate extended metadata-- for example, Positive integer of value between 0 and 1

cqgi.parse currently reads a script and produces a model object.-- this model object, in my opinion, is the right abstraction. It contains metadata about the parmeters, and contains an executable function.

What i took away from our last discussion on the topic was that i need to make it possible to have several adapaters between a canonical model , and other things ( like a cqparts Component, or a python script that exposes a method like the above).

If we imagine it this way, then we can imagine that somebody has to write the code for converting cqparts components into a cqparts.model. Then, its really just a matter of where this code is located, but it doesnt actually matter.

[fragmuffin]

Just brainstorming here...

import cadquery
import cqparts
from cqparts.params import FloatRange, Bool

class MyComponent(cqparts.Part):
    size = FloatRange(0, 10, default=5)
    width = Bool(True)
    def make(self):
        ... return a cadquery.Workplane

@cadquery.part(**MyComponent.get_cqgi_params())
def obj(**kwargs):
    return MyComponent(**kwargs).local_obj

I'm not implying I like that model better than any other.
And that would definitely make the canonical display model messy:

@cadquery.part(size=10, width=True)
def cube(**kwargs):
    size = kwargs.pop('size')
    width = kwargs.pop('width')
    # yuck
    ... return a cadquery.Workplane

@dcowden : What's your preferred canonical model?
Forget about how it would be implemented; what would you like to see as the base cadquery template?

[fragmuffin]

PS: I love your new avatar @dcowden 👍

[dcowden]

@fragmuffin My apologies for this long post.

I've been thinking through this the last couple of days, in preparation for releasing a new cqgi proposal. I really want cqgi to work well with cqparts, because I think the abstractions you've created are excellent. I think people will relate well to them. But i also need to make things work in a non-cqparts environment too.

I think I have the execution side down, but I'm struggling with discovery of input parameters.In particular, i'm trying to figure out how to allow cqgi to gather cqparts parameters without the need to re-bind them as globals.

zwn said:

That is what hit me when looking at cqgi -- it is trying really hard to reimplement function calling! Only the parameters appear to be global variables, the return value is accumulated to a list during the execution and the function has no name.

But that's not true for two reasons:

(1) we're trying to capture variable types and constraints ( PositiveFloat), and
(2) we're trying to gather the information BEFORE the model runs, not AS it runs.
(3) i'd like to support the notion of validation, in which we check all of the inputs without trying to run the model. This becomes important when models are complex and you have a GUI-- you can't wait for a re-built to tell user the vars are wrong.

I can see a solution that uses decorators, along with the assumption that a cqparts project ultimately must contain a single, top-level component that's capable of listing all its parameters: Something like this:

import cqparts
from cadquery.cqgi import  FloatRange, Bool
import cqqi.model

class MyComponent(cqparts.Part):
    size = FloatRange(0, 10, default=5)
    width = Bool(True)
    def make(self):
        ... return a cadquery.Workplane

@cqgi.model(**MyComponent.get_params())
def build_project(**kwargs):
    return MyComponent(**kwargs).local_obj

The executing environment would compile the module, and then look for any functions decorated with @cqgi.model. The cqgi.model function would look kind of like this:

def model(cqgi_parameters, execute=False

When we need to read the parameters or validate, we'd run the method with execute=False, inspecting the variables we get, and then returning the appropriate result. In this case, the underlying build function would not be called.

Non-cqparts libraries would need to provide a static list of parameters, like so:

import cadquery
from cadquery.cqgi import cqgi_build, FloatRange

@cqgi.parameters({"size": FloatRange(0,500)  })
def cube(size=10):
    obj = cadquery.Workplane('XY').box(size, size, size)
    debug(cube.faces(">Z").workplane())
    return obj

Requiring the user to write that top level function also opens the possibility to use PEP-484 type hints instead of our own type system, which would give us better supports inside of editors.

This works, but has a couple of disadvantages:

(1) its more code to write for the user, because of the need to define a function.
(2) it is complicated for the executing environment, and hard to understand the flow
(3) there is going to be confusion about what happens when you define more than one build method. Do they both run? in what order? what's the result of running two?

If we're ok requiring the user to define an entry point function in the top level script, another alternative that's maybe easier to explain and understand is this:

import cadquery
from cadquery.cqgi import FloatRange

 parameters =  {"size": FloatRange(0,500)  }
 def build(size=10):
    obj = cadquery.Workplane('XY').box(size, size, size)
    debug(cube.faces(">Z").workplane())
    return obj

Here, the executing environment uses the parameters var to get the list of vars, and invokes the build method. This clears up the confusion of #3 above-- there can be only one build method. Its simpler to implement. This is what the original cq did long ago. As with the above, we could eliminate the parameters variable by using PEP 484 types:

import cadquery
from cadquery.cqgi import FloatRange

 def build(size: FloatRange = 10):
    obj = cadquery.Workplane('XY').box(size, size, size)
    debug(cube.faces(">Z").workplane())
    return obj

This seems pretty straightfoward to me. The executing environment compiles the script as a module, looks for a build method, and gets the types and the defaults from the build method. for cqparts, the user is required to write a top-level wrapper function, that can grab the parameters off of the model.

Thoughts?

[zbynekwinkler]

@dcowden As you probably figured, I like the "function calling" interface 👍. The result looks very pythonic to me which I consider a major benefit.

The implementation of CQGI would import the module, inspect the type annotations of the build function and call it with the right arguments? If that is the plan then see many 👍 coming your way from me.

Maybe it would be useful to support another common pattern:

if __name__ == '__main__':
    do_something() # like starting simple object viewer

That way the script really can be executed as a standalone script without modifications.

[fragmuffin]

@dcowden

I think people will relate well to [cqparts]. But i also need to make things work in a non-cqparts environment too.

I absolutely agree, I think cadquery should always remain independent of cqparts.
During joint development (such as this discussion) they'll be built to compliment each other, but to an end user this won't matter if they choose to use cadquery without cqparts

we could eliminate the parameters variable by using PEP 484 types.

That does look fantastic (re: your last code example)
This is the great thing about leaving python 2.x behind;
we get to use all the new python 3.x toys! 🎈

The implementation gives the user the power to pass a type, or other callable to return a valid value.

The executing environment compiles the script as a module, looks for a build method, and gets the types and the defaults from the build method

Can I also assume this will work with external modules?... could you potentially point cqgi at a pre-compiled module and say "show me that"?

vanilla cadquery -> cadquery.cqgi
Working from your last example

import cadquery
from cadquery.cqgi import debug  # I'm assuming debug is imported

def positive(value):
    assert value >= 0, "value must be positive"
    return value

def build(size: positive = 10):
    cube = cadquery.Workplane('XY').box(size, size, size)
    debug(cube.faces(">Z").workplane())
    return cube

Can you make it so build can return:

• cadquery.Workplane instance (singular)
• iterator of cadquery.Workplane instances (eg: list or generator)

If cqgi can't take more than one cadquery.Workplane from the build output, then we can't display cqparts.Assembly results.

cqparts -> cadquery.cqgi
cqparts components could automatically apply PEP 484 annotations.

@MyComponent.annotate  # populates build.__annotations__
def build(**kwargs):
    component = MyComponent(**kwargs)
    return component.iterobjects()  # iterator of cadquery.Workplane objects

But to make it simpler, the build method could be made by the component itself:

build = MyComponent.get_cqgi_build()

colours / transparencies?
How does the user give their render options in this case?

[fragmuffin]

@zwn

I like the "function calling" interface 👍. [...] That way the script really can be executed as a standalone script without modifications.

The module aproach is still very pythonic...
From @dcowden above

The executing environment compiles the script as a module, looks for a build method, and gets the types and the defaults from the build method

So you could definitely still put that code at the bottom of the file...

if __name__ == '__main__':
    import myviewer
    myviewer.render(build())

everybody wins 👍

[dcowden]

Yes, ok thanks for the thoughts guys. I'm liking this direction. A few answers about the above:

Can you make it so build can return:

cadquery.Workplane instance (singular)
iterator of cadquery.Workplane instances (eg: list or generator)

Definitely. The existing show_object accepts only one, but that's because it was imagined you could call it multiple times at multiple points in the code. If we use functions, it requires the script author to collect all of the results and supply them as the return value, of course requiring ability to return >1

That does look fantastic (re: your last code example)
This is the great thing about leaving python 2.x behind;
we get to use all the new python 3.x toys! balloon

Ok i didnt know that you could use callables as types-- that is really sweet. Given that, i'm going to assume we'll use this overall strategy. I think it will work really well. Honestly this is the first example I can think of where i'm actually excited about moving to python 3 vs sad because of the work

Maybe it would be useful to support another common pattern:

if name == 'main':
do_something() # like starting simple object viewer
That way the script really can be executed as a standalone script without modifications.

I like this.

Ok let me play around ( probably wont get to it till this weekend), and post some examples after I've worked it through.

The deployment path for this is a bit more complex than i thought. I initially wanted to get something quick done on master, but that can't happen since master is python 2.x. So i'll probably propose one version that doesnt handle types really well, but's functional, with the idea that we'll get the full type-oriented version with the OCC/cq 2.0 branch.

[jmwright]

So, this is maybe only tangentially related, but I'll put it out there anyway.

Currently if I import my cqparts part modules and use them in an assembly, they never un-import/re-import when changes are made to the underlying part modules. So for instance, if I import the chassis of a robot into an assembly, execute the assembly script, and then add cutouts to the chassis part file, those chassis changes will not be reflected when I re-execute the assembly script. I will see the chassis in the assembly as it was when I first imported it. A few attempts have been made at correcting this in CQFM, but we're still not getting the proper result. CQGI uses an environment builder currently. Is there a way to sandbox each script execution and then throw the (virtual?) environment away and start fresh on the next execution? This would be a huge help when developing cqparts assemblies in CQFM. Here's an example script showing what I'm talking about.

import cadquery as cq
import cqparts
from cqparts import Assembly
from cqparts.params import *
from cqparts.display import display
from cqparts.utils import CoordSystem
from cqparts.constraint import Fixed, Coincident
from parts.chassis.chassis import Chassis
from parts.cots.futaba_s3003_servo import Servo


class RobotAssembly(Assembly):
    def make_components(self):
        chassis = Chassis()

        return {'chassis': chassis}

    def make_constraints(self):
        return [
            Fixed(self.components['chassis'].mate_origin, CoordSystem())
        ]

robot_assembly = RobotAssembly()
# display(robot_assembly)
for (name, component) in robot_assembly.components.items():
    show_object(component.local_obj, options={"rgba": (204, 204, 204, 0.0)})

[fragmuffin]

This sounds like the same issue fixed for the non-cqgi interface:
CadQuery/cadquery-freecad-workbench#103

[dcowden]

@jmwright yes, I certainly wouldn't mind doing this, but I don't currently know how. I've never researched how to force python to reload modules and classes from inside the same interpreter. Adam compiles a script as a module in cadquery-gui. Does his approach achieve the desired result?

[fragmuffin]

Re-loading a module is quit easy, it's all in that PR

If you have a module called mod.py

# mod.py
print(__name__)

And you run the code...

import mod  # prints 'mod'
import mod  # no output; module is already imported

import sys
del sys.modules['mod']

import mod  # prints 'mod' (re-imported)

the output will be:

mod
mod

python is awesome

[jmwright]

103 introduced a bug outlined in 104 though. I'm not sure if that's intrinsic to the module unloading/releasing process, or if its just a FreeCAD thing though.

[fragmuffin]

@jmwright I don't know... I remember spending a lot of time on that issue, but I never did figure out the root cause. 😢

[dcowden]

@fragmuffin oh wow, i feel kind of dumb that its that easy. i didnt know that trick. well then consider that feature in for sure!

I should have guessed, delete is the new hotness everywhere. At work i get to play with kubernetes, and you delete pods to redeploy them. but that's like a whole application, and 'delete myapp' on a production server feels-- a little weird.

[adam-urbanczyk]

@dcowden not sure if this is still useful:
This is another way of reloading

from imp import reload
reload(yourmodule)

If you execute a code into a module you can inspect the globals and look for objects of type module and reload them at your will:

import types
m = types.ModuleType('tmp')
exec(SOME_CODE,m.__dict__)
loaded_modules = [obj for obj in dir(m) if isinstance(obj, types.ModuleType)]

If you want to reload all modules recursively then I guess you need observe sys.modules change/install an import hook or maybe us modulefinder module

[dcowden]

Useful, thanks!

[zbynekwinkler]

While reloading modules in python is possible it may not always yield the same result as executing from scratch (python has very complicated/powerful runtime). More reliable method might be using something like multiprocessing and actually execute the script in a throw-away python process and communicating the results back to the parent process for display.

[dcowden]

@zwn yes, and for the server side this has the benefit that there is better isolation against malicious scripts.

When I had the parametrics.com server running, each model was executing in a separate proccrss, in addition to being inside of a resirticted python environment with a lot of dangerous things removed.

Even though of course it has been mostly proven that a totally sandboxed python is impossible, it kept people honest.

[dcowden]

This has been closed in favor of the CQ 2.0 implemention at https://github.com/cadquery/cadquery