[suggestion] 2D plotter / drawing / printing option

[ES-Alexander]

It'd be nifty for people to be be able to put in some text or an image and get a meaningful gcode output, for things like drawing with a pen (or painting) on a page, or printing a design onto some fabric, or printing as a top layer that then gets painted or used as a stamp.

A couple of spit-balled ideas:

1. allow text input that draws the letters with single width passes (instead of the doubled borders a slicer would tend to make)
   • would require one or more fonts to be included, which could either be manually defined as points, or (more advanced) automatically converted from common computer font files
   • to some extent this is covered by SVG import #11, but I'm thinking of filled fonts rather than letter outlines
2. allow (raster) image input that gets drawn in some interesting way
   • Canny edge detection would be a quite simple option, to draw just the detected edges from the image
   • more advanced/artistic approaches could do some form of shading using things like
     • variable width parallel lines with even spacing
     • squiggly non-crossing fixed width lines that use some form of iterative point adjustments to create lighter and darker regions by density
     • variably spaced hatches
     • variably oriented straight lines that each cover the darkest remaining line in the image
     • variably sized filled circles, either in a grid or haphazardly packed
     • something Voronoi based
   • this kind of thing might require Pillow or opencv as a dependency, in which case it could be useful to create multiple install targets depending on what features people want (e.g. pip install fullcontrol[all] vs pip install fullcontrol[minimal] and such)

[fullcontrol-xyz]

These are great ideas! I've been thinking about single-line text for some stuff I want to do with pen drawing. There are some fonts out there that are single line. But there would need to be some method to convert the ttf to svg in python unless it's acceptable to have a workflow where other software like inkscape is used to create an svg of the single-line-font text.

The image stuff would also be really cool and very achievable. I think the best bet would be to initially get a parallel line version working by simply varying z-position and using a soft-tip pen. You'd just create a load of line segments with x/y following a square-wave or square concentric spiral or similar, and for each point z would be set proportional to greyscale value (or some fancy non-linear relationship for z:greyscale. If each line segment ended in the 'middle' of a pixel, this would be so simple to do. Or if not, it might still work by sampling the image data, if the appropriate resolutions for image/toolpath are used.

If someone has a pen-plotter for which this would work, comment here and let's get something up and running

[ES-Alexander]

A few initial results from a bit of playing. Left image was the input, the others are some different outputs that could potentially be 'drawn' by a printer (rightmost is your square concentric spiral idea):

I should hopefully be able to upload a gist in a few days - my code is not yet using any of the fullcontrol codebase.
One of the fiddly parts will be in displaying the expected output (rather than just the toolpath), because plotting libraries don't tend to like displaying points or lines with their sizes/widths in plot units, so we'll need to either plot the line segments as individual shapes (e.g. rectangles and trapezoids), or draw things in a rasterised format instead (which is likely less desirable, but better than nothing).

[fullcontrol-xyz]

Looking good - that makes sense about line widths. It's something I really wanted to do in the basic FullControl plot, but ran into challenges when trying to have multiple lines widths on the same line, as you've mentioned.
In the future, it will be really valuable to have a way to plot continuously varying line-widths for 2D and 3D line plots. But it also becomes valuable to have some kind of shading to avoid lines merging visually into big regions of solid colour. It's possible with threejs, but difficult to make versatile for all the different types of toolpaths that might be designed. Hence the simpler plot style in FullControl initially.

For this 2D approach in the short term, what about plotting each line as a (long) closed polygon? A series of points offset from the line medial axis by width/2. This would be possible directly in plotly - generating the points is where the most effort is required. There'd be a new polygon for each new line (and each time the pen lifts off the paper due to white pixels). Alternatively, if the whole plot is one single polygon with infinitely narrow connections between lines, that would make plotly run much quicker. It just depends whether plotly shows those infinitely narrow connections unintentionally.

[ES-Alexander]

In the future, it will be really valuable to have a way to plot continuously varying line-widths for 2D and 3D line plots.

Agreed. What kind of profile are you envisioning for the 3D side of things? 2D width is at least conceptually straightforward, but 3D might need some concept of collisions and flattening in order to be visually accurate, which gets complicated fast. I suppose it could be possible to do cylinders+cones as an analog of the rectangles and trapezoids in the 2D domain, but I'm unsure how limiting that will be for more complicated designs that involve connections and/or some form of layer stacking.

For this 2D approach in the short term, what about plotting each line as a (long) closed polygon? A series of points offset from the line medial axis by width/2.

Yeah, that's the kind of thing I was thinking - I'll have to look at what's available.

As is I did the rightmost plot as a matplotlib LineCollection (and the others were just defined as pixels in raster images), but I don't like needing to manually interpolate steps at the width changes, so using polygons is a better choice if it's feasible / not too computation heavy (not to mention that matplotlib's linewidths are in pixel units, so aren't good for physically consistent plots). Ideally it'll be possible to define something that's easy to use and interoperates nicely with plotly, but (having not looked into it yet) I'm not sure whether that's best handled on the Python side or the javascript side / within plotly itself.

I suppose if performance is an issue we could make a variable_widths flag in the plot function or something, that falls back to the current line approach if it's disabled.

[fullcontrol-xyz]

For the 3D stuff, I know threejs has some easy-to-use 'tube geometry' options that shade nicely to give clear visual representation even if the tubes overlap a lot. But the js would need to be dug into to make that work since the default threejs functions work on 'paths' and these don't handle print paths of discrete points well. It's probably easy for a threejs expert to do, but would take a fair bit of time for me (Andy). The maths has already been done by someone to get the cylinders (actually polygons) to connect neatly at the ends, etc., so it's probably not that difficult. But this route obviously has some complexity involved in communicating between python and js. But that's something that FullControl can simplify for end-users. I'm sure there are loads of other options too. It may depend on whether someone comes along who interested in FullControl and has some relevant experience to be able to knock up a really great visualiser. At that point, it'll probably be best to consider more general stuff too (beyond just 3D variable-width lines), like animations, etc.

I think the 2D polygon idea should be fine in either python or javascript, since there probably won't be toooo many polygons. So I'd do as much as possible in python so it can easily transfer over to other visualisation packages if people prefer not to use plotly. E.g. it'd be easy for someone to convert those polygon points into an SVG.

[ES-Alexander]

Noting for future reference: Plotly Streamtubes look promising.

[fullcontrol-xyz]

Ah it does! And it has some shading which is important

[ES-Alexander]

Quick update:

1. I tried the streamtube approach and didn't make much headway with it
   • given what it's designed for it seems unlikely to play nicely with sharp corners and path intersections, both of which are common in machine toolpaths
2. I spent a bunch more time starting a path points -> tube mesh converter
   • I've sorted out a suitable code structure for generating cylinders / cut cones with variable end diameters, but
   • there are some intricacies with determining reasonable mesh points at the path ends and corners that I'm still working through
   • I'm hopeful I'll complete that over the weekend, and be able to start applying it to paths to see how it goes performance wise
     • I have put at least a little bit of effort into minimising redundant operations and memory usage, as well as avoiding unnecessary polygons in the generated mesh (while also being able to turn up the resolution on demand)
     • I've just had the thought that I might be able to add a "ribbon" mode or function, which would conveniently also serve as a high performance option for flat plots as well... - I'll have a play

... it has some shading which is important

Thankfully this seems to be the case for plotly's mesh rendering in general - you can even specify things like the location and orientation of the light source :-)

[fullcontrol-xyz]

Shame about streamtube, but the other stuff sounds great! I presume a lot of the work is for the geometry side of things rather than plotly-specific stuff? If so, for the corners, check out the threejs tubegeometry code. I haven't dug beyond the top-level stuff, but I think all the maths for points must be accessible since it's open source.
I'm sure you already know better than me, but in the ideal case, your code would be easy to reuse for other visualisation approaches that may come in the future. E.g. to output an stl file

[fullcontrol-xyz]

And thanks so much much for all the effort!!!

[aapolipponen]

Do you guys have a plotter already? I have a 3d printer that has an plotter attachment I built, so I can test the system if needed.

[ES-Alexander]

FYI, current state of my visualisation code is in this gist. I decided to change approach from my initial one to simplify some of the required maths by using global coordinates, so there's now working functionality, but it still needs some revising of the mesh generation before it'll be properly ready/nice - specifically at the corners / changeover points between path sections.

[ES-Alexander]

Do you guys have a plotter already? I have a 3d printer that has an plotter attachment I built, so I can test the system if needed.

@aapolipponen I don't have a plotter, and I believe @fullcontrol-xyz don't have one either.

I'm currently focusing on this visualisation thing before I get more properly into realising my plotter ideas, but if there's something in particular you're interested in testing I can potentially throw together a code snippet to generate some points or gcode or something (like I was playing with earlier).

[aapolipponen]

The 3d printer is actually not even working right now, but the parts for the repair are coming in something like two weeks, so no rush. I just though that if the code needs testing, I could help with that.

[fullcontrol-xyz]

Yeh no plotter here, but mrdrbernd on reddit uses one with FullControl. Still, the more people trying it out, the better, to identify any problems, etc.

[fullcontrol-xyz]

FYI, current state of my visualisation code is in this gist. I decided to change approach from my initial one to simplify some of the required maths by using global coordinates, so there's now working functionality, but it still needs some revising of the mesh generation before it'll be properly ready/nice - specifically at the corners / changeover points between path sections.

Great stuff!
For your R3 image, I can see some kinda chamfer on a corner. But your code suggests that tube_sep and chamfer_type are not implemented. From a quick glance through your code, I wouldn't expect to see a chamfer. So, why is it there?

My first thought is that a lot of complexity is coming from the N-1 option for tube diameters. So consider putting that on hold for now (or include it with an instant diameter change) and doing some speed tests for large point lists (if you haven't already). I think it's very common to be at 100k points. And maybe common up to 1 million points, but plotly itself will potentially hold things up then. Still, it's easy to switch from plotly to something else if your code is much faster than plotly.

It'll also be interesting to test the code with some tight helixes, concentric paths, crossing paths, etc., to figure out how well the shading works to differentiate between lines. I'm sure you're able to generate such points but let me know if you want me to send you a gist with various FullControl designs that you easily can convert to a point list for you function.

[ES-Alexander]

Great stuff!

Thanks! :-)

For your R3 image, I can see some kinda chamfer on a corner. But your code suggests that tube_sep and chamfer_type are not implemented. From a quick glance through your code, I wouldn't expect to see a chamfer. So, why is it there?

There's no chamfer - that's partly some illusory ambiguity, but also partly because the tubes are currently naively constructed using a polygon at the end of each path segment that's orthogonal to that segment. Accordingly, if there's a 90 degree turn from one path segment to the next then the end of the second tube starts parallel to its path segment (from the end of the previous tube), which means it starts out flat before widening out to the end.

My original corner approach (having the corner plane oriented halfway between sequential path segment pairs) is more physically logical, and that's the main thing I'm planning to implement before I start testing performance and whatnot. I hadn't planned to do the current corner approach at all, but the original one was harder to get working as a starting point (and has some extra complexities around collinear path segments), so I ended up doing this as a simple initial approach that at least allowed testing the mesh triangle generation and the plotting side of things.

My first thought is that a lot of complexity is coming from the N-1 option for tube diameters.

That should actually be quite straightforward once my planned corner change is implemented, because both of the proposed chamfer options just involve changing the input points to have two points per "corner" (so any size changes happen at the corners), after which everything would go through the "normal" generation process.

I'm not planning to focus on adding those though (yet at least), and I'll likely save the constant diameters + chamfer corner options for either a classmethod constructor or a subclass - I just left them in the initial docstring because I wanted to write down the useful feature ideas I'd thought of as options to be implemented at a later date.

It'll also be interesting to test the code with some tight helixes, concentric paths, crossing paths, etc., to figure out how well the shading works to differentiate between lines.

We'll see how the shading goes, but even if it doesn't resolve things clearly enough it's possible to use a colorscale and assign numbers to the mesh points to assign colours to them, so we can always improve differentiability by just applying a cyclic colour scheme to sequential path segments, and tweaking the cycle frequency and/or the number of colours in the colormap until nearby lines tend to be different colours :-)

... let me know if you want me to send you a gist with various FullControl designs that you easily can convert to a point list for you function.

I'm sure I can come up with some interesting test cases, but if there's something in particular you want tested and/or are concerned about then feel free to provide some point lists / point generation code to include in testing.

[fullcontrol-xyz]

That all makes sense thanks. In terms of geometry to test, I think the stuff in the overview tutorial gives a nice range of structures.

For the shading, the cyclic idea is a good option, but it does have some limitations. When you cycle frequently, you lose that sense of depth that you get with z-gradient or a single colour-cycle. So you gain definition between lines/layers, but often lose clarity of the overall object. I found the the threejs MeshNormalMaterial was really well designed and versatile in my trials. Threejs was also much more capable that plotly for larger data sets. So keep in mind the option to added some extra complexity in terms of communication between js and python to save efforts overall. Or it may be possible to use pythreejs (I haven't tested it), which does have the same material option I believe.

[ES-Alexander]

I haven't done my corner change yet, but that won't affect the number of triangles. This was with 100k path points and 6-point tubes (so 12 triangles per cylinder -> 1.2M triangles), using plotly's default colorscale, and I had no issues manoeuvring it around or seeing which line was what. The 'filament' wobbliness is because I made it have a randomly varying diameter.

I also briefly tried running with 1M path points, which took my computer ~25s from run to when the browser tab opened, but then the page was taking a long time to load so I aborted it (most likely my computer was running out of memory - the resource monitor said the tab was using ~10.2GB when I closed it).

That said, a computer with more RAM may have been fine, and at that many path points it likely makes sense to use fewer points for the tubes (3-point tubes with the 100k path point helix still looked fine to me).

[ES-Alexander]

Some without the varying diameter for reference

• 6-point
  
• 3-point (the inside looks a bit janky)
  
• 2-point (ribbon), with 1M path points - took a bit to load the plot, but no issues with manipulation
  

[fullcontrol-xyz]

Nice. So a 3-point one is basically a triangular prism? 6-point = hexagonal? How much of the computation time is your python code generating data and how much is from plotly creating the plot? And the 100k one just takes a few seconds?

For the 6 point one without varying diameter. Do you know why there is a mottling like pattern (some tubes are lighter, some darker). I remember this occurred in threejs sometimes too. Are all your tubes oriented identically about the medial axis. E.g. For a hexagonal prism, does the flat side always point up/to-the-side, or does the orientation vary?

The shading is good. It's really easy to see the different lines. Even with just three points.

[ES-Alexander]

So a 3-point one is basically a triangular prism? 6-point = hexagonal?

Yep :-)

How much of the computation time is your python code generating data and how much is from plotly creating the plot? And the 100k one just takes a few seconds?

Timing on my computer for a 100k path with 6-point tube:

Generating path...        DONE [0.012s]
Generating mesh data...   DONE [0.106s]
Generating plotly mesh... DONE [0.031s]
Creating plot...          DONE [0.119s]
Displaying figure...      DONE [3.069s]

Worth noting that the "Displaying figure" time is how long it takes before the program ends, at which point it has dispatched the relevant data out, but the browser still needs to load and display it (which takes another few seconds).

Do you know why there is a mottling like pattern (some tubes are lighter, some darker).

Not sure on that one I'm afraid - the default colour scale doesn't have small fluctuations like that, and it doesn't look consistent enough to be a shadow, but my best guess is it's some form of shadow rendering attempt 🤷‍♂️

Are all your tubes oriented identically about the medial axis. E.g. For a hexagonal prism, does the flat side always point up/to-the-side, or does the orientation vary?

With the current approach they're mostly consistent, but there are some "bad" turning directions that twist that tube section (which is part of what I'm hoping to fix with my other approach):

In contrast, the approach used in threejs doesn't care about global orientation (so there's no consistent top side), and instead twists each section in any direction by the smallest possible amount, based on the previous section. I suspect that approach may not be nice to do in a performant way in Python (without adding a compiler like numba as a dependency).

[ES-Alexander]

Visualisation stuff is now up to R6 (which fixes the main things I was planning to fix near-term), and I've made a PR for it, so now planning to move on/back to 2D plotter stuff, pending any requested changes to the PR.

[ES-Alexander]

Ended up not being able to get the chamfering corners out of my mind, so implemented the initial ChamferedTubeMesh in R8 today, which fixes the sharp corners issue and significantly alleviates the twisting issue (twists should only occur in the added corner pieces now, which are generally negligible).

[fullcontrol-xyz]

Thanks for all this. I've begun comments on the pr now. Shall I close this issue?

[ES-Alexander]

Shall I close this issue?

Nah - the visualisation discussion has been quite the detour from why this issue was raised (albeit at least a tangentially related one), but it can go back to being a discussion about 2D plotter and drawing options now :-)

[ES-Alexander]

Noting for future reference, difference of gaussians looks like a potentially interesting candidate for getting aesthetically pleasing lines from an image.

[ES-Alexander]

@aapolipponen, out of interest, is your plotter set up to just draw points/lines with a fixed width, or do you have some form of soft tipped pen/brush that allows you to draw thicker lines by pressing harder?

[aapolipponen]

It's just a pen strapped to a 3d printer. Nothing fancy. I think it could be possible to draw thicker lines with more pressure because the pen I have used is soft tipped, but I haven't tested that because I have just generated the gcode using an Inkscape plugin which doesn't have support for that. Actually it could be cool to try that.

[ES-Alexander]

Some potentially interesting plotter ideas in this video (more image-based stuff).