Skip to content

Latest commit

 

History

History
139 lines (117 loc) · 12.9 KB

Notes.md

File metadata and controls

139 lines (117 loc) · 12.9 KB
Raw

Notes on Midcurve NN

LLM Prompt based approach

Here is the prompt I propose to test any new LLM that gets available to check how good it is in predicting midcurve. Use src\utils\prepare_plots.py for plotting the output.

You are a geometric transformation program that transforms input 2D polygonal profile to output 1D polyline profile. Input 2D polygonal profile is defined by set of connected lines with the format as: ...

Below are some example transformations, specified as pairs of 'input' and the corresponding 'output'. After learning from these examples, predict the 'output' of the last 'input' specified.
Do not write code or explain the logic but just give the list of lines with point coordinates as specified for the 'output' format. 

input:{"Points": [[22.5, 22.5], [45.0, 22.5], [45.0, 135.0], [157.5, 135.0], [157.5, 157.5], [22.5, 157.5]], "Lines": [[0, 1], [1, 2], [2, 3], [3, 4], [4, 5], [5, 0]], "Segments": [[0, 1, 2, 3, 4, 5]]}"
output: {"Points": [[33.75, 22.5], [33.75, 146.25], [157.5, 146.25]], "Lines": [[0, 1], [1, 2]], "Segments": [[0, 1]]}

input: {"Points": [[0.0, 125.0], [125.0, 125.0], [125.0, 100.0], [75.0, 100.0], [75.0, 0.0], [50.0, 0.0], [50.0, 100.0], [0.0, 100.0]], "Lines": [[0, 1], [1, 2], [2, 3], [3, 4], [4, 5], [5, 6], [6, 7], [7, 0]], "Segments": [[0, 1, 2, 3, 4, 5, 6, 7]]}
output: {"Points": [[62.5, 0.0], [62.5, 112.5], [125.0, 112.5], [0.0, 112.5]], "Lines": [[0, 1], [1, 2], [3, 1]], "Segments": [[0], [1], [2]]}

input: {"Points": [[-4.77, 24.54], [5.05, 26.45], [1.23, 46.08], [6.14, 47.04], [9.95, 27.4], [19.77, 29.31], [20.72, 24.4], [10.91, 22.49], [14.72, 2.86], [9.82, 1.91], [6.0, 21.54], [-3.82, 19.63]], "Lines": [[0, 1], [1, 2], [2, 3], [3, 4], [4, 5], [5, 6], [6, 7], [7, 8], [8, 9], [9, 10], [10, 11], [11, 0]], "Segments": [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]]}
output: {"Points": [[12.27, 2.39], [7.98, 24.47], [3.68, 46.56], [-4.29, 22.09], [20.25, 26.86]], "Lines": [[0, 1], [4, 1], [2, 1], [3, 1]], "Segments": [[0], [1], [2], [3]]}

input:{"Points": [[1.47, 6.92], [5.66, 9.64], [-2.51, 22.22], [-6.7, 19.5]], "Lines": [[0, 1], [1, 2], [2, 3], [3, 0]], "Segments": [[0, 1, 2, 3]]}
output:

Compare the output with 'gold' result given below and let us know how much they differ:

gold: {"Points": [[3.57, 8.28], [-4.6, 20.86]], "Lines": [[0, 1]], "Segments": [[0]]}

A wider research domain: CAD with Deep Learning

Vectorization is a must for preparing input for Machine/Deep Learning

Comparison of NLP and CAD wrt vectorization:

NLP CAD
Words Entities
Sentences composed of words Features composed of entities
Paragraphs composed of sentences Bodies composed of features
Document composed of sentences Assemblies composed of bodies

FAQs

  • Which of the following is the correct result?

    • Right one

  • When do you get better results?

    • First, clean the pkl files from model directory. changed the optimizer to Adam just now and in 10 epochs, it's giving good results. So I think after training for 200 epochs as originally, it should do the trick
    • Normalization had to be added because of the binary crossentropy loss. I saw that the loss was falling below 0.
    • As our predicted y was between 0 and 1 because of sigmoid and actual y was 0 and 255

  • These are some results from the Simple Encoder Decoder approach, trained with Adam on 100 epochs. Are these results ok, with newer dataset? I think that this model is actually overfitting, and won't be able to generalize well.Using the weighted binary crossentropy for UNets improved the results significantly, so we can try weighted BCE or balanced BCE for other approaches as well, and get the results.
    For improving the unet results I have some things to try

    1. Change the loss function from weighted BCE to a distance based loss as used in the unet paper
    2. Add activations from the auxiliary decoder to the main decoder branch
    • Yes they are..little less noise though.

  • Are different thicknesses ok? Should these widths be same? I remember you once telling me something about the polygon should have same thickness or something?

    • Yes, my standard set followed that. Most of these shapes are from other researchers papers..although they relax it a bit.but still keep THINNESS ..although of different widths...wineglass shape is another such example

  • I modified the current approach such that the input to the model is the middle image, instead of the first one. This solved the issue that the model was drawing lines where it wasn't supposed to. The right most one is the result of current model which wasn't trained on this shape. . I was thinking for inference, we could use opencv's floodfill or some algorithm to fill the closed polygon given as input image. That's what I have done here. The only issue is that we have to find the gray level for image thresholding. That would be constant if we have images coming from the same source

    • Good point. Even filled polygons are theoretically correct inputs. Filled region represents material side. The input shapes are simple enough to be drawn in Paint with filled in. Then transformations can be applied to generate variations. If all approaches work well on this...We can have another directory for them and point to it from all approaches.

  • I don't think the dense/simple encoder decoder would work well on the new data.. that's because it's difficult to do image to image translation as is, on top of that fully connected networks lack the structure to encode useful image properties Which is why I had earlier suggested that the simple encoder decoder has overfit the data which is why it was giving good results and would not be able to generalize CNN encoder decoder might give better/comparable results with the UNet approach using the filled image as input. pix2pix as you said is theoretically incorrect to model this problem Which is also the reason why we don't really use dense networks even for image classification So I think we should focus on just the UNet approach, and maybe the CNN encoder decoder The only issue with current unet is with the joints, which I think would be solved with the weight maps trick weighing the region near the joints more than the other parts.. I'm trying to figure out the implementation details for the same I also wanted to ask one more thing - is there any python script or something that could be used to get the midcurves for the approaches currently being used? So that we could show the comparison of the current approaches vs our approach

    • No, unfortunately not. As there is no standard/one algorithm for it. Even things that are not working is important information and should be kept as demonstration. So all approaches can be tried on both hollow profiles and filled ones. Results can be gathered and even mentioned in papers to come. Looks ok? This also should be tried and documented, along with note that for such and such reasons the approach is not appropriate...imho..let others agree/counter...

References