Blender Integration

[yuanming-hu]

Continuing discussion (https://github.com/yuanming-hu/taichi/issues/299)

@PavelBlend said:
"You are planning to add a blender addon to this repository?:
https://github.com/taichi-dev/elements
Or will a separate repository be created for the blender addon?
I looked at my list of nodes and realized that there are a lot of them and at this stage of development it is too early to discuss all the nodes.
I will try later to reduce the number of nodes in the list to an acceptable number and publish this list.
And I had questions:
why was the repository named elements? Do you plan to rename it in the future?"

[yuanming-hu]

I suggest to have an addon inside this repo, since that's more convenient than manually synchronizing two repos. I guess both the simulator and the blender interface will evolve quickly, so keeping them together will help to make them consistent with each other. That's just my personal opinion - please feel free to propose yours :-)

On my end, Blender integration/users will be the top priority. I would also make the API reusable for other purposes, to get more people/contributors involved.

Regarding your question:
Elements is borrowed from Five Elements, a concept from traditional Chinese philosophy. ~2000 years ago Chinese people believe the world is composed of five elements and they interact with each other. This solver supports multiple materials and they can interact, so I picked this name.

It's just a temporary name - if you have a better proposal I'm happy to change it in the future.

[yuanming-hu]

3D simulation working: https://youtu.be/XAwL_QLP7uQ
You will need Taichi v0.3.20 to run it. Sorry about the poor visualization - hopefully you can tell it's 3D from rotations! :-)

[PavelBlend]

https://github.com/PavelBlend/blender-taichi-elements-concept
I added two new nodes: Set and Merge. Set node creates a list of objects. Similar to this action in python:

result = []
result.append(input_1)
result.append(input_2)
...
result.append(input_n)

Merge node must combine lists into one list. Which is equivalent to the following python code:

result = []
result.extend(input_1)    # type(input_1) = list
result.extend(input_2)
...
result_extend(input_n)

Here is an example of using these nodes:

Do you have any ideas how to change this system of nodes? Or maybe change the parameter names and node names? And one more question:

Do you have any experience with blender?

[yuanming-hu]

Looks cool! I think the current nodes are pretty good and, if it makes sense, maybe we should start trying to have a basic simulation node graph running, so that we can learn more about how exactly the simulation node system work in Blender and how the simulator should be designed to fit this interface. I'll try to play with your addon - could you point me to a tutorial or a few hints to have that running in my local Blender 2.81a? :-)

I only have one small suggestion now: is "make list" a better name for "set"?

In the future, I think the node graph can be more customizable. For example, users can specify varying stiffness/viscosity using textures. We can even let the users specify material constitutive models using a sub node graph (just like BRDF graphs in rendering), and have that JIT-compiled into a component of the simulator, to allow more flexibility.

To be honest I don't have much experience with Blender. I followed a few Blender toturials 3 years ago, and implemented a Blender addon for my old renderer, however it's been a while and what I have done with Blender is quite basic. The node system in Blender is quite new to me. I've been using nodes in Houdini. My experiences are more in building high-performance simulators and compiler, and I think that's exactly why we should work together to combine our expertise in both high-performance simulator building and Blender development :-)

[yuanming-hu]

I installed the addon and saw this info in the Blender logging window:

Modules Installed () from '/home/yuanming/Downloads/blender-taichi-elements-concept-master/taichi_elements/elements.zip' into '/home/yuanming/.config/blender/2.81/scripts/addons'

Trying to figure out how to open up a window and play with the nodes...

[PavelBlend]

You need to activate the addon in the settings (menu Edit > Preferences > Addons).
Enter the elements text in the search field. Check the box next to the Elements addon.
Next, open the Elements window:

Click the New button (to create a new node tree).
Add nodes using the Add menu or using the Shift + A keys

[yuanming-hu]

For some reason, I couldn't find the elements item in either official/community/testing...
I see the log as follows:

bpy.data.window_managers["WinMan"].addon_search = "elements"
bpy.data.window_managers["WinMan"].addon_support = {'TESTING'}
bpy.data.window_managers["WinMan"].addon_search = "elements"
bpy.data.window_managers["WinMan"].addon_support = {'COMMUNITY', 'TESTING'}
bpy.data.window_managers["WinMan"].addon_support = {'OFFICIAL', 'COMMUNITY', 'TESTING'}
bpy.data.window_managers["WinMan"].addon_search = ""
bpy.data.window_managers["WinMan"].addon_search = ""
Modules Installed () from '/home/yuanming/Downloads/blender-taichi-elements-concept-master/taichi_elements/elements.zip' into '/home/yuanming/.config/blender/2.81/scripts/addons'

One possibility is that when installing, the zip file of the addon doesn't have the correct format...

[PavelBlend]

You need to manually copy the folder taichi_elements in /home/yuanming/.config/blender/2.81/scripts/addons.

Since the zip archive with github contains a gitignore file, which interferes with the normal installation.

[yuanming-hu]

awesome! it works now.

[PavelBlend]

I can now prepare a blend scene so that you can clearly see which nodes can be connected to each other.

[yuanming-hu]

It feels so cool when actually playing with it!! Great work.

[PavelBlend]

Here is the scene.
node_tree_example.zip
I connected all the nodes so that all the connections were correct.
And each hub can have its own gravity, which affects all hub emitters.

I only have one small suggestion now: is "make list" a better name for "set"?

well. I will change this name.

[yuanming-hu]

Cool!

I really like the scalability and composability of this design. I guess the next step is to

• I'll modify the simulator code so that the nodes can be expressed in the simulator
• I'll continue working on the performance aspects of the simulator, to make it run faster
• (I have no idea about this one) Figure out how to actually plug the simulator code into the blender environment and your node system, and bake a simple simulation to start with. Would you like to take care of this one?

[yuanming-hu]

Actually I discussed with Ton Roosendaal and Brecht Van Lommel a year ago, on the potential integration of taichi_mpm into Blender. They seem quite enthusiastic about having a new multi-material simulator. Later I realized that since taichi_mpm does not support GPUs, maybe it's better to redesign a more powerful simulator for Blender (i.e. this repo elements) and integrate it.

If we get stuck on making the node system work, maybe we can restart the conversation and ask them for help, if you would like to.

[PavelBlend]

I don’t know this at the moment. Now I’ll start to figure it out. There are ready-made node systems:
https://github.com/JacquesLucke/animation_nodes
https://github.com/hsab/GrowthNodes
https://github.com/Sverchok/Sverchok

You can look at how the implementation of the node tree in these add-ons is implemented.

And I created a Simulate button on a Simulation node. If you click on this button, the text Start simulation will be printed in the console. This button is an operator (bpy.types.Operator). You can bind a node analyzer (ordinary arbitrary function) to this operator. And after analysis, run the simulator with the settings from the node system. But I am not sure of the correctness of this decision. If I find the best solution, I will inform you about it.

[PavelBlend]

I think that you can collect information from all nodes and create a dict based on this information.
The code for collecting information can be placed here:
https://github.com/PavelBlend/blender-taichi-elements-concept/blob/8476eb9a29e00925256ef7e7483e2eec808b1017/taichi_elements/operators.py#L9

I'll try to do that. Having a list of all parameters, you can create instances of classes (for example, the Emitter class with parameters, time of appearance, source object, position...). After that, you can pass the Emitter class to the simulator.

[PavelBlend]

I wrote code that collects information from the node tree. The simulation class is created here:
https://github.com/PavelBlend/blender-taichi-elements-concept/blob/c6a27153506aae4bfd4995b1f35844b67dfe703b/taichi_elements/operators.py#L47

If you click the Simulate button on the Simulation node, then information about the simulation classes is printed to the console.

Classes are described here:
https://github.com/PavelBlend/blender-taichi-elements-concept/blob/c6a27153506aae4bfd4995b1f35844b67dfe703b/taichi_elements/types.py

[yuanming-hu]

Awesome! I'm starting to integrate the simulator to the addon now.

[yuanming-hu]

Apart from Blender, I think it will be helpful to release taichi_elements as a PyPI package, so that more people can get involved and contribute, e.g. those doing research in mechanical engineering who needs an MPM simulator.

I also propose to merge two repos. The pros are:

1. Both the development of taichi_elements and blender-taichi-elements-concept will be simplified since no version synchronization will be needed
2. Including the addon in the main simulator repo conveys a message, that the simulator is designed with the consideration that Blender users are one of the top priorities. This will encourage more Blender users to try it.

All your previous commits in blender-taichi-elements-concept are preserved in the commit history in the merged branch. If you are happy with this proposal, I'll make merged `master

[PavelBlend]

If you are happy with this proposal, I'll make merged `master

Yes, everything suits me.

But I found a problem in the node system. Extra classes are created if the node is used simultaneously in several relationships. You do not need to create a class for the node if this class already exists. I also noticed that the DiskCache class is not being created.

After merging the branches with the master branch, I will delete my repository in order not to mislead users.

[yuanming-hu]

Sounds good. I have updated the master branch.

Extra classes are created if the node is used simultaneously in several relationships.

Yeah, I have also been slightly confused by this: Is the node graph a tree or a directed acyclic graph? I thought it should be the latter yet a tree is exposed now.

Now I can read the gravity, resolution, material settings and can add a cube. I'll show you a demo in a minute :-)

[yuanming-hu]

Integrated a basic simulator: https://youtu.be/3kUp0GtOfEg
Cubes and nodes:

(I don't know where to write the particles, so I wrote the images to the GUI...)

[yuanming-hu]

Blender file with commit 4532d1f
simple_example_commit_4532d1f.zip

[PavelBlend]

There are problems with particles in the blender. The particle system has errors. Here is the bugreport:
https://developer.blender.org/T68436
If you use ParticleSystem.particles.foreach_set, the blender becomes unstable and the result is incorrect.

I think so far it’s not necessary to create particles, but to create a mesh that has only vertices.

[yuanming-hu]

Creating a mesh sounds good, as long as we can manipulate (e.g. render) the simulated particles.

[PavelBlend]

I made it possible to import simulation particles as an object mesh:
https://youtu.be/lRCqzcWIa2I

This is a temporary solution. In the future, you can add a particle system. When it will become more stable.

And I noticed that the scale of the particles is different from the scale of the original objects. With what it can be connected?

update:

I made the following bundles of nodes equal:

I saw in the code that you added a check for the number of gravities:
https://github.com/taichi-dev/elements/blob/4532d1fc6a48245320d8fcae920c503a857eee93/operators.py#L71
I have a suggestion:
Allow the simulator to have multiple gravitations, but make them stack.
For example, if the speed of the first gravity is 9.8, and the direction is 0, 0, -1, and the speed of the second gravity is 9.8 and the direction is 0, 0, 1, then these gravitational forces will cancel each other out. As a result, the total gravitational force will be equal to 0.0.

or another example:
gravity_1: speed=9.8, direction=0, 0, -1
gravity_2: speed=9.8, direction=0, 0, -1
gravity_final: speed=19.6, direction=normalized(0, 0, -2)

[yuanming-hu]

Looks cool!!

The scale issue is due to a redundant normalization I did here to make it fix Taichi's GUI system (with coordinates from 0 to 1): https://github.com/taichi-dev/elements/blob/fix-node-tree/operators.py#L67 I'll remove that.

I think the node simplification is great! For beginners often they only have one force or one emitter, in that case removing the Make List node will make their life much easier.

I'll implement the gravity stacking mechanism. Also, we may consider support nodes that locally appls gravity: e.g. that "local gravity" happens only within cube.03.

[PavelBlend]

I created a Hub node in order to group forces (gravity, wind, whirlwind ...) with emitters. Here is an example of nodes:

Two emitters have gravity with a value of 9.8, and the other two emitters have gravity with a value of 0.0. And all this within the limits of one simulation.

[yuanming-hu]

Oh, now I understand why the existence of the "Hubs". Do you think it is necessary to also have a "global gravity" connected to the MPM solver that works for all hubs?

[PavelBlend]

Do you think it is necessary to also have a "global gravity" connected to the MPM solver that works for all hubs?

I think you can do without global gravity. Perhaps in the future you can do this if it seems convenient. Now you can create one common hub.

[yuanming-hu]

OK, sounds good!

[PavelBlend]

I forgot to say:
I added a new parameter for the MPM Solver node. This parameter is called Domain Object. I think that it is possible to extract information for a domain from this object. You can calculate the resolution for all sides of the domain, domain size, position, etc. Here is a description of this object:
https://docs.blender.org/api/current/bpy.types.Object.html

[yuanming-hu]

Yeah, I figured that out. I'll temporarily use a box as the bounding domain. Ultimately we may want to make the simulation domain unbounded (i.e. grow as simulation goes on). #3

[PavelBlend]

It may be to change the name of the addon blender from taichi_elements to elements? To make the name more concise.

Or is it all left as is?

[yuanming-hu]

Actually I'm planning to rename the repo from elements to taichi_elements and then we can release a PyPI package named taichi_elements, to distinguish it from many other "elements"... I agree elements is more concise, but taichi_elements is probably more searchable, as there are too many packages called elements.

[yuanming-hu]

@PavelBlend Btw how do you visualize the .bin files?

[PavelBlend]

I create a particle system, and indicate the positions of the particles this way:

taichi_elements/handlers.py

Line 122 in 3c70ca1

particle_system.particles.foreach_set('location', particles_locations)

This code is run every time the value of the current frame changes:

taichi_elements/handlers.py

Line 144 in 3c70ca1

bpy.app.handlers.frame_change_pre.append(import_simulation_data)

[theois]

Are you guys aware of the the Simulation nodes proposal by Jaques Lucke at the Blender institute?
It is very similar to what you are developing.

https://devtalk.blender.org/t/unified-simulation-system-proposal/11394

[EgleXR]

I suggest to have an addon inside this repo, since that's more convenient than manually synchronizing two repos. I guess both the simulator and the blender interface will evolve quickly, so keeping them together will help to make them consistent with each other. That's just my personal opinion - please feel free to propose yours :-)

On my end, Blender integration/users will be the top priority. I would also make the API reusable for other purposes, to get more people/contributors involved.

Regarding your question:
Elements is borrowed from Five Elements, a concept from traditional Chinese philosophy. ~2000 years ago Chinese people believe the world is composed of five elements and they interact with each other. This solver supports multiple materials and they can interact, so I picked this name.

It's just a temporary name - if you have a better proposal I'm happy to change it in the future.

Everything is the same thing, in different states. So my suggestion would be to separate it into 5 main states:

• solid
• Liquid
• Gaseous
• Plasma
• Luminous

[PavelBlend]

Hello, @yuanming-hu
I want to ask: do you have the opportunity to develop this simulator?
I would like this simulator to be in a minimum working condition. Now there is not enough seeding of particles from arbitrary geometry. I think if there is a seed from arbitrary geometry, then the simulator can already be used. It will also be possible to advertise this simulator on blender forums to attract more blender-python programmers who could help develop the simulator.

[yuanming-hu]

Hi @PavelBlend - I think it would be a great idea to push this project to a minimal working condition so that people from the Blender community can help.

I guess the most tricky step in implementing #5 is to convert a Blender geometry into a 3D voxelized grid/3D texture, so that Taichi can simply read from a 3D bumpy array and seed particles on the voxels that are within the geometry.

I have been spawned by the development of Taichi itself for too long - sorry about that. The good news is that now Taichi is more stable and capable than before. These improvements have made elements compile/run faster automatically.

[PavelBlend]

I think you can close this issue. Since we integrated the simulator into a blender. The simulator works in a blender and this was the goal of this issue. I think that it is possible to create new issues as necessary, rather than writing to this general issue.