Blender to Luisa Render

[orarugogerincufelpattano]

Hi Mike, let's continue here with Questions regarding scenes and data coming from Blender.
I'm looking for a renderer (physically correct) which I could use for automotive lighting related data and therefore created a simple test scene. A bent light guide (with fake optics) and an area light to see how well the renderer can handle the complicated light distribution.
It looks like this in Cycles:

As you see, Cycles is not able to distribute the light.
I also tested Mitsuba,

but I'm not able to increase the samples as it hits a memory limit and either crashes or renders forever.
I analysed some of the sample scenes and started to manually assemble this scene for Luisa.
I exported the parts as obj and used the glass-of-water demo JSON file as a "base" to exchange with my data.
One thing which seems t be a problem, is the camera as in Blender it only has a position and a rotation but no "front".
I guess front in Luisa is a look at point or a vector pointing from the position towards the look at.
I used this code (what I found in the net) to separate the Blender camera matrix to the required data:
`if ob.type == "CAMERA":

    cam = ob.data

    # Camera transform matrix

    m = ob.matrix_world

    # Compute at, look, up, fov

    pos = m.to_translation()

    at = m @ Vector((0,0,-1,1))   # Point one unit in front of the camera

    up = m @ Vector((0,0,1,0))    # Vector in the z direction

    fov = degrees(ob.data.angle)`

It seems to generate valid numbers so I used those in my Luisa scene.
The rendered image is black however so something is wrong.

1. I noticed that all of the demo files I looked at, had very small camera positions (numbers smaller than 1 most of the time).
   Are the units used in Blender matter in Luisa? As an engineer I use millimeters in Blender with a unit scale of 0.001. (As opposed to Meters and 1 in default Blender settings.)

2. Also, in the demo files the matrices for the meshes have default (0) values. My objects have a translation matrix. is that a problem?

3. Anyway, just to test if I can see something at all, I copied an environment setting out of the Dining-Room demo scene.
   As soon as I do that, Luisa starts to render the image (no complaints about the JSON file or missing Textures) and around half the rendering suddenly reboots the machine (reproducible) . It has to do with my scene as the dining-room renders correctly.

4. I use the cuda backend. I also tried CPU but then I get another error message:
   thread '<unnamed>' panicked at luisa_compute_backend_impl\src\cpu\llvm.rs:372[2024-02-26 18:11:53.639] [console] [:info29] Geometry built with 17822 triangles. [D:\a\LuisaRender\LuisaRender\src\base\geometry.cpp:23] : Could not find clang. Please set LUISA_CLANG_PATH to the path of clang++ executable note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
   (I would like to use CUDA anyway, so this is not really a problem for me. I only wanted to know if the sudden reboot comes from the use of the RTX card.)

EDIT: I forgot to mention, I use Z up in Blender but other packages mostly use Y up.

Here is my scene if you want to take a look:

LightGuide_Test.zip
I hope it's not too many questions at once...

EDIT: #3 I could solve. It was my mistake: in the dining_room scene the environment had a rotation around the Y axis (Y up).
I changed now this to a rotation around Z ( transform : SRT { rotate { 0, 0, 1, -65 } } ) and now it no longer crashes and reboots the machine. I can see now the environment but not my objects, so the camera is wrong. For the translation it would be easier if the camera in Luisa could handle position & rotation instead of position and front. Mitsuba uses also position and rotation so there it is a bit easier to convert.

[Mike-Leo-Smith]

Cool scene! However, the light transport in the scene looks quite difficult and I suspect rendering techniques like Stochastic Progressive Photon Mapping might be necessary. But anyway, I'll first look into the geometry/camera settings in the scene file. It might also help if you would not mind providing the original blender file (if it's not private/proprietary).

Answers to your questions:

1. I noticed that all of the demo files I looked at, had very small camera positions (numbers smaller than 1 most of the time).
   Are the units used in Blender matter in Luisa? As an engineer I use millimeters in Blender with a unit scale of 0.001. (As opposed to Meters and 1 in default Blender settings.)

Luisa doesn't have a unit system inside. As long as the objects/cameras etc are using the same scale, the rendering should work fine.

2. Also, in the demo files the matrices for the meshes have default (0) values. My objects have a translation matrix. is that a problem?

It should not be a problem. The only requirement is that matrices need to be "affine", i.e., the last row being 0, 0, 0, 1.

3. Anyway, just to test if I can see something at all, I copied an environment setting out of the Dining-Room demo scene.
   As soon as I do that, Luisa starts to render the image (no complaints about the JSON file or missing Textures) and around half the rendering suddenly reboots the machine (reproducible) . It has to do with my scene as the dining-room renders correctly.

This rebooting seems abnormal, even with a wrong scene setup. I'll look into this.

4. I use the cuda backend. I also tried CPU but then I get another error message:
   thread '<unnamed>' panicked at luisa_compute_backend_impl\src\cpu\llvm.rs:372[2024-02-26 18:11:53.639] [console] [:info29] Geometry built with 17822 triangles. [D:\a\LuisaRender\LuisaRender\src\base\geometry.cpp:23] : Could not find clang. Please set LUISA_CLANG_PATH to the path of clang++ executable note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
   (I would like to use CUDA anyway, so this is not really a problem for me. I only wanted to know if the sudden reboot comes from the use of the RTX card.)

The cpu backend requires clang visible in the PATH. You may download the LLVM-17.0.1-win64.exe installer from here.

For the translation it would be easier if the camera in Luisa could handle position & rotation instead of position and front.

Yes, you can use position and rotation to specify the camera like this:

Camera camera : Pinhole {
  fov { 39.59 }
  spp { 16384 }
  filter : Gaussian {
    radius { 1 }
  }
  film : Color {
    resolution { 1920, 600 }
  }
  file { "render.exr" }
  transform : SRT {
    rotate { ... }
    translate { ... }
  }
}

[Mike-Leo-Smith]

Thanks! I'll look into the scenes.

[Mike-Leo-Smith]

Hi, @orarugogerincufelpattano

I tried the Blender -> export glTF (Blender Menu-File-Export-glTF 2.0, select the glTF Separate Format in the popup) -> Luisa (with luisa-render-export.exe <path-to-the-exported-gltf>) path, together with the camera conversion script. The geometry seems correctly set up now. I'll also try to see if there is a convenient way to match the materials.

Here is the converted scene:
test.zip

[orarugogerincufelpattano]

Cool, thanks! I have very limited time currently so I was not able to continue with my export script yet. (Only the raw obj export is done.)
As the blender materials can be super complex, I don't really see an easy way how to properly "convert" materials. As soon as there is a mix shader node, it's hard to judge what material property one should use for the conversion. I'm thinking about using naming for the conversion, like: LR_Metal. whatever or LR_Glass.whatever, etc.
This way one could create a blender asset library with Luisa specific materials which could be than easily replaced in the conversion with a Luisa metal, glass or other material. I need to check if it's possible to get the final colour value at the material output node in python. If yes, one could use that colour value to pass it to the Luisa material.
These materials would look unrealistic in Blender but would allow the easy conversion to Luisa.
As soon as I have more time, I will take a look.

[orarugogerincufelpattano]

Your JSON files look different from the sample files I looked at. Especially the materials in comparison to the glass-of-water demo scene. There the glass material had a format like this:

Surface mat_Glass : Glass {
  Kr : Constant {
    v { 1.0, 1.0, 1.0 }
    semantic { "albedo" }
  }
  Kt : Constant {
    v { 1.0, 1.0, 1.0 }
    semantic { "albedo" }
  }
  eta : Constant {
    v { 1.5 }
  }
  roughness : Constant {
    v { 0.001 }
  }
}

Now it's like this:

   "Surface:00000:Glass": {
        "impl": "Disney",
        "prop": {
            "color": "@Texture:00000",
            "metallic": "@Texture:Metallic:1",
            "remap_roughness": true,
            "roughness": "@Texture:Roughness:1"
        },
        "type": "Surface"
    },

Now I see double quotes and keywords like "type":"Surface" which were not present in the demo file.
Are these still interchangeable?

[Mike-Leo-Smith]

Your JSON files look different from the sample files I looked at. Especially the materials in comparison to the glass-of-water demo scene.

Hi, @orarugogerincufelpattano.

These are the two scene description formats that Luisa supports. The first one is text-based, suitable for hand-written tests. The second is JSON-based, more structured, and easier for programmatic generation/conversion. They both parse to the same internal representation, so equivalent in functionality. The correspondences are as follows:

[Text]
Type node_name : Impl {
  property_node : Impl { ... }
  reference_node { @name_of_some_previously_defined_node }
  property_value { ... } // strings or numbers
}

[JSON]
"node_name": {
  "type": "Type",
  "impl": "Impl",
  "prop": {
    "property_node": {
      "impl": "Impl",
      ...
    },
    "reference_node": "@name_of_some_previously_defined_node",
    "property_value": [list] or "string" or number
  }
}

You can choose either to use, but mixing them in one file is not supported as that would kill the parser.