Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let F3D take as input a model via streaming rather than filename #1100

Open
mrdeveloperdude opened this issue Dec 17, 2023 · 7 comments
Open
Labels
type:enhancement New feature or request
Milestone

Comments

@mrdeveloperdude
Copy link

mrdeveloperdude commented Dec 17, 2023

See related issue for streaming output.

F3D currently accepts a model name as a command-line argument.

F3D will then read the model from disk before generating a preview.

In many cases model files can be quite large and reading them from disk can incur significant IO penalty. Further, in many cases we already have the contents of the model file in memory from previous work.

This feature request is for having a sanctified and documented way of streaming the model data into F3D without having to save it to disk first to avoid said IO penalty.

There are ways to possibly do this (untested by issue author) by the use of tmpfs on linux platforms, and by passing /dev/stdin as the model filename.

There is also the possibility of using libf3d instead of the stand alone executable.

Here are some relevant comments from discord:

stream input is on the to-do list but requires changes in underlying libs so the ETA isn't entirely up to F3D (the changes are slowly but surely making their way into VTK as far as I know)
stream output should be doable on F3D's side

tracks that can be investigated without waiting for any new features:
using a ram disk, I'd be interested in seeing a benchmark of piping from stdout to stdin vs. writing and reading in a tmpfs directory (in theory piping should be better because it allows actual streaming, however in the case at hand we can't really render without buffering the whole input model and output image anyway so the streaming aspect won't matter and we just want fast/cheap IO)
using libf3d instead of the F3D application, for example through the python bindings, would let you read rendered images from memory and also allows to do multiple renders of the same model without reloading it

@mwestphal
Copy link
Contributor

Please note being able to stream would be great for two other issues:

This already support by the libf3d API however not all format supports it, this require major improvements in VTK to support all formats.

It also requires to identify how to do that with the F3D application in a cross platform way.

@mwestphal mwestphal added the type:enhancement New feature or request label Dec 17, 2023
@mwestphal mwestphal added the help wanted Please help with this issue! label Jan 7, 2024
@mwestphal mwestphal added workflow:discussion and removed help wanted Please help with this issue! labels Jan 26, 2024
@mwestphal mwestphal added this to the 2.4.0 milestone Feb 4, 2024
@joa-quim
Copy link

joa-quim commented Mar 9, 2024

I would love to give this a try from Julia. But C++ is difficult to interface with. It would much easier if a C API could be added as well. ... Just wishing.

@mwestphal
Copy link
Contributor

We thought about a C API but it would have been too restrictive. Maybe use the python API instead ?

@joa-quim
Copy link

joa-quim commented Mar 9, 2024

OK, it's perhaps a bit too soon to talk about it since this feature doesn't exist yet, but a python API is no option for me. What I'm interested is in a way to exchange data, already in memory, between Julia and F3D. I mentioned C for two reasons. First because that's what I know and second because Julia has a very convenient way of communicating with C that doesn't depend on any 3rd party tools.

@mwestphal
Copy link
Contributor

Sure, C API is not planned for F3D I'm afraid.

@Meakk
Copy link
Contributor

Meakk commented Mar 9, 2024

I agree with @joa-quim, C bindings are required. Not only for Julia, but other language like Rust could use it.

@mwestphal
Copy link
Contributor

mwestphal commented Mar 9, 2024

C bindings seems indeed required in order to support interop with rust and julia. Lets see if there is interest from the community: #1307

@mwestphal mwestphal modified the milestones: 2.4.0, 2.5.0 Apr 4, 2024
@mwestphal mwestphal modified the milestones: 2.5.0, 3.0.0 Jul 3, 2024
@mwestphal mwestphal modified the milestones: 3.0.0, 3.1.0 Jul 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:enhancement New feature or request
Projects
Status: Investigate
Development

No branches or pull requests

4 participants