Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can I write WebGL with FShade ? an integrate it with Fable.io? #14

Closed
goswinr opened this issue Nov 16, 2018 · 17 comments
Closed

Can I write WebGL with FShade ? an integrate it with Fable.io? #14

goswinr opened this issue Nov 16, 2018 · 17 comments

Comments

@goswinr
Copy link

goswinr commented Nov 16, 2018

This is a question not an issue. I am new to Shaders (but not F#) . Can I use this to write WebGL? What would be the best strategy to integrate it with Fable.io?

@voronoipotato
Copy link

This sounds like a fun exercise to test lol.

@krauthaufen
Copy link
Owner

Hi,
I think it would be great to have FShade available in Fable, but sadly i don't think that this will be easy (or even possible)
FShade internally uses F#'s Expr and a ton of reflection things, which are most likely not available in Fable.
The only way I currently see would be to integrate parts of FShade's compiler directly in Fable.

An OpenGL ES backend should be fairly easy to implement, and (when compositions are statically known) the resulting code could somehow be 'injected' into the resulting javascript code.

I'll definitely look into that when I have the time.
Cheers

@voronoipotato
Copy link

https://github.com/fable-compiler/Fable/blob/master/tests/Main/ReflectionTests.fs

For the record I think this is super cool but we also shouldn't derail your project with it even if it is possible or easy. Don't be blinded by the beauty of feature creep @krauthaufen , let the community explore FShade for fable.

@voronoipotato
Copy link

First attempt

Missing Quotations

Seems like the major barring factor to even generating GLSL is that quotations aren't supported in fable.

Fable can't find or resolve things,

This is likely caused by the framework mismatch

Conclusion

Less stuff broke than I thought, but most things broke.

module App

(**
 The famous Increment/Decrement ported from Elm.
 You can find more info about Emish architecture and samples at https://elmish.github.io/
*)

open Elmish
open Elmish.React
open Fable.Helpers.React
open Fable.Helpers.React.Props
open Aardvark.Base
open FShade
// MODEL

type Model = int

type Msg =
| Increment
| Decrement

type Data = {pos: V4d; tc : V2d}
let shader (d: Data) = 
  vertex {
    let mvp : M44d = uniform?ModelViewProjMatrix
    return { pos = mvp * d.pos; tc = 2.0 * d.tc }
  }
let effect = Effect.ofFunction shader

let s = 
    effect
    |> Effect.toModule { 
        EffectConfig.empty with 
            lastStage = ShaderStage.Vertex
            outputs = Map.ofList ["pos", (typeof<V4d>, 0)]
        }
    |> ModuleCompiler.compileGLSL glsl410
   // |> GLSL.Assembler.assemble glsl410


let init() : Model = 0

// UPDATE

let update (msg:Msg) (model:Model) =
    match msg with
    | Increment -> model + 10
    | Decrement -> model - 1

// VIEW (rendered with React)

let view (model:Model) dispatch =

  div []
      [ button [ OnClick (fun _ -> dispatch Increment) ] [ str "+" ]
        div [] [ str (string model) ]
        div [] [str (string s)]
        button [ OnClick (fun _ -> dispatch Decrement) ] [ str "-" ] ]

// App
Program.mkSimple init update view
|> Program.withReact "elmish-app"
|> Program.withConsoleTrace
|> Program.run

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(1,1): error FABLE: Cannot find FShade.EffectConfig constructor
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(24,2): (24,8) error FABLE: Cannot resolve FShade.ShaderBuilders.vertex
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(24,2): (24,8) error FABLE: Quotes are not currently supported by Fable
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(28,13): (28,37) error FABLE: Cannot resolve FShade.EffectModule.ofFunction
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(32,23): (36,9) error FABLE: Cannot resolve FShade.EffectConfigModule.empty
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(32,23): (36,9) error FABLE: Cannot resolve FShade.EffectConfigModule.empty
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(32,7): (36,9) error FABLE: Cannot resolve FShade.EffectModule.toModule
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(37,7): (37,41) error FABLE: Cannot resolve FShade.Backends.ModuleCompilerModule.compileGLSL
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

ERROR in ./src/App.fs
Module Error (from ./node_modules/fable-loader/index.js):
C:/Users/Vorotato/projects/FableFShade/src/App.fs(37,7): (37,41) error FABLE: Cannot resolve FShade.Backends.glsl410
@ ./src/App.fsproj 1:0-25 1:0-25
@ multi (webpack)-dev-server/client?http://localhost:8080 ./src/App.fsproj

C:\Users\Vorotato\projects\FableFShade\src\App.fsproj : warning NU1701: Package 'Aardvark.Base 3.0.34' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
C:\Users\Vorotato\projects\FableFShade\src\App.fsproj : warning NU1701: Package 'Aardvark.Base.FSharp 3.0.34' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
C:\Users\Vorotato\projects\FableFShade\src\App.fsproj : warning NU1701: Package 'DevILSharp 0.0.13' was
restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
C:\Users\Vorotato\projects\FableFShade\src\App.fsproj : warning NU1701: Package 'FShade.Core 3.2.4' was
restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
C:\Users\Vorotato\projects\FableFShade\src\App.fsproj : warning NU1701: Package 'FShade.GLSL 3.2.4' was
restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
C:\Users\Vorotato\projects\FableFShade\src\App.fsproj : warning NU1701: Package 'FShade.Imperative 3.2.4' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
C:\Users\Vorotato\projects\FableFShade\src\App.fsproj : warning NU1701: Package 'FShade.SpirV 3.2.4' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
  Restore completed in 14.92 ms for C:\Users\Vorotato\projects\FableFShade\src\App.fsproj.
  Restore completed in 27.5 ms for C:\Users\Vorotato\projects\FableFShade\src\App.fsproj.

@krauthaufen
Copy link
Owner

Hi, it would be extremely awesome to see FShade working in Fable. Here's a little outline of the things FShade currently uses internally:

  • Aardvark.Base for vector/matrix types (should be relatively easy to replace)
  • Aardvark.Base.FSharp for some datastructures (also possible to replace with F#'s default ones mostly)
  • FsPickler for hashing exprs which is not necessary for a first prototype
  • F# quotations which are the core of the entire system. These would need to be available in Fable

The real problem here is, that we need runtime-type information for exprs in order to generate GLSL.
I cannot tell how hard these things are in fable.

@voronoipotato
Copy link

Maybe something more near term that could be fun is using say Giraffe and FShade to generate GLSL and incorporate that into a page to be sent as a page request. Building that would require no changes to either fable or fshade, and using Fable.Bridge you could have the GLSL change without page refresh. I doubt adding quotations is extremely high on Fable's priority list.

@krauthaufen
Copy link
Owner

Hi,
Generating GLSL code currently involves executing the user-code in order to get the Quotation Expressions, which is a deal-breaker for a server-side API since it basically allows users to execute arbitrary code on the server.
I'm currently exploring possibilities to generate FShade's AST directly from the FSharp.Compiler.Service TAst. This is quite huge change since FShade uses the Quotations everywhere and these would need to be replaced with something different.
It would be nice to have an intermediate language that can be created from Quotations/TAst/custom parsers, etc.
This might take some time to implement, but when its done FShade could be running as a simple web-service and possibly even parse GLSL input shaders.

Cheers

@goswinr
Copy link
Author

goswinr commented Nov 26, 2018

Looks like quotation are not coming anytime soon to Fabel: https://twitter.com/alfonsogcnunez/status/1066996863449075714?s=21

@7sharp9
Copy link

7sharp9 commented Nov 26, 2018

quotations completely depend on reflection, you can sort of fool things a little but it depends on the use case.

@7sharp9
Copy link

7sharp9 commented Nov 26, 2018

Quotations can be transformed to AST, I do this in Falanx, and AST becomes TAST on compile, you can compile the AST too via an interface into the compiler that takes the AST as input. (Here AST refers to the parse tree)

@voronoipotato
Copy link

voronoipotato commented Nov 26, 2018

@krauthaufen I wouldn't worry about removing quotations and increasing the complexity of FShade. I didn't mean a website where users submit fsharp and get shaders and canvas out but just an example where GLSL gets generated from F# (that you wrote) and then injected into a page which gets rendered and the fsharp and glsl gets shown in the example. I wouldn't focus on making FShade fable compliant right now, more full documentation, and more examples on the website would be preferred. Some code with some associated pictures for example of things you can render with it would be nice.

What would the value of the intermediate language provide?

I noticed you remarked on the site that DU's aren't easily supported, I wonder if that could be hacked up via faux enums, aka consts with the GetHashCode of the DU cases. Since we already have the compile time checks in F# I would think we don't have to worry so much about the generated GLSL being insane.

@krauthaufen
Copy link
Owner

@7sharp9 @voronoipotato I couldn't stop thinking about the idea of FShade running in a browser and after approximately 4 weeks of work i finally got fshade running in fable 😁

I extended fable with quotation support (see fable-compiler/Fable#1818), ported some components of aardvark.base and aardvark.rendering and FShade.

The thing is currently very fragile and various features don't work, but here's a video showing the basic functionality: https://www.youtube.com/watch?v=s5rO0RUXXmo

The implementation is private atm. but I intend to make it open-source when it's done.
Cheers

@voronoipotato
Copy link

Any hopes opening it up so we can hack on it :)?

@krauthaufen
Copy link
Owner

Hey, I'll try to make it public, but currently it's very interwoven with a commercial project which we can't open-source yet.

@voronoipotato
Copy link

I appreciate it, and I also understand. Sometimes it's not quite so easy :).

@markpattison
Copy link

This would be awesome!

@krauthaufen
Copy link
Owner

Pretty late, but now there is something using FShade in the browser (not fable though).
https://github.com/aardworx/aardworx.webassembly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants