Specifications #1
Comments
|
For feedbacks @alfonsogarciacaro @ncave @Zaid-Ajaj @dbrattli @alexswan10k |
|
This looks great. Couple of minor observations
|
Originally, there was a note to tell that this tag probably will never exist in reality.
I don't think this is true because you can use compiler directives if you need to shadow/mock API from different target. Imagine, you are creating a let readFile (filePath : string) =
#if FABLE_JAVASCRIPT
node.api.readFileSync(filePath)
#endif
#if FABLE_RUST
rust.readFile(filePath) // Pseudo code as I don't know the rust instruction
#endif
// etc.In the documentation, I think we will teach library authors which tag to use.
The old tag is harmless indeed, it just doesn't allow for precise search.
I know I was forgetting one language in my list but couldn't find which one ahah. Hum, well theory any package written I think it is would be easier to make only |
|
I think this makes sense. All packages are basically javascript with potential typescript type annotations, so I guess for a binding, it would always be js. With supported languages though, this is a little more vague since Emit can now export type information. Type information will obviously not compile if you are targeting javascript and webpack-ing .js files. With respect to conditional compilation, I do not think it is best to encourage package authors to tag with For compilation switches, I would suggest authors explicitly tag the languages they support ( |
|
Testing how tag search work on NuGet.org is actually possible with their website/URL. On their website, you can use
The corresponding URL is This discovery made me remove the part of the section
Because the rules for tags doesn't seems to be contains but exact. If you search for the tag
Also if you search using 2 tags the search seems to be an OR and sort the result by relevance but even if
With this new discovery I think in order to have a user experience which allow to search by:
we are going to have specialised tags. FableUse the tag Package type tagsThe packages should have one of these two tags to make it possible search by package type:
Target tags
If we want to be really declarative we could go with Package & Target tags
Tests the search scenario
I used Example: User experienceIn regard of the user experience when using We will not propose to the users the different tags but more general search option like:
By default, the tool will search for package with the tag If the user select If the user select If the user select If the user select Author experienceFor the author, the complexity cannot really be hidden but we can work on having great 😊 documentation. And we can provide a similar tool as for the standard user search, which ask the author what his library does and generate the tags to add to the package. |
You are right indeed, I was not thinking far enough in the future. |
|
Just made a discovery. Looks like you can do And with the following query:
Perhaps composition is an option after all |
|
@alexswan10k Nice find. Indeed, if we can do AND query I think we can we can go back to the original specification. Ironically, there is one target that we kept forget which is For example, FsToolkit.ErrorHandling works both on JavaScript and dotnet currently. And I believe it will also work for all the others targets too. |
|
This sounds like a great idea! I've been unsure if I should call my packages |
|
@dbrattli In your case, I would probably go with
But indeed, the tags will help us know what a package does and make it easier to discover them (I hope so). |
|
I started the work on the website and the more I think about it the more I like my second proposition with specialised tags.
This also make it really easy to implements the query generation: [<RequireQualifiedAccess>]
type Target =
| Dotnet
| JavaScript
| Rust
| Python
| Dart
| All
type SearchFormResult = {
SearchForBinding: bool
SearchForLibrary: bool
Targets: Target list
}
let targetToTag (target : Target) =
match target with
| Target.Dotnet -> "fable-dotnet"
| Target.JavaScript -> "fable-javascript"
| Target.Rust -> "fable-rust"
| Target.Python -> "fable-python"
| Target.Dart -> "fable-dart"
| Target.All -> "fable-all"
let searchToUrl (request : SearchFormResult) =
let tags =
[
"fable"
if request.SearchForBinding then
"fable-binding"
if request.SearchForLibrary then
"fable-library"
yield! List.map targetToTag request.Targets
]
|> List.map (fun tag ->
$"\"%s{tag}\""
)
|> String.concat ","
|> window.encodeURIComponent
$"https://azuresearch-usnc.nuget.org?q=Tags%%3A%s{tags}"If the user select If the user select If the user select If the user select We could probably simplify Example: But I kind of like the |
|
Today, I was able to make great progress on designing the interface for the tool. Not everything is functional yet, but I think the design is starting to look good. And is enough to give you an idea of the direction I took. fable_packages_demo.mp4 |
It looks like this syntax is actually not doing a "AND query over the tags". But looking for package with the tag I released a version of Glutinum.Fuse with one of the suggestion I made for the naming to give it a try.
The tags applied to the package are:
If you query for
If you query for
If you query for I tried other syntaxes like NuGet v3 API is not Rate limited so I guess we are going to need to handle search on our side.
My idea is to use the search syntax for what works well like searching in Text + options like "include prerelease" etc. And then once we get that result, fetch all the packages informations and apply the tag filters on the client side. I will make a naive implementation to test this idea. If needed later it is always possible to add caching mechanism etc. |
|
Hello, My suspicion about AND queries not working on tags has been confirmed by the NuGet team. During the last days I made a prototype which export the search responsibility to the user browser. fable_packages_indexing_demo.mp4Indexing is taking a bit of time around 10 sec here for 360 packages. It is possible to reduce the time by sending more request in parallel but I think having a progress bar instead of a generic loading animation is better for the user. Also, I am planning to use Web Worker to cache the request result so on future visit the indexing should be much faste. Last point, my vision for the tool is that in theory user will not have to go to There are a few limitations that exist but in theory we have some time before reaching them. The limitation will be documented both in the code and in the README or an issue. |
|
Hmm interesting. So to be precise, they are not actually saying that it is OR. They are saying each term is independently scored and aggregated (lucene query behaviour), so adding two terms Tag:A, Tag:B means each tag contributes to the score. If this is accurate, you do not need an absolute filter such as AND, you simply need something with multiple matched terms to be scored higher, so these results appear at the top. According to the response, this appears to be the case. What I believe is confusing matters is the fact the download count is massively contributing to the score also, thus throwing off the results. It would be nice to have an additional filter to throw out everything non-fable, but I guess this gets us into the And/Or dilemma discussed in the response. I am not sure this is the end of the world though since the fundamental behaviour for scoring tags is expected. As to keeping an index of the whole nuget repository locally - it's a nice POC, but not something I would generally recommend unless there really is no other way, as there are a lot of moving parts to go wrong. Looking good other than that though. |
We don't have any other way to filter correctly the packages. Their search engine gives too much weight to the download count and because it doesn't do an AND on field scoped term this cause problem. The main query which retrieve all the Fable tagged package is never cached and will always be up to date with NuGet repository when the user visit the website. That query currently takes around one second to execute, we are forced to do it in one query because of when using the pagination the retrieved list of packages is not stable and you end up with duplicates and missing packages... The response that are going to be cached is the response for a specific version of a package because we don't have access to the Because we are going to use a stale while re-validate cache this should not cause any problem. |
|
I just realised a version of Fable.Packages online. You can find it here https://fable.io/packages/ This version has more or less all the features I could imagine. It will also serve as a V1 to gather feedbacks and see what features needs improvement or is missing. V2 will be a complete rewrite of the project using a server to fix the performance problem we have with the indexing process. I will take the time in the next days to write the specification that has been implemented. And also the list of all the features that this tools has. This will serve a base for the V2 rewrite and help keep track of if something is supported or not. |
|
Looks great. Indexing is not as slow as I expected actually, although 350 packages is not exactly a big dataset, so if it grows substantially I guess the V2 bit is more interesting. Nice work. |
|
I published a new version which has a really basic cache which works for the duration of the tab life. Previously, when the user was doing:
Then he was having the indexing process happen twice (on step 1 & 4). Now, it will only happen on step 1 and not on step 4. If the user reload the tab, the cache is destroyed. |
|
Documentation about how the tags system works is available here |
|
Closing as the specification is now hosted on Fable docs and the announcement will be made tomorrow |
Introduction
Finding Fable packages has always been a difficult tasks.
Several solutions have been tried to try fixing this problem:
But both of this project had flows, they require external actions beside pushing the code to NuGet in order to make it available. This makes them easily outdated and would require additional work from the package author when they are already have more important stuff to focus on.
A tool to
ruleunify them allHosted under: https://fable.io/packages
The idea behind this tool is to use the NuGet API to find all the Fable packages. This means the list is always up to date with what is published on NuGet.org.
The only requirements is going to add the right tags to the published packages.
There are 2 types of Fable packages:
Libraries
A library is a collections of pre-written code that user can use.
Example: Elmish, Feliz, Thoth.Json.
Bindings
A binding is just here to expose native API to F# world. Sometimes it can also contains some helper code to unwrap/wrap Union types for examples.
For example, a binding can expose the Browser API to F# world so user can write
document.querySelector(...)Example: Fable.Browser.*, Glutinum.Fuse
Both libraries or bindings can be:
Runtime agnostics works for (.NET, JavaScript, Python, Rust, etc.)
Examples:
- Fable.Core
- FsToolkit.ErrorHandling
Runtime specific
Tags are going to be used to identify if a package is a binding or a library and what runtime is supported
Tags
Libraries
Format:
fable:<target>Supported tags:
fable:allfable:javascriptfable:pythonfable:rustfable:dartBindings
Format:
fable-binding:<target>Supported tags:
fable-binding:allfable-binding:javascriptfable-binding:pythonfable-binding:rustfable-binding:dartExample:
If a library support both Python and JavaScript, then the NuGet package will be published with:
Notes
Contrary to what I have said in the past, I don't think we should have a
fabletag.With the proposed tags I think we can support the following scenarios:
fablefable-bindingfable:javascriptfable-binding:javascriptfable:javascriptANDfable-binding:javascriptThe text was updated successfully, but these errors were encountered: