docs: add documentation for the change of native library loading.

docs/AdvancedTutorials/CustomizeNativeLibraryLoading.md

@@ -0,0 +1,196 @@
+# Customize the native library loading process
+
+In [the tutorial of native library config](../Tutorials/NativeLibraryConfig.md), we introduces how to auto-select a best native library as backend with your configurations. In this tutorial, we're taking a step further, to customize the native library loading process in a much more flexible way.
+
+## Customize the policy of the selection
+
+`INativeLibrarySelectingPolicy` is responsible for selecting the native libraries to load and sort them in order of priority. You could implement this interface and register your implementation by calling `NativeLibraryConfig.WithSelectingPolicy`.
+
+Note that when you're using your own implementation, the native libraries returned from `INativeLibrarySelectingPolicy.Apply` are all the libraries that are possibly used. It might spend lots of time of you to complete a robust implementation, but you can always fallback to the default behavior by calling `DefaultNativeLibrarySelectingPolicy` when no suitable library is found.
+
+```cs
+public interface INativeLibrarySelectingPolicy
+{
+    /// <summary>
+    /// Select the native library.
+    /// </summary>
+    /// <param name="description">The description of the user's configuration.</param>
+    /// <param name="systemInfo">The system information of the current machine.</param>
+    /// <param name="logCallback">The log callback.</param>
+    /// <returns>The information of the selected native library files, in order by priority from the beginning to the end.</returns>
+    IEnumerable<INativeLibrary> Apply(NativeLibraryConfig.Description description, SystemInfo systemInfo, NativeLogConfig.LLamaLogCallback? logCallback = null);
+}
+```
+
+## Customize the native library
+
+The native library in LLamaSharp is not just a file path. Instead, it's abstracted to `INativeLibrary`, which contains the information of it and allows finer-grained control of the loading process.
+
+```cs
+public interface INativeLibrary
+{
+    /// <summary>
+    /// Metadata of this library.
+    /// </summary>
+    NativeLibraryMetadata? Metadata { get; }
+
+    /// <summary>
+    /// Prepare the native library file and returns the local path of it.
+    /// If it's a relative path, LLamaSharp will search the path in the search directies you set.
+    /// </summary>
+    /// <param name="systemInfo">The system information of the current machine.</param>
+    /// <param name="logCallback">The log callback.</param>
+    /// <returns>
+    /// The relative paths of the library. You could return multiple paths to try them one by one. If no file is available, please return an empty array.
+    /// </returns>
+    IEnumerable<string> Prepare(SystemInfo systemInfo, NativeLogConfig.LLamaLogCallback? logCallback = null);
+}
+```
+
+`INativeLibrary.Metadata` contains the information of the native library. When you implement the interface, you can ignore it and return `null`. However, the best practice will always include implemenbting this peoperty because it will provide much more information when there's an anexpected behavior.
+
+`INativeLibrary.Prepare` is a method which allows you to do some preparations before loading the library. For example, you can move the file, download the file, output some logs, etc. It is supposed to return a list of file paths in order of priority. 
+
+There are several implementations inside LLamaSharp, which are `NativeLibraryFromPath`, `NativeLibraryWithAvx`, `NativeLibraryWithCuda` and `NativeLibraryWithMacOrFallback`. Please refer to their implementations if you find it difficult to implement `INativeLibrary`.
+
+## How does the loading work
+
+Here're the steps for the native library loading in LLamaSharp.
+
+1. Gather and validate the user's configuration for native library loading. (`NativeLibraryConfig`)
+2. Gather the system information, mainly including platform and device.
+3. Apply the selecting policy to get the list of native libraries to load.
+4. For each of the selected native library, call the `INativeLibrary.Prepare` method to get the file paths. Then Try to load the library from the paths in turn. If the library is loaded successfully with any path of them, the loop will be broken.
+
+## Example: use remote library files
+
+Though the native library downloading feature will be introduced as discussed in https://github.com/SciSharp/LLamaSharp/issues/670, it might have some security concerns. Here's an example to implement the remote native library downloading.
+
+Firstly, the native library with downloading process should be implemented.
+
+```cs
+public class AutoDownloadedLibraries
+{
+    // Wrap a cuda native library
+    public class Cuda: INativeLibrary
+    {
+        // the default cuda native library implementation in LLamaSHarp
+        private NativeLibraryWithCuda _cudaLibrary;
+        // Some download settings
+        private NativeLibraryDownloadSettings _settings;
+
+        public Cuda(NativeLibraryWithCuda cudaLibrary, NativeLibraryDownloadSettings settings)
+        {
+            _cudaLibrary = cudaLibrary;
+            _settings = settings;
+        }
+
+        public NativeLibraryMetadata? Metadata => _cudaLibrary.Metadata;
+
+        public IEnumerable<string> Prepare(SystemInfo systemInfo, NativeLogConfig.LLamaLogCallback? logCallback = null)
+        {
+            foreach(var relativePath in _cudaLibrary.Prepare(systemInfo, logCallback))
+            {
+                // try to use the default path first. If loaded successfully, the download will not be triggered.
+                yield return relativePath;
+                // download the file.
+                // NOTE: be sure to complete the downloading process here. You CANNOT make `Prepare` as an async method.
+                var path = NativeLibraryDownloader.DownloadLibraryFile(_settings, relativePath, logCallback).Result;
+                // if the downloading is successful, return the path of the downloaded file.
+                if (path is not null)
+                {
+                    yield return path;
+                }
+            }
+        }
+    }
+}
+```
+
+Then, implement the selecting policy for the native libraries above
+
+```cs
+public class SelectingPolicyWithAutoDownload: INativeLibrarySelectingPolicy
+{
+    // making this class a wrapper for the default policy.
+    private DefaultNativeLibrarySelectingPolicy _defaultPolicy = new();
+    // record the download settings.
+    private NativeLibraryDownloadSettings _downloadSettings;
+
+    internal SelectingPolicyWithAutoDownload(NativeLibraryDownloadSettings downloadSettings)
+    {
+        _downloadSettings = downloadSettings;
+    }
+
+    public IEnumerable<INativeLibrary> Apply(NativeLibraryConfig.Description description, SystemInfo systemInfo, NativeLogConfig.LLamaLogCallback? logCallback)
+    {
+        foreach(var library in _defaultPolicy.Apply(description, systemInfo, logCallback))
+        {
+            // check the return type and returns the corresponding wrapper
+            if(library is NativeLibraryWithCuda cudaLibrary)
+            {
+                yield return new AutoDownloadedLibraries.Cuda(cudaLibrary, _downloadSettings);
+            }
+            else if(library is NativeLibraryWithAvx avxLibrary)
+            {
+                yield return new AutoDownloadedLibraries.Avx(avxLibrary, _downloadSettings);
+            }
+            else if(library is NativeLibraryWithMacOrFallback macLibrary)
+            {
+                yield return new AutoDownloadedLibraries.MacOrFallback(macLibrary, _downloadSettings);
+            }
+            // Generally, you don't need to download the DLL if the user specify a path.
+            // But if you want, you can certainly add a wrapper for it.
+            else if(library is NativeLibraryFromPath)
+            {
+                yield return library;
+            }
+            // It's also reasonable to throw an exception here.
+            else
+            {
+                yield return library;
+            }
+        }
+    }
+}
+```
+
+Finally, add an extension to make an API for users to enable this feature.
+
+```cs
+ public static class NativeLibraryAutoDownloadExtension
+{
+    public static NativeLibraryConfig WithAutoDownload(this NativeLibraryConfig config, bool enable = true, NativeLibraryDownloadSettings? settings = null)
+    {
+        if (config.LibraryHasLoaded)
+        {
+            throw new Exception("The library has already loaded, you can't change the configurations. " +
+                "Please finish the configuration setting before any call to LLamaSharp native APIs." +
+                "Please use NativeLibraryConfig.DryRun if you want to see whether it's loaded successfully " +
+                "but still have chance to modify the configurations.");
+        }
+        if (enable)
+        {
+            if(settings is null)
+            {
+                settings = NativeLibraryDownloadSettings.Create();
+            }
+            // If you want to return an relative path in `INativeLibrary.Prepare`, 
+            // be sure to add the downloading directory to the search directories.
+            var defaultLocalDir = NativeLibraryDownloadSettings.GetDefaultLocalDir(settings);
+
+            // When using auto-download, this should be the only search this directory.
+            List<string> searchDirectoriesForDownload = [settings.LocalDir!];
+            // unless extra search paths are added by the user.
+            searchDirectoriesForDownload.AddRange(settings.ExtraSearchDirectories ?? []);
+            config.WithSearchDirectories(searchDirectoriesForDownload);
+
+            // register you selecting policy.
+            config.WithSelectingPolicy(new SelectingPolicyWithAutoDownload(settings));
+        }
+        return config;
+    }
+}
+```
+
+Now your users are now able to enable this feature by calling `NativeLibraryConfig.All.WithAutoDownload`!

docs/Tutorials/NativeLibraryConfig.md

@@ -1,4 +1,4 @@
-# Customize the native library loading
+# Configure the native library loading
 
 As indicated in [Architecture](../Architecture.md), LLamaSharp uses the native library to run the LLM models. Sometimes you may want to compile the native library yourself, or just dynamically load the library due to the environment of your user of your application. Luckily, since version 0.7.0, dynamic loading of native library has been supported! That allows you to customize the native library loading process.
 
@@ -21,22 +21,39 @@ We provide `LLama.Native.NativeLibraryConfig` class with singleton mode to allow
 All you need to do is adding the following code to the very beginning of your code.
 
 ```cs
-NativeLibraryConfig.Instance.WithLibrary("<Your native library path>");
+NativeLibraryConfig.All.WithLibrary("<Your native library path>");
+```
+
+If you want to configure the loading for LLama library or llava library respectively, please call the following APIs.
+
+```cs
+NativeLibraryConfig.LLama.WithLibrary("<Your llama native library path>");
+NativeLibraryConfig.LLava.WithLibrary("<Your llava native library path>");
 ```
 
 ### Automatically select one from multiple native library files
 
 Let's consider this case: you don't know your user's device when distributing your application, so you put all the possible native libraries in a folder and want to select the best one depending on the user's device. LLamaSharp allows you to define the strategy to do it.
 
-- `NativeLibraryConfig.Instance.WithCuda`: decide if you want to use cuda if possible.
-- `NativeLibraryConfig.Instance.WithAvx`: decide the highest AVX level you want to use if possible.
-- `NativeLibraryConfig.Instance.WithSearchDirectory`: specify the directory to search the native library files.
-- `NativeLibraryConfig.Instance.WithAutoFallback`: whether to allow fall back to other options if no native library that matches your specified settings could be found.
+- `NativeLibraryConfig.All.WithCuda`: decide if you want to use cuda if possible.
+- `NativeLibraryConfig.All.WithAvx`: decide the highest AVX level you want to use if possible.
+- `NativeLibraryConfig.All.WithSearchDirectory`: specify the directory to search the native library files.
+- `NativeLibraryConfig.All.WithAutoFallback`: whether to allow fall back to other options if no native library that matches your specified settings could be found.
+
+### Skip the check in case of false trigger of validation
+
+`NativeLibraryConfig.All.SkipCheck` allows you to skip the checks of the device/environment compatibility. This API will be very useful if your users have an unusual environment. For example, if the user installed the cuda toolkit in a customized path and didn't set the environment variables correctly, it might be regarded as invalid even though the loading could actually succeed.
+
+### Dry run your configuration
+
+In some conditions, you or your users are not sure if the native library could be successfully loaded with the configuration. To address this issue, `NativeLibraryConfig.All.DryRun` is provided to try to run the native library loading with the current configuration. It will return whether the loading is successfully, and the loaded library. However, the loaded library is not assigned to the handle of `NativeApi`. So you could try different configurations until one of them could be loaded successfully.
+
+NOTE: Similar to other APIs, `NativeLibraryConfig.All.DryRun` is only available before calling an arbitrary API in `NativeApi`, too.
 
 ### Set the log level of native library loading
 
 ```cs
-NativeLibraryConfig.Instance.WithLogs();
+NativeLibraryConfig.All.WithLogs();
 ```
 
 There are four log levels, which are error, warning, info and debug. If you are not sure if the correct library is selected, please set log level to `info` to see the full logs.

mkdocs.yml

@@ -6,13 +6,16 @@ nav:
     - FAQ: FAQ.md
     - Contributing Guide: ContributingGuide.md
     - Tutorials:
-        - Customize the native library loading: Tutorials/NativeLibraryConfig.md
+        - Configure the native library loading: Tutorials/NativeLibraryConfig.md
         - Use executors: Tutorials/Executors.md
         - Use ChatSession: Tutorials/ChatSession.md
         - Understand LLamaContext: Tutorials/UnderstandLLamaContext.md
         - Get embeddings: Tutorials/GetEmbeddings.md
         - Quantize the model: Tutorials/Quantization.md
 
+    - Advanced Tutorials:
+        - Customize the native library loading: AdvancedTutorials/CustomizeNativeLibraryLoading.md
+
     - Integrations:
         - semantic-kernel integration: Integrations/semantic-kernel.md
         - kernel-memory integration: Integrations/kernel-memory.md