feat: added contents for testing of wasmedge apps

docs/contribute/overview.md

@@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi
 
 - [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source)
 - [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system)
-- [Test WasmEdge](test.md)
+- [Test WasmEdge](./testing/intro.md)
 - [WasmEdge Fuzzing](fuzzing.md)
 - [WasmEdge internal explanation](internal.md)
 - [WasmEdge installer system explanation](installer.md)

docs/contribute/testing/_category_.json

@@ -0,0 +1,8 @@
+{
+ "label": "Testing",
+ "position": 4,
+ "link": {
+ "type": "generated-index",
+ "description": "We will learn how to test WasmEdge applications"
+ }
+}

docs/contribute/testing/best-practises.md

@@ -0,0 +1,36 @@
+---
+sidebar_position: 3
+---
+
+# Best Practises for WasmEdge Application Testing
+
+Writing WasmEdge applications involves a blend of best practices from WebAssembly development, cloud computing, and edge computing. These best practices can serve as a guide to building robust, scalable, and efficient WasmEdge applications.
+
+## Design Principles
+
+1. **Modular Code:** Keep your WebAssembly modules small and focused, making them easier to test.
+
+2. **Mock External Dependencies:** Use mocking to isolate your tests from external services or modules.
+
+3. **Documentation:** Document your tests to explain what each test does and why specific assertions are made.
+
+4. **Balance Cost and Performance:** While it's important to cover as many test cases as possible, also consider the time and computational resources needed to run extensive tests.
+
+5. **Version Control:** Keep both your code and tests in version control, ensuring that they evolve together.
+
+6. **Continuous Integration:** Integrate testing into your CI/CD pipeline to catch issues early.
+
+## Tools and Frameworks
+
+- **Google Test:** For C/C++ unit testing.
+
+- **WasmEdge API:** For testing WebAssembly modules directly.
+
+- **Selenium/Puppeteer:** For web-based end-to-end testing.
+
+- **Jenkins/GitHub Actions:** For continuous integration and automated testing.
+
+## Optional Packages and Third-Party Integrations
+
+- **Code Coverage:** Tools like lcov for C/C++.
+- **Static Analysis:** Tools like cppcheck for C/C++.

docs/contribute/testing/intro.md

@@ -0,0 +1,59 @@
+---
+sidebar_position: 1
+---
+
+# Testing WasmEdge
+
+Testing is an indispensable aspect of software development that aims to ensure the reliability, security, and performance of an application. For WasmEdge applications, which are designed to run in various environments ranging from edge devices to cloud servers, testing takes on added significance and confidence while deploying production grade application or tools
+
+## Why is Testing Important for WasmEdge Applications?
+
+### Reliability
+
+WasmEdge applications often run in environments where failure is not an option, such as IoT devices or edge servers. Rigorous testing ensures that the application can handle various scenarios reliably.
+
+### Performance 
+
+Given that WasmEdge is designed for both edge and cloud computing, performance is a critical factor. Testing helps in identifying bottlenecks and optimizing code for better performance.
+
+### Security
+
+WebAssembly, and by extension WasmEdge, offers a sandboxed execution environment. Testing helps in ensuring that the security features are implemented correctly and that the application is resistant to common vulnerabilities.
+
+### Compatibility
+
+WasmEdge applications can run on various platforms. Testing ensures that your application behaves consistently across different environments.
+
+### Maintainability
+
+Well-tested code is easier to maintain and extend. It provides a safety net that helps developers make changes without fear of breaking existing functionality.
+
+### Quality Assurance
+
+Comprehensive testing is key to ensuring that the application meets all functional and non-functional requirements, thereby assuring quality.
+
+## Testing a Simple Function in a WebAssembly Module
+
+Let's assume you have a WebAssembly module that contains a function add which takes two integers and returns their sum. Here's how you can test it,
+
+```cpp
+#include <gtest/gtest.h>
+#include <wasmedge.h>
+
+TEST(WasmEdgeIntegrationTest, TestAddFunction) {
+ WasmEdge::Configure Conf;
+ WasmEdge::VM VM(Conf);
+
+ // Load Wasm module
+ ASSERT_TRUE(VM.loadWasm("path/to/your/add_module.wasm"));
+ ASSERT_TRUE(VM.validate());
+ ASSERT_TRUE(VM.instantiate());
+
+ // Execute function
+ auto Result = VM.execute("add", {3_u32, 4_u32});
+
+ // Validate the result
+ ASSERT_TRUE(Result);
+ EXPECT_EQ(Result.value().get<uint32_t>(), 7);
+}
+```

docs/contribute/testing/testing-types.md

@@ -0,0 +1,218 @@
+---
+sidebar_position: 2
+---
+
+# Types of Testing
+
+Testing WasmEdge applications and Wasm code written in C++ involves a multi-layered approach to ensure robustness, performance, and compatibility. Below are the types of testing you can perform,
+
+## Unit Testing
+
+### Native Code
+- Use native testing frameworks like **Google Test** for C++ to test your code before compiling it to WebAssembly.
+ ```cpp
+ #include <gtest/gtest.h>
+ TEST(MyFunctionTest, HandlesPositiveInput) {
+ EXPECT_EQ(my_function(1, 2), 3);
+ }
+
+### Wasm Code
+
+Use WasmEdge API to write tests that load and execute WebAssembly modules, verifying their behavio
+
+```cpp
+#include <wasmedge.h>
+TEST(WasmEdgeTest, RunMyProgram) {
+ WasmEdge::Configure Conf;
+ WasmEdge::VM VM(Conf);
+ EXPECT_TRUE(VM.loadWasm("my_program.wasm"));
+ EXPECT_TRUE(VM.validate());
+ EXPECT_TRUE(VM.instantiate());
+ auto Result = VM.execute("my_function", {3_u32, 4_u32});
+ EXPECT_TRUE(Result);
+ EXPECT_EQ(Result.value().get<uint32_t>(), 7);
+}
+```
+
+## Integration Testing
+
+### API Testing
+
+Test the interaction between your WebAssembly modules and the host environment, whether it's a web browser, Node.js, or a cloud-based application.
+
+Here's a C++ code snippet that uses the WasmEdge API to load a WebAssembly module and execute a function. This function could be one that interacts with an external API.
+
+```cpp
+#include <wasmedge.h>
+#include <gtest/gtest.h>
+
+TEST(WasmEdgeIntegrationTest, TestAPIInteraction) {
+ WasmEdge::Configure Conf;
+ WasmEdge::VM VM(Conf);
+
+ // Load Wasm module
+ ASSERT_TRUE(VM.loadWasm("path/to/your_module.wasm"));
+ ASSERT_TRUE(VM.validate());
+ ASSERT_TRUE(VM.instantiate());
+
+ // Execute a function that is supposed to interact with an external API
+ auto Result = VM.execute("call_external_api_function", { /* parameters */ });
+
+ // Validate the result
+ ASSERT_TRUE(Result);
+ EXPECT_EQ(Result.value().get<int>(), /* expected_value */);
+}
+```
+
+### Testing Data Flow with Mock Database
+
+Suppose your WebAssembly module interacts with a database. You can mock this database interaction to test the data flow.
+
+```cpp
+#include <wasmedge.h>
+#include <gtest/gtest.h>
+#include <mock_database_library.h>
+
+TEST(WasmEdgeIntegrationTest, TestDataFlow) {
+ // Setup mock database
+ MockDatabase mockDB;
+ mockDB.setReturnValue("some_query", "expected_result");
+
+ WasmEdge::Configure Conf;
+ WasmEdge::VM VM(Conf);
+
+ // Load Wasm module
+ ASSERT_TRUE(VM.loadWasm("path/to/your_module.wasm"));
+ ASSERT_TRUE(VM.validate());
+ ASSERT_TRUE(VM.instantiate());
+
+ // Execute a DB function
+ auto Result = VM.execute("call_database_function", { /* parameters */ });
+
+ // Validation
+ ASSERT_TRUE(Result);
+ EXPECT_EQ(Result.value().get<std::string>(), "expected_result");
+}
+```
+
+### Error Handling
+
+Let's assume you have a WebAssembly function divide that takes two integers and returns the result of division. This function should handle division by zero gracefully. Here's how you can test it,
+
+```cpp
+#include <gtest/gtest.h>
+#include <wasmedge.h>
+
+TEST(WasmEdgeIntegrationTest, TestDivideFunction) {
+ WasmEdge::Configure Conf;
+ WasmEdge::VM VM(Conf);
+
+ // Load Wasm module
+ ASSERT_TRUE(VM.loadWasm("path/to/your/divide_module.wasm"));
+ ASSERT_TRUE(VM.validate());
+ ASSERT_TRUE(VM.instantiate());
+
+ // Test zero division
+ auto Result = VM.execute("divide", {4_u32, 0_u32});
+
+ // Validate that the function handles the error gracefully
+ ASSERT_FALSE(Result);
+}
+
+```
+
+## End-to-End Testing
+
+End-to-End testing involves testing the flow of an application from start to finish to ensure the entire process of a user's interaction performs as designed. In the context of WasmEdge, this means testing not just the WebAssembly modules but also their interaction with the host environment, external APIs, databases, and even user interfaces if applicable.
+
+In the below example, we will try to mock an external API and Database interaction
+
+```cpp
+#include <gtest/gtest.h>
+#include <wasmedge.h>
+#include <map> // For mocking the database
+
+// Mock for the external API
+std::string mock_fetch_data() {
+ return "sample_data";
+}
+
+// Mock for the database
+std::map<int, std::string> mock_database;
+
+void mock_store_data(int id, const std::string& processed_data) {
+ mock_database[id] = processed_data;
+}
+
+TEST(WasmEdgeE2ETest, TestProcessDataFunction) {
+ WasmEdge::Configure Conf;
+ WasmEdge::VM VM(Conf);
+
+ // Load Wasm Module
+ ASSERT_TRUE(VM.loadWasm("path/to/your/process_data_module.wasm"));
+ ASSERT_TRUE(VM.validate());
+ ASSERT_TRUE(VM.instantiate());
+
+ // Register the mock API and Database functions
+ VM.registerHostFunction("mock_fetch_data", mock_fetch_data);
+ VM.registerHostFunction("mock_store_data", mock_store_data);
+
+ // Run the 'process_data' function
+ auto Result = VM.execute("process_data", {1});
+
+ // Validate that the data is processed and stored correctly
+ ASSERT_TRUE(Result);
+ ASSERT_EQ(mock_database[1], "processed_sample_data");
+}
+
+int main(int argc, char **argv) {
+ ::testing::InitGoogleTest(&argc, argv);
+ return RUN_ALL_TESTS();
+}
+
+```
+
+## Performance Testing
+
+Performance testing aims to evaluate the system's responsiveness, stability, and speed under different conditions. For WasmEdge applications, this could mean testing:
+
+1. **Execution Time:** How long it takes for a WebAssembly function to execute.
+2. **Resource Utilization:** How much CPU, memory, and other resources are used.
+3. **Scalability:** How well the application performs as the load increases.
+4. **Concurrency:** How the system behaves with multiple users or requests.
+
+For example, Let's consider you have a Wasm function `compute` that performs some complex calculations
+
+```cpp
+#include <benchmark/benchmark.h>
+#include <wasmedge.h>
+
+static void BM_WasmEdgeComputeFunction(benchmark::State& state) {
+ WasmEdge::Configure Conf;
+ WasmEdge::VM VM(Conf);
+
+ // Load Wasm module
+ VM.loadWasm("path/to/your/compute_module.wasm");
+ VM.validate();
+ VM.instantiate();
+
+ // Benchmark the 'compute' function
+ for (auto _ : state) {
+ auto Result = VM.execute("compute", {1000_u32}); 
+ if (!Result) {
+ state.SkipWithError("Function execution failed");
+ break;
+ }
+ }
+}
+BENCHMARK(BM_WasmEdgeComputeFunction);
+
+BENCHMARK_MAIN();
+```
+
+
+## Further Testing
+
+1. **Environment-Specific Behavior:** Given that WasmEdge can run on various environments (cloud, edge, browser, etc.), your tests may need to cover these different scenarios as well.
+
+2. **Security:** Testing of vulnerabilities that can be exploited when the WebAssembly module interacts with external services or databases may also be needed for your use cases.

i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md

@@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi
 
 - [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source)
 - [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system)
-- [Test WasmEdge](test.md)
+- [Test WasmEdge](./testing/intro.md)
 - [WasmEdge Fuzzing](fuzzing.md)
 - [WasmEdge internal explanation](internal.md)
 - [WasmEdge installer system explanation](installer.md)

i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/_category_.json

@@ -0,0 +1,8 @@
+{
+ "label": "Testing",
+ "position": 4,
+ "link": {
+ "type": "generated-index",
+ "description": "We will learn how to test WasmEdge applications"
+ }
+}