From 5f89bd23e2da1d4013bdba33f2ae935e73029f23 Mon Sep 17 00:00:00 2001 From: Adithya Krishna Date: Sat, 21 Oct 2023 15:52:12 +0530 Subject: [PATCH 1/6] feat: added contents for testing of wasmedge apps Signed-off-by: Adithya Krishna --- docs/contribute/test.md | 10 - docs/contribute/testing/_category_.json | 8 + .../contribute/testing/best-practises copy.md | 36 +++ docs/contribute/testing/intro.md | 59 +++++ docs/contribute/testing/testing-types.md | 218 ++++++++++++++++++ 5 files changed, 321 insertions(+), 10 deletions(-) delete mode 100644 docs/contribute/test.md create mode 100644 docs/contribute/testing/_category_.json create mode 100644 docs/contribute/testing/best-practises copy.md create mode 100644 docs/contribute/testing/intro.md create mode 100644 docs/contribute/testing/testing-types.md diff --git a/docs/contribute/test.md b/docs/contribute/test.md deleted file mode 100644 index ad778399c..000000000 --- a/docs/contribute/test.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -sidebar_position: 4 ---- - -# Testing - - -:::info -Work in Progress -::: diff --git a/docs/contribute/testing/_category_.json b/docs/contribute/testing/_category_.json new file mode 100644 index 000000000..b54e56ba9 --- /dev/null +++ b/docs/contribute/testing/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Testing", + "position": 4, + "link": { + "type": "generated-index", + "description": "We will learn how to test WasmEdge applications" + } +} diff --git a/docs/contribute/testing/best-practises copy.md b/docs/contribute/testing/best-practises copy.md new file mode 100644 index 000000000..a240ff46d --- /dev/null +++ b/docs/contribute/testing/best-practises copy.md @@ -0,0 +1,36 @@ +--- +sidebar_position: 3 +--- + +# Best Practises + +Writing WasmEdge applications involves a blend of best practices from WebAssembly development, cloud computing, and edge computing. These best practices can serve as a guide to building robust, scalable, and efficient WasmEdge applications. + +## Design Principles + +1. **Modular Code:** Keep your WebAssembly modules small and focused, making them easier to test. + +2. **Mock External Dependencies:** Use mocking to isolate your tests from external services or modules. + +3. **Documentation:** Document your tests to explain what each test does and why specific assertions are made. + +4. **Balance Cost and Performance:** While it's important to cover as many test cases as possible, also consider the time and computational resources needed to run extensive tests. + +5. **Version Control:** Keep both your code and tests in version control, ensuring that they evolve together. + +6. **Continuous Integration:** Integrate testing into your CI/CD pipeline to catch issues early. + +## Tools and Frameworks + +1. **Google Test:** For C/C++ unit testing. + +2. **WasmEdge API:** For testing WebAssembly modules directly. + +3. **Selenium/Puppeteer:** For web-based end-to-end testing. + +4. **Jenkins/GitHub Actions:** For continuous integration and automated testing. + +## Optional Packages and Third-Party Integrations + +1. **Code Coverage:** Tools like lcov for C/C++. +2. **Static Analysis:** Tools like cppcheck for C/C++. \ No newline at end of file diff --git a/docs/contribute/testing/intro.md b/docs/contribute/testing/intro.md new file mode 100644 index 000000000..1c696c505 --- /dev/null +++ b/docs/contribute/testing/intro.md @@ -0,0 +1,59 @@ +--- +sidebar_position: 1 +--- + +# Testing WasmEdge + +Testing is an indispensable aspect of software development that aims to ensure the reliability, security, and performance of an application. For WasmEdge applications, which are designed to run in various environments ranging from edge devices to cloud servers, testing takes on added significance and confidence while deploying production grade application or tools + +## Why is Testing Important for WasmEdge Applications? + +### Reliability + +WasmEdge applications often run in environments where failure is not an option, such as IoT devices or edge servers. Rigorous testing ensures that the application can handle various scenarios reliably. + +### Performance + +Given that WasmEdge is designed for both edge and cloud computing, performance is a critical factor. Testing helps in identifying bottlenecks and optimizing code for better performance. + +### Security + +WebAssembly, and by extension WasmEdge, offers a sandboxed execution environment. Testing helps in ensuring that the security features are implemented correctly and that the application is resistant to common vulnerabilities. + +### Compatibility + +WasmEdge applications can run on various platforms. Testing ensures that your application behaves consistently across different environments. + +### Maintainability + +Well-tested code is easier to maintain and extend. It provides a safety net that helps developers make changes without fear of breaking existing functionality. + +### Quality Assurance + +Comprehensive testing is key to ensuring that the application meets all functional and non-functional requirements, thereby assuring quality. + +## Testing a Simple Function in a WebAssembly Module + +Let's assume you have a WebAssembly module that contains a function add which takes two integers and returns their sum. Here's how you can test it, + +```cpp +#include +#include + +TEST(WasmEdgeIntegrationTest, TestAddFunction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your/add_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Execute function + auto Result = VM.execute("add", {3_u32, 4_u32}); + + // Validate the result + ASSERT_TRUE(Result); + EXPECT_EQ(Result.value().get(), 7); +} +``` diff --git a/docs/contribute/testing/testing-types.md b/docs/contribute/testing/testing-types.md new file mode 100644 index 000000000..001787244 --- /dev/null +++ b/docs/contribute/testing/testing-types.md @@ -0,0 +1,218 @@ +--- +sidebar_position: 2 +--- + +# Types of Testing + +Testing WasmEdge applications and Wasm code written in C++ involves a multi-layered approach to ensure robustness, performance, and compatibility. Below are the types of testing you can perform, + +## Unit Testing + +### Native Code +- Use native testing frameworks like **Google Test** for C++ to test your code before compiling it to WebAssembly. + ```cpp + #include + TEST(MyFunctionTest, HandlesPositiveInput) { + EXPECT_EQ(my_function(1, 2), 3); + } + +### Wasm Code + +Use WasmEdge API to write tests that load and execute WebAssembly modules, verifying their behavio + +```cpp +#include +TEST(WasmEdgeTest, RunMyProgram) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + EXPECT_TRUE(VM.loadWasm("my_program.wasm")); + EXPECT_TRUE(VM.validate()); + EXPECT_TRUE(VM.instantiate()); + auto Result = VM.execute("my_function", {3_u32, 4_u32}); + EXPECT_TRUE(Result); + EXPECT_EQ(Result.value().get(), 7); +} +``` + +## Integration Testing + +### API Testing + +Test the interaction between your WebAssembly modules and the host environment, whether it's a web browser, Node.js, or a cloud-based application. + +Here's a C++ code snippet that uses the WasmEdge API to load a WebAssembly module and execute a function. This function could be one that interacts with an external API. + +```cpp +#include +#include + +TEST(WasmEdgeIntegrationTest, TestAPIInteraction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Execute a function that is supposed to interact with an external API + auto Result = VM.execute("call_external_api_function", { /* parameters */ }); + + // Validate the result + ASSERT_TRUE(Result); + EXPECT_EQ(Result.value().get(), /* expected_value */); +} +``` + +### Testing Data Flow with Mock Database + +Suppose your WebAssembly module interacts with a database. You can mock this database interaction to test the data flow. + +```cpp +#include +#include +#include + +TEST(WasmEdgeIntegrationTest, TestDataFlow) { + // Setup mock database + MockDatabase mockDB; + mockDB.setReturnValue("some_query", "expected_result"); + + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Execute a DB function + auto Result = VM.execute("call_database_function", { /* parameters */ }); + + // Validation + ASSERT_TRUE(Result); + EXPECT_EQ(Result.value().get(), "expected_result"); +} +``` + +### Error Handling + +Let's assume you have a WebAssembly function divide that takes two integers and returns the result of division. This function should handle division by zero gracefully. Here's how you can test it, + +```cpp +#include +#include + +TEST(WasmEdgeIntegrationTest, TestDivideFunction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your/divide_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Test zero division + auto Result = VM.execute("divide", {4_u32, 0_u32}); + + // Validate that the function handles the error gracefully + ASSERT_FALSE(Result); +} + +``` + +## End-to-End Testing + +End-to-End testing involves testing the flow of an application from start to finish to ensure the entire process of a user's interaction performs as designed. In the context of WasmEdge, this means testing not just the WebAssembly modules but also their interaction with the host environment, external APIs, databases, and even user interfaces if applicable. + +In the below example, we will try to mock an external API and Database interaction + +```cpp +#include +#include +#include // For mocking the database + +// Mock for the external API +std::string mock_fetch_data() { + return "sample_data"; +} + +// Mock for the database +std::map mock_database; + +void mock_store_data(int id, const std::string& processed_data) { + mock_database[id] = processed_data; +} + +TEST(WasmEdgeE2ETest, TestProcessDataFunction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm Module + ASSERT_TRUE(VM.loadWasm("path/to/your/process_data_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Register the mock API and Database functions + VM.registerHostFunction("mock_fetch_data", mock_fetch_data); + VM.registerHostFunction("mock_store_data", mock_store_data); + + // Run the 'process_data' function + auto Result = VM.execute("process_data", {1}); + + // Validate that the data is processed and stored correctly + ASSERT_TRUE(Result); + ASSERT_EQ(mock_database[1], "processed_sample_data"); +} + +int main(int argc, char **argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +``` + +## Performance Testing + +Performance testing aims to evaluate the system's responsiveness, stability, and speed under different conditions. For WasmEdge applications, this could mean testing: + +1. **Execution Time:** How long it takes for a WebAssembly function to execute. +2. **Resource Utilization:** How much CPU, memory, and other resources are used. +3. **Scalability:** How well the application performs as the load increases. +4. **Concurrency:** How the system behaves with multiple users or requests. + +For example, Let's consider you have a Wasm function `compute` that performs some complex calculations + +```cpp +#include +#include + +static void BM_WasmEdgeComputeFunction(benchmark::State& state) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + VM.loadWasm("path/to/your/compute_module.wasm"); + VM.validate(); + VM.instantiate(); + + // Benchmark the 'compute' function + for (auto _ : state) { + auto Result = VM.execute("compute", {1000_u32}); + if (!Result) { + state.SkipWithError("Function execution failed"); + break; + } + } +} +BENCHMARK(BM_WasmEdgeComputeFunction); + +BENCHMARK_MAIN(); +``` + + +## Further Testing + +1. **Environment-Specific Behavior:** Given that WasmEdge can run on various environments (cloud, edge, browser, etc.), your tests may need to cover these different scenarios as well. + +2. **Security:** Testing of vulnerabilities that can be exploited when the WebAssembly module interacts with external services or databases may also be needed for your use cases. From 490b64a51ffdb1cf9b13abdc5f20369899f6f0ac Mon Sep 17 00:00:00 2001 From: Adithya Krishna Date: Sat, 21 Oct 2023 15:55:40 +0530 Subject: [PATCH 2/6] chore: updates lists Signed-off-by: Adithya Krishna --- docs/contribute/testing/best-practises copy.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/contribute/testing/best-practises copy.md b/docs/contribute/testing/best-practises copy.md index a240ff46d..abd388b06 100644 --- a/docs/contribute/testing/best-practises copy.md +++ b/docs/contribute/testing/best-practises copy.md @@ -22,15 +22,15 @@ Writing WasmEdge applications involves a blend of best practices from WebAssembl ## Tools and Frameworks -1. **Google Test:** For C/C++ unit testing. +- **Google Test:** For C/C++ unit testing. -2. **WasmEdge API:** For testing WebAssembly modules directly. +- **WasmEdge API:** For testing WebAssembly modules directly. -3. **Selenium/Puppeteer:** For web-based end-to-end testing. +- **Selenium/Puppeteer:** For web-based end-to-end testing. -4. **Jenkins/GitHub Actions:** For continuous integration and automated testing. +- **Jenkins/GitHub Actions:** For continuous integration and automated testing. ## Optional Packages and Third-Party Integrations -1. **Code Coverage:** Tools like lcov for C/C++. -2. **Static Analysis:** Tools like cppcheck for C/C++. \ No newline at end of file +- **Code Coverage:** Tools like lcov for C/C++. +- **Static Analysis:** Tools like cppcheck for C/C++. \ No newline at end of file From d5b9ad3596a3f2cdd637e949d997f98e6d1dba73 Mon Sep 17 00:00:00 2001 From: Adithya Krishna Date: Sat, 21 Oct 2023 16:24:40 +0530 Subject: [PATCH 3/6] chore: add zh docs for testing Signed-off-by: Adithya Krishna --- docs/contribute/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contribute/overview.md b/docs/contribute/overview.md index 524d85e1f..581ac3e3d 100644 --- a/docs/contribute/overview.md +++ b/docs/contribute/overview.md @@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi - [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source) - [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system) -- [Test WasmEdge](test.md) +- [Test WasmEdge](./testing/intro.md) - [WasmEdge Fuzzing](fuzzing.md) - [WasmEdge internal explanation](internal.md) - [WasmEdge installer system explanation](installer.md) From 72404a1323888a80093355dbde0be5c22327ea1f Mon Sep 17 00:00:00 2001 From: Adithya Krishna Date: Sat, 21 Oct 2023 16:24:48 +0530 Subject: [PATCH 4/6] chore: add zh docs for testing - v2 Signed-off-by: Adithya Krishna --- .../current/contribute/overview.md | 2 +- .../current/contribute/test.md | 10 - .../contribute/testing/_category_.json | 8 + .../contribute/testing/best-practises copy.md | 36 +++ .../current/contribute/testing/intro.md | 59 +++++ .../contribute/testing/testing-types.md | 218 ++++++++++++++++++ 6 files changed, 322 insertions(+), 11 deletions(-) delete mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md create mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/_category_.json create mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises copy.md create mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/intro.md create mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/testing-types.md diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md index 524d85e1f..581ac3e3d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md @@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi - [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source) - [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system) -- [Test WasmEdge](test.md) +- [Test WasmEdge](./testing/intro.md) - [WasmEdge Fuzzing](fuzzing.md) - [WasmEdge internal explanation](internal.md) - [WasmEdge installer system explanation](installer.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md deleted file mode 100644 index ad778399c..000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -sidebar_position: 4 ---- - -# Testing - - -:::info -Work in Progress -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/_category_.json b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/_category_.json new file mode 100644 index 000000000..b54e56ba9 --- /dev/null +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Testing", + "position": 4, + "link": { + "type": "generated-index", + "description": "We will learn how to test WasmEdge applications" + } +} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises copy.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises copy.md new file mode 100644 index 000000000..abd388b06 --- /dev/null +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises copy.md @@ -0,0 +1,36 @@ +--- +sidebar_position: 3 +--- + +# Best Practises + +Writing WasmEdge applications involves a blend of best practices from WebAssembly development, cloud computing, and edge computing. These best practices can serve as a guide to building robust, scalable, and efficient WasmEdge applications. + +## Design Principles + +1. **Modular Code:** Keep your WebAssembly modules small and focused, making them easier to test. + +2. **Mock External Dependencies:** Use mocking to isolate your tests from external services or modules. + +3. **Documentation:** Document your tests to explain what each test does and why specific assertions are made. + +4. **Balance Cost and Performance:** While it's important to cover as many test cases as possible, also consider the time and computational resources needed to run extensive tests. + +5. **Version Control:** Keep both your code and tests in version control, ensuring that they evolve together. + +6. **Continuous Integration:** Integrate testing into your CI/CD pipeline to catch issues early. + +## Tools and Frameworks + +- **Google Test:** For C/C++ unit testing. + +- **WasmEdge API:** For testing WebAssembly modules directly. + +- **Selenium/Puppeteer:** For web-based end-to-end testing. + +- **Jenkins/GitHub Actions:** For continuous integration and automated testing. + +## Optional Packages and Third-Party Integrations + +- **Code Coverage:** Tools like lcov for C/C++. +- **Static Analysis:** Tools like cppcheck for C/C++. \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/intro.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/intro.md new file mode 100644 index 000000000..1c696c505 --- /dev/null +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/intro.md @@ -0,0 +1,59 @@ +--- +sidebar_position: 1 +--- + +# Testing WasmEdge + +Testing is an indispensable aspect of software development that aims to ensure the reliability, security, and performance of an application. For WasmEdge applications, which are designed to run in various environments ranging from edge devices to cloud servers, testing takes on added significance and confidence while deploying production grade application or tools + +## Why is Testing Important for WasmEdge Applications? + +### Reliability + +WasmEdge applications often run in environments where failure is not an option, such as IoT devices or edge servers. Rigorous testing ensures that the application can handle various scenarios reliably. + +### Performance + +Given that WasmEdge is designed for both edge and cloud computing, performance is a critical factor. Testing helps in identifying bottlenecks and optimizing code for better performance. + +### Security + +WebAssembly, and by extension WasmEdge, offers a sandboxed execution environment. Testing helps in ensuring that the security features are implemented correctly and that the application is resistant to common vulnerabilities. + +### Compatibility + +WasmEdge applications can run on various platforms. Testing ensures that your application behaves consistently across different environments. + +### Maintainability + +Well-tested code is easier to maintain and extend. It provides a safety net that helps developers make changes without fear of breaking existing functionality. + +### Quality Assurance + +Comprehensive testing is key to ensuring that the application meets all functional and non-functional requirements, thereby assuring quality. + +## Testing a Simple Function in a WebAssembly Module + +Let's assume you have a WebAssembly module that contains a function add which takes two integers and returns their sum. Here's how you can test it, + +```cpp +#include +#include + +TEST(WasmEdgeIntegrationTest, TestAddFunction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your/add_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Execute function + auto Result = VM.execute("add", {3_u32, 4_u32}); + + // Validate the result + ASSERT_TRUE(Result); + EXPECT_EQ(Result.value().get(), 7); +} +``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/testing-types.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/testing-types.md new file mode 100644 index 000000000..001787244 --- /dev/null +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/testing-types.md @@ -0,0 +1,218 @@ +--- +sidebar_position: 2 +--- + +# Types of Testing + +Testing WasmEdge applications and Wasm code written in C++ involves a multi-layered approach to ensure robustness, performance, and compatibility. Below are the types of testing you can perform, + +## Unit Testing + +### Native Code +- Use native testing frameworks like **Google Test** for C++ to test your code before compiling it to WebAssembly. + ```cpp + #include + TEST(MyFunctionTest, HandlesPositiveInput) { + EXPECT_EQ(my_function(1, 2), 3); + } + +### Wasm Code + +Use WasmEdge API to write tests that load and execute WebAssembly modules, verifying their behavio + +```cpp +#include +TEST(WasmEdgeTest, RunMyProgram) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + EXPECT_TRUE(VM.loadWasm("my_program.wasm")); + EXPECT_TRUE(VM.validate()); + EXPECT_TRUE(VM.instantiate()); + auto Result = VM.execute("my_function", {3_u32, 4_u32}); + EXPECT_TRUE(Result); + EXPECT_EQ(Result.value().get(), 7); +} +``` + +## Integration Testing + +### API Testing + +Test the interaction between your WebAssembly modules and the host environment, whether it's a web browser, Node.js, or a cloud-based application. + +Here's a C++ code snippet that uses the WasmEdge API to load a WebAssembly module and execute a function. This function could be one that interacts with an external API. + +```cpp +#include +#include + +TEST(WasmEdgeIntegrationTest, TestAPIInteraction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Execute a function that is supposed to interact with an external API + auto Result = VM.execute("call_external_api_function", { /* parameters */ }); + + // Validate the result + ASSERT_TRUE(Result); + EXPECT_EQ(Result.value().get(), /* expected_value */); +} +``` + +### Testing Data Flow with Mock Database + +Suppose your WebAssembly module interacts with a database. You can mock this database interaction to test the data flow. + +```cpp +#include +#include +#include + +TEST(WasmEdgeIntegrationTest, TestDataFlow) { + // Setup mock database + MockDatabase mockDB; + mockDB.setReturnValue("some_query", "expected_result"); + + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Execute a DB function + auto Result = VM.execute("call_database_function", { /* parameters */ }); + + // Validation + ASSERT_TRUE(Result); + EXPECT_EQ(Result.value().get(), "expected_result"); +} +``` + +### Error Handling + +Let's assume you have a WebAssembly function divide that takes two integers and returns the result of division. This function should handle division by zero gracefully. Here's how you can test it, + +```cpp +#include +#include + +TEST(WasmEdgeIntegrationTest, TestDivideFunction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + ASSERT_TRUE(VM.loadWasm("path/to/your/divide_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Test zero division + auto Result = VM.execute("divide", {4_u32, 0_u32}); + + // Validate that the function handles the error gracefully + ASSERT_FALSE(Result); +} + +``` + +## End-to-End Testing + +End-to-End testing involves testing the flow of an application from start to finish to ensure the entire process of a user's interaction performs as designed. In the context of WasmEdge, this means testing not just the WebAssembly modules but also their interaction with the host environment, external APIs, databases, and even user interfaces if applicable. + +In the below example, we will try to mock an external API and Database interaction + +```cpp +#include +#include +#include // For mocking the database + +// Mock for the external API +std::string mock_fetch_data() { + return "sample_data"; +} + +// Mock for the database +std::map mock_database; + +void mock_store_data(int id, const std::string& processed_data) { + mock_database[id] = processed_data; +} + +TEST(WasmEdgeE2ETest, TestProcessDataFunction) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm Module + ASSERT_TRUE(VM.loadWasm("path/to/your/process_data_module.wasm")); + ASSERT_TRUE(VM.validate()); + ASSERT_TRUE(VM.instantiate()); + + // Register the mock API and Database functions + VM.registerHostFunction("mock_fetch_data", mock_fetch_data); + VM.registerHostFunction("mock_store_data", mock_store_data); + + // Run the 'process_data' function + auto Result = VM.execute("process_data", {1}); + + // Validate that the data is processed and stored correctly + ASSERT_TRUE(Result); + ASSERT_EQ(mock_database[1], "processed_sample_data"); +} + +int main(int argc, char **argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +``` + +## Performance Testing + +Performance testing aims to evaluate the system's responsiveness, stability, and speed under different conditions. For WasmEdge applications, this could mean testing: + +1. **Execution Time:** How long it takes for a WebAssembly function to execute. +2. **Resource Utilization:** How much CPU, memory, and other resources are used. +3. **Scalability:** How well the application performs as the load increases. +4. **Concurrency:** How the system behaves with multiple users or requests. + +For example, Let's consider you have a Wasm function `compute` that performs some complex calculations + +```cpp +#include +#include + +static void BM_WasmEdgeComputeFunction(benchmark::State& state) { + WasmEdge::Configure Conf; + WasmEdge::VM VM(Conf); + + // Load Wasm module + VM.loadWasm("path/to/your/compute_module.wasm"); + VM.validate(); + VM.instantiate(); + + // Benchmark the 'compute' function + for (auto _ : state) { + auto Result = VM.execute("compute", {1000_u32}); + if (!Result) { + state.SkipWithError("Function execution failed"); + break; + } + } +} +BENCHMARK(BM_WasmEdgeComputeFunction); + +BENCHMARK_MAIN(); +``` + + +## Further Testing + +1. **Environment-Specific Behavior:** Given that WasmEdge can run on various environments (cloud, edge, browser, etc.), your tests may need to cover these different scenarios as well. + +2. **Security:** Testing of vulnerabilities that can be exploited when the WebAssembly module interacts with external services or databases may also be needed for your use cases. From 7cbe458793407f9dc6f83d84524c57f9aa7cf4f1 Mon Sep 17 00:00:00 2001 From: Adithya Krishna Date: Wed, 15 Nov 2023 17:36:31 +0530 Subject: [PATCH 5/6] chore: rename best-practises Signed-off-by: Adithya Krishna --- .../testing/{best-practises copy.md => best-practises.md} | 0 .../testing/{best-practises copy.md => best-practises.md} | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename docs/contribute/testing/{best-practises copy.md => best-practises.md} (100%) rename i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/{best-practises copy.md => best-practises.md} (100%) diff --git a/docs/contribute/testing/best-practises copy.md b/docs/contribute/testing/best-practises.md similarity index 100% rename from docs/contribute/testing/best-practises copy.md rename to docs/contribute/testing/best-practises.md diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises copy.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises.md similarity index 100% rename from i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises copy.md rename to i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises.md From 2c4cef1fb69e69e527a4a4d7443c26a353a5a165 Mon Sep 17 00:00:00 2001 From: Adithya Krishna Date: Fri, 22 Dec 2023 12:17:08 +0530 Subject: [PATCH 6/6] chore: updated test docs Signed-off-by: Adithya Krishna --- docs/contribute/testing/best-practises.md | 2 +- .../current/contribute/testing/best-practises.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/contribute/testing/best-practises.md b/docs/contribute/testing/best-practises.md index abd388b06..4e88c6d03 100644 --- a/docs/contribute/testing/best-practises.md +++ b/docs/contribute/testing/best-practises.md @@ -2,7 +2,7 @@ sidebar_position: 3 --- -# Best Practises +# Best Practises for WasmEdge Application Testing Writing WasmEdge applications involves a blend of best practices from WebAssembly development, cloud computing, and edge computing. These best practices can serve as a guide to building robust, scalable, and efficient WasmEdge applications. diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises.md index abd388b06..4e88c6d03 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/testing/best-practises.md @@ -2,7 +2,7 @@ sidebar_position: 3 --- -# Best Practises +# Best Practises for WasmEdge Application Testing Writing WasmEdge applications involves a blend of best practices from WebAssembly development, cloud computing, and edge computing. These best practices can serve as a guide to building robust, scalable, and efficient WasmEdge applications.