Skip to content

Unit tests that aim to verify the behavior and correctness of the sampling pipeline under various conditions #1107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 35 commits into
base: master
Choose a base branch
from

Conversation

csharpconsultant
Copy link

@csharpconsultant csharpconsultant commented Feb 21, 2025

This pull request adds several unit tests to the LlamaSharp library's SamplingTests.cs file. The added tests aim to verify the behavior and correctness of the sampling pipeline under various conditions. The tests include:

SamplingWithTopK:

This test checks the behavior of the LLamaSamplerChainHandle when using different TopK values during sampling.

Ensures the sampled tokens and resulting text differ when using TopK=5 and TopK=50.

FrequencyPenalty_ThrowsException_WhenValueIsLessThanMinusTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the frequency penalty is less than -2.

FrequencyPenalty_ThrowsException_WhenValueIsGreaterThanTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the frequency penalty exceeds 2.

PresencePenalty_ThrowsException_WhenValueIsLessThanMinusTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the presence penalty is less than -2.

PresencePenalty_ThrowsException_WhenValueIsGreaterThanTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the presence penalty exceeds 2.

DefaultValues_AreSetCorrectly:

Verifies that the default values in the DefaultSamplingPipeline are set correctly.

Seed_HasLowProbabilityOfCollision:

This test verifies that the DefaultSamplingPipeline generates seeds with a low collision probability.

Seed_IsInitializedWithSpecificValue:

Verifies that the pipeline seed can be initialized with a specific value.

SetMinKeep:

Tests setting the MinKeep property to a specific value.

GetMinKeepDefault:

Verifies that the default value of the MinKeep property is set correctly.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added Unit tests for the Sampling Pipeline

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added Unit test for seed initialization on Default sampling pipeline …

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Upstream made changes to a smaller model for testing

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added Unit test MinKeep Test Default Get Default Sampling Pipeline

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Tony dev

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Addressed reviewer comments and updated with new changes
@csharpconsultant
Copy link
Author

I made the following updates:

Removed the non-deterministic sampling with temperature test.
Made the TopK test deterministic.
Removed the uint seed range test that would always pass.
Added a test to verify that the DefaultSamplingPipeline generates seeds with a low collision probability.

csharpconsultant and others added 9 commits February 20, 2025 22:42

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Tony dev

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
seth2396 and others added 3 commits February 23, 2025 12:35

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Adding LLama.Rag project to solution for implementation of RAG
@csharpconsultant csharpconsultant marked this pull request as draft February 24, 2025 02:58
seth2396 and others added 5 commits February 27, 2025 21:29

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Created an interface IWebScraper

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
GreedySampling and Grammar unit tests for project

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Unit phone
Copy link

This pull request has been automatically marked as stale due to inactivity. If no further activity occurs, it will be closed in 7 days.

@github-actions github-actions bot added the stale Stale issue will be autoclosed soon label Jul 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Stale issue will be autoclosed soon
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants