Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix p value in paired T-test #99

Merged
merged 2 commits into from
Apr 17, 2023
Merged

Conversation

ylansegal
Copy link
Contributor

Thank you for making this gem available!

I believe I've found (and fixed) a bug. In the Student's T-test (Paired T-Test) wiki page, we see the following example:

left_group
# => [0.12819915256260872, 0.24345459073897613, 0.27517650565714014, 0.8522185144081152, 0.05471111219486524]
right_group
# => [0.3272414061985621, 0.2989306116723194, 0.642664937717922, 0.9476073892620895, 0.7050008194345182]
# ...
StatisticalTest::TTest.paired_test(alpha = 0.01, :two_tail, left_group, right_group)
# => {:t_score=>-2.520215812331144, :probability=>0.032670873044446366, :p_value=>1.9346582539111072, :alpha=>0.01, :null=>true, :alternative=>false, :confidence_level=>0.99}

Notice the p-value returned is greater greater than 1.

If we do the same calculation in R:

left_group <- c(0.12819915256260872, 0.24345459073897613, 0.27517650565714014, 0.8522185144081152, 0.05471111219486524)
right_group <- c(0.3272414061985621, 0.2989306116723194, 0.642664937717922, 0.9476073892620895, 0.7050008194345182)

t.test(left_group, right_group, alternative = 'two.sided', paired = TRUE, conf.level = 0.99)

#   Paired t-test

# data:  left_group and right_group
# t = -2.5202, df = 4, p-value = 0.06534
# alternative hypothesis: true mean difference is not equal to 0
# 99 percent confidence interval:
# -0.7732524  0.2261783
# sample estimates:
# mean difference
#     -0.2735371

The p-value is calculated as 0.06534.

I believe the root cause is the same as what was fixed for .perform in #24. In this PR I use that same fix for .paired_test and the specs corroborate the numbers in the R calculations.

Assuming this is accepted, the wiki should also be changed:

# => [0.12819915256260872, 0.24345459073897613, 0.27517650565714014, 0.8522185144081152, 0.05471111219486524]
right_group
# => [0.3272414061985621, 0.2989306116723194, 0.642664937717922, 0.9476073892620895, 0.7050008194345182]


StatisticalTest::TTest.paired_test(alpha = 0.01, :two_tail, left_group, right_group)
# =>
# {:t_score=>-2.520215812331144,
#  :probability=>0.03267087304444638,
#  :p_value=>0.06534174608889276,
#  :alpha=>0.01,
#  :null=>true,
#  :alternative=>false,
#  :confidence_level=>0.99}

Thank you!

@estebanz01
Copy link
Owner

Thank you for you contribution! I'll take a look.

@estebanz01
Copy link
Owner

Can you please rebase this PR with latest master (just dependabot changes) and post a full output of bundle exec rake ? I'm not sure why the Github workflows are not running for this.

@ylansegal
Copy link
Contributor Author

Can you please rebase this PR with latest master (just dependabot changes) and post a full output of bundle exec rake ? I'm not sure why the Github workflows are not running for this.

@estebanz01 Of course! I rebased, and ran the command:

$ bundle exec rake
/Users/ylansegal/.asdf/installs/ruby/3.2.1/bin/ruby -I/Users/ylansegal/.asdf/installs/ruby/3.2.1/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.1/lib:/Users/ylansegal/.asdf/installs/ruby/3.2.1/lib/ruby/gems/3.2.0/gems/rspec-support-3.12.0/lib /Users/ylansegal/.asdf/installs/ruby/3.2.1/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.1/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
....................................................................................................................................This is an alpha version code. The generated sample is similar to an uniform distribution
.This is an alpha version code. The generated sample is similar to an uniform distribution
*..............................The composite simpson's rule needs even intervals!
......................................{:d_max=>(3/25), :d_critical=>0.18152976342488275, :total_samples=>550, :alpha=>0.05, :null=>true, :alternative=>false, :confidence_level=>0.95}
...................

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) Statistics::Distribution::TStudent#random does not generate a random sample that follows an uniform distribution
     # random sample follows an uniform and t-student distribution
     Failure/Error: expect(test[:null]).to be false

       expected false
            got true
     # ./spec/statistics/distribution/t_student_spec.rb:83:in `block (3 levels) in <top (required)>'

Finished in 9.87 seconds (files took 0.07437 seconds to load)
221 examples, 0 failures, 1 pending

Thank you,

@estebanz01 estebanz01 merged commit c445127 into estebanz01:master Apr 17, 2023
@estebanz01
Copy link
Owner

Thank you very much for your efforts! I'm going to release a new version later today with this fix.

estebanz01 added a commit that referenced this pull request Apr 17, 2023
This release includes the fix reported on #99 which is another
manifestation of the good ol' #23 which was fixed on #24.

Thanks @ylansegal :yay:.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants