Implementation of `dropcollinear` feature in `GeneralizedLinearModel` #488

mousum-github · 2022-07-15T17:08:32Z

This pull request supersedes #340

We are looking for the dropcollinear feature in GLM and found #340. The PR, which opened in Oct'2019, looks very close to complete.

Apart from the changes covered in PR 340, we have added

A test case to compare LM with equivalent GLM
Two test cases to match output with R
Another implementation of dof for CholeskyPivoted since the dof will be calculated based on rank, instead of the number of coefficients.
update API documentation: Added dropcollinear in the list of keyword arguments.

We have performed logistic regression with 10 independent variables, 10,000 rows and dropcollinear feature true and false; the dataset does not have a multicollinearity issue. The machine configuration is as follows: -
OS – Linux,
CPU - AMD Ryzen 5 3600 6-Core Processor
RAM – 64 GB

With dropcollinear = true: average time 4.381 ms
With dropcollinear = false: average time 4.396 ms

Adding PowerLink link function in GLM

Test was failing in Julia Nightly as: 1) GLM.linkinv(InverseLink(), 10) was 0.01 while GLM.linkinv(PowerLink(-1), 10) was 0.010000000000000002 2) GLM.linkinv(InverseSquareLink(), 10) was -10.01 while GLM.linkinv(PowerLink(-2), 10) was -0.010000000000000002 Rounding off to 2 digits should solve this. Note: These tests were passing in other versions of Julia.

…ed it for the docs

…(-2), 10 to 2 digits

…lia Nightly

Correcting order of PowerLink, ProbitLink, SqrtLink. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Using ≈ instead of isapprox. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Using alphabetical order in references. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Writing the example description in plain text. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Removing extra space to be consistent with the style. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Using a better variable name for 1 by lambda. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Using inline function. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Making the doctest code concise. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Removing the R code. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Removing test of hashes. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

…few more metrics to test, also replaced all `=` by `≈` for real numbers

Since we are storing the link - the `GLM.Link` function can be defined uniformly for all link functions Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

mousum-github · 2022-10-10T12:05:18Z

Thanks @ayushpatnaikgit. Not sure why GitHub doesn't support that, but thanks to the access to the xKDR repo I could push my merge commit. Tests still appear to fail though.

Usually we only give commit access to people who have made several PRs over a relatively long period, but given that xKDR is invested in improving the stats ecosystem I guess we can accelerate that process. I've just sent an invitation to @mousum-github. Help reviewing other PRs would of course be welcome!

Thanks @nalimilan

…ome m/c

mousum-github · 2022-10-17T19:55:34Z

Hi @nalimilan,
We have updated the code and documentation.

We have updated the newly added nulldeviance and nullloglikelihood functions in glm. Passed the dropcollinear argument value to fit function for null models.

We also added a test case to check the PosDefException exception when dropcollinear is false for a rank-deficient model. Unfortunately, the test case was not passed in a few systems. So, we have commented the test case for timing.

nalimilan · 2022-10-17T20:36:13Z

We also added a test case to check the PosDefException exception when dropcollinear is false for a rank-deficient model. Unfortunately, the test case was not passed in a few systems. So, we have commented the test case for timing.

Maybe another example would pass tests more reliably? We really need to check that path. How about trying with categorical variables, like in the "rankdeficient" testset for linear models?

mousum-github · 2022-10-17T21:01:46Z

Okay. Let me check.

mousum-github · 2022-10-17T21:57:52Z

Okay. Let me check.

Added two test cases similar to rankdeficient.

test/runtests.jl

nalimilan

Seems almost ready! Just a few stylistic comments.

src/glmfit.jl

src/linpred.jl

test/runtests.jl

src/linpred.jl

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

…t case

src/glmfit.jl

Co-authored-by: Bogumił Kamiński <bkamins@sgh.waw.pl>

nalimilan

Thanks!

DilumAluthge and others added 30 commits March 25, 2022 23:26

Add support for rank deficient Generalized Linear Models

363b74c

remove unexectued code

c415886

solve issue about converges slower

1fd0790

remove file

2eb96e5

PowerLink refers to a class of techniques that use a power function

405fe93

added a few more testcases

742455f

Fixing doctests for PowerLink

f788854

Making subheading smaller than heading.

f65b842

Merge pull request #1 from xKDR/powerlink

6b80352

Adding PowerLink link function in GLM

adding Optim dependancy to docs

2e3b67a

No need to have optim as a dependancy for the entire package. Only ne…

5b09ae1

…ed it for the docs

Rounding off GLM.linkinv(PowerLink(-1), 10) and GLM.linkinv(PowerLink…

fddeebe

…(-2), 10 to 2 digits

Rounding off GLM.mueta(PowerLink(-1), 10) to make the test work on Ju…

e1e7631

…lia Nightly

Using inexact equality comparison instead of rounding.

b9ffe36

Apply suggestions from code review

ad4c986

Correcting order of PowerLink, ProbitLink, SqrtLink. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Apply suggestions from code review

f9dcc4c

Using ≈ instead of isapprox. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Apply suggestions from code review

9dae795

Using alphabetical order in references. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update docs/src/examples.md

83e7b12

Writing the example description in plain text. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update docs/src/examples.md

9d703a5

Removing extra space to be consistent with the style. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Adding the missing comma.

a0571a6

Update src/glmtools.jl

99aa23f

Using a better variable name for 1 by lambda. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update src/glmtools.jl

60bcde3

Using inline function. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Apply suggestions from code review

f8c7809

Making the doctest code concise. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Follwing alphabetical order in references

aef3048

Adding suggested changes by nalimilan

93a607a

Update test/runtests.jl

9620cf9

Removing the R code. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update test/runtests.jl

0d94609

Removing test of hashes. Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

replaced isapprox function by ≈ symbol whereever possible, added …

7943818

…few more metrics to test, also replaced all `=` by `≈` for real numbers

Update src/glmfit.jl

3cf14f4

Since we are storing the link - the `GLM.Link` function can be defined uniformly for all link functions Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Mousum added 4 commits October 17, 2022 20:11

Added test cases and changed model_rank to linpred_rank

f0e01a4

Removing PosDefException test for timing. It is throwing error in s…

a2bd6ec

…ome m/c

Added test cases and changed model_rank to linpred_rank

dbba47d

Changed glm model desciptions example.md file

69a1015

Added rankdeficient test cases

353449f

bkamins reviewed Oct 19, 2022

View reviewed changes

test/runtests.jl Outdated Show resolved Hide resolved

nalimilan reviewed Oct 20, 2022

View reviewed changes

mousum-github and others added 10 commits October 20, 2022 14:41

Update src/glmfit.jl

3201d4c

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update src/glmfit.jl

371d01f

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update src/linpred.jl

3c31d26

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update test/runtests.jl

85cbb52

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update test/runtests.jl

b989aae

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update test/runtests.jl

9c86701

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update test/runtests.jl

779ee29

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update src/linpred.jl

601cfc6

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Update test/runtests.jl

4f8c5b4

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>

Shifted the dropcollinear test case to the end. Removed commneted tes…

a4a2466

…t case

bkamins reviewed Oct 20, 2022

View reviewed changes

src/glmfit.jl Outdated Show resolved Hide resolved

Update src/glmfit.jl

07e44de

Co-authored-by: Bogumił Kamiński <bkamins@sgh.waw.pl>

nalimilan approved these changes Oct 21, 2022

View reviewed changes

bkamins approved these changes Oct 21, 2022

View reviewed changes

nalimilan merged commit 1459737 into JuliaStats:master Oct 21, 2022

nalimilan deleted the dropcollinearity branch October 21, 2022 07:50

andreasnoack mentioned this pull request May 7, 2024

Add support for rank deficient GeneralizedLinearModel #340

Closed

andreasnoack mentioned this pull request Nov 19, 2024

Failed to "solve" logistic regression that R can #449

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of `dropcollinear` feature in `GeneralizedLinearModel` #488

Implementation of `dropcollinear` feature in `GeneralizedLinearModel` #488

mousum-github commented Jul 15, 2022 •

edited

Loading

mousum-github commented Oct 10, 2022

mousum-github commented Oct 17, 2022

nalimilan commented Oct 17, 2022

mousum-github commented Oct 17, 2022 •

edited

Loading

mousum-github commented Oct 17, 2022

nalimilan left a comment

nalimilan left a comment

Implementation of dropcollinear feature in GeneralizedLinearModel #488

Implementation of dropcollinear feature in GeneralizedLinearModel #488

Conversation

mousum-github commented Jul 15, 2022 • edited Loading

mousum-github commented Oct 10, 2022

mousum-github commented Oct 17, 2022

nalimilan commented Oct 17, 2022

mousum-github commented Oct 17, 2022 • edited Loading

mousum-github commented Oct 17, 2022

nalimilan left a comment

Choose a reason for hiding this comment

nalimilan left a comment

Choose a reason for hiding this comment

Implementation of `dropcollinear` feature in `GeneralizedLinearModel` #488

Implementation of `dropcollinear` feature in `GeneralizedLinearModel` #488

mousum-github commented Jul 15, 2022 •

edited

Loading

mousum-github commented Oct 17, 2022 •

edited

Loading