POUNDERS/MATLAB GitHub Testing Action #146

jared321 · 2024-04-25T21:54:02Z

The work associated with the branch has been finished. The following will be done as part of a self-review of the PR

All manual MATLAB testing was done on GCE/compute-004 with MATLAB R2023b.

@jmlarson1
The miss_hit action presently logs the following without the action indicating failure

In pounders/m/general_h_funs/emittance_combine.m, line 1
| function [G, H] = emittance(Cres, Gres, Hres)
|                   ^^^^^^^^^ check (low): function name does not match the filename emittance_combine [filename_primary_entity_name]
MISS_HIT Lint Summary: 57 file(s) analysed, 1 check(s)

@jmlarson1 @mmenickelly Can you both please review this work.

pounders.m and its tests often add folders to the MATLAB path but do so using relative paths, which can influence from where you can call the code. The changes here adopt a different path strategy using paths relative to the file that is adding to the path. This is related to Issue #127. A second goal is to make it so that users don't have to add too many paths. Confirmed that the full test suite can be run as detailed in its inline documentation. This includes creating an HTML coverage report. Confirmed that changes to the path made by tests are not present upon termination. Ran each single test individually with a clean MATLAB environment to confirm that they run through with and only with the documented path requirements. Confirmed that they leave the path as found upon entry.

The output of this file can vary based on the local compute environment used to run the test. The differences do not necessarily indicate a failure or problem with the test. As such there is no single baseline and we remove the file.

coveralls · 2024-04-25T21:55:39Z

Pull Request Test Coverage Report for Build 8854673898

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+4.4%) to 100.0%

Totals
Change from base Build 8787010077:	4.4%
Covered Lines:	15
Relevant Lines:	15

💛 - Coveralls

The matrix seems to work OK, so let's just stick to Ubuntu for now.

Changes analogous to those made for POUNDERS in commit ba4fbb1. Tested in similar way. See reference commit's message for more details. Additionally confirmed that if I run MATLAB tests with coverage enabled from the root folder, that all MATLAB tests in IBCDFO were run and all results included in a single coverage report.

I am getting a Commit creating failed: ["Service not found: none"] failure in CodeCov. Some issues online make it sound like this is due to git not having the correct safe directory set. The checkout action is setting safe directories to both of the repos. However, these are not being set for the default folder due to my custom installation of two repos. Setting the default workspace as safe to see if this helps. This is where the reports are and where code cov is running from (AFAIK).

codecov · 2024-04-26T21:31:00Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

Ran with BenDFO in default location and saw it pass. Moved clone to different location and saw it fail. Set BenDFO env var to wrong path and saw it fail. Set env var to correct location and saw it pass.

wildsm · 2024-08-05T15:54:30Z

As discussed, we should hold off on this until the cell2mat(allcomb(-)) have been eliminated.

This will mean that allcomb is not referenced in the readme and will allow the tests/actions to complete without the additional external dependency.

I was able to run the full manifold sampling test suite on GCE using MATLAB/2023b. All three tests passed even after I deleted my local copy of allcomb.

Also try to get MATLAB coverage to include all tests.

In particular, we should not include coverage of the MINQ submodule in coverage.

Each test log can be quite long. This makes it easier to review the logs and quickly zero in on failures.

If all goes well, this would be addressed in Issue #157.

Ran this successfully on GCE/compute-01. Coverage report contains all subpackages and seems reasonable.

Update `145MatlabCI` pre-merge

jared321 added 4 commits April 25, 2024 12:54

POUNDERS benchmark solved.txt removed from repo.

bf28cbd

The output of this file can vary based on the local compute environment used to run the test. The differences do not necessarily indicate a failure or problem with the test. As such there is no single baseline and we remove the file.

Fix my bad markdown formatting

4371594

First try at a MATLAB-based CI action.

39e8ca9

jared321 self-assigned this Apr 25, 2024

The first of many CI action fixes.

0a13af0

jared321 added 21 commits April 25, 2024 16:59

macOS failing to install MATLAB with strange error.

67b44c6

The matrix seems to work OK, so let's just stick to Ubuntu for now.

Simplify...

5f8e890

Please addpath...

c65994b

Take two on addpath

cc16ce7

Two addpaths?! Now you're getting cocky...

ae479bc

Try a different YAML line continuation

bbaef61

Try out the full command sequence again.

81cfc94

Make sure we get MINQ.

dc3ad5e

Seems to work for 2021a. Try other releases.

b1dbae6

(Issue #145) Run code coverage and generate artifacts.

de2107c

(Issue #145) See if we can find the reports for artifact.

791d048

(Issue #145) Trying again...

b4e90a6

(Issue #145) Fix typos

3655f17

(Issue #145) Try to get MATLAB report in CodeCov.

a869c4a

(Issue #145) Help CodeCov find report?

6917bde

(Issue #145) See if I can get more information for debugging

e339e17

Try idea of last commit in a different way.

fff319e

Take three.

1367d48

Try nested repos instead of side-by-side

f88dcf8

(Issue #145) Clean-up a bit.

79de7e0

jared321 added 4 commits August 5, 2024 09:54

(Issue #145) Provide MATLAB script for users to test all subpackages.

02de8dc

Ran with BenDFO in default location and saw it pass. Moved clone to different location and saw it fail. Set BenDFO env var to wrong path and saw it fail. Set env var to correct location and saw it pass.

Add in Python test script to match new MATLAB script.

47b844c

(Issue #145) Placate miss hit.

bb8932b

Black is forcing me to have really long lines.

3ca8d3c

jmlarson1 and others added 13 commits August 20, 2024 16:48

typos

f5db8e0

Using product of cells

82d9d0d

mh

388f632

Removing another allcomb

b3bd568

(Issue #150) Removed all other ManifoldSampling/m references to allcomb.

b6051a9

I was able to run the full manifold sampling test suite on GCE using MATLAB/2023b. All three tests passed even after I deleted my local copy of allcomb.

(Issue #150) Removed all mentions of allcomb from READMEs.

b200f42

(Issue #145) MATLAB testing action now runs all tests that it finds.

fe59f4a

(Issue #145) Use env vars and fix broken action.

057d847

Also try to get MATLAB coverage to include all tests.

(Issue #145) Fix typo and copy/paste typo.

86fc96e

(Issue #145) See if we can load the optimization toolbox for MS.

04d3a27

(Issue #145) MATLAB coverage on pounders and manifold sampling only.

57bf1e3

In particular, we should not include coverage of the MINQ submodule in coverage.

(Issue #145) No testing on MINQ please.

8fe6ae4

(Issue #145) Run each subpackage in its own subaction.

a3e3d1b

Each test log can be quite long. This makes it easier to review the logs and quickly zero in on failures.

jared321 mentioned this pull request Sep 9, 2024

MATLAB CI Manifold Sampling Failure #157

Open

(Issue #145) Exclude failing manifold sampling test setup.

3ca698b

If all goes well, this would be addressed in Issue #157.

jmlarson1 approved these changes Sep 10, 2024

View reviewed changes

jmlarson1 and others added 4 commits September 10, 2024 08:50

isort

bb9e1e9

IBCDFO MATLAB testing script can now run manifold sampling as well.

7fc086d

Ran this successfully on GCE/compute-01. Coverage report contains all subpackages and seems reasonable.

Make miss-hit happy. Test MATLAB on Ubuntu 24.04 as well.

e1f6e76

Run Python tests in Ubuntu 24.04 as well.

1b4771e

jared321 mentioned this pull request Sep 16, 2024

Update 145MatlabCI pre-merge #158

Merged

Merge pull request #158 from POptUS/develop

737f582

Update `145MatlabCI` pre-merge

jared321 merged commit b0ea680 into develop Sep 16, 2024
73 checks passed

jared321 mentioned this pull request Sep 16, 2024

Remove allcomb dependence #150

Closed

jmlarson1 deleted the 145MatlabCI branch September 30, 2024 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

POUNDERS/MATLAB GitHub Testing Action #146

POUNDERS/MATLAB GitHub Testing Action #146

jared321 commented Apr 25, 2024 •

edited

Loading

coveralls commented Apr 25, 2024 •

edited

Loading

codecov bot commented Apr 26, 2024

wildsm commented Aug 5, 2024

POUNDERS/MATLAB GitHub Testing Action #146

POUNDERS/MATLAB GitHub Testing Action #146

Conversation

jared321 commented Apr 25, 2024 • edited Loading

coveralls commented Apr 25, 2024 • edited Loading

Pull Request Test Coverage Report for Build 8854673898

Details

💛 - Coveralls

codecov bot commented Apr 26, 2024

Welcome to Codecov 🎉

wildsm commented Aug 5, 2024

jared321 commented Apr 25, 2024 •

edited

Loading

coveralls commented Apr 25, 2024 •

edited

Loading