Name: beets
URL: https://github.com/beetbox/beets
Number of lines of code and the tool used to count it: 49267 lines counted with lizard
Programming language: Python
We ran Coverage.py, found at https://coverage.readthedocs.io/en/7.5.4/. It was executed by running it on the testall.py file using the command ‘coverage run test/testall.py’.
This is a screenshot of the coverage report generated by the command ‘coverage html’. The repository had a coverage of 76.75% without any of our changes. However, this varies between devices. Harman and Collin both got this 76.75% figure using Ubuntu via WSL, but Griffin and Sam used Macs and got slightly higher numbers.
This screenshot includes the coverage percentages for each of the files we worked on, highlighted in yellow. Note that the percentages are for the files as a whole, not the individual functions.
Collin
configure() in beets/autotag/mb.py
https://github.com/beetbox/beets/commit/4da5fbb67184ebc687f9fe34d4a5352580672971 is the commit I made adding the coverage information to my two functions. It was the same commit for both functions.
This is a screenshot of the original coverage for configure(), before tests were added.
track_info() in beets/autotag/mb.py
https://github.com/beetbox/beets/commit/4da5fbb67184ebc687f9fe34d4a5352580672971 is the same commit as above containing the coverage information.
This is a screenshot of the original coverage for track_info(), before tests were added.
Sam
Both functions were adapted in the same commit, the following: https://github.com/ark3ll/beets/commit/5b0cdaa354a757edda2c32209177f056ffd123cb
match_by_id() in beets/autotag/match.py
This is the coverage percentage for match_by_id() as attained by my own coverage tool, before new tests were added.
has_program() in beets/test/helper.py
This is the coverage percentage for has_program() as attained by my own coverage tool, before new tests were added.
Griffin
add() in beets/autotag/hooks.py Both function’s coverage edits were in the same commit: https://github.com/ark3ll/beetsSEP/commit/c63095b386b2df30e8b6b2fe0290dbf7d4446aec
This is the branch coverage for distance.add() in beets/autotag/hooks
update() in beets/autotag/hooks.py
This is the branch coverage for distance.update() in beets/autotag/hooks.py
Harman match_album()
The link for the commit below contains the code required to find the cover measurements for the respective functions for which an increase in coverage was aimed for: https://github.com/ark3ll/beetsSEP/commit/9e110adb6cbe5fdea2740fccae3be0490ff176e9
In the above screenshot we find that the coverage for the match_album function is 44.44% with quite a few parts of the code not being covered.
TrackInfo.decode()
The link for the commit below contains the code required to find the cover measurements for the respective functions for which an increase in coverage was aimed for: https://github.com/ark3ll/beetsSEP/commit/9e110adb6cbe5fdea2740fccae3be0490ff176e9
In the above screenshot we find that the coverage for the TrackInfo.decode() function is 75% with only one part of the code not hitting
Collin
test_my_configure() in test/test_mb.py
https://github.com/beetbox/beets/commit/c9f695b342e0f465d70645e0125639af68a0efcf is the commit I made creating new tests for my functions. It is the same commit for both functions.
This is the old coverage for configure() first and the new coverage second.
The coverage doubled from 50% to 100% by adding a test to check if I was able to change the hostname away from musicbrainz.org.
test_my_track_info() in test/test_mb.py
https://github.com/beetbox/beets/commit/c9f695b342e0f465d70645e0125639af68a0efcf is the same commit as above for my tests.
This is the old coverage for track_info() first and the new coverage second.
The coverage almost doubled from 46% to 89% primarily by adding tests that made sure artists who worked on specific tracks could have their names stored in the track information. The tests also made sure ISRC information can be stored. The coverage did not reach 100%, mostly because there is also extra track data that can be handled by a plugin that I did not know how to test. An increase to 89% branch coverage is still more than good enough. Sam
As with the coverage tool, both tests were added in the same commit: https://github.com/ark3ll/beets/commit/5b0cdaa354a757edda2c32209177f056ffd123cb
Test 1: test_match_by_id.py
As shown above, the coverage improved by 50%, to 100%. This was done by writing new tests, specifically the following: ‘test_no_album_id’: Checks that the function returns None when all items lack an album ID. Covers the StopIteration branch. ‘test_consensus’: Ensures that the function correctly identifies and returns album information when all items have the same album ID. Covers the try and else branches when there is a consensus. ‘test_no_consensus’: Makes sure the function returns None when items have different IDs. Covers the try and if branches when there is no consensus. ‘test_loop_execution’: Confirms the function returns the correct information when multiple items share the same ID. Covers the try, for loop and else branches when there is a consensus with multiple items.
Test 2: test_has_program.py
As shown, coverage was improved by 25%, again to 100%. This was also done by writing new tests, ones that ensured all branches were hit. The function itself includes calling ‘subprocess.check_call’, to check if it is possible to run the commands passed to it. As such, each test also involves this. Each test covers the try branch, and their own respective branches. ‘test_called_process_error’: Ensures the function returns False when a process error is encountered. Covers the Process Error branch. ‘Test_os_error’: Checks that the function returns False when an OSError is raised. Covers the OSError branch. ‘test_success’: Ensures that the function returns True when the command is executed without errors. Covers the else branch.
Griffin
<Test 1> Both tests were added in the same github commit: https://github.com/ark3ll/beets/commit/4189b38549d7f788d8a08fd04b1bfea16f972ede
The coverage improves by 50 percent. This is because I added a test that passes an illegal value of a variable “dist” to check that its failsafe state would not cause any other major problems in the code.
<Test 2>
The coverage was improved by 25% by adding a test that attempts to add an empty data structure to a list of data structures. This correctly raises an error and does not mutate the data of the original structure.
Harman
test_match_album_extra_tags and test_match_album_none
The link below is for the commit after committing the new tests to increase coverage of their respective functions: https://github.com/ark3ll/beetsSEP/commit/b9b1940b7992be2ebe7f77c83101cc5f00a390dc
In the first photo we see the old coverage results which are 44.44%. In the second photo we have the new coverage results after implementing more tests and see that the coverage now increases to 83.33%. This improvement in coverage was seen with 2 new tests being implemented. Firstly it is important to note that this function had an input where 4 arguments were taken, however only 2 of these were required and hence tests were only made around these 2 required arguments. However a large part of the code which was not being covered was made for handling one of the optional arguments hence a test was made in which the optional argument was given and then handled by the code thereby increasing coverage. Another test had to be made however as this one test alone did not satisfy the coverage threshold hence another one was made. This new test checked whether one of the required arguments existed or if it was NULL. In the code there was a check to see if this argument was NULL however there was no test hence this check never ran. The test made simply gave an input where the argument was NULL and that part of the code was covered hence increasing coverage.
test_decode_with_bytes
The link below is for the commit after committing the new tests to increase coverage of their respective functions: https://github.com/ark3ll/beetsSEP/commit/b9b1940b7992be2ebe7f77c83101cc5f00a390dc
In the first photo we see the old coverage results which are 75%. In the second photo we have the new coverage results after implementing more tests and see that the coverage now increases to 100%. The function that the test is implemented for, TrackInfo.decode() gets string inputs and then assigns them to certain attributes, however there is a check in the function for whether the input is in bytes and not strings however there is not test for it in bytes hence the bytes conversion code is never ran, The implemented test essentially gives the function an input in bytes and expects a return of strings signifying that the function converted from bytes to a string. With this additional test the coverage increased to 100%
These were the screenshots of our original coverage in Coverage.py, before any additions to the code or the tests.
These are the screenshots of our new coverage in Coverage.py after adding our tests. The total coverage of our repository went up from 76.75% to 77.18%. The coverage of three of our files also increased. Oddly, the coverage of beets/autotag/match.py went down for some reason. The coverage of our specific function we worked on, match_by_id(), did go up, as evidenced by Sam’s first set of tests, but for some reason, adding new tests reduced the coverage of the distance() function, which reduced the coverage of the file as a whole.
Sam:
In this assignment I focused on improving code coverage for the ‘match_by_id’ and ‘has_program’ functions by implementing and developing my own coverage tool, along with new unit tests. I specifically developed tests to cover all branches of these functions. I also developed a code coverage percentage function to dynamically calculate and track coverage improvement, alongside implementing the write-to-file function that allowed us to log the coverage data. By making these utilities available for each team member to call in their respective functions, we aim to reduce code repetition and enhance overall quality across the project.
Harman:
In this assignment, I looked into functions which had limited coverage and could be increased to have better coverage. I ended up working on improving the coverage of the functions TrackInfo.decode() and match_album(). I focused on finding which branches in these functions were not being accessed due to lack of testing and from there proceeded to increase the coverage of both functions by implementing tests with a wider spread of inputs which allowed for more branches to be run.
Collin:
In this assignment, I worked on improving the coverage for the functions configure() and track_info(), both in beets/autotag/mb.py. Configure was a very simple function to increase, with just the one if statement. Track info was much longer, which was advised against, but when I read it, it looked very intuitive and easy to test. All I had to do was make sure information about artists related to the track was stored properly. Additionally, I coordinated the writing of the readme, doing things such as taking the screenshots of the overall coverage information and writing about the coverage tool we used.
Griffin:
In this assignment, I worked on improving the coverage for the functions add() and update() in the class distance in beets/autotag/hooks. The functions themselves were simple but integral to the functionality of the autotag system and there were no tests checking if activating their fail state could mutate the data structures in an illegal way. I created some tests for these functions to evaluate if activating the error states mid-operation would edit any variable code and improved the branch coverage of both functions to 100%.