Fix tests

[faysal-ishtiaq]

No description provided.

[faysal-ishtiaq]

@isedwards there is one more failing test which is not related to plots but a table. This is the failing test

FAILED tests/test_cdms_products.py::test_inventory_table - AssertionError: assert False

I have checked the generated and actual files manually, and they do not look the same.

You can reproduce the result by running pytest

Then you can put this file https://github.com/faysal-ishtiaq/opencdms-process/blob/rpy2Test/tests/data/daily_niger.csv
and generated file ( tests/results_actual/inventory_table_actual010.csv ) side by side and see the difference. Looking at the files, it seems that the bug's origin is in cdms.products.

[lloyddewit]

@faysal-ishtiaq If it helps, then I could look at results_actual/inventory_table_actual010.csv to see if I have an idea what's causing the difference.

[isedwards]

@faysal-ishtiaq If it helps, then I could look at results_actual/inventory_table_actual010.csv to see if I have an idea what's causing the difference.

@lloyddewit, yes please - let us know if you can see what is causing the difference

[isedwards]

Was the formatting change here accidental? It looks a bit odd being >79 chars and also the unnecessary concatenation

[faysal-ishtiaq]

Yes, it was due to black, not formatting strings. I have added a make command in the makefile with a preview flag for black so that black takes care of strings as well.

[faysal-ishtiaq]

@lloyddewit

Here is the generated csv file

inventory_table_actual010.csv

[isedwards]

@lloyddewit

Here is the generated csv file

inventory_table_actual010.csv

@lloyddewit did you get a chance to look at what may be causing the difference with the inventory table?

[lloyddewit]

@isedwards @faysal-ishtiaq
I've been sick the last 2 days. Sorry for the slow response.
The difference is due to how null values are displayed in the csv file. In the linked file, they are shown as 'NA_character_' whereas in the expected results file they are shown as empty strings. The other values all look correct.
I'll investigate further when I'm recovered, I hope that's OK. Apologies again for the inconvenience.

[lloyddewit]

@isedwards @faysal-ishtiaq
I ran the inventory_table test directly in the R environment. The results were functionally the same as the linked file and the expected results in the repo. The only differences are how months with less than 31 days are represented (February, April, June etc.). For example 31 April is shown as below in the R environment, expected results file and linked file respectively:

I suspect that our development environments are behaving slightly differently in the line below:

In summary, I think your results are correct as far as the user is concerned. The test comparison fails because missing values are written differently to the csv file (or potentially represented differently in the pandas data frame).

In the context of your current work, how important is this?

This code requires Python, R and many third-party R/Python packages. The dependencies are complex and it's hard to get them identical in different development environments. In the future, should we consider having a reference installation and use this to run the unit tests?

[isedwards]

Just for info: the name of this repo recently change from github.com/opencdms-processes/opencdms-process to github.com/opencdms-components/opencdms-rclimatic in order to reflect that this repo will be a component/plugin for OpenCDMS for working with the cdms.products R Climatic library. The old links will continue to redirect to the new repo, so you can continue to work with the old name if this helpful.