Warnings have been turned off

[aidanheerdegen]

Capturing of warnings is turned on here:

cosima-cookbook/cosima_cookbook/database.py

Line 45 in 19808e0

logging.captureWarnings(True)

It occurred in this commit c196861

It means users no longer get warnings that were designed to tell users there was potentially a problem with their query:

cosima-cookbook/cosima_cookbook/querying.py

Lines 307 to 322 in 19808e0

	# check whether the results are unique
	unique_files = set(os.path.basename(f.NCFile.ncfile) for f in ncfiles)
	if len(unique_files) > 1:
	warnings.warn(
	f"Your query gets a variable from differently-named files: {unique_files}. "
	"This could lead to unexpected behaviour! Disambiguate by passing "
	"ncfile= to getvar, specifying the desired file."
	)
	unique_freqs = set(f.NCFile.frequency for f in ncfiles)
	if len(unique_freqs) > 1:
	warnings.warn(
	f"Your query returns files with differing frequencies: {unique_freqs}. "
	"This could lead to unexpected behaviour! Disambiguate by passing "
	"frequency= to getvar, specifying the desired frequency."
	)

Arguably the multiple filenames warning is no longer fit for purpose. Since the diagnostics have been split out into individual files with dates in the filename this warning is misleading and unhelpful.

However there is currently an issue with queries returning incorrect data (#252) and warning the user about this might be one potential approach.

[angus-g]

Agreed about the multiple filename warning, but maybe we should be showing "warn" level warnings by default anyway? It's curious that they'd be hidden!

[aidanheerdegen]

Agree definitely should be emitting warnings.

Weirdly in #252 I am getting warnings. Hrm. Might have jumped the gun on this. Will check more thoroughly tomorrow.

[aidanheerdegen]

They're being captured by logging due to the logging.captureWarnings(True) in database.py.

Simple example:

import cosima_cookbook as cc
from cosima_cookbook import querying

import warnings
import logging

logging.captureWarnings(False)
warnings.simplefilter("always")

session = cc.database.create_session()

# Define experiment in database
expt = '01deg_jra55v140_iaf_cycle2'
start_time = '1990-01-01'
end_time   = '1990-12-31'

files = querying._ncfiles_for_variable(expt,
                                       'u',
                                       session,
                                       start_time=start_time,
                                       end_time=end_time,
                                       frequency='1 monthly',
                                       attrs_unique={'cell_methods': 'area: time: mean'},
                                       )

With the latest version of the cookbook I get this output (it loads the files because I've reset the warnings filter, otherwise it would thrown an exception which does make it to the console)

$ python test.py 
/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.07/lib/python3.9/site-packages/cosima_cookbook/querying.py:357: QueryWarning: Your query returns variables from files with different cell_methods: {'time: mean', 'time: mean_pow(02)'}. This could lead to unexpected behaviour! Disambiguate by passing attrs={'cell_methods'=''} to getvar, specifying the desired attribute value.
  warnings.warn(
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-mean-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-pow02-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-mean-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-mean-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-mean-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-mean-ym_1990_10.nc

If I comment out logging.captureWarnings(False) no warning makes it to stderr

$ python test.py 
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-mean-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-pow02-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-mean-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-mean-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-mean-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-mean-ym_1990_10.nc

Consequently I think this should be removed from database.py.

[aidanheerdegen]

Created a custom warning for query errors and reset filter for it to throw exception in #257