CMIP7: Remove model_type for CMIP7 processing.

[mo-jareddrayton]

The request.metadata.model_type is redundant for CMIP7 processing but needs some thought as to how best to handle it.

https://github.com/MetOffice/CDDS/blob/main/cdds/cdds/common/request/metadata_section.py#L72

Also need to decouple the validator.

CDDS/cdds/cdds/common/request/validations/cv_validators.py

Lines 85 to 107 in 6332a0e

	@classmethod
	def model_types_validator(cls) -> Callable[[CVConfig, 'Request'], None]:
	"""
	Returns validator to validate the model types against the allowed and required
	model types in the CV.
	:return: validate function
	:rtype: Callable[[CVConfig, 'Request'], None]
	"""
	def validate(cv_config: CVConfig, request: 'Request'):
	allowed_model_types = cv_config.allowed_source_types(request.metadata.experiment_id)
	model_types = request.metadata.model_type
	valid_model_types = set(model_types).issubset(set(allowed_model_types))
	if not valid_model_types:
	raise CVEntryError('Not all model types are allowed by the CV')
	required_model_types = cv_config.required_source_type(request.metadata.experiment_id)
	if required_model_types:
	contain_required_model_types = set(required_model_types).issubset(set(model_types))
	if not contain_required_model_types:
	raise CVEntryError('not all required model types are given.')
	return validate

Configure usage.

CDDS/cdds/cdds/configure/user_config.py

Line 130 in 6332a0e

user_config['cmor_dataset']['model_type'] = ' '.join(user_config['cmor_dataset']['model_type'])

[matthew-mizielinski]

MIP Convert is tightly coupled to this field too -- need to adjust the following too

CDDS/mip_convert/mip_convert/save/cmor/cmor_dataset.py

Lines 92 to 141 in 6332a0e

	def validate_source_type_values(self):
	"""
	Ensure the 'source_type' provided by the
	|user configuration file| conforms to the CVs.
	Note the current version of CMOR (3.4.0) doesn't perform this
	check.
	Raises
	------
	RuntimeError: if any |model type| does not exist in the list of
	valid |model types| from the CVs.
	RuntimeError: if the required |model types| specified for the
	|experiment| from the CVs are not present in the
	|user configuration file|.
	RuntimeError: if any additional |model types| specified in the
	|user configuration file| are inconsistent with the values
	specified for the |experiment| from the CVs.
	"""
	logger = logging.getLogger(__name__)
	user_source_types = self._user_config.source_type
	for source_type in user_source_types:
	# First ensure the value of 'source_type' provided by the 'user configuration file' is a valid value.
	self._cv_config.validate_with_error(source_type, 'source_types')
	if 'experiment_id' in self._cv_config._get_values_from_cv('required_global_attributes'):
	experiment_id = self._user_config.experiment_id
	# Then ensure the required values for the experiment from the CVs are provided by the
	# 'user configuration file'.
	required_source_types = self._cv_config.required_source_type(experiment_id)
	for required_source_type in required_source_types:
	if required_source_type not in self._user_config.source_type:
	message = 'Required model type "{}" for experiment "{}" not present'
	raise RuntimeError(message.format(required_source_type, experiment_id))
	# Finally ensure any additional values for the experiment from the CVs are consistent with the values
	# provided by the 'user configuration file'.
	additional_source_types = self._cv_config.additional_source_type(experiment_id)
	invalid = set(user_source_types).difference(
	set(required_source_types)).difference(set(additional_source_types))
	if invalid:
	raise RuntimeError(
	'"{}" is inconsistent with the additional values specified for the experiment "{}" '
	'from the CVs ("{}")'.format(' '.join(sorted(invalid)), experiment_id,
	' '.join(additional_source_types))
	)
	else:
	logger.debug('Could not validate source type for experiment as experiment_id is not '
	'a required global attribute')

Checking for source_type in the required_global_attributes list could be done to avoid failures here.