-
Notifications
You must be signed in to change notification settings - Fork 183
DIRAC v6r15
The main new feature in v6r15 release
- New class for handling errors in S_ERROR() return structure
- Changes in the File Catalog client to have all the methods defined locally
- Changes FileCatalog container class for simplifications
- Updates in the File Catalog: recursive operations, dynamic module loading and others
- Changes in the DIRAC installation tools to allow installation from CVMFS if available
- RSS updates adding the CE availability policies
- Enhanced Parametric jobs with multiple parameter sequences
- Initial support for using multi-processor computing slots
- Making use of the Machine-Job-features information if available on worker nodes or in the virtual machines;
The PR https://github.com/DIRACGrid/Externals/pull/8 provided several changes to the externals shipped with DIRAC. The main change introduced within this version is that MySQL is NOT anymore part of the externals. If your installation has been using the DIRAC version of MySQL that has been included up to DIRAC v6r14, then you will have to intervene on your databases BEFORE deploying version v6r15. Some suggestions on the best way to proceed may be advertised later in this wiki. Alternatively, users may find suggestions by writing on the DIRAC google forum.
The new externals tag for servers (v6r4) are compiled for few (more modern) platforms (e.g. for CentOS 7).
Important change: The values for registration and third party protocols have to be moved from Resources to Operation, and new lists are needed :
DataManagement/RegistrationProtocols # Comes from Resources/FileCatalogs
DataManagement/ThirdPartyProtocols # Comes from Resources/FileCatalogs
DataManagement/AccessProtocols
DataManagement/WriteProtocols
For the extensions, the Resource Helpers getRegistrationProtocols and getThirdPartyProtocols are replaced with methods (same name) in the DMSHelpers class
These lists can be overwritten at the level of a StorageElement, and they should be if it is a special case. For example, for LHCb, the OnlineRunDB storageElements needs to have "http" as WriteProtocols
New procedures need to be added (BEFORE THE CODE):
ps_set_dir_gid_recursive
ps_set_dir_uid_recursive
ps_set_dir_mode_recursive
And two procedures need to be recreated for bug fix (https://github.com/DIRACGrid/DIRAC/pull/2806) :
ps_delete_replica_from_file_and_se_ids
ps_delete_files
Simply copy paste the code of the mentioned procedures, with eventually the drop statement before, from DataManagementSystem/DB/FileCatalogWithFkAndPsDB.sql
The DirectoryUsage tables will have to be rebuilt at a convenient time by calling the ps_rebuild_directory_usage procedure
Added a new table HostLogging to the already existing InstalledComponentsDB database. Once the software has been updated, this new table can be automatically created by restarting the ComponentMonitoring service.
This new table will store logging information on the hosts periodically and will now replace the CLI's getHostInfo call ( since checking the database is faster than obtaining the fields for every call ). In order to ensure that the data starts logging into the database it is necessary to restart the SystemAdministrator service once the software has been updated for every host that should have its information logged.
TransformationDB TaskInputs table change:
ALTER TABLE TaskInputs CHANGE COLUMN InputVector InputVector MEDIUMTEXT;
The general "from DIRAC.Core.Utilities import *" in the top-level init.py is removed. The DIRAC code is updated to import utilities from where they are defined. However in the extensions such imports can be still done. This should be fixed by the extension developers
The S_ERROR function returns a more complicated structure than it used to before and have 2 alternative ways of calling:
1. S_ERROR( "Error message" )
2. S_ERROR( Errno, "Error message" )
Therefore it retains backward compatibility with all the codes that used this function before. The new form 2. has the first integer argument which represents an Error code. Numerical Error codes are defined in the /DIRAC/Core/Utilities/DErrno.py module. DIRAC extensions can define new specific Error codes in the corresponding DErrno.py module which will be automatically incorporated. The return dictionary contains two additional fields "Errno" and "CallStack". "Errno" contains the numerical error. The "CallStack" contains the calling stack of the line in which the S_ERROR is called. The calling stack is printed out in the logger.debug() function if the new S_ERROR return dictionary is passed to it. In order to facilitate comparison of return structures to a numerical Error a cmpError( inErr, compErr ) function is provided, where compErr is a numerical Erro code and inErr is the result of the new S_ERROR output.
The new VOMS2CSAgent agent performs synchronization of the user data in the VOMS service with the DIRAC Registry. It replaces the UserAndGroupAgent. It is working with multiple VOs in multi-VO installations. It also uses the VO data from the CS located in the /Registry/VO section. The /Registry/VOMS section is obsoleted, please, update your Configuration.
On a similar note: the Bdii2CSAgent now completely replaces the CE2CSAgent, which has been removed.
The new GOCDB2CSAgent is used to synchronize information between GOCDB and DIRAC configuration System (CS). For now, it will update the PERFSonar endpoints
SiteDirector Flag to download pilot output to PilotAgentsDB moved to False (was: True)
Since version v6r14p11, DIRAC pilots included by default 2 more commands: CheckCECapabilities and CheckWNCapabilities. These commands have the role to enrich the local dirac pilot configuration file with information including possible tags (e.g. a MultiProcess Tag, if given), the number of available processors, and the total amount of memory of the worker node.
In order to specify the notification Groups, the old field has moved its CS location from
Operations-->Defaults-->ResourceStatus-->PolicyActions-->notificationGroups
to
Operations-->Defaults-->ResourceStatus-->Config-->notificationGroups
There is also a new agent to install:
[user@diracClient ~]$ dirac-admin-sysadmin-cli --host=yourhost.some.where
Pinging yourhost.some.where...
[yourhost.some.where]> install agent ResourceStatus EmailAgent
...