forked from DIRACGrid/DIRAC
-
Notifications
You must be signed in to change notification settings - Fork 0
/
release.notes
4892 lines (3557 loc) · 166 KB
/
release.notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
[v6r15-pre8]
Removed general "from DIRAC.Core.Utilities import *" in the top-level __init__.py
*Core
NEW: New class DError for reporting errors in output results of DIRAC functions
CHANGE: gMonitor instantiation removed from DIRAC.__init__.py to avoid problems in
documentation generation
NEW: dirac-install - makes us of the DIRAC tar files in CVMFS if available
NEW: dirac-install-client - a guiding script to install the DIRAC client from A to Z
NEW: MJFTimeLeft - using Machine JOb features in the TimeLeft utility
FIX: BaseClient - only give warning log message "URL banned" when one of the
service URLs is really banned
NEW: dirac-rss-policy-manager - allows to interactively modify and test only the
policy section of Dirac.cfg
FIX: XXXTimeLeft - do not mix CPU and WallTime values
*Configuration
NEW: GOCDB2CSAgent agent to synchronize GOCDB and CS data about perfSONAR services
NEW: VOMS2CSAgent to synchronize VOMS user data with the DIRAC Registry
*Framework
CHANGE: SystemAdministratorIntegrator - make initial pinging of the hosts in parallel
to speed up the operation
CHANGE: InstalledComponentsDB - table to cache host status information populated
by a periodic task
NEW: ComponentInstaller Client class
*RSS
NEW: CE Availability policy, closing #2373
*DMS
CHANGE: FileCatalogClient - make explicit methods for all service calls
CHANGE: DataManager, StorageElement - move physical accounting the StorageElement
CHANGE: FileCatalog - added recursive changePathXXX operations
NEW: ConsistencyInspector class to perform data consistency checks between
different databases
CHANGE: FileCatalog(Client) - refactored to allow clients declare which interface
they implement
*WMS
NEW: TaskQueueDB - possibility to present requirements in a form of tags from the
site( pilot ) to the jobs to select ones with required properties
FIX: JobWrapper - the InputData optimizer parameters are now DEncoded
CHANGE: JobAgent - add Processors and WholeNode tags to the resources description
[v6r14p12]
*Core
FIX: allow a renormalization of the estimated CPU power
FIX: dirac-install: Make hashlib optional again (for previous versions of python, since the pilot may end up on old machines)
*Framework
FIX: allow to install agents with non-standard names (different from the module name)
*DMS
CHANGE: Consider files to reschedule and submit when they are Failed in FTS
*WMS
CHANGE: Move getCEStatus function back to using the ARC API
[v6r14p11]
*Core
FIX: XXXTimeLeft - set limit to CPU lower than wall clock if unknown
FIX: Logger - fix exception printing in gLogger.exception()
CHANGE: InstallTools - added more info about the process in getStartupComponentStatus()
CHANGE: Time - better report from timeThis() decorator
*DMS
CHANGE: FTSAgent - wait some time between 2 monitorings of each job
*WMS
NEW: pilotCommands - added CheckCECapabilities, CheckWNCapabilities commands
NEW: Added dirac-wms-get-wn-parameters command
*TS
NEW: Added dirac-production-runjoblocal command
FIX: TransformationAgent(Plugin) - clean getNextSite() and normalizeShares()
FIX: TransformationPlugin - added setParameters() method
*RSS
FIX: dirac-rss-sync - move imports to after the Script.getPositionalArguments()
*Resources
NEW: Added dirac-resource-get-parameters command
[v6r14p10]
*Configuration
FIX: Resources - getQueue() is fixed to get properly Tag parameters
*Framework
FIX: SecurityFileLog - fix for zipping very large files
*Resources
NEW: added dirac-resource-get-parameters command
*WMS
NEW: JobMonitoringHandler - add getJobsParameters() method
NEW: pilotCommands - added CheckCECapabilities, CheckWNCapabilities
NEW: Added dirac-wms-get-wn-parameters command
NEW: Matcher - generate internal tags for MaxRAM and NumberOfProcessors parameters
CHANGE: SiteDirector does not pass Tags to the Pilot
FIX: Matcher(Handler) - do not send error log message if No match found,
fixed Matcher return value not correctly interpreted
[v6r14p9]
*Core
FIX: BaseClient - enhance retry connection logic to minimize the overall delay
FIX: MessageBroker - fix of calling private __remove() method from outside
of the class
*Framework
BUGFIX: dirac-(un)install-component - bug in importing InstallTools module
*WMS:
FIX: JobWrapper - fix in getting the OutputPath defined in the job
*Resources
FIX: ARCComputingElement - add queue to the XRSL string
[v6r14p8]
*Core
FIX: XXXTimeLeft - minor fixes plus added the corresponding Test case
FIX: ReturnValues - fixes in the doc strings to comply with the sphinx syntax
FIX: SocketInfoFactory - in __sockConnect() catch exception when creating a
socket
*Interfaces
FIX: Job.py - fixes in the doc strings to comply with the sphinx syntax
*RSS
NEW: Configurations.py - new possible configuration options for Downtime Policies
*WMS
CHANGE: StatesAccountingAgent - retry once and empty the local messages cache
in case of failure to avoid large backlog of messages
CHANGE: SiteDirector - do not send SharedArea and ClientPlatform as pilot
invocation arguments
CHANGE: Matcher - allow matching by hosts in multi-VO installations
[v6r14p7]
*Core
CHANGE: XXXTimeLeft utilities revisited - all return real seconds,
code refactoring - use consistently always the same CPU power
*WMS
FIX: JobAgent - code refactoring for the timeLeft logic part
*Resources
BUGFIX: ComputingElement - get rid of legacy getResourcesDict() call
[v6r14p6]
*Configuration
FIX: Bdii2CSAgent - refresh configuration from Master before updating
FIX: Bdii2CSAgent - distinguish the CE and the Cluster in the Glue 1.0 schema
*DMS
CHANGE: FTSAgent - make the amount of scheduled requests fetched by the
FTSAgent a parameter in the CS
CHANGE: RMS Operations - check whether the always banned policy is applied for SEs
to a given access type
*RMS
FIX: RequestClient(DB,Manager) - fix bulk requests, lock the lines when selecting
the requests to be assigned, update the LastUpdate time, and expose the
assigned flag to the client
*WMS
FIX: JobAgent - when the application finishes with errors but the agent continues
to take jobs, the timeLeft was not evaluated
FIX: JobAgent - the initial timeLeft value was always set to 0.0
[v6r14p5]
*Core
FIX: X509Certificate - protect from VOMS attributes that are not decodable
*Resources
FIX: GFAL2_StorageBase - fixed indentation and a debug log typo
*WMS
BUGFIX: Matcher - only the first job was associated with the given pilot
FIX: pilotTools - 0o22 is only a valid int for recent python interpreters,
replaced by 18
[v6r14p4]
*Core
FIX: DictCache - fix the exception in the destructor preventing the final
cache cleaning
*Framework
FIX: SystemAdministratorClientCLI - corrected info line inviting to update
the pilot version after the software update
*DMS
FIX: FTSAgent - Add recovery of FTS files that can be left in weird statuses
when the agent dies
CHANGE: DataManager - allow to not get URLs of the replicas
CHANGE: FTSJob - keep and reuse the FTS3 Context object
*Storage
CHANGE: StorageManagerClient - don't fail getting metadata for staging if at
least one staged replica found
*WMS
FIX: CPUNormalization - protect MJF from 0 logical cores
FIX: JobScheduling - fix printout that was saying "single site" and "multiple sites"
in two consecutive lines
NEW: pilotTools,Commands - added CEType argument, e.g. to specify Pool CE usage
FIX: WatchDog - added checks of function return status, added hmsCPU initialization to 0,
removed extra printout
*Resources
FIX: GFAL2 plugins - multiple bug fixes
[v6r14p3]
*Core
BUGFIX: small bug fixed in dirac-install-component, dirac-uninstall-component
BUGFIX: VOMS - remove the temporary file created when issuing getVOMSProxyInfo
FIX: FileHelper - support unicode file names
FIX: DictCache - purges all the entry of the DictCache when deleting the DictCache object
*Framework
BUGFIX: dirac-populate-component-db - avoid return statement out of scope
*Interfaces
BUGFIX: Dirac - in submitJob() faulty use of os.open
*WMS
FIX: JobWrapper - avoid evaluation of OutputData to ['']
FIX: Matcher - the Matcher object uses a VO dependent Operations helper
CHANGE: JobAgent - stop agent if time left is too small (default 1000 HS06.s)
FIX: CPUNormalization - use correct denominator to get power in MJF
*Resources
FIX: ARCComputingElement - changed implementation of ldap query for getCEStatus
[v6r14p2]
*Core
FIX: Use GSI version 0.6.3 by default
CHANGE: Time - print out the caller information in the timed decorator
CHANGE: dirac-install - set up ARC_PLUGIN_PATH environment variable
*Framework
FIX: dirac-proxy-info - use actimeleft VOMS attribute
*Accounting
CHANGE: Removed SRMSpaceTokenDeployment Accounting type
*RSS
CHANGE: ResourceStatus - re-try few times to update the RSS SE cache before giving up
FIX: XXXCommand, XXXAction - use self.lof instead of gLogger
CHANGE: Added support for all protocols for SEs managed by RSS
*RMS
FIX: Request - produce enhanced digest string
FIX: RequestDB - fix in getDigest() in case of errors while getting request
*Resources
CHANGE: Propagate hideExceptions flag to the ObjectLoader when creating StorageElements
FIX: ARCComputingElement - multiple fixes after experience in production
*WMS
FIX: Pilot commands - fixed an important bug, when using the
dirac-wms-cpu-normalization script
[v6r14p1]
The version is buggy when used in pilots
*Core
NEW: dirac-install-component command replacing dirac-install-agent/service/executor
commands
*Resources
NEW: FileStorage - plugin for "file" protocol
FIX: ARCComputingElement - evaluate as int the job exit code
*RSS
FIX: CSHelpers - several fixes and beautifications
[v6r14]
*Core
NEW: CSGlobals - includes Extensions class to consistently check the returned
list of extensions with proper names
NEW: ProxyManagerXXX, ProxyGeneration, X509XXX - support for RFC proxies
NEW: ProxyInfo - VOMS proxy information without using voms commands
NEW: LocalConfiguration - option to print out license information
FIX: SocketInfo.py - check the CRL lists while handshaking
Configuration
NEW: ConfigurationClient - added getSectionTree() method
*Framework
NEW: InstalledComponentsDB will now store information about the user who did the
installation/uninstallation of components.
*Resources
NEW: ARCComputingElement based on the ARC python API
*RSS
FIX: Improved logging all over the place
*DMS
NEW: New FileCatalog SecurityManager with access control based on policies,
VOMSPolicy as one of the policy implementations.
NEW: lfc_dfc_db_copy - script used by LHCb to migrate from the LFC to the DFC with
Foreign Keys and Stored Procedures by accessing the databases directly
NEW: FileManagerPs.py - added _getFileLFNs() to serve info for the Web Portal
CHANGE: Moving several tests to TestDIRAC
*Interfaces
CHANGE: use jobDescription.xml as a StringIO object to avoid multiple disk
write operations while massive job submission
*WMS
FIX: Watchdog - review for style and pylint
CHANGE: Review of the Matcher code, extracting Limiter and Matcher as standalone
utilities
*Transformation
NEW: New ported plugins from LHCb, added unit tests
[v6r13p21]
*TS
FIX: Registering TargetSE for Standard TransformationAgent plugin
[v6r13p20]
*DMS
FIX: DMSHelpers - allow for more than one Site defined to be local per SE
*Resources
FIX: XRootStorage - fix in getURLBase()
[v6r13p19]
FIX: changes incorporated from v6r12p53 patch
[v6r13p18]
*WMS
FIX: JobWrapper - ported back from v6r14p9 the fix for getting OutputPath
[v6r13p17]
FIX: changes incorporated from v6r12p52 patch
[v6r13p16]
FIX: changes incorporated from v6r12p51 patch
[v6r13p15]
Included patches from v6r12p50 release
[v6r13p14]
*DMS
FIX: ReplicateAndRegister - fix a problem when a file is set Problematic
in the FC but indeed doesn't exist at all
*Resources
CHANGE: StorageFactory - enhance the logic of BaseSE inheritance in the
SE definition in the CS
*WMS
CHANGE: CPUNormalization, dirac-wms-cpu-normalization - reading CPU power
from MJF for comparison with the DIRAC evaluation
FIX: SiteDirector - create pilot working directory in the batch system working
directory and not in "/tmp"
[v6r13p13]
*DMS
BUGFIX: FileCatalogClient - bug fixed in getDirectoryMetadata()
[v6r13p12]
*Resources
FIX: StorageElement - bug fixed in inValid()
CHANGE: StorageFactory - do not interpret VO parameter as mandatory
[v6r13p11]
*DMS
BUGFIX: RemoveReplica - fix in singleRemoval()
FIX: dirac-dms-user-lfns - increased timeout
[v6r13p10]
CHANGE: Use sublogger to better identify log source in multiple places
*Core
CHANGE: Review / beautify code in TimeLeft and LSFTimeLeft
FIX: LSFTimeLeft - is setting shell variables, not environment variables,
therefore added an "export" command to get the relevant variable
and extract then the correct normalization
*Accounting
FIX: DataOperationPlotter - add better names to the data operations
*DMS:
FIX: DataManager - add mandatory vo parameter in __SEActive()
CHANGE: dirac-dms-replicate-and-register-request - submit multiple requests
to avoid too many files in a single FTS request
FIX: FileCatalog - typo in getDirectoryMetadata()
FIX: FileCatalog - pass directory name to getDirectoryMetadata and not file name
FIX: DataManager - in __SEActive() break LFN list in smaller chunks when
getting replicas from a catalog
*WMS
FIX: WMSAdministratorHandler - fix in reporting pilot statistics
FIX: JobScheduling - fix in __getSitesRequired() when calling self.jobLog.info
CHANGE: pilotCommands - when exiting with error, print out current processes info
[v6r13p9]
*Framework
FIX: SystemLoggingDB - schema change for ClientIPs table to store IPv6 addresses
*DMS
BUGFIX: DMSRequestOperationsBase - bug fix in checkSEsRSS()
FIX: RemoveFile - in __call__(): bug fix; fix in the BannedSE treatment logic
*RMS
BUGFIX: Operation - in catalogList()
BUGFIX: ReqClient - in printOperation()
*Resources
FIX: GFAL2_StorageBase - added Lost, Cached, Unavailable in getSingleFileMetadata() output
BUGFIX: GFAL2_StorageBase - fixed URL construction in put(get)SingleFile() methods
*WMS
FIX: InputDataByProtocol - removed StorageElement object caching
[v6r13p8]
*Framework
FIX: MonitoringUtilities - minor bug fix
*DMS
FIX: DataManager - remove local file when doing two hops transfer
*WMS
FIX: SandboxStoreClient - get the VO info from the delegatedGroup argument to
use for the StorageElement instantiation
*TMS
CHANGE: Transformation(Client,DB,Manager) - multiple code clean-up without
changing the logic
[v6r13p7]
*Core
NEW: X509CRL - class to handle certificate revocation lists
*DMS
FIX: RequestOperations/RemoveFile.py - check target SEs to be online before
performing the removal operation.
FIX: SecurityManager, VOMSPolicy - make the vomspolicy compatible with the old client
by calling in case of need the old SecurityManager
*Resources
BUGFIX: Torque, GE - methods must return Message field in case of non-zero return status
FIX: SRM2Storage - when used internaly, listDirectory should return urls and not lfns
*WMS
FIX: ConfigureCPURequirements pilot command - add queue CPU length to the extra local
configuration
FIX: JobWrapper - load extra local configuration of any
*RMS
FIX: RequestDB - fix in getRequestSummaryWeb() to suit the Web Portal requirements
*Transformation
FIX: TransformationManagerHandler - fix in getTransformationSummaryWeb() to suit
the Web Portal requirements
[v6r13p6]
*Core
FIX: X509Chain - use SHA1 signature encryption in all tha cases
*Resources
FIX: ComputingElement - take CPUTime from its configuration defined in the
pilot parameters
*WMS
FIX: SiteDirector - correctly configure jobExecDir and httpProxy Queue parameters
[v6r13p5]
*Resources
BUGFIX: Torque - getCEStatus() must return integer job numbers
FIX: StorageBase - removed checking the VO name inside the LFN
*WMS
FIX: InputData, JobScheduling - StorageElement needs to know its VO
*DMS
FIX: ReplicateAndRegister - Add checksumType to RMS files when adding
checksum value
FIX: DataManager - remove unnecessary access to RSS and use SE.getStatus()
FIX: DMHelpers - take into account Alias and BaseSE in site-SE relation
*RMS
FIX: Request - bug fixed in optimize() in File reassignment from one
Operation to another
*Transformation
FIX: TransformationDB - set derived transformation to Automatic
[v6r13p4]
*Core
FIX: VOMSService - treat properly the case when the VOMS service returns no result
in attGetUserNickname()
*DMS
FIX: FTSAgent, ReplicateAndRegister - make sure we use source replicas with correct
checksum
*RMS
FIX: Request - minor fix in setting the Request properties, suppressing pylint
warnings
CHANGE: File, Reques, Operation, RequestDB - remove the use of sqlalchemy on
the client side
*Resources
FIX: StorageElement - import FileCatalog class rather than the corresponding module
FIX: SLURM - proper formatting commands using %j, %T placeholders
FIX: SSHComputingElement - return full job references from getJobStatus()
*RSS
FIX: DowntimeCommand - checking for downtimes including the time to start in hours
*Workflow
CHANGE: FailoverRequest - assign to properties rather than using setters
*Transformation
FIX: TransformationClient(DB,Utilities) - fixes to make derived transformations work
[v6r13p3]
*DMS
FIX: DataManager - in putAndRegister() specify explicitly registration protocol
to ensure the file URL available right after the transfer
*Resources
FIX: SRM2Storage - use the proper se.getStatus() interface ( not the one of the RSS )
[v6r13p2]
*Framework
FIX: SystemAdministratorHandler - install WebAppDIRAC extension only in case
of Web Portal installation
CHANGE: dirac-populate-component-db - check the setup of the hosts to register
into the DB only installations from the same setup; check the MySQL installation
before retrieving the database information
*DMS
FIX: FTSAgent - fix in parsing the server result
FIX: FTSFile - added Waiting status
FIX: FTSJob - updated regexps for the "missing source" reports from the server;
more logging message
*Resources
FIX: SRM2Storage - fix in treating the checksum type
FIX: StorageElement - removed getTransportURL from read methods
*RMS
FIX: Request - typo in the optimize() method
[v6r13p1]
*Framework
CHANGE: SystemAdminstratorIntegrator - can take a list of hosts to exclude from contacting
*DMS
FIX: DataManager - fix in __getFile() in resolving local SEs
FIX: dirac-dms-user-lfns - sort result, simplify logic
*RMS
FIX: Request - Use DMSHelper to resolve the Failovers SEs
FIX: Operation - treat the case where the SourceSE is None
*WMS
FIX: WMSAdministratorHandler - return per DN dictionary from getPilotStatistics
[v6r13]
CHANGE: Separating fixed and variable parts of error log messages for multiple systems
to allow SystemLogging to work
*Core
FIX: MySQL.py - treat in detailed way datetime functions in __escapeString()
FIX: DictCache.get() returns now None instead of False if no or expired value
NEW: InstallTools - allow to define environment variables to be added to the component
runit run script
NEW: Changes to make the DISET protocol IP V6 ready
CHANGE: BaseClient - retry service call on another instance in case of failure
CHANGE: InnerRPCClient - retry 3 times in case of exception in the transport layer
CHANGE: SocketInfo - retry 3 times in case of handshaking error
CHANGE: MySQL - possibility to specify charset in the table definition
FIX: dirac-install, dirac-distribution - removed obsoleted defaults
NEW: Proxy utility module with executeWithUserProxy decorator function
*Configuration
NEW: CSAPI,dirac-admin-add-shifter - function, and script, for adding or modifying a
shifter in the CS
*Framework
FIX: NotificationDB - escape fields for sorting in getNotifications()
NEW: Database, Service, Client, commands for tracking the installed DIRAC components
*Interfaces
CHANGE: Dirac - changed method names, keeping backward compatibility
CHANGE: multiple commands updated to use the new Dirac API method names
*DMS
NEW: Native use of the FTS3 services
CHANGE: Removed the use of current DataLogging service
CHANGE: DataManager - changes to manage URLs inside StorageElement objects only
FIX: DataManager - define SEGroup as accessible at a site
CHANGE: DirectoryListing - extracted from FileCatalogClientCLI as an independent utility
CHANGE: MetaQuery - extracted from FileCatalogClientCLI as an independent utility
CHANGE: FileCatalogClientCLI uses external DirectoryListing, MetaQuery utilities
CHANGE: FileCatalog - replace getDirectoryMetadata by getDirectoryUserMetadata
NEW: FileCatalog - added new getDirectoryMetadata() interface to get standard directory metadata
NEW: FileCatalog - possibility to find files by standard metadata
NEW: FileCatalog - possibility to use wildcards in the metadata values for queries
NEW: DMSHelpers class
NEW: dirac-dms-find-lfns command
*WMS
NEW: SiteDirector - support for the MaxRAM queue description parameter
CHANGE: JobScheduling executor uses the job owner proxy to evaluate which files to stage
FIX: DownloadInputData - localFile was not defined properly
FIX: DownloadInputData - could not find cached files (missing [lfn])
*RMS
CHANGE: Removed files from the previous generation RMS
CHANGE: RMS refactored based on SQLAlchemy
NEW: ReqClient - added options to putRequest(): useFailoverProxy and retryMainServer
CHANGE: DMSRequestOperationsBase - delay execution or cancel request based on SE statuses
from RSS/CS
FIX: Fixes to make use of RequestID as a unique identifier. RequestName can be used in
commands in case of its uniqueness
*Resources
NEW: Computing - BatchSystem classes introduced to be used both in Local and SSH Computing Elements
CHANGE: Storage - reworked Storage Element/Plugins to encapsulate physical URLs
NEW: GFAL2_StorageBase.py, GFAL2_SRM2Storage.py, GFAL2_XROOTStorage.py
*RSS:
NEW: dirac-admin-allow(ban)-se - added RemoveAccess status
CHANGE: TokenAgent - added more info to the mail
*TS
CHANGE: Task Manager plugins
[v6r12p53]
*DMS
CHANGE: FileCatalogClientCLI - ls order by size, human readable size value
FIX: DirectoryMetadata - enhanced error message in getDirectoryMetadata
*WMS
BUGFIX: JobAgent - bug when rescheduling job due to glexec failure
*TS
NEW: TransformationCLI - added getOutputFiles, getAllByUser commands
NEW: Transformation - added getAuthorDNfromProxy, getTransformationsByUser methods
*Resources
CHANGE: GlobusComputingElement - simplify creating of pilotStamp
[v6r12p52]
*DMS
NEW: dirac-dms-directory-sync - new command to synchronize the contents of a
local and remote directories
FIX: DataManager - in removeFile() return successfully if empty input file list
*TS
NEW: TransformationCLI - getInputDataQuery command returning inputDataQuery
of a given transformation
[v6r12p51]
*Core
FIX: dirac-install - fix to work with python version prior to 2.5
*DMS
CHANGE: FileCatalogClientCLI - possibility to set multiple metadata with one command
*Resources
FIX: HTCondorComputingElement - multiple improvements
[v6r12p50]
*Core
FIX: dirac-install - define TERMINFO variable to include local sources as well
*Framework
FIX: SystemAdministratorHandler - show also executors in the log overview
*DMS
FIX: FileCatalogClientCLI - use getPath utility systematically to normalize the
paths passed by users
*WMS
FIX: PilotStatusAgent - split dynamic and static parts in the log error message
*Resources
NEW: HTCondorCEComputingElement class
[v6r12p49]
*Resources
FIX: GlobusComputingElement - in killJob added -f switch to globus-job-clean command
FIX: ARCComputingElement - create working directory if it does not exist
*DMS
CHANGE: DataManager - added XROOTD to registration protocols
*TMS
FIX: TransformationCLI - doc string
[v6r12p48]
*DMS
FIX: DirectoryTreeBase - fix in changeDirectoryXXX methods to properly interpret input
[v6r12p47]
*DMS
BUGFIX: FileCatalogClientCLI - wrong signature in the removeMetadata() service call
[v6r12p46]
*Core
FIX: GraphData - check for missing keys in parsed_data in initialize()
*WMS
CHANGE: PilotStatusAgent - kill pilots being deleted; do not delete pilots still
running jobs
*RSS
CHANGE: Instantiate RequestManagementDB/Client taking into account possible extensions
*Resources
FIX: GlobusComputingElement - evaluate WaitingJobs in getCEStatus()
FIX: SRM2Storage - error 16 of exists call is interpreted as existing file
FIX: XROOTStorage - added Lost, Cached, Unavailable in the output of getSingleMetadata()
*WMS
FIX: pilotCommands - removed unnecessary doOSG() function
[v6r12p45]
*Resources
FIX: SRM2Storage - error 22 of exists call is interpreted as existing file
( backport from v6r13 )
[v6r12p44]
*WMS
FIX: SiteDirector - consider also pilots in Waiting status when evaluating
queue slots available
*Resources
NEW: SRM2Storage - makes use of /Resources/StorageElements/SRMBusyFilesExist option
to set up the mode of interpreting the 22 error code as existing file
[v6r12p43]
*DMS:
FIX: DirectoryTreeBase - avoid double definition of FC_DirectoryUsage table
in _rebuildDirectoryUsage()
[v6r12p42]
FIX: added fixes from v6r11p34 patch release
[v6r12p41]
*WMS
CHANGE: dirac-wms-job-submit - "-r" switch to enable job repo
[v6r12p40]
*DMS
FIX: DirectoryTreeBase.py - set database engine to InnoDB
[v6r12p39]
FIX: imported fixes from rel-v6r11
[v6r12p38]
*DMS
CHANGE: DataManager - enhanced real SE name resolution
*RMS
FIX: Request - fixed bug in the optimization of requests with failover operations
*Resources
CHANGE: StorageFactory - allow for BaseSE option in the SE definition
[v6r12p37]
*Core
FIX: InstallTools - force $HOME/.my.cnf to be the only defaults file
[v6r12p36]
*Configuration
FIX: Utilities.py - bug fix getSiteUpdates()
[v6r12p35]
*Core
CHANGE: VOMSService - add URL for the method to get certificates
*DMS
FIX: DataManager - in __replicate() set do not pass file size to the SE if no
third party transfer
FIX: RemoveFile, ReplicateAndRegister - regular expression for "no replicas"
common for both DFC and LFC
*WMS
FIX: WMSHistoryCorrector - make explicit error if no data returned from WMSHistory
accounting query
[v6r12p34]
*DMS
BUGFIX: FileCatalogWithFkAndPsDB - fix storage usage calculation
[v6r12p33]
*Core
NEW: VOMSService - added method admListCertificates()
*DMS
BUGFIX: dirac-dms-put-and-register-request - missing Operation in the request
*Resources
FIX: sshce - better interpretation of the "ps" command output
[v6r12p32]
*RMS
FIX: ReqManager - in getRequest() possibility to accept None type
argument for any request
[v6r12p31]
*WMS
FIX: pilotCommands - import json module only in case it is needed
[v6r12p30]
*Core
FIX: InstallTools - 't' file is deployed for agents installation only
FIX: GOCDBClient - creates unique DowntimeID using the ENDPOINT
*Framework
FIX: SystemAdministratorHandler - use WebAppDIRAC extension, not just WebApp
*DMS:
FIX: FileCatalogComponents.Utilities - do not allow empty LFN names in
checkArgumentDict()
[v6r12p29]
*CS
CHANGE: CSCLI - use readline to store and resurrect command history
*WMS
FIX: JobWrapper - bug fixed in the failoverTransfer() call
CHANGE: dirac-wms-job-submit - added -f flag to store ids
*DMS
FIX: DataManager - make successful removeReplica if missing replica
in one catalog
*RMS
FIX: Operation, Request - limit the length of the error message
[v6r12p28]
*RMS
FIX: Request - do not optimize requests already in the DB
[v6r12p27]
*Core
CHANGE: InstallTools - install "t" script to gracefully stop agents
*DMS
FIX: FileCatalog - return GUID in DirectoryParameters
*Resource
CHANGE: DFC/LFC clients - added setReplicaProblematic()
[v6r12p26]
*DMS
BUGFIX: FileCatalog - getDirectoryMetadata was wrongly in ro_meta_methods list
*RMS
FIX: Operation - temporary fix in catalog names evaluation to smooth
LFC->DFC migration - not to forget to remove afterwards !
*WMS
CHANGE: JobWrapper - added MasterCatalogOnlyFlag configuration option
[v6r12p25]
*DMS
BUGFIX: PutAndRegister, RegitserFile, RegisterReplica, ReplicateAndRegister - do not
evaluate the catalog list if None
[v6r12p24]
*DMS:
FIX: DataManager - retry RSS call 5 times - to be reviewed
[v6r12p23]
*DMS
FIX: pass a catalog list to the DataManager methods
FIX: FileCatalog - bug fixed in the catalog list evaluation
[v6r12p22]
*DMS
FIX: RegisterFile, PutAndRegister - pass a list of catalogs to the DataManager instead of a comma separated string
FIX: FTSJob - log when a job is not found in FTS
CHANGE: dropped commands dirac-admin-allow(ban)-catalog
*Interfaces
CHANGE: Dirac, JobMonitoringHandler,dirac-wms-job-get-jdl - possibility to retrieve original JDL
*WMS
CHANGE: JobManifest - make MaxInputData a configurable option
[v6r12p21]
*RMS
BUGFIX: File,Operation,RequestDB - bug making that the request would always show
the current time for LastUpdate
*WMS
FIX: JobAgent - storing on disk retrieved job JDL as required by VMDIRAC
( to be reviewed )
[v6r12p20]
*DMS
FIX: DataManager - more informative log messages, checking return structure
FIX: FileCatalog - make exists() behave like LFC file catalog client by checking
the unicity of supplied GUID if any
FIX: StorageElementProxyHandler - do not remove the cache directory
*Framework
FIX: SystemAdministratorClient - increase the timeout to 300 for the software update
*RMS
FIX: Operation.py - set Operation Scheduled if one file is Scheduled
CHANGE: Request - group ReplicateAndRegister operations together for failover
requests: it allows to launch all FTS jobs at once
*Resources
FIX: LcgFileCatalogClient - fix longstanding problem in LFC when several files
were not available (only one was returned)
*TS
BUGFIX: TransformationCleaning,ValidateOutputDataAgent - interpret correctly
the result of getTransformationParameters() call
FIX: TaskManager - fix exception in RequestTaskAgent