Open XDMoD 8.0.0

• Features
  • General
    • Added a beta version of the Cloud realm to provide metrics relevant to cloud computing resources.
    • Added a beta version of the Storage realm to provide metrics relevant to storage systems installed at a center.
    • Federated XDMoD has been released for production. Federated XDMoD allows individual, locally managed, XDMoD instances to report all or a subset of their accounting data to a central Hub which provides a global view of the federation.
    • All XDMoD user profiles are now associated with an organization. Previously, this was only required for Campus Champions.
    • Added support for automatically detecting / assigning a new SSO User's organization.
    • Added support for automatically detecting if a user's organization has changed and updating their accounts accordingly. This may include, but is not limited to, the removal of elevated privileges.
    • Hardened the login and password reset process as a result of a security audit by University of Cambridge.
    • Improved support for resource manager job arrays.
    • Many improvements to the documentation.
  • ETL
    • Reorganized several ETL pipelines.
    • Improved data sanitization for tighter checks present in MySQL 5.7.
    • Refactored Jobs realm ingestion to utilize ETLv2.
    • Standardize action names to follow the format module.pipeline.action. For example, xdmod.acls.manage-tables.
    • Added character set and coalition to table definitions.
    • Added support for foreign key constraints.
    • Added support for the definition of ETL variables on the command line using -d variable=value.
    • Add ingestion of node hostname data from SGE logs.
    • Various ETL performance improvements.
• Bug Fixes
  • User Interface
    • Deep linking when logged in using SSO has been restored.
    • Update the logrotate configuration to use the su and create options.
  • ETL
    • Add primary keys to select ETL source queries.
    • When modifying an existing table, preserve the order of the columns in the definition file.
    • Ensure that file handles are flushed before inserting the final chunk of data.
  • Misc
    • Fixed several exceptions that were outside of a namespace.
    • Fixed an issue where ACLs were not properly created on upgrade.
    • Several minor bugfixes

Open XDMoD 7.5.1

• Bug Fixes
  • Properly implement data access for non-feature ACLs (e.g., ACLs that provide access to data
    but not a feature such as a tab) that are not part of the hierarchy. For example, the Value
    Analytics ACL.

This RPM was missing the user manual, it has been attached a separate file that you can download and extract to /usr/share/xdmod/html/

Open XDMoD 7.5.0

• Features
  • General
    • Added support for Globus as a federated authentication provider
    • Improvements to the user login dialog and matching of users to institutions
    • Added the ability to use fine-grained ACLs for controlling access to features such as tabs and realms as well as data
    • Code pertaining to creating/sending emails has been moved to a central location.
    • Improved online documentation for installations and upgrades
    • Improve performance of Utilization statistic
    • Do not embed JavaScript in chart objects returned by the back end, instead
      include this code directly in the user interface code
    • General improvements to the performance of the REST stack
    • Updated Google Captcha to v2 (v1 is now discontinued)
    • Added numerous component and unit tests
    • Removed unused code paths
    • Update greenlion/PHP-SQL-Parser to newer release that does not autoload itself
  • Internal Admin Dashboard
    • Updated user management functionality in the administrative dashboard to support fine-grained ACLs for individual users
    • Many stability improvements to the administrator dashboard, especially for managing users
  • Storage Realm
    • Added alpha version of the Storage realm to track resource storage utilization
  • User Interface
    • Legend item edit box is now displayed next to the legend item that is being edited and is larger to accommodate longer strings
    • Added a "Select All" button to the filter dialog
    • Added PDF export capability, which greatly improves the quality of images included in LATEX documents
  • ETL
    • When ingesting Slurm data, return duplicate jobs such as those that were resubmitted due to node fail conditions
    • Improve performance of filter list generation
    • Added an ETL pipeline for ingesting log files generated by Eucalyptus clouds
    • Added support for PBS/Torque logs where the host of a job is not included in the job's ID string
    • Support references into complex source records such as JSON objects
  • Application Kernels
    • Several minor bug fixes and UI improvements
• Bug Fixes
  • General
    • Improved error reporting
    • Fixed several issues where JSON was not properly encoded
    • For end dates that fall on the current date or in the future, do not automatically adjust the date to the end of the aggregation period that it falls into.
    • Fix the Show Guidelines button in the Usage tab
    • Fixed an issue with data filtering when an unprivileged user tries to view timeseries data for a restricted realm or statistic
  • Report Generator
    • Make timeframes editable for individual charts
  • User Interface
    • Fix the "TypeError: element is undefined" error when plotting Pie chart in metric explorer
  • ETL
    • Ignore duplicate hosts found in LSF accounting log files
    • Don't verify data endpoints associated with disbaled actions
    • Always regenerate source data queries prior to execution to ensure that any modified ETL variables are properly applied
    • Update host list parser to ensure that empty host names are not returned
    • Remove PHP memory limit when running ETL pipelines

v7.5.0-rc.2: Disallow Updating of Federated User's Passwords (#444)

* Disallow editing passwords for federated users.

v7.5.0-rc1

Merge pull request #440 from smgallo/etl/flush-before-load

Flush data file before calling LOAD DATA INFILE

Open XDMoD 7.0.1

• Bug Fixes
  • General
    • Fixed compatibility with PHP 5.3.3 (#269)

Open XDMoD 7.0.0

• Features
  • General
    • Enhanced authorization framework (#97, #146, #206)
    • Improved login prompt to automatically appear when an unauthenticated user attempts to access any private tab (#110)
    • Improved design of Metric Explorer Load Chart menu (#144)
    • Re-enabled aggregate mode for "Wall Hours: Per Job" metric (#186)
    • Added quarterly report template for center directors (#199)
    • Improved support for third-party PHP libraries used by modules (#205)
  • ETLv2
    • Added support for RFC-6901 JSON References (#100, #166)
    • Added file directory data endpoint (#154)
    • Improved support for order_id columns in Open XDMoD database (#201)
    • Refactored various components to support new features (#138, #145, #151, #167, #174, #180, #196)
  • Job Viewer
    • Added Gantt chart view of job peers (#153, #164)
    • Added ability to link directly to jobs (#156)
    • Improved user feedback while job data is loading (#168)
• Bug Fixes
  • General
    • Fixed various compatibility issues with PHP 7 (#101, #183)
    • Fixed handling of 4-byte UTF-8 characters during XRAS ingestion (#122)
    • Improved handling of invalid start and end dates received by API (#160)
    • Fixed validation of length of names in contact forms (#175)
    • Improved handling of jobs with "0" start or end time (#197)
    • Fixed case where About tab would display a blank page when loaded using Chrome (#232)
  • Job Viewer
    • Fixed directions not always appearing when all jobs are closed (#155)
    • Fixed memory leak (#149)
    • Fixed duplicate search nodes being created when opening jobs from Metric Explorer charts with "#" in the title (#236)
  • Metric Explorer
    • Fixed handling of "%" characters in Metric Explorer options when using Firefox (#114)
    • Fixed XSS vulnerability involving chart names (#239)
  • Usage
    • Fixed error bars option being enabled when error bars are not available (#188)
    • Fixed metrics appearing to be available to users that do not have access (#189)
• Miscellaneous
  • Moved Node.js ETL framework to Open XDMoD repository (#106)
  • Fixed build script running out of memory allocated by PHP (#118)
  • Performed work in anticipation of federated instances (#148)
  • Improved development workflow (#124, #157, #195)
  • Improved quality assurance (#107, #116, #134, #143, #150, #163, #165, #169, #173, #184, #185, #187, #190, #193, #194, #198, #212, #235)
  • Cleaned up old and/or unused code (#104, #105, #109, #112, #117, #128, #158, #159, #182, #191, #213)
  • Improved documentation (#161, #202, #203, #229, #247)

Open XDMoD 6.6.0

• Features
  • General
    • Added ability to group realms together under categories (#60)
      • Categories have taken the place of realms in the user interface
    • Improved support for browser client assets provided by modules (#82, #113)
    • Improved upgrade messaging (#86)
  • ETLv2
    • Added per-pipeline ETL locks (#10)
    • Enhanced multi-host aggregation (#13)
    • Added read support for Oracle endpoints (#34)
    • Improved support for running pipelines over all dates (#77)
    • Improved transformation support (#80)
    • Added a tool for comparing SQL tables (#78)
    • Added other improvements (#43, #45, #84, #90, #92)
• Bug Fixes
  • General
    • Fixed warning that could appear when using federated authentication (#19)
    • Fixed unnecessary rounding in processor count statistics (#25)
    • Fixed errors when attempting to preview an unsaved report (#26)
    • Fixed aggregators excluding data on certain time boundaries (#47)
    • Fixed handling of backslashes when using certain MySQL features (#52, #53)
    • Fixed non-aggregate metrics being allowed in aggregate datasets (#74)
    • Fixed REST API throwing non-standard errors (#87)
    • Fixed display error with password reset form that occurred when using federated authentication (#108)
    • Fixed "Show chart title" option in Usage tab not working as expected (#139)
  • Metric Explorer
    • Fixed Y-axis context menu disappearing after changing between linear and log scales (#12)
    • Fixed typing certain characters causing chart options menu to close when using Firefox (#119)
    • Fixed chart errors not displaying if help graphic is active (#121)
    • Fixed chart last modified time not updating (#140)
• Miscellaneous
  • Performed work in anticipation of allocations/accounts data in Open XDMoD (#11, #50)
  • Performed work in anticipation of cloud data in Open XDMoD (#68, #75)
  • Cleaned up old and/or unused code (#54, #55, #56, #57, #73, #88)
  • Improved logging (#29, #40, #46, #50, #66, #98)
  • Improved quality assurance (#21, #27, #28, #35, #41, #48, #58, #67, #76, #79, #93, #94)
  • Improved documentation (#32, #37, #44, #71, #103, #115, #123, #130, #132, #135)

Open XDMoD 6.5.0

• Features
  • General
    • Modified the Summary tab to reload automatically after its charts are
      modified in Metric Explorer.
    • Modified REST stack to support multiple configuration files, allowing
      modules to supply their own REST resources.
    • Increased maximum size of node lists in database, allowing jobs
      running across more nodes to be more accurately tracked.
    • Improved efficiency of job host list parser and ingestor.
    • Added ability to disable Basic Auth in REST API.
    • Improved email validation.
  • ETLv2
    • Added new PHP-based ETL system to Open XDMoD.
    • Vastly improved logging and debugging.
    • Supports individual ETL actions as well as ordered sets of actions
      called pipelines.
    • Optimized for performance when importing large numbers of records.
    • Predefined actions support flexible ingestion of data from multiple
      sources.
    • ETL is configured via JSON files and requires far less code
      modification when customizing to more easily support flexibility at
      customer installations.
• Bug Fixes
  • General
    • Fixed stacked area charts not handling empty data points correctly.
    • Stopped inaccurate warning about Adobe Flash content in browsers that
      block Flash. (XDMoD does not make use of Flash.)
    • Fixed various typos and grammatical errors.
    • Fixed incorrectly-oriented subpanel titles in Internet Explorer 10.
    • Downgraded PHP packages that required PHP 5.3.9 instead of
      Open XDMoD's current minimum PHP version, 5.3.3.
    • Fixed inconsistency between labels used for users and PIs on a job.
    • Fixed automatic aggregation unit selection for charts not working
      correctly on servers running PHP 5.3.3.
    • Stopped package builds from being logged to an Open XDMoD database.
    • Fixed documentation files from modules overwriting core files when
      installing using tarballs.
    • Fixed UGE shredder.
    • Fixed Slurm job array index parsing.
• Refactors and Miscellaneous
  • Added documentation for using LDAP for federated authentication.
  • Spun the App Kernels, SUPReMM, and XSEDE modules out into separate
    code repositories.
  • Began transition to a more flexible, more efficient data warehouse that
    will support innovative HPC resources including cloud computing resources.
  • Consolidated third-party JavaScript libraries into one library directory.
  • Improved development setup process for external contributors.
  • Migrated website from a single SourceForge site to multiple GitHub sites.
  • Cleaned up and reorganized numerous other assets.
  • Added reqgres and reqtres to the list of Slurm fields that are
    shredded and stored in the Slurm job table.

Open XDMoD 6.0.0

• Important Notes
  • Updated Highcharts from v3.0.9 to v4.2.5.
    • Commercial users (as defined by the Highcharts license terms) will
      need to acquire a new Highcharts license if their current license does
      not cover the new version.
• Features
  • Added demo user type.
  • Added roadmap link to feature request dialog.
  • Shredders
    • Fixed calculation of missing end times from the start time and wall
      time.
    • LSF
      • Now storing the node list, exit code and exit status for SUPReMM
        support.
    • Slurm
      • Now allowing null eligible times.
      • Failed parsing of datetimes are no longer a fatal error.
    • SGE
      • Now supporting older versions of SGE that contain only 43 fields
        in their accounting logs.
• Bug Fixes
  • General
    • Fixed some charts not rendering if more than 1000 points were plotted.
    • Improved consistency of chart aesthetics across tabs.
    • Fixed report generator not cleaning up old report files.
  • Metric Explorer
    • Fixed Add Filter list only including filter types applicable to the
      realm of the first data series on a chart.
  • Shredders
    • PBS
      • Fixed parsing of newer exec_host formats (e.g. host/0-3,
        host/0*8).
  • xdmod-import
    • Fixed importing of hierarchy and group-to-hierarchy files.
• Refactors and Miscellaneous
  • Refactored Open XDMoD to be the code base upon which XDMoD is built
    instead of the other way around.
  • Updated jQuery from v1.9.1 to v1.12.4.
  • Updated Node.js Mongo driver from v1.4 to v2.1.
  • Improved tab loader to allow tabs to be defined in configuration files
    instead of being hard-coded into the loader.
  • Merged public and private versions of the main page into a single page.
  • Added options to more quickly build Open XDMoD for development purposes.
  • Reduced duplicate code in asset setup script.