Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unused and deprecated queue_options #8471

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/reference/configuration/keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1832,9 +1832,9 @@ There are configuration options for the various queue systems, described in deta
in :ref:`queue-system-chapter`. In brief, the queue systems have the following options:

* :ref:`LOCAL <local-queue>` — no queue options.
* :ref:`LSF <lsf-systems>` — ``LSF_SERVER``, ``LSF_QUEUE``, ``LSF_RESOURCE``,
* :ref:`LSF <lsf-systems>` — ``LSF_QUEUE``, ``LSF_RESOURCE``,
``BSUB_CMD``, ``BJOBS_CMD``, ``BKILL_CMD``,
``BHIST_CMD``, ``BJOBS_TIMEOUT``, ``SUBMIT_SLEEP``, ``PROJECT_CODE``, ``EXCLUDE_HOST``,
``BHIST_CMD``, ``SUBMIT_SLEEP``, ``PROJECT_CODE``, ``EXCLUDE_HOST``,
``MAX_RUNNING``
* :ref:`TORQUE <pbs-systems>` — ``QSUB_CMD``, ``QSTAT_CMD``, ``QDEL_CMD``,
``QSTAT_OPTIONS``, ``QUEUE``, ``CLUSTER_LABEL``, ``MAX_RUNNING``, ``NUM_NODES``,
Expand Down
20 changes: 0 additions & 20 deletions docs/reference/configuration/queue.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,14 +140,6 @@ The following is a list of available LSF configuration options:

QUEUE_OPTION LSF BHIST_CMD command

.. _bjobs_timeout:
.. topic:: BJOBS_TIMEOUT

Determines how long-lived the job cache is. Default: ``0`` (no cache).
To change it to 60 s::

QUEUE_OPTION LSF BJOBS_TIMEOUT 60

.. _submit_sleep:
.. topic:: SUBMIT_SLEEP

Expand All @@ -156,18 +148,6 @@ The following is a list of available LSF configuration options:

QUEUE_OPTION LSF SUBMIT_SLEEP 1

.. _lsf_server:
.. topic:: LSF_SERVER

This options tells ERT which server should be used when submitting.
So when your configuration file has the setting::

QUEUE_OPTION LSF LSF_SERVER be-grid01

ERT will use ``ssh`` to submit your jobs using shell commands on the server
``be-grid01``. For this to work you must have passwordless ``ssh`` to the
server.

.. _lsf_queue:
.. topic:: LSF_QUEUE

Expand Down
1 change: 0 additions & 1 deletion docs/reference/configuration/site_wide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ properties of the cluster. This could be an example site config file: ::
-- Set some properties of the local LSF system
QUEUE_OPTION LSF LSF_QUEUE hmqueue
QUEUE_OPTION LSF MAX_RUNNING 100
QUEUE_OPTION LSF LSF_SERVER lsf-front01.company.com
QUEUE_OPTION LSF BSUB_CMD /path/lsf/bin/bsub
QUEUE_OPTION LSF BJOBS_CMD /path/lsf/bin/bjobs
QUEUE_OPTION LSF BKILL_CMD /path/lsf/bin/bkill
Expand Down
23 changes: 0 additions & 23 deletions src/ert/config/parsing/config_schema_deprecations.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
]
RSH_KEYWORDS = ["RSH_HOST", "RSH_COMMAND", "MAX_RUNNING_RSH"]
USE_QUEUE_OPTION = [
"LSF_SERVER",
"LSF_QUEUE",
"MAX_RUNNING_LSF",
"MAX_RUNNING_LOCAL",
Expand Down Expand Up @@ -178,22 +177,6 @@
"for the Ensemble Smoother update algorithm. "
"Please use ENKF_ALPHA and STD_CUTOFF keywords instead.",
),
DeprecationInfo(
keyword="QUEUE_OPTION",
message="LSF_LOGIN_SHELL as QUEUE_OPTION to the LSF system will be removed in "
"the future, and it is not recommended to use this QUEUE_OPTION. "
"It has been used in the past to force the bsub command to use a "
"specific shell. The current ERT default is to use local shell.",
check=lambda line: "LSF_LOGIN_SHELL" in line,
),
DeprecationInfo(
keyword="QUEUE_OPTION",
message="LSF_RSH_CMD as QUEUE_OPTION to the LSF system will be removed in "
"the future, and it is not recommended to use this QUEUE_OPTION. "
"It has been used in the past to set the remote shell command. "
"The ERT default is to use /usr/bin/ssh.",
check=lambda line: "LSF_RSH_CMD" in line,
),
DeprecationInfo(
keyword="QUEUE_OPTION",
message="QUEUE_QUERY_TIMEOUT as QUEUE_OPTION is ignored. "
Expand All @@ -206,12 +189,6 @@
"Please remove the line.",
check=lambda line: "QSTAT_OPTIONS" in line,
),
DeprecationInfo(
keyword="QUEUE_OPTION",
message="LSF_SERVER as QUEUE_OPTION is not needed and will be removed in "
"the future. Please remove the configuration line.",
check=lambda line: "LSF_SERVER" in line,
),
DeprecationInfo(
keyword="QUEUE_OPTION",
message="NUM_CPUS_PER_NODE as QUEUE_OPTION to Torque is deprecated and will removed in "
Expand Down
13 changes: 1 addition & 12 deletions src/ert/config/queue_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,15 @@
LSF_DRIVER_OPTIONS = [
"BHIST_CMD",
"BJOBS_CMD",
"BJOBS_TIMEOUT",
"BKILL_CMD",
"BSUB_CMD",
"DEBUG_OUTPUT",
"EXCLUDE_HOST",
"LSF_LOGIN_SHELL",
"LSF_QUEUE",
"LSF_RESOURCE",
"LSF_RSH_CMD",
"LSF_SERVER",
"PROJECT_CODE",
]
OPENPBS_DRIVER_OPTIONS: List[str] = [
"CLUSTER_LABEL",
"DEBUG_OUTPUT",
"JOB_PREFIX",
"KEEP_QSUB_OUTPUT",
"MEMORY_PER_JOB",
Expand Down Expand Up @@ -73,10 +67,7 @@
queue_string_options: Mapping[str, List[str]] = {
"LSF": [
"LSF_RESOURCE",
"LSF_SERVER",
"LSF_QUEUE",
"LSF_LOGIN_SHELL",
"LSF_RSH_CMD",
"BSUB_CMD",
"BJOBS_CMD",
"BKILL_CMD",
Expand All @@ -102,15 +93,13 @@
"QUEUE",
"CLUSTER_LABEL",
"JOB_PREFIX",
"DEBUG_OUTPUT",
"PROJECT_CODE",
],
"LOCAL": ["PROJECT_CODE"],
"GENERIC": [],
}
queue_positive_int_options: Mapping[str, List[str]] = {
"LSF": [
"BJOBS_TIMEOUT",
"MAX_RUNNING",
],
"SLURM": [
Expand All @@ -132,7 +121,7 @@
"GENERIC": ["SUBMIT_SLEEP"],
}
queue_bool_options: Mapping[str, List[str]] = {
"LSF": ["DEBUG_OUTPUT"],
"LSF": [],
"SLURM": [],
"TORQUE": ["KEEP_QSUB_OUTPUT"],
"LOCAL": [],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -230,15 +230,11 @@ def test_that_suggester_gives_schedule_prediciton_migration(tmp_path):

def test_that_suggester_gives_job_prefix_migration(tmp_path):
(tmp_path / "config.ert").write_text(
"NUM_REALIZATIONS 1\nQUEUE_OPTION TORQUE JOB_PREFIX foo\nQUEUE_OPTION LSF LSF_RSH_CMD foo\n"
"NUM_REALIZATIONS 1\nQUEUE_OPTION TORQUE JOB_PREFIX foo\n"
)
suggestions = make_suggestion_list(str(tmp_path / "config.ert"))

assert any(
"JOB_PREFIX as QUEUE_OPTION to the TORQUE system is deprecated" in str(s)
for s in suggestions
)
assert any(
"LSF_RSH_CMD as QUEUE_OPTION to the LSF system will be removed" in str(s)
for s in suggestions
)
1 change: 0 additions & 1 deletion tests/unit_tests/config/test_ert_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@
"MAX_SUBMIT": 13,
"QUEUE_SYSTEM": "LSF",
"LSF_QUEUE": "mr",
"LSF_SERVER": "simulacrum",
"LSF_RESOURCE": "select[x86_64Linux] same[type:model]",
"MAX_RUNNING": "100",
"DATA_FILE": "eclipse/model/SNAKE_OIL.DATA",
Expand Down
16 changes: 0 additions & 16 deletions tests/unit_tests/config/test_parser_error_collection.py
Original file line number Diff line number Diff line change
Expand Up @@ -1024,22 +1024,6 @@ def test_that_executable_directory_errors(dirname):
["RSH_HOST", "RSH_COMMAND", "MAX_RUNNING_RSH"]
)
],
*[
ExpectedErrorInfo(
line=20 + i,
column=1,
end_column=1 + len(kw),
match=f"The {kw} keyword has been removed",
)
for i, kw in enumerate(
[
"LSF_SERVER",
"LSF_QUEUE",
"MAX_RUNNING_LSF",
"MAX_RUNNING_LOCAL",
]
)
],
*[
ExpectedErrorInfo(
line=25 + i,
Expand Down
1 change: 0 additions & 1 deletion tests/unit_tests/config/test_queue_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,6 @@ def test_initializing_empty_config_queue_options_resets_to_default_value(
@pytest.mark.parametrize(
"queue_system, queue_option, queue_value, err_msg",
[
("LSF", "BJOBS_TIMEOUT", "-3", "is not a valid positive integer"),
("SLURM", "SQUEUE_TIMEOUT", "5a", "is not a valid integer or float"),
("TORQUE", "NUM_NODES", "3.5", "is not a valid positive integer"),
],
Expand Down