Run lint benchmarks w/ all the lints.

[pq]

• enables all the lints for benchmarking
• (note that all.yaml is guaranteed to be complete by the 'all.yaml' test in integration_test.dart)

Follow-up from #670.

@devoncarew

[coveralls]

Coverage remained the same at 96.173% when pulling 947246b on timing_bot_tweak into d9ddbf1 on master.

[devoncarew]

lgtm!

For the -s output, we may want to swap the order of the Counts and Timings sections? Or not show counts, or de-emphasize them somehow? It's a longer section than the timings one, but I'm much more interested in the relative lint running times.

[devoncarew]

Adding timings in for posterity:

----------------------------------------------------
Timings                                           ms
----------------------------------------------------
public_member_api_docs                           210
invariant_booleans                               140
close_sinks                                       78
unnecessary_this                                  58
annotate_overrides                                57
package_api_docs                                  51
always_require_non_null_named_parameters          45
avoid_positional_boolean_parameters               44
avoid_returning_null                              44
directives_ordering                               43
cancel_subscriptions                              41
prefer_final_fields                               40
always_specify_types                              38
always_declare_return_types                       36
non_constant_identifier_names                     35
prefer_is_empty                                   35
unnecessary_overrides                             34
cascade_invocations                               33
prefer_constructors_over_static_methods           30
prefer_initializing_formals                       30
prefer_contains                                   30
prefer_is_not_empty                               29
always_put_control_body_on_new_line               28
prefer_conditional_assignment                     27
prefer_final_locals                               27
type_annotate_public_apis                         26
overridden_fields                                 25
use_setters_to_change_properties                  25
avoid_returning_this                              25
use_to_and_as_if_applicable                       25
unnecessary_lambdas                               24
prefer_interpolation_to_compose_strings           24
join_return_with_assignment                       24
unnecessary_getters_setters                       23
camel_case_types                                  23
unrelated_type_equality_checks                    23
omit_local_variable_types                         23
parameter_assignments                             23
sort_constructors_first                           23
no_duplicate_case_values                          23
recursive_getters                                 23
iterable_contains_unrelated_type                  22
avoid_slow_async_io                               22
avoid_setters_without_getters                     22
constant_identifier_names                         21
avoid_types_on_closure_parameters                 21
slash_for_doc_comments                            21
avoid_function_literals_in_foreach_calls          21
avoid_null_checks_in_equality_operators           21
prefer_expression_function_bodies                 20
avoid_classes_with_only_static_members            20
avoid_init_to_null                                20
use_string_buffers                                20
control_flow_in_finally                           20
hash_and_equals                                   19
unawaited_futures                                 19
prefer_adjacent_string_concatenation              19
sort_unnamed_constructors_first                   19
list_remove_unrelated_type                        19
package_prefixed_library_names                    19
throw_in_finally                                  19
library_names                                     19
prefer_collection_literals                        19
avoid_annotating_with_dynamic                     19
literal_only_boolean_expressions                  19
only_throw_errors                                 19
unnecessary_brace_in_string_interps               19
valid_regexps                                     18
avoid_return_types_on_setters                     18
avoid_as                                          18
prefer_function_declarations_over_variables       18
implementation_imports                            18
prefer_const_constructors                         18
prefer_foreach                                    18
library_prefixes                                  18
type_init_formals                                 17
empty_statements                                  17
await_only_futures                                17
avoid_catches_without_on_clauses                  17
empty_constructor_bodies                          17
use_rethrow_when_possible                         17
super_goes_last                                   17
test_types_in_equals                              17
one_member_abstracts                              17
no_adjacent_strings_in_list                       17
comment_references                                17
unnecessary_null_aware_assignments                17
avoid_empty_else                                  17
unnecessary_null_in_if_null_operators             17
avoid_catching_errors                             17
empty_catches                                     16
----------------------------------------------------
Total                                           2578
----------------------------------------------------

[alexeieleusis]

Invariant booleans was already ~80s (oh! in my workstation), and I just ran the script locally with 138ms. You think there is a regression?

[pq]

@devoncarew we could collapse these lists into one and embed count info

--------------------------------------------------------
Lint                          count                   ms
--------------------------------------------------------
public_member_api_docs         1200                  210

Or add more specific flags (-t for timings and , ummm, something for counts, -c is taken... maybe -n for number?).

[pq]

@alexeieleusis : maybe? It's consistently around ~80ms for me (w/ analyzer-0.30.0). What version of the analyzer are you using?

[pq]

Timings from the bot are a little different (and there invariant_booleans is a bit high).

But notice parameter_assignments!

----------------------------------------------------
Timings                                           ms
----------------------------------------------------
parameter_assignments                            186
public_member_api_docs                           179
invariant_booleans                               134
unnecessary_this                                  50
close_sinks                                       47
annotate_overrides                                45
directives_ordering                               42
package_api_docs                                  41
avoid_positional_boolean_parameters               39
avoid_returning_null                              37
always_require_non_null_named_parameters          37
cancel_subscriptions                              36
non_constant_identifier_names                     34
always_specify_types                              33
always_declare_return_types                       31
unnecessary_overrides                             31
cascade_invocations                               29
prefer_is_empty                                   27
prefer_contains                                   26
prefer_is_not_empty                               25
always_put_control_body_on_new_line               25
prefer_final_fields                               23
prefer_initializing_formals                       23
use_setters_to_change_properties                  22
prefer_final_locals                               22
type_annotate_public_apis                         22
omit_local_variable_types                         21
use_to_and_as_if_applicable                       21
overridden_fields                                 21
unnecessary_lambdas                               20
prefer_interpolation_to_compose_strings           20
recursive_getters                                 20
camel_case_types                                  19
constant_identifier_names                         19
join_return_with_assignment                       19
avoid_returning_this                              19
sort_constructors_first                           19
avoid_slow_async_io                               19
unrelated_type_equality_checks                    19
unnecessary_getters_setters                       19
no_duplicate_case_values                          19
literal_only_boolean_expressions                  18
avoid_types_on_closure_parameters                 18
avoid_setters_without_getters                     18
iterable_contains_unrelated_type                  18
control_flow_in_finally                           18
unawaited_futures                                 17
use_string_buffers                                17
avoid_classes_with_only_static_members            17
package_prefixed_library_names                    17
avoid_init_to_null                                17
avoid_function_literals_in_foreach_calls          17
avoid_annotating_with_dynamic                     17
avoid_null_checks_in_equality_operators           17
implementation_imports                            17
prefer_conditional_assignment                     17
slash_for_doc_comments                            17
only_throw_errors                                 16
sort_unnamed_constructors_first                   16
prefer_function_declarations_over_variables       16
prefer_collection_literals                        16
list_remove_unrelated_type                        16
prefer_expression_function_bodies                 16
prefer_foreach                                    15
avoid_empty_else                                  15
unnecessary_null_in_if_null_operators             15
unnecessary_brace_in_string_interps               15
unnecessary_null_aware_assignments                15
valid_regexps                                     15
prefer_adjacent_string_concatenation              15
hash_and_equals                                   15
prefer_const_constructors                         15
throw_in_finally                                  15
library_names                                     15
await_only_futures                                15
avoid_as                                          15
avoid_return_types_on_setters                     15
no_adjacent_strings_in_list                       15
use_rethrow_when_possible                         15
prefer_constructors_over_static_methods           15
library_prefixes                                  15
type_init_formals                                 14
avoid_catching_errors                             14
one_member_abstracts                              14
avoid_catches_without_on_clauses                  14
empty_constructor_bodies                          14
super_goes_last                                   14
test_types_in_equals                              14
empty_catches                                     14
comment_references                                14
empty_statements                                  14
----------------------------------------------------
Total                                           2323
----------------------------------------------------

[pq]

I'm seeing some variation in runs. For example:

unnecessary_this                                199
package_api_docs                                168
public_member_api_docs                          117
invariant_booleans                              104
close_sinks                                      34

...

We should consider multiple runs...

[alexeieleusis]

I did pub get and upgrade and ran
dart bin/linter.dart -s -q -c example/all.yaml . like in the script, though I am seeing the variations you mention too, now it was 107ms.