Tagging certain tests as "lint only"

[MikePopoloski]

Some classes of tests, specifically thinking of hdlconverter and utd-sv, don't even intend to be valid SystemVerilog designs. For example, they instantiate modules without providing definitions for those modules. I think it's worth tagging these kinds of tests separately so that tools can be configured differently to be more permissive. For example, I recently added a --lint-only flag to slang that would make most of these tests pass. I wouldn't want to enable this flag everywhere as it reduces the strength of checking, so hopefully we can mark the tests that require it. I assume other tools would like to know this as well.

[tgorochowik]

There are actually multiple test types:

• preprocessing
• parsing
• simulation

and most of the tests run in the parsing mode. It is kind of misleading, maybe we should add a separate mode, like elaboration and make it the new default? The current parsing mode could then be used for those more permissive runs (like your --lint-only flag).

CC @hzeller

[MikePopoloski]

Yeah, I think that makes sense. Nearly all of the existing "simulation" tests don't actually require simulation in any way, as evidenced by the fact that slang passes nearly all of those tests but can't actually simulate anything. slang also has a --parse-only flag but that doesn't do type checking or name resolution, so it would dilute the quality of the testing if it applied to the vast majority of tests that are otherwise well formed and able to be elaborated.

[MikePopoloski]

I ended up taking care of this for the slang runner by just checking specific tags, so I'm going to close this for now. I wouldn't object to a separate mode but I'm not going to take on doing that myself.