There are still two different variables. My idea was to have one, so a user can specify TESTS=cpu or TESTS=ClassName or TESTS=ClassName.methodName. No filters - just a list of tests to run. If it helps, make test specifier case-sensitive, so it's easy to disambiguate between directory names and class names.

I think you overcomplicated it a bit. There is no division into directories, names and filters. Furthermore, there are no filters. Everything passed as an argument is just a test name. The rules to interpret a test name can be the following (you may suggest your version, but as usual - the simpler the better).

1. If it only consists of lowercase letters - it's a directory name (alternatively, you may check for directory existence, but this is not necessary). A directory name is unambiguously translated to the class name: alloc -> test.alloc.AllocTests.
2. Otherwise, if it ends with Tests, it's a class name. It may be a fully qualified class name or short class name. In the latter case, the package is prepended using the same basic rules: AllocTests -> test.alloc.AllocTests.
3. Otherwise, if it contains a dot, it's a class name with a method name.

That directory-name division was already in the code, I made it explicit by splitting args into testNames and testDirectories. testFilters is a new addition.

I am hoping to implement a contains matching. Current PR does not implement exact matching.

Please, format the code. There is always a space between a keyword and (.

To avoid ambiguity, I would not convert names to lowercase and would check for exact match rather than contains. Otherwise, AllocTests.alloc will match both alloc and allocTotal; liveness will also select livenessJfrHasStacks and so on. This is probably not what a user expects.

That is what I expected :) We can have both contains and exact match modes. Exact match to be implemented.

Having multiple top-level classes in one source file is discouraged (in favor of nested classes or one class per file).
That said, in this particular case, I think we don't all these classes at all. The code could be shorter and simpler with lambdas. Alternatively, what I would personally prefer, is using a standard j.u.regex.Pattern, which can be easily constructed from a glob expression.

This condition is never met. Class names that have a period (that is, a package) always start with a lower case character.

This condition is met e.g. for CstackTests.vmStructs.

Yes, but this does not work: the subsequent Class.forName will fail anyway, since the actual class name is test.cstack.CstackTests.

Thanks, simplified and refactored.

As I mentioned in a previous review, division by directories, classes and filters is confusing, especially when they are mixed together.
E.g., in the current implementation, TESTS='allo CstackTests.vmStructs' selects one test, whereas TESTS='alloc CstackTests.vmStructs' selects nothing, which is counterintuitive.

Users should not rack their brains keeping this division in mind; they just provide a list of test specifiers, and the runner's job is to make a best guess how to interpret this specifier. Specifiers are independent. So, alloc CstackTests.vmStructs should mean "run all tests in the alloc directory and CstackTests.vmStructs".

This conditional branch makes it even more confusing.
Maybe we can always get a list of all available tests and then let specifiers (or filters, whatever you call it) select matched ones?

Since Runner already scans test directories, Makefile no longer needs to do so. Perhaps TESTS can be initialized with *

It is initialized as empty now.

It's now a single-method utility class; I'd just move the method to RunnerDeclaration.

RunnerDeclaration already gets a list of all directories elsewhere. Let's use list (or better, set) here.

We need to check directory existence to distinguish between testFilters and includeDirs.

You get complete list of test directories when Runner starts, no need to do filesystem checks again. includeDirs will go away as per my other comment.
So, the algorithm I imagine works as follows:

1. Obtain a set of test directories by listing test/test/.
2. Parse test specifiers passed in command line arguments, converting each one to Pattern.
3. Similarly, parse skip definition to get a list of Patterns to be skipped.
4. Use reflection to find all runnable tests from directories obtained at (1).
5. Go through a sorted list of runnable tests and execute them if the name matches any pattern from (2) and does not match any pattern from (3).

Does this make sense?

I think all these 4 structures can be replaced with a single list of patterns as follows:

1. glob is a test dir => GlobTests.*
2. glob looks like a class name => Glob.*
3. glob looks like a method name => *.glob
4. otherwise => glob

Not readable. Streams are usually formatted as follows:

return globs.stream()
        .filter(g -> g.equals(g.toLowerCase()))
        .flatMap(g -> getRunnableTests(g).stream())
        .sorted(Comparator.comparing(RunnableTest::testName))
        .collect(Collectors.toList());

Given that List<String> globs is always a list of directories here, filter can be removed.

Also, getRunnableTests method has too many overloads, I'd inline this one in getRunnableTests() below.

continue; ?

But I'd rewrite all this loop without continue using a plain one-level if-else chain - this will be easier to follow, IMO:

if (g.contains(".") || g.contains("*")) {
   ...
} else if (...) {
   ...
} else if ...

Never reached because of the above condition?

In order to avoid confusion with a method name consisting of all lowercase letters, I'd check against allDirs instead.

nit: returning List here and calling stream() in the context of the stream below will look a bit nicer.

Unused import

Dead code

!anyMatch -> noneMatch

What's the purpose of these case manipulations?

It was a misunderstanding, correcting to result.add(g + ".*");.

Why do we convert it to lowercase? AFAICS, we use case sensitive patterns.

It was a misunderstanding, correcting to result.add("*." + g);.