-
-
Notifications
You must be signed in to change notification settings - Fork 3k
Description
Issue
Some of the builtins fixtures are sparse to the point that they're missing even the declaration of dict, list, tuple etc. types. The austerity is justified by improved tests performance, and it's understood that specific tests can specify a broader fixture without slowing down other tests.
However, some core features rely on the presence of those types, e.g.
- functions'
**kwargs - modules'
__annotations__ namedtuple's attributes like_field_types
I've observed 3 different strategies used to handle the types absence:
- crash (example:
**kwargshandling) - give up (example:
__annotations__not being added) - fallback to
object(example:NamedTupleattributes)
I think we should recommend and, ideally, consolidate on one strategy.
Motivation
Case in point: Recently while implementing #14526, I had to "play" a whack-a-mole with unrelated tests breaking. I've used **kw which required me to add dict to a number of builtins fixtures, which in turn caused __annotations__ to materialize where they didn't before, and failed some other tests.
I think ideally we shouldn't have this whack-a-mole, and tests should be less brittle.
Recommendation
Which strategy to adopt? While it's good practice to structure code to be more testable, I think it's normally discouraged to have code paths that are only due to testing. Since those builtins should universally exist in "production", I think crash should be our strategy.
Since code typically relies only on the presence of those classes (not their methods), we can put in place a rule (validated by tests) that any builtins fixture must define a baseline of types, even if empty. This shouldn't affect performance greatly, and provide a comfortable baseline.