[mypyc] Use defined __new__ method in tp_new and constructor #19739
Conversation
Co-authored-by: Brian Schubert <brianm.schubert@gmail.com>
| val: int | ||
|
|
||
| def __new__(cls, val: int): | ||
| obj = super().__new__(cls) |
There was a problem hiding this comment.
Test what happens if you call e.g. super().__new__(str). The specialization should only trigger if the first argument is the cls argument of the surrounding method, I think.
There was a problem hiding this comment.
Right, good point. Restricted the transformation to this case only and added a test.
|
|
||
| t0 = Test0() | ||
| assert t0.val == 0 | ||
| t0 = Test0.__new__(Test0, 1) |
There was a problem hiding this comment.
Do you have a test for calling ClsName.__new__(...) in compiled code? Also add an irbuild test for this.
There was a problem hiding this comment.
Yes, there's some calls like this in the run test:
def test_arg_mismatch() -> None:
t0 = Test0()
assert t0.val == 0
t0 = Test0.__new__(1)
assert t0.val == 1
with assertRaises(TypeError, "__new__() missing required argument 'val'"):
t1 = Test1()
t1 = Test1.__new__(Test1, 2)
assert t1.val == 2
with assertRaises(TypeError, "__new__() takes at most 0 arguments"):
t2 = Test2(42)
t2 = Test2.__new__(Test2)
with assertRaises(AttributeError, "attribute 'val' of 'Test2' undefined"):
print(t2.val)
i've added an irbuild test too.
mypyc/irbuild/prepare.py
Outdated
|
|
||
| new_node: SymbolNode | None = None | ||
| new_typeinfo = cdef.info.get("__new__") | ||
| if new_typeinfo and new_typeinfo.fullname and not new_typeinfo.fullname.startswith("builtins"): |
There was a problem hiding this comment.
Add comment explaining what is happening here.
This name seems a bit confusing. new_typeinfo refers to a symbol table node? If that's the case, maybe rename it as new_symbol or similar.
The prefix should be "builtins." (with a dot at the end), since there can be a user-defined module such as builtinsx.
#19739 introduced support for compiling `__new__` methods in native classes. Native classes differ internally from regular python types which results in `TypeError`s being raised when `object.__new__(cls)` is called when `cls` is a native class. To avoid making this call, `super().__new__(cls)` is transformed into a call to an internal setup function. I forgot to replicate this for calls to equivalent `object.__new__(cls)` calls so this PR fixes that. This introduced a regression because before my changes, `__new__` methods with `object.__new__(cls)` were effectively ignored at runtime, and after my changes they started raising `TypeError`s. Note that these calls are left as-is outside of `__new__` methods so it's still possible to trigger the `TypeError` but that is not a regression as this was the case before. For example this code: ``` class Test: pass t = object.__new__(Test) ``` results in `TypeError: object.__new__(Test) is not safe, use Test.__new__()`. This differs from interpreted python but the error message is actually correct in that using `Test.__new__(Test)` instead works.
3 participants
Fixes #16012
mypyc ignored custom implementations of
__new__because, even though a C function representing the method was generated, it was called neither in the type constructor nor in the method assigned to thetp_newpointer.Now if there's a
__new__method defined for a type, the corresponding function is called in place of the setup function which is responsible for allocating memory for new objects and initializing their attributes to default values.The setup function is still called when creating instances of the type as calls resolving to
object.__new__()are transformed to call the setup function. This way,__new__can return instances of other types and instances of the type of the class where__new__is defined are setup correctly.There are a couple of limitations:
super().__new__()calls in__new__methods of non-native classes are rejected because it's more difficult to resolve the setup function for non-native classes but this could probably be supported in the future.tp_newmethod of the parent type results in an error because cpython expects the sub type to use a wrapper fortp_newwhich compiled classes don't. Allowing this would require compiled types to be initialized more closely to the way cpython does it which might need a lot of work.__new__is annotated with@classmethod, calling it without the type parameter works in compiled code but raises an error in interpreted. I'm not sure of the reason and it's difficult to make it a compiler error because it's outside of what mypyc sees.