-
-
Notifications
You must be signed in to change notification settings - Fork 33.2k
Description
PEP614 introduces relaxed decorator grammar in 3.9: https://peps.python.org/pep-0614/
Right now, ast.parse with feature_version=(3, 8) and bellow behave differently from specified python versions themself.
For example, main branch is able to parse this code:
Python 3.12.0a0 (heads/main-dirty:e860e521ec, Aug 28 2022, 21:36:18) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> code = """
... @decs[0].func
... def some(): pass
... """
>>> ast.parse(code)
<ast.Module object at 0x102db8150>
>>> ast.parse(code, feature_version=(3, 8))
<ast.Module object at 0x102d9d320>While python==3.8.x is not able to:
Python 3.8.9 (default, May 3 2021, 12:15:25)
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> code = """
... @decs[0].func
... def some(): pass
... """
>>> ast.parse(code)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/sobolev/.pyenv/versions/3.8.9/lib/python3.8/ast.py", line 47, in parse
return compile(source, filename, mode, flags,
File "<unknown>", line 2
@decs[0].func
^
SyntaxError: invalid syntaxWhen looking at we-love-parsers/pegen - it does have a version check for this: https://github.com/we-like-parsers/pegen/blame/599255a5ba3930e86367cd3fc78fae91ee184dd5/data/python.gram#L660-L662
Pros and Cons
I think that there are several key points to this, pros:
- Correct parsing of older versions
- Consistency with other parser like
we-love-parsers/pegen
However, there are some serious cons to it:
- Grammar is made quite significanly more complex
- There might be new bugs in new behaviour: locations / performance / etc
I think it is up to @pablogsal and other core devs to decide.
Proposed solution
I've modified an existing we-love-parsers/pegen's solution a bit, here's what it looks like:
decorators[asdl_expr_seq*]: a[asdl_expr_seq*]=decorator+
decorator[expr_ty]:
| a=('@' f=dec_maybe_call NEWLINE { f }) { a }
| a=('@' f=named_expression NEWLINE { f }) {
CHECK_VERSION(expr_ty, 9, "Relaxed decorator syntax is", a) }
dec_maybe_call[expr_ty]:
| a=dec_primary '(' b=[arguments] ')' {
_PyAST_Call(a,
(b) ? ((expr_ty) b)->v.Call.args : NULL,
(b) ? ((expr_ty) b)->v.Call.keywords : NULL,
EXTRA) }
| dec_primary
dec_primary[expr_ty]:
| a=dec_primary '.' b=NAME { _PyAST_Attribute(a, b->v.Name.id, Load, EXTRA) }
| a=NAME
I will send a PR with all the changes / tests / auto-generated files, so we can take a closer look and decide whether we want this or not.