KEMBAR78
Breaking changes in 0.10.0 and future changes · Issue #170 · theskumar/python-dotenv · GitHub
Skip to content

Breaking changes in 0.10.0 and future changes #170

@bbc2

Description

@bbc2

As reported by several users in #148, version 0.10.0 changed how some .env files are parsed in a breaking way, especially with regards to escapes. I think some of those changes were necessary, but some other may have been unexpected.

Experiment

It was unclear what exactly broke and whether Python-dotenv is consistent with other parsers, so I ran an experiment to compare versions and packages.

The scripts to generate these tables are in bbc2/dotenv-parser-comparisons. I may update them if I find new interesting behavior.

Basic

foo=ab
parser output
python-dotenv-0.9.1 a b
python-dotenv-0.10.1 a b
bash-5.0.0 a b
js-dotenv-6.2.0 a b
ruby-dotenv-2.6.0 a b

Escaped z

foo=a\zb
parser output
python-dotenv-0.9.1 a \ \ z b
python-dotenv-0.10.1 a \ z b
bash-5.0.0 a z b
js-dotenv-6.2.0 a \ z b
ruby-dotenv-2.6.0 a \ z b

Escaped and single-quoted z

foo='a\zb'
parser output
python-dotenv-0.9.1 a \ z b
python-dotenv-0.10.1 a \ z b
bash-5.0.0 a \ z b
js-dotenv-6.2.0 a \ z b
ruby-dotenv-2.6.0 a \ z b

Escaped and double-quoted z

foo="a\zb"
parser output
python-dotenv-0.9.1 a \ z b
python-dotenv-0.10.1 a \ z b
bash-5.0.0 a \ z b
js-dotenv-6.2.0 a \ z b
ruby-dotenv-2.6.0 a z b

Escaped n

foo=a\nb
parser output
python-dotenv-0.9.1 a \ \ n b
python-dotenv-0.10.1 a \ n b
bash-5.0.0 a n b
js-dotenv-6.2.0 a \ n b
ruby-dotenv-2.6.0 a \ n b

Escaped and single-quoted n

foo='a\nb'
parser output
python-dotenv-0.9.1 a \ n b
python-dotenv-0.10.1 a \n b
bash-5.0.0 a \ n b
js-dotenv-6.2.0 a \ n b
ruby-dotenv-2.6.0 a \ n b

Escaped and double-quoted n

foo="a\nb"
parser output
python-dotenv-0.9.1 a \ n b
python-dotenv-0.10.1 a \n b
bash-5.0.0 a \ n b
js-dotenv-6.2.0 a \n b
ruby-dotenv-2.6.0 a \n b

Quoted newline

foo="a
b"
parser output
python-dotenv-0.9.1 " a
python-dotenv-0.10.1 a \n b
bash-5.0.0 a \n b
js-dotenv-6.2.0 a
ruby-dotenv-2.6.0 a \n b

Non-escaped space

foo=a b
parser output
python-dotenv-0.9.1 a b
python-dotenv-0.10.1 a b
bash-5.0.0 a
js-dotenv-6.2.0 a b
ruby-dotenv-2.6.0 a b

Non-escaped #

foo=a#b
parser output
python-dotenv-0.9.1 a # b
python-dotenv-0.10.1 a
bash-5.0.0 a # b
js-dotenv-6.2.0 a # b
ruby-dotenv-2.6.0 a

Non-escaped spaced #

foo=a #b
parser output
python-dotenv-0.9.1 a # b
python-dotenv-0.10.1 a
bash-5.0.0 a
js-dotenv-6.2.0 a # b
ruby-dotenv-2.6.0 a

Escaped #

foo="a#b"
parser output
python-dotenv-0.9.1 a # b
python-dotenv-0.10.1 a # b
bash-5.0.0 a # b
js-dotenv-6.2.0 a # b
ruby-dotenv-2.6.0 a # b

UTF-8

foo=é
parser output
python-dotenv-0.9.1 \ x e 9
python-dotenv-0.10.1 é
bash-5.0.0 é
js-dotenv-6.2.0 é
ruby-dotenv-2.6.0 é

Quoted UTF-8

foo="é"
parser output
python-dotenv-0.9.1 é
python-dotenv-0.10.1 é
bash-5.0.0 é
js-dotenv-6.2.0 é
ruby-dotenv-2.6.0 é

Conclusion

  1. Non-quoted escapes (valid or invalid): All up-to-date parsers except Bash do the same thing. Bash removes the \. 0.9.1 added a \, which was obviously incorrect.
  2. Single-quoted invalid escapes: Everything is fine.
  3. Single-quoted valid escapes: 0.10.1 is the only parser that interprets them as control characters.
  4. Double-quoted invalid escapes: It's fine except for Ruby.
  5. Double-quoted valid escapes: All up-to-date parsers except Bash do the same thing. Bash and 0.9.1 keep the original characters instead of interpreting them as control characters.
  6. Pound sign #: Interpreted as a comment delimiter since 0.10.0 if unquoted, even if there is no whitespace preceding it. When quoted or prefixed with whitespace, everything is fine except for JavaScript.
  7. Non-quoted UTF-8: Fixed in 0.10.0. When quoted, everything is fine.
  8. Non-escaped space: Only Bash ignores everything after it (or treats the rest as a command). Other parsers include everything until the end of the line.

My opinion:

  • (2, 4, 7) are OK.
  • (1, 5, 8) are where Bash differs from other parsers. It isn't obvious what we should do.
  • (3, 6) are where python-dotenv 0.10.0 is quite obviously broken and should be fixed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions