[3.13] gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520) #120945
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…related issues (pythonGH-120520) * Add an InternalDocs file describing how interning should work and how to use it. * Add internal functions to *explicitly* request what kind of interning is done: - `_PyUnicode_InternMortal` - `_PyUnicode_InternImmortal` - `_PyUnicode_InternStatic` * Switch uses of `PyUnicode_InternInPlace` to those. * Disallow using `_Py_SetImmortal` on strings directly. You should use `_PyUnicode_InternImmortal` instead: - Strings should be interned before immortalization, otherwise you're possibly interning a immortalizing copy. - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI. * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery. * Make sure the statically allocated string singletons are unique. This means these sets are now disjoint: - `_Py_ID` - `_Py_STR` (including the empty string) - one-character latin-1 singletons Now, when you intern a singleton, that exact singleton will be interned. * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic). * Intern `_Py_STR` singletons at startup. * For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup. * Beef up the tests. Cover internal details (marked with `@cpython_only`). * Add lots of assertions Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
encukou
requested review from
berkerpeksag,
ericsnowcurrently,
erlend-aasland,
iritkatriel,
isidentical,
kumaraditya303,
lysnikolaou,
markshannon,
methane,
pablogsal and
rhettinger
as code owners
This was referenced Jun 24, 2024
encukou
added
the
🔨 test-with-buildbots
bedevere-bot
removed
the
🔨 test-with-buildbots
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(cherry-picked from 6f1d448)
Add an InternalDocs file describing how interning should work and how to use it.
Add internal functions to explicitly request what kind of interning is done:
_PyUnicode_InternMortal_PyUnicode_InternImmortal_PyUnicode_InternStaticSwitch uses of
PyUnicode_InternInPlaceto those.Disallow using
_Py_SetImmortalon strings directly. You should use_PyUnicode_InternImmortalinstead:_Py_SetImmortaldoesn't handle theSSTATE_INTERNED_MORTALtoSSTATE_INTERNED_IMMORTALupdate, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI.Add private
_only_immortalargument forsys.getunicodeinternedsize, used in refleak test machinery.Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
_Py_ID_Py_STR(including the empty string)Now, when you intern a singleton, that exact singleton will be interned.
Add a
_Py_LATIN1_CHRmacro, use it instead of_Py_ID/_Py_STRfor one-character latin-1 singletons everywhere (including Clinic).Intern
_Py_STRsingletons at startup.For free-threaded builds, intern
_Py_LATIN1_CHRsingletons at startup.Beef up the tests. Cover internal details (marked with
@cpython_only).Add lots of assertions