-
-
Notifications
You must be signed in to change notification settings - Fork 33.2k
Open
Labels
extension-modulesC modules in the Modules dirC modules in the Modules dirinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Bug report
There are a a few non thread-safe libc functions used in Python that can be an issue for free threading, isolated subinterpreters, or sometimes even with the GIL.
In #126316, @vstinner fixed the use setgrent / getgrent. It's probably a good time to look for other similar issues.
clang-tidy
clang-tidy has a concurrency-mt-unsafe check that looks for "known-to-be-unsafe functions".
clang-tidy notes
Prerequisites: install clang-tidy-18 and bear.
./configure -C --with-pydebug --disable-gil
# generate compile_commands.json
bear -- make -j
run-clang-tidy-18 -checks='-*,concurrency-mt-unsafe' -p .Unsafe libc functions
localeconv(): not thread-safe, see glibc's manual. Isnl_langinfoa substitue?setlocalesetpwent,getpwent, andendpwentinpwdmodule.c. These are similar togrpmodule.cand can likely be addressed the same way.getservbyname,getservbyport,getprotobynameinModules/socketmodule.c: usegetservbyname_r, etc.? Note these thread-safety issues affect the default build because we release the GIL around the relevant calls.dbm_open,dbm_close, etc. inModules/_dbmmodule.cgetlogin: usegetlogin_rif available?
Unfixable by us?
getenv,setenv,unsetenv,putenv: environment modification isn't thread-safe andgetenvis used extensively in other C libraries, so putting a lock around our accesses doesn't do much. (Might finally be fixed in glibc, see https://inbox.sourceware.org/libc-alpha/cover.1722193092.git.fweimer@redhat.com/).login_tty: not sure it matters aslogin_ttyis usually called after afork()
Safe due to our usage
getc_unlocked, safe because we use it within aflockfile()call.mbrtowc()- safe as long as the passed inmbstate_t *is non-NULL, which is the case in CPython.
Safe in glibc
These functions are flagged by clang-tidy because they are not guaranteed to be safe by POSIX, but they are safe in glibc. It'd be nice to verify that they are safe in other libc implementations. I don't think it's worth changing them:
dlerror()strerror(): see https://man7.org/linux/man-pages/man3/strerror.3.htmlsystem(): see https://man7.org/linux/man-pages/man3/system.3.htmlreaddir(DIR *dirp): see https://man7.org/linux/man-pages/man3/readdir.3.html: "...in modern implementations (including the glibc implementation), concurrent calls to readdir() that specify different directory streams are thread-safe."wcstombs: see https://man7.org/linux/man-pages/man3/wcstombs.3.htmlnl_langinfo: https://www.man7.org/linux/man-pages/man3/nl_langinfo.3.html (as long as locale doesn't change)tcsendbreak,tcflow: see https://www.man7.org/linux/man-pages/man3/termios.3.html#ATTRIBUTESsetlogmask: safe since glibc 2.33strsignal: not documented as thread-safe, but uses glibc uses TLS internally since 2.32
Other
exit(): apparently concurrent calls toexit()are not thread-safe, but I don't think it matters for our usages.ptsname(): we already useptsname_r(), but the static analyzer gets confused
Linked PRs
- gh-127081: fix some un-thread-safe use of libc functions #132591
- gh-127081: lock non-re-entrant
*pwentcalls #132748 - gh-127081: add critical sections to
dbmobjects #132749 - gh-127081: use re-entrant variants of
get{proto,serv}by{name,port}#132750 - gh-127081: use
getlogin_rif available #132751 - [3.14] gh-127081: use
getlogin_rif available (gh-132751) #135097 - [3.13] gh-127081: use
getlogin_rif available (gh-132751) #135098 - [3.14] gh-127081: add critical sections to
dbmobjects (gh-132749) #139996
Metadata
Metadata
Assignees
Labels
extension-modulesC modules in the Modules dirC modules in the Modules dirinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error