Repository navigation
marshal.dumps() raises TypeError for a set containing several complex NaNs #159184
Description
Bug report
marshal writes the elements of a set in the order of a sort of
(marshalled bytes, value) pairs. When two elements have the same
marshalled bytes, the sort compares the values. Complex numbers are not
orderable, so this fails:
>>> import marshal
>>> marshal.dumps(frozenset({complex('nan'), complex('nan')}))
TypeError: '<' not supported between instances of 'complex' and 'complex'Two distinct elements only have the same bytes when they are NaNs that are
different objects (or containers of such NaNs).
The compiler can produce such a constant by folding, so the code object
cannot be marshalled and no .pyc can be written for the module:
>>> code = compile("x in {(1e999-1e999)*1j, (1e999-1e999)*2j}", "m.py", "exec")
>>> code.co_consts
(inf, frozenset({(nan+nanj), (nan+nanj)}))
>>> marshal.dumps(code)
TypeError: '<' not supported between instances of 'complex' and 'complex'A related problem in the same code: float NaNs do compare, but neither way,
so elements with equal bytes stay in set iteration order, which depends on
their addresses. If one of them is also referenced elsewhere in the dump,
the output differs for the same value:
import marshal
nans = [float('nan') for _ in range(3)]
# A list of a NaN and a set of three NaNs, one of which is that NaN.
print(len({marshal.dumps([n, set(nans)]) for n in nans})) # 3, expected 1Proposed fix
Sort the elements on their marshalled bytes only and never compare the
values. That fixes the TypeError on its own. Making the order of equal
elements independent of addresses additionally needs the writer to know
which objects it has already written; that part depends on the fix for gh-129724
(coming).
Discovered using Claude while doing a deep dive on pyc compilation determinism.
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Linked PRs
- gh-159184: Do not compare set elements when marshalling a set #159196
- gh-159184: Make the marshal order of set elements with equal dumps stable #159197
- [3.15] gh-159184: Do not compare set elements when marshalling a set (GH-159196) #159205
- [3.14] gh-159184: Do not compare set elements when marshalling a set (GH-159196) #159206
- gh-159184: Do not intern NaN constants on the free-threaded build #161276
- [3.14] gh-159184: Do not intern NaN constants on the free-threaded build (GH-161276) #161286
- [3.15] gh-159184: Do not intern NaN constants on the free-threaded build (GH-161276) #161287
Activity
pythongh-159184: Do not compare set elements when marshalling a set
pythongh-159184: Do not compare set elements when marshalling a set
pythongh-159184: Make the marshal order of set elements with equal du…
gh-159184: Do not compare set elements when marshalling a set (GH-159196
pythongh-159184: Make the marshal order of set elements with equal du…
[3.14] gh-159184: Do not compare set elements when marshalling a set (G…
[3.15] gh-159184: Do not compare set elements when marshalling a set (G…
gpshead commentedon Oct 11, 2026
#159197 is needed to fix this but requires #159192 first.
pythongh-159184: Make the marshal order of set elements with equal du…
gh-159184: Make the marshal order of set elements with equal dumps st…
hugovk commentedon Nov 3, 2026
Re-opening, looks like this caused refleaks:
gh-159184: Do not intern NaN constants on the free-threaded build (GH…
gpshead commentedon Nov 4, 2026
I think #161276 fixed that (cody ran buildbot tests). thanks!
[3.15] gh-159184: Do not intern NaN constants on the free-threaded bu…
[3.14] gh-159184: Do not intern NaN constants on the free-threaded bu…