format(Fraction(1, 3), '.016f') raises ValueError

[skirpichev]

Bug report

Bug description:

c.f.

>>> format(float(Fraction(1, 3)), '.016f')
'0.3333333333333333'

Looking on docs, I think that float formatting better conforms to the specification.

Similar issue is valid for the width:

>>> format(float(Fraction(1, 3)), '0030.016f')
'0000000000000.3333333333333333'
>>> format(Fraction(1, 3), '0030.016f')
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    format(Fraction(1, 3), '0030.016f')
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/fractions.py", line 577, in __format__
    raise ValueError(
    ...<2 lines>...
    )
ValueError: Invalid format specifier '0030.016f' for object of type 'Fraction'

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-130662: accept leading zeros in precision/width for Fraction's formatting #130663
• gh-130662: refine width/precision specification in format-spec #130717
• gh-130662: accept leading zeros in precision/width for Decimal's formatting #132549

[skirpichev]

CC @mdickinson

BTW, I like more strict formatting rules for Fraction's. Maybe we should keep them, then this should be documented as now docs says: "If the format_spec format specification string ends with one of the presentation types 'e', 'E', 'f', 'F', 'g', 'G' or '%' then formatting follows the rules outlined for the float type in the Format Specification Mini-Language section." Adding more strict processing of width and precision for floats - probably will be too severe compatibility break.

PR is ready for review: #130663

[gvanrossum]

I'm not crazy about the proposed fix. It's harmless, but it perpetuates the questionable support for meaningless extra leading zeros.

I agree we can't tighten float's formatting language, it's not worth breaking working code over.

As far as the docs, maybe we can change the float docs to disallow the redundant leading zeros and explain in a note that CPython does support those for backwards compatibility reasons, but only for float, not for Fraction? Or otherwise mention the difference in the Fraction docs.

[skirpichev]

I agree we can't tighten float's formatting language, it's not worth breaking working code over.

Maybe we can. I doubt someone rely on this "feature". Thus, deprecating this behavior will not affect too much code.

On another hand, it seems that more strict processing - slightly complicates parsing in C (get_integer() helper) and the grammar rules. So, maybe those zeros aren't a big problem: it's easy to support them in the fractions module, see pr. Support this - also not a big problem for external modules, see the mpmath issue.

maybe we can change the float docs to disallow the redundant leading zeros and explain in a note that CPython does support those for backwards compatibility reasons, but only for float, not for Fraction? Or otherwise mention the difference in the Fraction docs.

Formatting docs already (IMO) big and complex. I would prefer to avoid adding more special cases (there is already float vs Decimal differences, etc).

Maybe we could just document more strict requirements for width/precision and soft-deprecate old behavior in WhatsNew?

[serhiy-storchaka]

I think it is better to allow leading zeroes in "width" and "precision" in the Fraction format. The current specification allows leading zeroes, forbidding them would make it even more complex. AFAIK, leading zeroes are allowed in all other programming languages. So forbidding them would complicate the documentation, the implementation, will create unnecessary difference from other programming languages.

[gvanrossum]

I note that Python numbers cannot have leading zeros (to avoid confusion with octal notation in C, C++ and in ancient Python).

But as the simplest (and fully backwards compatible) solution is to allow redundant leading zeros for Fraction, let's just do that.

For consistency I would also make the same change in _GENERAL_FORMAT_SPECIFICATION_MATCHER.

[skirpichev]

I note that Python numbers cannot have leading zeros (to avoid confusion with octal notation in C, C++ and in ancient Python).

Note, that in the given context we have only decimal notation.

BTW, Decimal's seems to be partially affected by this issue (width processing):

>>> f"{Decimal(1.25):0010f}"
Traceback (most recent call last):
  File "<python-input-9>", line 1, in <module>
    f"{Decimal(1.25):0010f}"
      ^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid format string
>>> f"{Decimal(1.25):.010f}"
'1.2500000000'

[gvanrossum]

Yeah, so I am now against “fixing” this. It is easy for users to avoid the issue: don’t use redundant leading zeros.

If there is concern about the docs, let’s change the float docs to no longer promise support for redundant leading zeros. Then users who discover that they work can file issues about getting float fixed, to which we can respond that it is an accidental historic bug that they are supported, and we recommend not using that, it we are reluctant to fix it for fear of unnecessary breaking working code.

[serhiy-storchaka]

I suspect that the pattern for width and precision in _FLOAT_FORMAT_SPECIFICATION_MATCHER was just copied from the pattern for width in _GENERAL_FORMAT_SPECIFICATION_MATCHER, as it was the nearest place. There was reason for not supporting leading zeroes in _GENERAL_FORMAT_SPECIFICATION_MATCHER (the zeropad flag is not supported), but that does not apply for _FLOAT_FORMAT_SPECIFICATION_MATCHER.

[skirpichev]

let’s change the float docs to no longer promise support for redundant leading zeros.

Alternative pr: #130717 (docs only)

[gvanrossum]

Okay, this is much more of a quagmire than I had assumed. In the end I don't care enough about this issue. I will withdraw from the discussion and let you all decide (you might simply vote on it).

[ant1sir]

Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import decimal
>>> val = decimal.Decimal(1.25)
>>> f"{val:06f}"
'001.25'
>>> decimal.__version__
'1.70'

[serhiy-storchaka]

@rhettinger, @tim-one, what are your thoughts?

There is a reason of not supporting leading zeroes in width in _GENERAL_FORMAT_SPECIFICATION_MATCHER -- because the first zero is the zeropad flag, but the zeropad flag is not supported here. This does not apply to _FLOAT_FORMAT_SPECIFICATION_MATCHER, where the zeropad flag is supported. I suspect that the pattern was just copied without further ado. I think that it would be good to harmonize format for float, Decimal and Fraction, and don't introduce a difference which is not necessary.

[serhiy-storchaka]

Yet one example: complex numbers. Leading zeroes are allowed in precision, but not allowed in width, because the zeropadding flag is not supported:

>>> format(1j, '20.03f')
'        0.000+1.000j'
>>> format(1j, '020.3f')
Traceback (most recent call last):
  File "<python-input-22>", line 1, in <module>
    format(1j, '020.3f')
    ~~~~~~^^^^^^^^^^^^^^
ValueError: Zero padding is not allowed in complex format specifier

[mdickinson]

I suspect that the pattern was just copied without further ado.

Just for the historical record, that's not what happened: the choice to exclude leading zeros for both the width and the precision was carefully considered and very much deliberate. That said, I'm happy to leave it to the active core devs to decide how to take this forward.

[skirpichev]

For a context, this bug (and two other) is an outcome of the mpmath issue: f"{mpf('0.1'):.016f}" triggered a ValueError (format specifier parsing followed rather to Fraction's code). Lets focus on 'e' and 'f' format types first, I think that for 'g' format (or for None) we can accept more differences for Fraction's, float's and Decimal's.

I think that it would be good to harmonize format for float, Decimal and Fraction, and don't introduce a difference which is not necessary.

Exactly. Documenting all these tiny differences will be a nightmare both for documentation writers and for readers. And it's not clear for external projects (gmpy2/mpmath) to which convention follow.

In my view, more permissible syntax for format specifiers is good for compatibility with other languages (read: C). That's all.

More strict syntax rules (used in the fractions module and, partially, in the decimal module) - seems better (to me), as they are more economical and don't require explanation for some ambiguity ('0' flag vs leading zero in width field).

So, #130717 seems better than #130663. I think that silent change (and a compatibility break) of the format specification language (without real changes in the code) is acceptable. I never seen those meaningless 0's in real code (maybe only something, coming from a typo).

[serhiy-storchaka]

Thank you for your information @mdickinson.

Arguments for and against support of the leading zeros for the width and the precision are solid, but this left us with inconsistency. We should either allow or deprecate leading zeros in formats of all other types for consistency, for simplifying the documentation and the mental model.

There is other issue related to leading zeros. The leading zero can be interpreted as the zeropad flag or as the fill character.

>>> format(-1, '010')
'-000000001'
>>> format(-1, '>010')
'00000000-1'

The Decimal format is more restricted. The leading zero is only accepted as the fill character before the align flag or if there is no align flag. But any other fill character except zero and space require the align flag.

>>> format(Decimal(-1), '010')
'-000000001'
>>> format(Decimal(-1), '0>10')
'00000000-1'
>>> format(Decimal(-1), '>010')
Traceback (most recent call last):
  File "<python-input-77>", line 1, in <module>
    format(Decimal(-1), '>010')
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid format string
>>> format(Decimal(-1), '~10')
Traceback (most recent call last):
  File "<python-input-78>", line 1, in <module>
    format(Decimal(-1), '~10')
    ~~~~~~^^^^^^^^^^^^^^^^^^^^
ValueError: invalid format string
>>> format(Decimal(-1), '~>10')
'~~~~~~~~-1'

[skirpichev]

The leading zero can be interpreted as the zeropad flag or as the fill character.

I think that existing documentation is clear here: "When no explicit alignment is given, preceding the width field by a zero ('0') character enables sign-aware zero-padding for numeric types, excluding complex. This is equivalent to a fill character of '0' with an alignment type of '='."

Are there reasons to prefer break this contract versus fixing Fraction/Decimal behavior?

BTW, let me know if someone think this (alongside with other small format()'s incompatibilities for float/Fraction/Decimal) require a more wide discussion on DPO.

[skirpichev]

There is other issue related to leading zeros. The leading zero can be interpreted as the zeropad flag or as the fill character.

@serhiy-storchaka, I think that issues with zeropad flag & fill/alignment should be treated separately, see #130716 and #131915. Lets focus on whether multiple leading zeros are allowed in width/precision.

State of art:

>>> format(float(0.25), '.02f')
'0.25'
>>> format(complex(0.25), '.02f')
'0.25+0.00j'
>>> format(Decimal(float(0.25)), '.02f')  # note that pure-Python version raises ValueError
'0.25'
>>> format(Fraction(float(0.25)), '.02f')
Traceback (most recent call last):
  File "<python-input-11>", line 1, in <module>
    format(Fraction(float(0.25)), '.02f')
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sk/src/cpython/Lib/fractions.py", line 600, in __format__
    raise ValueError(
    ...<2 lines>...
    )
ValueError: Invalid format specifier '.02f' for object of type 'Fraction'
>>> format(float(0.25), '02f')
'0.250000'
>>> format(float(0.25), '002f')
'0.250000'
>>> format(complex(0.25), '02f')
Traceback (most recent call last):
  File "<python-input-8>", line 1, in <module>
    format(complex(0.25), '02f')
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^
ValueError: Zero padding is not allowed in complex format specifier
>>> format(Decimal(float(0.25)), '02f')
'0.25'
>>> format(Decimal(float(0.25)), '002f')
Traceback (most recent call last):
  File "<python-input-13>", line 1, in <module>
    format(Decimal(float(0.25)), '002f')
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid format string
>>> format(Fraction(float(0.25)), '02f')
'0.250000'
>>> format(Fraction(float(0.25)), '002f')
Traceback (most recent call last):
...
ValueError: Invalid format specifier '002f' for object of type 'Fraction'

So, it seems both pure-Python version of the Decimal type and the Fraction type - reject leading zeros. The C-code Decimal type allows leading zeros in precision field.

CC @ericvsmith

[skirpichev]

CC @vstinner

[vstinner]

I agree with @serhiy-storchaka: #130662 (comment). I'm fine with accepting leading zeros in Fraction format spec for consistency with other numeric types (int, float, Decimal).

[skirpichev]

Ok, then PR #132549 - for Decimal's

[skirpichev]

It seems, alternative proposal #130717 - not interested anyone, I'm closing this pr.