XATTR_SIZE_MAX and XATTR_LIST_MAX undefined on kfreebsd/debian with eglibc

[zbysz]

BPO	13669
Nosy	@pitrou, @benjaminp

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-12-28.13:25:14.009>
labels = ['extension-modules']
title = 'XATTR_SIZE_MAX and XATTR_LIST_MAX undefined on kfreebsd/debian with eglibc'
updated_at = <Date 2014-05-13.21:42:05.012>
user = 'https://bugs.python.org/zbysz'

bugs.python.org fields:

activity = <Date 2014-05-13.21:42:05.012>
actor = 'skrah'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2011-12-28.13:25:14.009>
creator = 'zbysz'
dependencies = []
files = []
hgrepos = []
issue_num = 13669
keywords = []
message_count = 10.0
messages = ['150289', '150295', '150297', '150298', '150299', '150302', '150305', '150308', '150318', '175005']
nosy_count = 5.0
nosy_names = ['pitrou', 'benjamin.peterson', 'Arfrever', 'zbysz', 'pino']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue13669'
versions = ['Python 3.3']

[zbysz]

Extended attribute support was added in bpo-12720. Doesn't compile on kfreebsd/debian, which uses eglibc and gcc. The error is that the symbols XATTR_LIST_MAX and XATTR_SIZE_MAX are not defined.

After http://hg.python.org/cpython/rev/f325439d7f84 xattr compilation tests for 'defined(HAVE_SYS_XATTR_H) && defined(GLIBC)', but in this case this test doesn't work. Anyway, XATTR_SIZE_MAX is defined (on Linux) in <linux/limits.h>, and <sys/xattr.h> doesn't know anything about it.

(When defined XATTR_{SIZE,LIST}_MAX to 65536 like on linux to go through with the compilation, and os.listxattr('/') returns 'OSError: [Errno 78] Function not implemented'...)

[benjaminp]

Are you telling me that XATTR_SIZE_MAX is defined nowhere on eglibc?

[zbysz]

Unless I'm completely confused, XATTR_SIZE_MAX is defined by linux kernel headers, not the libc.

On my linux debian box:
$ grep -r XATTR_SIZE_MAX -I /usr
include/linux/limits.h:#define XATTR_SIZE_MAX 65536
$ dpkg -l libc6
libc6 2.11.2-10 Embedded GNU C Library: Shared libraries
$ dpkg -S /usr/include/linux/limits.h
linux-libc-dev: /usr/include/linux/limits.h

On debian/kfreebsd grep XATTR_SIZE_MAX doesn't find anything.

On fedora 16:
$ grep -r XATTR_SIZE_MAX -I /usr
/usr/include/linux/limits.h:#define XATTR_SIZE_MAX 65536
$ rpm -qf /usr/include/linux/limits.h
kernel-headers-3.1.0-0.rc10.git0.1.fc16.x86_64

[zbysz]

Forgot to add (on the fedora box):
$ rpm -q glibc
glibc-2.14.90-13.x86_64
(The GNU libc libraries from http://www.gnu.org/software/glibc/)

So the glibc/eglibc split is not important here.

[pitrou]

On my linux debian box:
$ grep -r XATTR_SIZE_MAX -I /usr
include/linux/limits.h:#define XATTR_SIZE_MAX 65536
$ dpkg -l libc6
libc6 2.11.2-10 Embedded GNU C Library: Shared libraries
$ dpkg -S /usr/include/linux/limits.h
linux-libc-dev: /usr/include/linux/limits.h

But linux/limits.h is included by the glibc:

$ \grep -r linux/limits.h /usr/include/
/usr/include/sys/param.h:#include <linux/limits.h>
[...]
$ \rpm -qf /usr/include/sys/param.h
glibc-devel-2.12.1-11.2.mga1

[zbysz]

Yes, it must be, because XATTR_SIZE_MAX is only defined in
linux/limits.h. The problem is that with the kfreebsd kernel,
/usr/include/sys/limits.h doesn't define or include anything that
defines XATTR_SIZE_MAX.

Maybe the test should be 'defined(HAVE_SYS_XATTR_H) &&
defined(XATTR_SIZE_MAX)'?

[skrah]

So the problem occurs on:

http://www.debian.org/ports/kfreebsd-gnu/

Did I get that right? Is __FreeBSD__ defined on that system?

I'm not sure though if we should start supporting hybrid systems.

[zbysz]

That's the one.

No. I'm putting the complete list below.

Actually python2.5-7 and 3.2 is normally packaged by debian for
is arch, so it mostly works.

$ gcc -dM -E - < /dev/null
#define __DBL_MIN_EXP__ (-1021)
#define __UINT_LEAST16_MAX__ 65535
#define __FLT_MIN__ 1.17549435082228750797e-38F
#define __UINT_LEAST8_TYPE__ unsigned char
#define __INTMAX_C(c) c ## L
#define __CHAR_BIT__ 8
#define __UINT8_MAX__ 255
#define __WINT_MAX__ 4294967295U
#define __ORDER_LITTLE_ENDIAN__ 1234
#define __SIZE_MAX__ 18446744073709551615UL
#define __WCHAR_MAX__ 2147483647
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 1
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 1
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 1
#define __DBL_DENORM_MIN__ ((double)4.94065645841246544177e-324L)
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 1
#define __FLT_EVAL_METHOD__ 0
#define __unix__ 1
#define __x86_64 1
#define __UINT_FAST64_MAX__ 18446744073709551615UL
#define __SIG_ATOMIC_TYPE__ int
#define __DBL_MIN_10_EXP__ (-307)
#define __FINITE_MATH_ONLY__ 0
#define __GNUC_PATCHLEVEL__ 2
#define __UINT_FAST8_MAX__ 255
#define __DEC64_MAX_EXP__ 385
#define __INT8_C(c) c
#define __UINT_LEAST64_MAX__ 18446744073709551615UL
#define __SHRT_MAX__ 32767
#define __LDBL_MAX__ 1.18973149535723176502e+4932L
#define __UINT_LEAST8_MAX__ 255
#define __UINTMAX_TYPE__ long unsigned int
#define __DEC32_EPSILON__ 1E-6DF
#define __unix 1
#define __UINT32_MAX__ 4294967295U
#define __LDBL_MAX_EXP__ 16384
#define __WINT_MIN__ 0U
#define __SCHAR_MAX__ 127
#define __WCHAR_MIN__ (-__WCHAR_MAX__ - 1)
#define __INT64_C(c) c ## L
#define __DBL_DIG__ 15
#define __SIZEOF_INT__ 4
#define __SIZEOF_POINTER__ 8
#define __USER_LABEL_PREFIX__ 
#define __GLIBC__ 1
#define __STDC_HOSTED__ 1
#define __LDBL_HAS_INFINITY__ 1
#define __FLT_EPSILON__ 1.19209289550781250000e-7F
#define __LDBL_MIN__ 3.36210314311209350626e-4932L
#define __DEC32_MAX__ 9.999999E96DF
#define __INT32_MAX__ 2147483647
#define __SIZEOF_LONG__ 8
#define __UINT16_C(c) c
#define __DECIMAL_DIG__ 21
#define __LDBL_HAS_QUIET_NAN__ 1
#define __GNUC__ 4
#define __MMX__ 1
#define __FLT_HAS_DENORM__ 1
#define __SIZEOF_LONG_DOUBLE__ 16
#define __BIGGEST_ALIGNMENT__ 16
#define __DBL_MAX__ ((double)1.79769313486231570815e+308L)
#define __INT_FAST32_MAX__ 9223372036854775807L
#define __DBL_HAS_INFINITY__ 1
#define __DEC32_MIN_EXP__ (-94)
#define __INT_FAST16_TYPE__ long int
#define __LDBL_HAS_DENORM__ 1
#define __DEC128_MAX__ 9.999999999999999999999999999999999E6144DL
#define __INT_LEAST32_MAX__ 2147483647
#define __DEC32_MIN__ 1E-95DF
#define __DBL_MAX_EXP__ 1024
#define __DEC128_EPSILON__ 1E-33DL
#define __SSE2_MATH__ 1
#define __PTRDIFF_MAX__ 9223372036854775807L
#define __amd64 1
#define __LONG_LONG_MAX__ 9223372036854775807LL
#define __SIZEOF_SIZE_T__ 8
#define __SIZEOF_WINT_T__ 4
#define __GCC_HAVE_DWARF2_CFI_ASM 1
#define __GXX_ABI_VERSION 1002
#define __FLT_MIN_EXP__ (-125)
#define __INT_FAST64_TYPE__ long int
#define __DBL_MIN__ ((double)2.22507385850720138309e-308L)
#define __LP64__ 1
#define __DEC128_MIN__ 1E-6143DL
#define __REGISTER_PREFIX__ 
#define __UINT16_MAX__ 65535
#define __DBL_HAS_DENORM__ 1
#define __UINT8_TYPE__ unsigned char
#define __NO_INLINE__ 1
#define __FLT_MANT_DIG__ 24
#define __VERSION__ "4.6.2"
#define __UINT64_C(c) c ## UL
#define __FLOAT_WORD_ORDER__ __ORDER_LITTLE_ENDIAN__
#define __INT32_C(c) c
#define __DEC64_EPSILON__ 1E-15DD
#define __ORDER_PDP_ENDIAN__ 3412
#define __DEC128_MIN_EXP__ (-6142)
#define __INT_FAST32_TYPE__ long int
#define __UINT_LEAST16_TYPE__ short unsigned int
#define unix 1
#define __INT16_MAX__ 32767
#define __SIZE_TYPE__ long unsigned int
#define __UINT64_MAX__ 18446744073709551615UL
#define __INT8_TYPE__ signed char
#define __ELF__ 1
#define __FLT_RADIX__ 2
#define __INT_LEAST16_TYPE__ short int
#define __LDBL_EPSILON__ 1.08420217248550443401e-19L
#define __UINTMAX_C(c) c ## UL
#define __SSE_MATH__ 1
#define __k8 1
#define __SIG_ATOMIC_MAX__ 2147483647
#define __SIZEOF_PTRDIFF_T__ 8
#define __x86_64__ 1
#define __DEC32_SUBNORMAL_MIN__ 0.000001E-95DF
#define __INT_FAST16_MAX__ 9223372036854775807L
#define __UINT_FAST32_MAX__ 18446744073709551615UL
#define __UINT_LEAST64_TYPE__ long unsigned int
#define __FLT_HAS_QUIET_NAN__ 1
#define __FLT_MAX_10_EXP__ 38
#define __LONG_MAX__ 9223372036854775807L
#define __DEC128_SUBNORMAL_MIN__ 0.000000000000000000000000000000001E-6143DL
#define __FLT_HAS_INFINITY__ 1
#define __UINT_FAST16_TYPE__ long unsigned int
#define __DEC64_MAX__ 9.999999999999999E384DD
#define __CHAR16_TYPE__ short unsigned int
#define __PRAGMA_REDEFINE_EXTNAME 1
#define __INT_LEAST16_MAX__ 32767
#define __DEC64_MANT_DIG__ 16
#define __INT64_MAX__ 9223372036854775807L
#define __UINT_LEAST32_MAX__ 4294967295U
#define __INT_LEAST64_TYPE__ long int
#define __INT16_TYPE__ short int
#define __INT_LEAST8_TYPE__ signed char
#define __DEC32_MAX_EXP__ 97
#define __INT_FAST8_MAX__ 127
#define __INTPTR_MAX__ 9223372036854775807L
#define __SSE2__ 1
#define __LDBL_MANT_DIG__ 64
#define __DBL_HAS_QUIET_NAN__ 1
#define __SIG_ATOMIC_MIN__ (-__SIG_ATOMIC_MAX__ - 1)
#define __k8__ 1
#define __INTPTR_TYPE__ long int
#define __UINT16_TYPE__ short unsigned int
#define __WCHAR_TYPE__ int
#define __SIZEOF_FLOAT__ 4
#define __UINTPTR_MAX__ 18446744073709551615UL
#define __DEC64_MIN_EXP__ (-382)
#define __INT_FAST64_MAX__ 9223372036854775807L
#define __FLT_DIG__ 6
#define __UINT_FAST64_TYPE__ long unsigned int
#define __INT_MAX__ 2147483647
#define __amd64__ 1
#define __INT64_TYPE__ long int
#define __FreeBSD_kernel__ 1
#define __FLT_MAX_EXP__ 128
#define __ORDER_BIG_ENDIAN__ 4321
#define __DBL_MANT_DIG__ 53
#define __INT_LEAST64_MAX__ 9223372036854775807L
#define __DEC64_MIN__ 1E-383DD
#define __WINT_TYPE__ unsigned int
#define __UINT_LEAST32_TYPE__ unsigned int
#define __SIZEOF_SHORT__ 2
#define __SSE__ 1
#define __LDBL_MIN_EXP__ (-16381)
#define __INT_LEAST8_MAX__ 127
#define __SIZEOF_INT128__ 16
#define __LDBL_MAX_10_EXP__ 4932
#define __DBL_EPSILON__ ((double)2.22044604925031308085e-16L)
#define _LP64 1
#define __UINT8_C(c) c
#define __INT_LEAST32_TYPE__ int
#define __SIZEOF_WCHAR_T__ 4
#define __UINT64_TYPE__ long unsigned int
#define __INT_FAST8_TYPE__ signed char
#define __DBL_DECIMAL_DIG__ 17
#define __DEC_EVAL_METHOD__ 2
#define __UINT32_C(c) c ## U
#define __INTMAX_MAX__ 9223372036854775807L
#define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__
#define __FLT_DENORM_MIN__ 1.40129846432481707092e-45F
#define __INT8_MAX__ 127
#define __UINT_FAST32_TYPE__ long unsigned int
#define __CHAR32_TYPE__ unsigned int
#define __FLT_MAX__ 3.40282346638528859812e+38F
#define __INT32_TYPE__ int
#define __SIZEOF_DOUBLE__ 8
#define __FLT_MIN_10_EXP__ (-37)
#define __INTMAX_TYPE__ long int
#define __DEC128_MAX_EXP__ 6145
#define __GNUC_MINOR__ 6
#define __UINTMAX_MAX__ 18446744073709551615UL
#define __DEC32_MANT_DIG__ 7
#define __DBL_MAX_10_EXP__ 308
#define __LDBL_DENORM_MIN__ 3.64519953188247460253e-4951L
#define __INT16_C(c) c
#define __STDC__ 1
#define __PTRDIFF_TYPE__ long int
#define __UINT32_TYPE__ unsigned int
#define __UINTPTR_TYPE__ long unsigned int
#define __DEC64_SUBNORMAL_MIN__ 0.000000000000001E-383DD
#define __DEC128_MANT_DIG__ 34
#define __LDBL_MIN_10_EXP__ (-4931)
#define __SIZEOF_LONG_LONG__ 8
#define __LDBL_DIG__ 18
#define __FLT_DECIMAL_DIG__ 9
#define __UINT_FAST16_MAX__ 18446744073709551615UL
#define __GNUC_GNU_INLINE__ 1
#define __UINT_FAST8_TYPE__ unsigned char

[zbysz]

[Why does roundup remove quoted text being replied to???]

On 12/28/2011 06:05 PM, Stefan Krah wrote:

http://www.debian.org/ports/kfreebsd-gnu/
That's the one.

Is __FreeBSD__ defined on that system?
No. I'm putting the complete list below.

I'm not sure though if we should start supporting hybrid systems.
Actually python2.5-7 and 3.2 is normally packaged by debian for
is arch, so it mostly works.

[pino]

*_MAX constants are usually defined when the system declares that the maximum limit for that option exist and is already known at compile time.
Python should simply not rely on XATTR_LIST_MAX and XATTR_SIZE_MAX being defined, but just grow the buffers as needed until *getxattr / *listxattr succeed or fail with errno != ERANGE; it seems the case already, although the two *_MAX are used as kind of "upper bound" limit.

Instead of using short lists with sizes to try, what about starting from a size (e.g. 128 for getxattr and 256 for listxattr), and grow the buffer by adding the same value each iteration (or doubling the size)?