Incorrect struct definition with bitfields

[berthin]

BPO	41932
Nosy	@FFY00, @berthin
Files	bitfields.zip

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-04.15:56:03.606>
labels = ['ctypes', 'type-bug', '3.8', '3.9', '3.10']
title = 'Incorrect struct definition with bitfields'
updated_at = <Date 2020-10-04.15:56:03.606>
user = 'https://github.com/berthin'

bugs.python.org fields:

activity = <Date 2020-10-04.15:56:03.606>
actor = 'berthin'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['ctypes']
creation = <Date 2020-10-04.15:56:03.606>
creator = 'berthin'
dependencies = []
files = ['49493']
hgrepos = []
issue_num = 41932
keywords = []
message_count = 1.0
messages = ['377953']
nosy_count = 2.0
nosy_names = ['FFY00', 'berthin']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41932'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[berthin]

Hi all,

I found one issue related to bitfields and ctypes, and although there is a similar issue already opened (https://bugs.python.org/issue29753), that only covers the case with pragmas.

Please, consider the following scenario:

>>> from ctypes import *
>>> class X(Structure):
...     _fields_ = [("a", c_uint8, 8), ("b", c_uint8, 8), ("c", c_uint32, 16)]
...
>>> sizeof(X)
5

Which seems to be incorrect, because what natively C does is to generate a 4bytes struct.

root@0bc6a647abaa:/target/cpython-fix# cat sample.c

#include <stdio.h>
#include <stdint.h>
struct X { 
    uint8_t a; 
    uint8_t b; 
    uint32_t c:16;
} X;

int main() { 
    printf("%u\n", sizeof(X)); 
}

root@0bc6a647abaa:/target/cpython-fix# gcc sample.c -o a && ./a
4

Also, if I use different databtypes I get what I expect (a struct of 4 bytes).

>>> X = type('X', (Structure,), {'_fields_': [('a', c_uint32, 8), ('b', c_uint32, 8), ('c', c_uint32, 16)]})
>>> sizeof(X)
4
>>> X = type('X', (Structure,), {'_fields_': [('a', c_uint16, 8), ('b', c_uint16, 8), ('c', c_uint16, 16)]})
>>> sizeof(X)
4
>>> X = type('X', (Structure,), {'_fields_': [('a', c_uint8, 8), ('b', c_uint8, 8), ('c', c_uint16, 16)]})
>>> sizeof(X)
4

I tried to debug a little bit, adding some prints before and after calling *PyCField_FromDesc* in *Modules/_ctypes/stgdict.c:L603*.=, if someone wants to look at it, the logs are attached / one log file per each test, using the following gdb init:

# before
break stgdict.c:601
commands
p field_size
p bitofs
p size
p align
p *dict
continue
end

# after
break stgdict.c:617
commands
p field_size
p bitofs
p size
p align
p *(CFieldObject*) prop
continue
end

Hope someone could spot the issue, I will also try to investigate it by my own.

Best,

P.S. For the tests, I worked on top of bpo-29753 (#19850)

[encukou]

Fixed in #97702, the reproducer is now part of tests (as the Example_gh_86098* classes)