Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[2.1.0] BaseCSVWidget.value_from_datadict — AttributeError: 'list' object has no attribute 'split' #1103

Closed
moseb opened this issue Jul 12, 2019 · 15 comments · Fixed by #1637
Closed

[2.1.0] BaseCSVWidget.value_from_datadict — AttributeError: 'list' object has no attribute 'split' #1103

moseb opened this issue Jul 12, 2019 · 15 comments · Fixed by #1637

Comments

@moseb
Copy link

moseb commented Jul 12, 2019

I ran into …

Traceback (most recent call last):
  [..]
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 393, in _clean_fields
    value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))
  File "/usr/local/lib/python3.7/site-packages/django_filters/widgets.py", line 201, in value_from_datadict
    return value.split(',')
AttributeError: 'list' object has no attribute 'split'

and found that the code at …

django-filter/django_filters/widgets.py

Lines 195 to 202 in 1f47e36

def value_from_datadict(self, data, files, name):
value = super().value_from_datadict(data, files, name)
if value is not None:
if value == '': # empty value should parse as an empty list
return []
return value.split(',')
return None

… is not robust with regard to super().value_from_datadict(data, files, name) returning a list. I need to debug this more but maybe you already know what's going on here?
@rpkilby
Copy link
Collaborator

rpkilby commented Jul 12, 2019

Hi @moseb. Can you paste the FilterSet code that you're using? The last few lines of the trace back are sufficient to determine why you'd hit this exception.

@moseb
Copy link
Author

moseb commented Jul 12, 2019 •
edited
Loading

Full traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python3.7/site-packages/django/contrib/staticfiles/handlers.py", line 65, in __call__
    return self.application(environ, start_response)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/wsgi.py", line 141, in __call__
    response = self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 75, in get_response
    response = self._middleware_chain(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django_global_request/middleware.py", line 15, in __call__
    return self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 145, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 143, in _get_response
    response = response.render()
  File "/usr/local/lib/python3.7/site-packages/django/template/response.py", line 106, in render
    self.content = self.rendered_content
  File "/usr/local/lib/python3.7/site-packages/rest_framework/response.py", line 72, in rendered_content
    ret = renderer.render(self.data, accepted_media_type, context)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 733, in render
    context = self.get_context(data, accepted_media_type, renderer_context)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 710, in get_context
    'filter_form': self.get_filter_form(data, view, request),
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 642, in get_filter_form
    html = backend().to_html(request, queryset, view)
  File "/usr/local/lib/python3.7/site-packages/rest_framework_filters/backends.py", line 52, in to_html
    return super().to_html(request, queryset, view)
  File "/usr/local/lib/python3.7/site-packages/django_filters/rest_framework/backends.py", line 105, in to_html
    return template.render(context, request)
  File "/usr/local/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/templatetags/crispy_forms_tags.py", line 199, in render
    c = self.get_render(context).flatten()
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/templatetags/crispy_forms_tags.py", line 118, in get_render
    actual_form.form_html = helper.render_layout(actual_form, node_context, template_pack=self.template_pack)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/helper.py", line 308, in render_layout
    template_pack=template_pack
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 140, in render
    return self.get_rendered_fields(form, form_style, context, template_pack, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 104, in get_rendered_fields
    for field in self.fields
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 104, in <genexpr>
    for field in self.fields
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/utils.py", line 148, in render_field
    html = template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 309, in render
    return nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 309, in render
    return nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 302, in render
    match = condition.eval(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 876, in eval
    return self.value.resolve(context, ignore_failures=True)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 671, in resolve
    obj = self.var.resolve(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 796, in resolve
    value = self._resolve_lookup(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 837, in _resolve_lookup
    current = getattr(current, bit)
  File "/usr/local/lib/python3.7/site-packages/django/forms/boundfield.py", line 74, in errors
    return self.form.errors.get(self.name, self.form.error_class())
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 180, in errors
    self.full_clean()
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 381, in full_clean
    self._clean_fields()
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 393, in _clean_fields
    value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))
  File "/usr/local/lib/python3.7/site-packages/django_filters/widgets.py", line 201, in value_from_datadict
    return value.split(',')
AttributeError: 'list' object has no attribute 'split'
[12/Jul/2019 13:49:06] "GET /XXXXXXXX/?ordering=first_name HTTP/1.1" 500 59

@moseb
Copy link
Author

moseb commented Jul 12, 2019 •
edited
Loading

It's from browsing a listing where django-rest-framework generates the code.

@moseb
Copy link
Author

moseb commented Jul 12, 2019

Hi @moseb. Can you paste the FilterSet code that you're using?

To be honest, I haven't found any in that area of the code, yet. It might be all by DRF. Not sure yet.
I'm aware my bug report is not ideal but it's the best I have right now. Sorry!

@rpkilby
Copy link
Collaborator

rpkilby commented Jul 12, 2019

Your API view should have either a filterset_class or filterset_fields, or if you're using an older version of django-filter, filter_class and filter_fields.

@moseb
Copy link
Author

moseb commented Jul 12, 2019 •
edited
Loading

I found use of filter_class — so I guess you just helped me find that the migration to django-filter 2.0 was not properly done to its end. Cool!
Is there something stopping django-filter from detecting pre-2.x leftovers and warning about them, conceptually? Does it warn but I missed it? Would code warning about these be a welcome addition for a pull request?

@rpkilby
Copy link
Collaborator

rpkilby commented Jul 12, 2019

v2.0 removed a lot of the existing deprecation warnings, so you might try running your test suite against v1.0 and then v1.1 of django-filter.

@carltongibson
Copy link
Owner

Assuming this will resolve itself when you update. Come back if not.

@moseb
Copy link
Author

moseb commented Jul 19, 2019

There is reproducible demo of the issue now at https://github.com/moseb/django-filter-issue-1103-demo. Please consider re-opening this issue. Thank you!

@rpkilby
Copy link
Collaborator

rpkilby commented Jul 19, 2019

Ah, this is related to ModelMultipleChoiceFilter not being compatible with the CSV mixin, as well as the in lookup. Two separate but related issues here.

You shouldn't need an in lookup for m2m fields, since the multiple choice filter already provides similar behavior.

@moseb
Copy link
Author

moseb commented Jul 22, 2019

Hi!

Ah, this is related to ModelMultipleChoiceFilter not being compatible with the CSV mixin, as well as the in lookup. Two separate but related issues here.

Is there existing tickets on GitHub for these issues?

It sounds like making BaseCSVWidget.value_from_datadict more robust as mentioned above alone will not fix the issue? (I don't fully understand the issue yet.)

You shouldn't need an in lookup for m2m fields, since the multiple choice filter already provides similar behavior.

in allows checking against multiple values over exact. If I cannot use ìn, how do I check for multiple values?

PS: Can we re-open this ticket?

@carltongibson carltongibson reopened this Jul 22, 2019
@rpkilby
Copy link
Collaborator

rpkilby commented Jul 22, 2019

It sounds like making BaseCSVWidget.value_from_datadict more robust as mentioned above alone will not fix the issue? (I don't fully understand the issue yet.)

Possibly. If the value is already a list, it should just go ahead and return that list. That said, I don't think it makes sense to mix the CSV-behavior with the SelectMultiple widget.

in allows checking against multiple values over exact. If I cannot use ìn, how do I check for multiple values?

The ModelMultipleChoiceFilter constructs an OR query from Q objects. So, if you have a query string like /api/mymodel?m2m=a&m2m=b, you would end up with a filter call like

MyModel.objects.filter(Q(m2m='a') | Q(m2m='b'))

With the 2.1 release, the lookup_expr is now applied to each Q object, so you'd end up with

MyModel.objects.filter(Q(m2m__in='a') | Q(m2m__in='b'))

The above is invalid and would break, however it makes sense for contains and other lookups that are intended to work with single values.

So, in short, all you need to do here is use the exact lookup for your m2m field.

I think the todo here is:

  • Make BaseCSVWidget compatible with SelectMultiple by returning the value if it's already a list. Or if it shouldn't be compatible, we should at least provide a useful error on init.
  • Fix Meta.fields for m2m. Generating an in lookup for ModelMultipleChoiceFilter is nonsensical. We should also test what happens for other lookups like isnull, which expects a boolean.

@carltongibson
Copy link
Owner

Yeah, this still doesn't look like a bug but how it all works is under documented I guess...

@marcosox
Copy link

I just had this error and its explanation is buried in this thread, so to recap and save a long search for future readers:

ManyToMany fields declared in viewsets' filterset_fields must not contain in

A warning at runtime would be very appreciated.

@carltongibson
Copy link
Owner

@marcosox Good idea. Happy to look at a PR adding that!

@sennierer sennierer mentioned this issue Oct 5, 2020
Closed
tony13tv added a commit to tony13tv/django-filter that referenced this issue Mar 7, 2021
@tony13tv
When value is a list it should be returned as is
4c4d3f6 
This change should help close carltongibson#1103
@tony13tv tony13tv mentioned this issue Mar 7, 2021
Closed
@bryan-brancotte bryan-brancotte mentioned this issue Jan 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@carltongibson @rpkilby @marcosox @moseb