Should not need to re-parse the request to get non-encoded file contents

Bug #1116954 reported by Steve Kowalik
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Low
Unassigned
lazr.restful
Fix Released
Low
Colin Watson

Bug Description

During the fix for a critical bug, it turned out that everything is sent through encoding in Zope's publisher, except if it looks like a file upload by having a non-None filename attribute. This results in corrupted uploads via the upload. We currently work around this by grabbing the request again and getting the non-encoded data.

Tags: tech-debt

Related branches

Revision history for this message
Colin Watson (cjwatson) wrote :

This was mostly fixed as part of wadllib 1.3.3, since that now arranges to always add a filename attribute to Content-Disposition for binary parameters:

- Implement a subset of MIME multipart/form-data encoding locally rather
  than using the standard library's email module, which doesn't have good
  handling of binary parts and corrupts bytes in them that look like line
  endings in various ways depending on the Python version. [bug=1729754]

However, versions of wadllib that lack this fix may still be in the wild (we only SRUed that back to xenial, and I hear that other distributions exist too), so we may need to wait a while longer before dropping this workaround.

Also, lazr.restful.testing.webservice has a similar issue, because it sends named POST requests as application/x-www-form-urlencoded rather than as multipart/form-data. We'll need to fix it to create a multipart/form-data document in a similar way to wadllib.

Changed in lazr.restful:
status: New → Triaged
importance: Undecided → Low
Revision history for this message
Colin Watson (cjwatson) wrote :

Fixed in lazr.restful 1.0.0:

Rework ``lazr.restful.testing.webservice.WebServiceCaller`` to send named
POSTs as ``multipart/form-data`` requests. Arguments that are instances of
``io.BufferedIOBase`` are sent as-is rather than being encoded as JSON,
allowing robust use of binary arguments on both Python 2 and 3
(bug 1116954).

Changed in lazr.restful:
milestone: none → 1.0.0
assignee: nobody → Colin Watson (cjwatson)
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.