Multipart Request Issue

[ulmentflam]

Hello, I am attempting to send a multipart/form-data request to a hug endpoint, however the file I am sending with the name image is not found therefore raising a 400 Bad Request exception. I am using python version 3.5.1

This is the route I am sending the multipart/form-data request:

@hug.post('/user/upload_photo', requires=token_key_authentication)
def upload_photo(user: hug.directives.user, image):
        #s3 upload code here

I am sending the request with PAW,
The request body I am sending looks like this:

POST /user/upload_photo HTTP/1.1
Authorization: KKbdeLa5pqaBNdu4j1AgrbhAY
Content-Type: multipart/form-data; boundary=__X_PAW_BOUNDARY__
Host: 192.168.99.100:8000
Connection: close
User-Agent: Paw/2.3.3 (Macintosh; OS X/10.11.6) GCDHTTPRequest
Content-Length: 13839

--__X_PAW_BOUNDARY__
Content-Disposition: form-data; name="image"; filename="294705_10201235674960027_1869713460_n.jpg"
Content-Type: image/jpeg

I believe the request is dying on cgi's parse_multipart which is returning an empty string, here in
input_format.py:
https://github.com/timothycrosley/hug/blob/develop/hug/input_format.py#L77

[ulmentflam]

I added a log statement in the decerator here:

@content_type('multipart/form-data')
def multipart(body, **header_params):
    """Converts multipart form data into native Python objects"""
    print('HITTING DECTORATOR')
    if header_params and 'boundary' in header_params:
        if type(header_params['boundary']) is str:
            print('IS STRING')
            print(header_params)
            header_params['boundary'] = header_params['boundary'].encode()
    print(body)
    form = parse_multipart(body, header_params)
    print(form)
    for key, value in form.items():
        if type(value) is list and len(value) is 1:
            form[key] = value[0]
    return form

This was the log:

HITTING DECTORATOR
IS STRING
{'boundary': '__X_PAW_BOUNDARY__', 'charset': 'utf-8'}
<falcon.request_helpers.Body object at 0x105a65358>
{} 

As you can see, when I print the form data after the parse_multipart function is called, nothing is returned. Hug is detecting that I am sending a multipart request, but something in cgi's parse_multipart is not finding the image in the body.

[ulmentflam]

It looks like instead of using parse_multipart, cgi's FieldStorage implementation seems to be the way other falcon-middleware implements multipart requests.

It looks like this issue has been addressed by falcon and they recommend using cgi FieldStorage as seen here falconry/falcon#469

There is a middleware that uses FieldStorage as well and is implemented here https://github.com/yohanboniface/falcon-multipart

[ulmentflam]

So in exploring the issue, I realized the stream object is in the body object passed into the multipart request. I made a modification to the multipart request because cgi's parse_multipart for some reason was not getting that file pointer/stream object. I passed the stream object of body into the request and the request was successful with no errors.

This is the body object in my case:

<falcon.request_helpers.Body object at 0x10616cf60>

This is the dict of the body object:

{'stream': <_io.BufferedReader name=7>, '_bytes_remaining': 13839, 'stream_len': 13839}

This is the change I made to https://github.com/timothycrosley/hug/blob/develop/hug/input_format.py#L77:

form = parse_multipart(body.stream, header_params)

By passing the stream object this is the retuned from parse_multipart:

{'image':[the image object]}

When you call body = request.stream at https://github.com/timothycrosley/hug/blob/develop/hug/interface.py#L449
Falcon upstream is giving you a falcon body object, which has a 'read' attribute. (viewable when dir(body) is called).

Hypothetically when cgi calls read on this object after it is passed to parse_multipart, it should read from the stream in the falcon object, but it does not and for some reason it will only read from body.stream.

In summery that makes this issue either an issue with cgi's parse_multipart function or a compatibility issue with falcon's upstream and the parse_multipart function. I don't know if calling body.stream is the solution, but hopefully figuring out why this issue is occurring will lead to a better solution.

[cag]

@thatGuy0923 Whoa nice. When I get a chance, I'll look into this as well. I originally wanted to use multipart with FieldStorage, so your finding a way that works is totally rad.

[ulmentflam]

@timothycrosley Do you have any input on this?

[timothycrosley]

@thatGuy0923 great work on this! Sorry I didn't get back sooner, I think your fix was the correct current term solution till we implement a more robust solution in the future (preferably, Falcon will add support for this as @cag mentioned - if they don't we can do it in a 3.0.0 release)

Your fix is now in development and will be pushed with the next release.

Thanks!

~Timothy

[timothycrosley]

2.2.0 has now been officially pushed to PYPI with this issue resolved

Thanks!

~Timothy