Handling uploads with App Engine and webapp2

I’ve been developing a prototype with Google App Engine, webapp2 and Python for a few weeks now, and I’m really enjoying it.

That said, some things have taken a bit more digging to work out.  This blog post is an attempt to explain one of those areas that I think could do with a bit of clarification — file uploads.

Accessing files uploaded via a form

To begin with, how do you actually read data from an file form field in webapp2?  Well, first make sure that your form tag includes the attribute enctype=”multipart/form-data” (w3).  This form encoding type is designed for file uploads.

If your form field is called “file”, you can then retrieve its contents in a webapp2 handler with

uploaded_file = self.request.POST.get("file")

Using self.request.POST like this is my recommended approach, but you’ll also come across examples just using:

uploaded_file = self.request.get("file")

This distinction between self.request.POST.get and self.request.get is crucial, and not very well documented.

What is the difference?  Well, the second one just gives you a string (or binary string) containing the contents of the file; the first gives you a cgi.FieldStorage object.

The FieldStorage object has some useful properties.  For example, you can do uploaded_file.filename to get the object’s filename and uploaded_file.type to get the mimetype.

Furthermore, if you access uploaded_file.file, it’s a StringO object, not just a string.  Why might this help you?  Read on…

Reading the file – CSV, TSV or just text

Getting hold of a StringO class which holds your uploaded file’s data is useful, because StringO objects can be treated like files…

# csv example
csv_reader = csv.reader(uploaded_file.file) # use the delimiter option to read tab-separated instead.
for row in csv_reader:
    # row is now a list containing all the column data in that row
    # do something with it here

# plain text example
for line in uploaded_file.file:
    # line is a string containing a line of the file
    # do something with it here

(If you’d tried the plain text example above with the result of self.request.get(“file”) (i.e. no POST part), it would have read it character by character.)

Uploading a file from webapp2 to Google Cloud Storage

Google example project for using Google Cloud Storage with App Engine has some good examples for reading and deleting files, but its example of writing files to GCS is a bit lacking – it simply creates a text file and writes directly to it.

I think it’s more likely that you – like me – will actually want to upload a file into Google Cloud Storage from an upload form.  Here’s an example of how to do it:

import os
import cloudstorage as gcs
import webapp2
from google.appengine.api import app_identity

class UploadHandler(webapp2.RequestHandler):
    def post(self)
        uploaded_file = self.request.POST.get("file")
        uploaded_file_content = uploaded_file.file.read()
        uploaded_file_filename = uploaded_file.filename
        uploaded_file_type = uploaded_file.type        
        bucket_name = os.environ.get('BUCKET_NAME', app_identity.get_default_gcs_bucket_name())
        
        # This write_retry_params params bit isn't essential, but Google's examples recommend it
        write_retry_params = gcs.RetryParams(backoff_factor=1.1)
        gcs_file = gcs.open(
            "/" + bucket_name + "/" + uploaded_file_filename,
            "w",
            content_type=uploaded_file_type,
            retry_params=write_retry_params
        )
        gcs_file.write(uploaded_file_content)
        gcs_file.close()

I hope that this article demystifies App Engine/webapp2 file uploading and processing for you.

2 Comments

  1. Name
    Posted 7 October 2016 at 2:29 pm | Permalink

    Thank youuuuu, it was drving me crazy not being able to read the mimetype. great example

  2. Sam
    Posted 7 October 2016 at 3:32 pm | Permalink

    Glad it helped! Thanks

Post a Comment

Your email address is never published nor shared. Required fields are marked *

Ready to talk?

Whether you want to create a new digital product or make an existing one even better, we'd love to talk it through.

Get in touch