Large file uploads in Seaside
Uploading files in a Seaside web application is easy. Unfortunately, there is a drawback to the easiness: entire files are loaded into the Seaside backend’s memory because an instance of
WAFile contains the entire file’s contents. In many situations, loading the file contents in the Seaside backend’s memory is not necessary (for example, if the file only needs to be stored on disk) or even impossible (e.g. in the case of extremely large files).
This post details an extension to Seaside that works together with the NGINX front-end web server and its file upload module. The uploaded file is stored to disk and NGINX subsequently only passes a file reference to the Seaside backend. This off-loads the heavy lifting to the web server and prevents memory overload in the Seaside backend while still keeping close to the ease of implementation of “traditional” file uploads in Seaside.
Many of you will notice that this solution is based on work by Nick Ager, whose blog post has unfortunately disappeared from the web. Since there have been several questions on the Seaside mailinglist on this topic, I thought it would a good idea to revisit our implementation (which has been working in production for years now) and make it usable as a separate Seaside extension. UPDATE: Nick’s blog post has reappeared.
The Seaside extension to support large file uploads is in the optional Seaside-ExternalFileUpload package. At the time of writing of this post, you need to load this package manually in your Seaside3.x image. The package should work well in 3.0, 3.1 and 3.2.
NGINX with file upload module
You need to compile NGINX from source because, like many of its modules, the file upload module is not included in the binary distributions. Also, since NGINX 1.3.0 or so, you need to make sure to use version 2.2 of the file upload module. Executing the following commands should work for you, but mind you might need to pass additional configuration options to the
configure command to fit your NGINX setup.
Once you get NGINX installed, you need to configure an upload location in the server block that concerns your Seaside app. The following configuration defines that location as the path
/fileupload, which means that the file upload plugin is listening at that location. Files uploaded to that location will be stored in the
upload_store directory on the server. In our case, the files will be uploaded to
Once the file is uploaded, the request that is sent to the Seaside back end (listening at location
/) has the additional fields
path that contain the respective properties of the uploaded file. These properties will be available in the Seaside callback attached to the file upload field. Finally, the configuration also ensures all (other) fields of the form are sent to Seaside such that all callbacks of the form are executed there.
Please see the file upload module documentation for more information on these and other configuration parameters.
Example File Upload
Seaside-ExternalFileUpload contains an example component
WAFileUploadExample that demonstrates “traditional” file uploads side-by-side with the new “external” file uploads. Here is the snippet for such an external file upload (i.e. where the upload is handled by the front-end web server NGINX):
Like any form with a file upload field, you need to set it to be
multipart. Next, you need to pass the
fileUploadLocation, which is the location configured in NGINX to handle file uploads. In our case, this is
fileupload, but you can choose any name you want for that location as long as you use the same name here and in the NGINX configuration.
The file upload field tag is
externalFileUpload (instead of
fileUpload). The callback block of this field
[ :ef | file := ef ] will be invoked with a
WAExternalFile instance instead of a
WAFile instance. A
WAExternalFile contains the uploaded file’s filename, its content type and path on disk. From here on, it’s up to you what to do with the file. A good idea is to move the file to its proper location, for example.
Seaside-ExternalFileUpload package is currently a preview package. It will evolve as we integrate it further, for example by adding support for the ajax file uploads (as they are part of Seaside 3.2) and the jQuery file upload plugin. More about this in upcoming posts.
Please contact us on the Seaside mailinglist in case you need help or for any additional comments and remarks.