File Management
One of the most common features for web applications I’ve built over the last 4 years doing rails is file management. Users download file attachments in almost every web application I use. Thankfully rails has a really capable suite of file management tools, and there are several great plugins to handle some of the more mundane functionality you’d need.
Over the next month or so I’m going to cover the set of techniques I use when building file management solutions for my clients, and some really exciting up and coming solutions which solve the last of my annoyances.
The rough order of business will be:
- File Downloads Done Right
- File Management Plugins
- Painless File Uploads
- Storing your Files
If there’s anything in particular that you’d like to see covered, let me know in the comments.

The Rails Way is all about teaching "best practices"
in 
Do you cover on upload part how to prevent overwriting of files with the same name? Or do you have a recipe how to upload the second as a second version of the former with the same name? Thanks in advance for your effort to cover the topic in general.
Great idea! Can’t wait.
I would like to know what is “the rails/right way” to manage temporary files uploaded on a request but assigned to AR models later on. A couple usage examples could be:
- the typical “preview” functionality when creating/updating a model with a file attribute: some sites forces the user to upload a file twice but keeping the file on a temporal state (without actually assigning it to any database-based model) can skip the second upload.
- submitting a creation form including a valid file but some invalid text fields: the user must submit the form again to provide valid text fields and sometimes is forced to upload the file a second time since no model has been stored on database on the first request.
I usually manage this situation by using session data, temporary flags on database, etc but I would like to know if there is some well-stablished pattern to solve this kind of problem.
Nice – looking forward to the painless uploading part.
I usually use attachment_fu and it makes all the usual picture stuff (like thumbnailing) easy, but I haven’t found/developed my own patterns for doing stuff like uploading a new picture to override the previously uploaded and so on.
Also I feel that changing things that are related to file attachments, like sizes of thumbnails or association from one-to-one to one-to-many, take special effort compared to other tasks.
Looking forward to this series. Great topic!
Let’s start with file uploading please =)
I’d like to see how to secure uploaded files to keep certain people from downloading them. A before_filter checking for authentication doesn’t seem like the right solution because an anonymous user can directly guess at the URL, bypassing the filter.
Maybe I’m just paranoid? Please enlighten me.
@Mike
You could always store the files outside of the public HTML directory, and then have Rails serve them up with x_sendfile. The request then can be routed to the rails app, which can check permissions, etc., but the file streaming is handled by the web server itself.
Ideas about producing reports or other cpu time-intensive files (eg rendering) would be appreciated. At first my app would grind away and then return the file. But that blocks the thread (and other users).
So my new method is to queue a job using delayed job, and then email the completed file to the requestor. Using email avoids issues of holding the completed files, timing them out, deleting them, etc.
Thoughts?
1. A progress bar that doesn’t suck
2. Upload a directory of files.
3. Checkboxes to tick off the items you want to upload within a directory (think facebook upload)
I’m about to start implementing file attachments in a project I was working on. It would be nice to see how Paperclip can be used for multiple file attachments instead of the tired examples of uploading user avatars.
Also, with Rails 2.3, will the nested forms support greatly simplify the way attachment forms can be placed on forms of their parent objects? I’d like to see that as well.
Maybe some javascript tricks on how to upload multiple attachments at the same time (Basecamp style) would be cool too.
Sounds good!
I’d be interested to see bulk uploading and attachment_fu playing together nicely.
The whole thing sounds like an interesting read anyway! Looking forward to it whichever part you tackle first.
Where to store uploaded attachments? Database or not?
Awesome! Can’t wait to read’m.
My wish was … how to easily create a stable file upload progress bar.
I would love to hear about files in a distributed environment with multiple webservers and/or using S3 or others for storage.
Storage of the attachments on AWS S3 and a tied in user system that is good for the use case where a service provider might want to provide client access to the files in a protected private client area.
Looking forward to the series!
Thanks, Kent
Progress bars that don’t suck sure sounds good.
I managed to use SWFUpload to POST directly to S3 and wrote some javascript to keep my rails app in the loop about which files were uploaded (using either AttachmentFu or Paperclip). I looked at a couple of the plugins that attempt this, but they either use a modified version of SWFUpload or are very simple and don’t solve the multi-upload scenario that is common nowadays.
Using SWFUpload as a replacement for a normal file upload sounds good at first, but when you use S3 for storage you find out that it actually doubles the upload/processing time for your users, since the file must first travel to your server and then passed along to S3. So, if it takes 5 mins to upload a 100MB file using normal HTTP, SWFUpload’s progress bar will work for the initial upload, but the site will appear to hang for the next few minutes while the file is pushed over to S3.
However, POSTing directly to S3 does not work if you’re wanting to do any sort of manipulation (for example, resizing / thumbnailing of an image) since your Rails app is completely out of the loop, but it’s a great solution if you’re simply accepting uploads and storing them as is.
I’ll be interested to see what you propose!
I look for forward to the painless uploads too. 2 things that would be awesome to see is a painless upload progress bar and dealing with the tmp files stored. tmp files or images can take up a lot of space after a while.
I would be interested in hearing about how to deal with testing of file upload. In particular, how to segregate files that are added/uploaded during testing, from those uploaded during development, when both testing and development use the same code.
Of course, this is only pertinent if files are stored in the file system, not the database.
Related is how to clean up files uploaded during test. Our database is typically cleaned up using transactions, but this does not remove the files from file system.
Base-camp style attachments would help my web application designs quite a bit.
Anyone using Amazon S3 who is interested in uploading multiple files (with progress bars, simultaneously) directly to S3 without hitting the server might be interested in my posts here:
http://www.railstoolkit.com/posts/fancyupload-amazon-s3-uploader-with-paperclip http://www.railstoolkit.com/posts/uploading-files-directly-to-amazon-s3-using-fancyupload