This is part 5 of my ongoing series on Ruby on Rails performance. Today’s topic is a bit more complicated that some of the previous parts.
I’m going to explore an alternative to using the method ActiveRecord::Base.to_xml.
Before I get into that, I probably should explain why. After all, ActiveRecord::Base.to_xml is really easy to use. The alternative I’m going to demonstrate isn’t very easy to use.
The problem is that ActiveRecord::Base.to_xml can be really slow—even after making the optimizations I’ve explored in the previous parts to this series. For fairly simple AR objects, this won’t be a problem. However, when you have a deep object hierarchy and need substantial portions of it to be included in the XML response, it becomes a problem.
One instance where I encountered this was a nested has_many tree. That is, the object tree looks something like this:
ParentObject has_many ChildObjects
Each ChildObject has_many OtherObjects
The XML output is often 100k or more, and representing a few dozen or more total objects.
In part 3 of my Ruby on Rails Performance Series we’ll look at how to improve the parsing speed of XML content.
Rails will automatically parse XML in two key places:
1. When XML content is sent to a controller/action pair via POST or PUT. (Your app is an API server.)
2. When XML content is received from an ActiveResource request. (Your app is an API client.)
The default XML parser is REXML. It’s written in pure Ruby which makes it very portable. And slow. Very slow.
So, we need to replace REXML with something faster. The key is the libxml-ruby gem which is just a binding to the very fast libxml2 C library.
libxml-ruby has a different set of methods from REXML and is not drop in compatible. If you’re planning on moving your own REXML parsing code to libxml-ruby, it will take some work. However, it’s worth it. In one particular project, I had about a 100x speed improvement!
Rails, as it often does, makes the switch to libxml-ruby super easy. First, install the gem (requires a functioning compiler, the libxml2 library, and headers for libxml2):
gem install libxml-ruby
Then, just add a new file to config/initializers/. Let’s call it config/initializers/xml_config.rb. In that file, put this:
ActiveSupport::XmlMini.backend = 'LibXML'
This was only added in recent versions of Rails, I believe in 2.3.
Restart your Rails app server(s) and off you go!
The standard way to build XML in Ruby is with the Builder gem.
Builder iterates over each character in the output text to determine if it needs to escape it. When generating larger XML documents, this can be quite slow.
Thankfully, the solution is already available via the fast_xs gem. This gem does compile an extension, so you’ll need a working compiler.
gem install fast_xs
Then just require it after Builder:
require 'builder'
require 'fast_xs'It will be used automatically by Builder going forward.
If your app is built on Rails, it’s even easier. Just install the gem—that’s it! Recent versions of Rails will auto-detect the existence of fast_xs and enable it.
Lately I’ve been doing a lot of API type work in Rails — both client and server. It mostly uses Rails’ default XML generation and consumption.
Along the way I’ve discovered that the default performance of XML and related things on Ruby and Rails is, shall we say, less than amazing. On the bright side, there are a number of ways to improve matters.
So, I’m going to do a series on improving the performance of Rails’ API-related stack.
I’ll start off with serialized columns in ActiveRecord.
Rails 2.3 made a lot of changes in how requests are handled. Most (all?) of this has to do with its new support for Rack. Anyway, all of these changes left Rails 2.3 incompatible with the last version of the Limited Sessions plugin.
So, here’s a shiny new version of Limited Sessions to go with your shiny new version of Rails.
I updated my limited_sessions plugin to improve support for Rails 2.1. It should maintain backwards compatibility.
The change surrounds Rails 2.1’s new partial updates support. Basically, it’s turned off for sessions so the session is kept current and doesn’t expire if the user is active.
it’s time to announce my second plugin for ruby on rails, limited_sessions.
it’s been publicly available for several days, so i guess it’s time that i actually talk about it.
this came out of a need to manage sessions more intelligently than rails does by default. all of these is built as an extension to ActiveRecordStore, so sessions must be stored in the db. features:
as usual, details are on the project page.
in ruby’s CGI::Session module, sessions are stored as a block of seeming junk, like this: “BMZWRlcm1hbiBCb25kaW5nIENvb”. it’s actually an encoded format which is all well and fine until you need to read something out of it for debugging purposes.
if you are using rails’ ActiveRecordStore, the contents of a session can be read fairly simply. since this relies on an AR model called Session, which your app most likely doesn’t have, we’ll create that too.
so, fire up script/console and input the following:
class Session < ActiveRecord::Base ; end
CGI::Session::ActiveRecordStore::Session.unmarshal(Session.find(:first).data)that will dump the contents of the first session. :first can be replaced with any valid option to AR’s #find method.
dump all of the sessions with something like:
Session.find(:all).collect {|s| CGI::Session::ActiveRecordStore::Session.unmarshal(s.data)}
i was recently sharing with someone how to make multiple uploads work with the attachment_fu plugin. as an aside, attachment_fu is the successor to acts_as_attachment. if you were planning to use acts_as_attachment, upgrade your plans and use attachment_fu instead.
multiple file uploads are actually fairly straight forward.
first, in the view:
<%= file_field_tag 'attachment_data[]' %>put as many of these in the view as desired. it’s also easy to cause an arbitrary number of them to be added via javascript by the user’s browser.
then, in the controller:
@attachment = Attachment.new
params[:attachment_data] ||= []
params[:attachment_data].each do |file|
@attachment = Attachment.create({:uploaded_data => file}) unless file == ""
endAttachment is my model for uploaded file—substitute as appropriate. as this isn’t a complete how-to for attachment_fu, i won’t get into configuring attachment_fu or the model here.
that’s it. that wasn’t so painful, eh?
i’ve been rewriting the software backend for this site. it has been my intention, once that was partly done, to begin to discuss more technical stuff here, including code samples—most likely for rails stuff.
that’s all fine and well, but you can’t just post code snippets in black and white. no, this is a technicolor world now and that would never do. so i’ve been on a quest to figure how how to make my shiny, new rails backend parse the code blocks and introduce them to my box of 64 (!) crayolas. (as an aside, i’m pretty sure i wasn’t the only one at age 7 who thought his world would be complete if only i could get a 64-color crayola set — much better than my set of 16.)
anyway, i won’t get into lots of details at this point (although i can if somebody wants me to), but in case you need to do the same thing, here are a few jumping off points.
the most common solution seems to be to use the syntax gem. you’ll need to roll your own interface between it and rails, although rails weenie can offer some help.
syntax will recognize ruby, yaml, and xml by default. if you need something more robust, UltraViolet looks interesting although it would certainly have a heavier memory footprint. it leverages textmate bundles so it should be able to interpret/colorize nearly anything that is even semi-common, and then some.
i’m sure there are more options—add them to the comments if you know of something.