Java The Hutt

I've been working on adding a data field or two on some of my user objects but these fields are really only ever editable by an administrator. You may argue that such fields belong in an entirely different table. Well in the interest of keeping things simple I chose to add it to the existing user object as it is directly applicable.

Anyway, illustrating by example let's say I have a reservation system and after a user creates a reservation an admin user needs to perform some manual operations and then update the reservation as being confirmed. Meanwhile some attributes on this reservation are still editable by the end user.

Ok so let's look at the active record API and we see there's a method called "update_attribute" that takes the name of the attribute and the value you'd like to set for the attribute.

Alright so I just call:

reservation.update_attribute('confirmed', true)

Right?

Well yes that does indeed update the targeted attribute, BUT (and you knew there'd be a but otherwise there's no blog post), it also saves the entire object which updates all attributes.

Here's the active record code for update_attribute

      def update_attribute(name, value)
        send(name.to_s + '=', value)
        save
      end

The send call of course sets the attribute value and save eventually boils down to this:

      def update
        connection.update(
          "UPDATE #{self.class.table_name} " +
          "SET #{quoted_comma_pair_list(connection, attributes_with_quotes(false))} " +
          "WHERE #{self.class.primary_key} = #{quote(id)}",
          "#{self.class.name} Update"
        )
        
        return true
      end

As you can see update simply updates all attributes for the current object.
The quoted_comma_pair_list simply takes the attribute values and builds up the SQL parameter list, e.g. "id = 15, `confirmed` = 1", etc

OK so what's the problem here? Well none if you never care about concurrent editing of the data. "But what about optimistic/pessimistic concurrency?" you ask. Indeed we could use either of those approaches to solve the race condition of two users editing this data simultaneously, but in this instance it's really too heavy handed and doesn't really fit the situation.

As I mentioned at the outset we have a single column named "confirmed", that we'd like to have updated by an administrator. Updating this field would never conflict with a user editing the other fields for this object. Ideally what we'd like is to have is some way to update a single attribute's value... hmm.... perhaps an instance method that when invoked only updates the specified attribute. Hmm... what would we call such a method? Oh, I know, "update_attribute". Uh oh! Who smells an active record mixin patch? In the words of Matt Puppet "I'm about to hack a bitch". I don't think a plugin is the right thing here as I think this is a bug as opposed to a feature.

# put in lib/active_record_ext/base.rb
# append "require 'active_record_ext/base'" to your config/environment.rb
 
module ActiveRecord
  class Base 
    # Updates a single attribute and saves the record. This is especially useful for boolean flags on existing records.
    # Note: This method is overwritten by the Validation module that'll make sure that updates made with this method
    # doesn't get subjected to validation checks. Hence, attributes can be updated even if the full object isn't valid.
    def update_attribute(name, value)
      send(name.to_s + '=', value)
      # safely escape the value before we update
      atts = attributes_with_quotes(false)
      connection.update(
        "UPDATE #{self.class.table_name} " +
        "SET #{quoted_comma_pair_list(connection, {name => atts[name]})} " +
        "WHERE #{self.class.primary_key} = #{quote(id)}",
        "#{self.class.name} Update"
      )
      
      return true
    end
  end
end

The above is my new implementation of "update_attribute" that performs as advertised. It will only update the specified column value for the given object. I haven't modified this in the rails codebase and run the tests to see if there's any breakage but I imagine anything that breaks as a result of this behavior change was using the API incorrectly to begin with as it means there would be reliance on the fact that an entire object save is performed when you update a single attribute value. I can't think of scenarios where this would be required behavior, please correct me if I'm not thinking of them.

So now when you add this patch you'll see "UPDATE reservations SET `confirmed` = 1 WHERE id = 20" instead of an update that includes all attributes.

I did a search and sure enough there are some others that agree that the behavior change I've prescribed is the correct thing to do:
http://dev.rubyonrails.org/ticket/8053

There is also a recent discussion on the rails core mailing list on this topic but mostly related to the lack of validation triggering for this call. On that very thread Peter Marklund discusses his need to write a custom method called "update_column" for the very same purpose I've described here.

Anyway, enjoy and happy updating.

I've read from a couple of folks that the log file data isn't completely accurate. Well sometimes you really don't learn just how true something is until you learn it yourself the hard way.

I've been using a VPS provider and for the most part things have been pretty good. Uptime is good, throughput is good, etc. But unfortunately I was experiencing intermittent slowness with my app. And when I say slowness we're talking about going from X req/sec to 40+ seconds for a single request. The problem would appear suddenly, and with no increased traffic on my box, and then leave just as suddenly.

With the help of my VPS provider we tracked down the slowness as being tied to heavy I/O usage by other users on the same machine. There were a few times where another user's xen instance (my host uses xen for virtualization) was completely pegged and hogging all the I/O. They would reboot that user's instance and suddenly my application would spring back to life.

Looking at the logfile the time for rendering appeared to be responsible for the entire slowdown and the DB access time looked exactly the same as when the app was performing normally.

After many weeks of on and off attempts at trying to figure out what's wrong and thinking that it was somehow related to fetching the template files from disk (even though they should be cached in memory), I believe that the problem may be finally solved with the help of Evan Weaver. I was on IRC and Evan overheard (overread?) me talking about maybe having to switch providers because of the inability to throttle I/O fairly among all users on a machine.

He asked what was happening and after explaining he said he'd seen this before and asked to see my MySQL configuration. I showed it to him and explained that I had taken one of the template configurations from the MySQL installation as a start. Well as it turns out that template configuration is way off in terms of memory allocation for InnoDB (I used the medium sized deployment template).



The MySQL configuration options are akin to the cockpit of the space shuttle (or so I'm guessing) but like you've seen in many a cartoon even with all those crazy switches and buttons there's always the one or two buttons that really do something important. Basically Evan took me aside and said "Hey doc, see this big red button right here and this blue one here, just press those." For me those big buttons were:

[mysqld]
innodb_buffer_pool_size = 384M
innodb_additional_mem_pool_size = 16M 

Those two were originally set to 16M and 2M respectively. I started with the "my-medium.cfg" template as it prescribed it for systems with 128MB of RAM that also ran a webserver. I figured "well I also have mongrel processes as well as backgroundrb processes so this might be a good starting point for my 512MB VPS". Boy was that ever wrong.

I also made a few modifications recommended by the MySQL performance blog

key_buffer = 16M
myisam_sort_buffer_size = 8M
thread_cache = 16
table_cache = 256
innodb_log_file_size = 64M
innodb_log_buffer_size = 8M

Since I've made these changes (about 3 days ago now) there has been no I/O blocking slowdown. I'm running the log file analysis tool pl_analyze to look for slow actions and it's been clean so I'm hopeful that in the end MySQL running with too little memory allocated ends up being the culprit. It does make sense that this would be the case as MySQL would have to go to disk quite a bit since it's unable to cache much of anything and of course when the disk is getting slammed that's going to negatively impact performance. We'll see if this keeps up but I'm optimistic that this was the problem.

Unfortunately the production log file led me to draw some early and incorrect conclusions about where things were bottlenecking. So add another voice to "the log file stats are inaccurate" with an exclamation mark and further stating that it's not just req/sec that can't be trusted it's the breakdown of DB and rendering time as well. Time to look into railsbench.

I'm in the market for a new mobile phone and today while I was shopping on Amazon I added a phone and plan to my cart. While browsing the items in my cart I saw the following link under my phone/plan entry:

http://www.amazon.com/Regulatory-Cost-Recovery-Fee/dp/B00020KCBK/ref=dp_return_1/102-0955861-5109717?ie=UTF8&n=301185&s=wireless

Now if only they'd let you review some of the other fees that get tacked on to various utility bills and bank accounts.

I'm going on a conference bender here in late April to mid-May. This weekend was the 2nd annual SV ruby conference which was a pretty good conference, despite the influx of vendors this year (SAP seems to run from conference to conference trying to convince people to use their stuff). The most interesting talks to me were Ezra's talk on merb and the Twitter scaling talk because they are both immediately applicable to me. Merb basically is a way to write parts of your app that require more speed and don't require all of rails, RSS feeds for example. It works as a mongrel handler which basically works exactly like a servlet in the Java world.

It was great to finally put some faces with some of the names. There are really some very smart folks working in the ruby community and they are all very friendly and fun to hang out with. I'm looking forward to RailsConf next month in Portland but next up for me is JavaOne and how I'm going is an interesting story.

The other day I was pitched via my PitchWire page (I have one of course but it's not because I consider myself an influencer and receive tons of PR inquiries, though perhaps times are a changin). It was somebody from Sun's PR department who presented me with an invitation to this year's JavaOne conference as a member of the press. I was pretty happy to receive the invite and immediately accepted. I thought for sure this would be the first time in 7 years that I would be unable to attend JavaOne but lady luck smiled upon me with a free invite. Regardless of the conference itself it's always fun to go and hang out at the parties :-)

It's been quite awhile since I've had the chance to see what's really new in the Java world and this is a good opportunity to see what's happening. Of course I'm well aware of the jruby project and watch with great interest as I'm hopeful that the JVM becomes a higher performance deployment option for rails. Interestingly, a quick search through the conference catalog for the word "ruby" yielded nearly a full conference schedule's worth of sessions to attend.

Of course there are the obligatory sessions on Swing and threading as well as EJB3 and SOA. I'll probably end up in a few of the webapp sessions like Gavin and Bob's webbeans talk, I'm looking forward to the advanced groovy as well as the various rails comparison sessions including what appears to be a real world case study of rails by a large company.

So if you're there look for me. I'll wearing my press pass and walking around lookin for the hot story to blog or twitter and basically playing the part of an influencer. I'm also looking forward to meeting legitimate influencers and informing them of PitchWire and helping build the community further. The response we've received thus far has been fantastic from both publicists and influencers.

Today's opening day in MLB and while I'm sure this is finally "next year" for my beloved Indians, today is also opening day for PitchWire. PitchWire is my latest business venture, a product that will help redefine the relationship between publicists and influencers (journalist, analysts, and bloggers).

The basic premise is that influencers are pitched via email quite a bit and dealing with that volume of information is difficult for a couple of reasons:

1. A large segment of email pitches (about 65%) are not relevant to where an influencer's interests lie.
   
2. Email is broken for managing processes, like the development of a story idea between two parties.
   

You can check it out here: http://www.pitchwire.com

As you might imagine the site is built entirely on rails (1.1.6 at the moment) and makes use of some of the following wonderful plugins:

• Acts as attachment (Soon to be attachment_fu for the s3 goodness)


• Backgroundrb Ezra's fine background processing creation. I use it for processing incoming emails and file uploads.


• Foreign Key Migrations Been using this one for awhile. Used it on LoanBack as well. Very handy.


• Live tree This fantastic little plugin make displaying and selecting tree based data nice and easy. I ended up doing some javascript hacking to customize behavior I needed but this is a great starting point for anyone requiring a tree for their data.


• Rails rcov Should be required for any rails project. In fact it should just come with rails. While rcov doesn't tell you the quality of your tests it does tell you "hey buddy this section of code here... completely untested"


• Responds to parentThis little gem makes it nice and easy to perform file uploads and be able to remain on the same page and track progress and display updates.


• Tabnav Nice simple plugin that makes it easy to add tab navigation to your app.


• SortableA plugin that I extracted out of my work with PitchWire. It makes it pretty easy to get a sortable, searchable, filterable, table of data up in a few minutes. And not just for a single object you can traverse relations and sort by attributes on those objects as well. Here's my blog post about it.

I'm also using YUI grids as the basis of my app's layout. It does a pretty good job of working the same across browsers. The only problem I've run into really was IE6 and min-width which I managed to fix with a hack.

I also make heavy use of a modified version of Lightbox gone wild. I'm basically using this as a modal dialog for my app in numerous places instead of either going to a new page or showing an inline div which has it's problems when you're trying to edit form items. More about my usage in rails can be found in this blog post.

And finally I also use really easy field validation with Prototype for my client side validation. It's really easy to add your own custom validations for the fields in your form.

I've blogged about some of the above in the past but I may get into a little more detail about how to extend and use some of these various plugins if you want to achieve some of the same results that I have in the PitchWire app. In the end there's certainly more ways than one to achieve some of what I've done and in fact I may find that there may be a better way (perhaps using the YUI widget library more extensively) but as it is now I've got all the pieces to a small app framework that allows me to easily whip up editable lists using dialogs as well as sortable, searchable, filterable, paginated tables.

I looked into the various autogeneration projets (streamlined specifically) but at the time I started building my app nothing was really there yet for my needs. I suspect the landscape is much better these days and I'll have a look to see if there's anything out there that I can use in conjunction with or as a replacement for what I have now.