Java The Hutt

OK I only posted once so far from my experience at RailsConf. I may post some more about some of the sessions I attended but overall it was a great time and I learned a few new things (namely some cool functionality that I didn't know firebug had :-)

One thing I was sporadically working on was adding a new feature to PitchWire and as I went further down the rabbit hole it became apparent that to add this new feature that it's probably best that I finally bite the bullet and upgrade from rails 1.1.6 to rails 1.2.3.

I'm actually still in the process of upgrading but I thought I'd blog about a couple of things that have tripped me up thusfar.

The first one that tripped me up was after updating rails and trying to run my tests I received tons of weird command line errors that I don't have anymore unfortunately but basically my rubygems was too old (.8.something) and I needed to upgrade to the latest (.9.something). I actually can't take credit for figuring it out. My new friend Jimmy (he doesn't have a blog yet but he should) thought of it as we were sitting there pair programming as he'd already gone through the upgrade exercise.

So after updating rubygems with:
"gem update system"

I reran my tests and was down to a total of 7 failures from 120something prior. I went to the first failure and it was an application failure. For some reason my profile update action test wasn't passing.

As it turns out the problem turns out to be my reliance on behavior that was deemed to be a bug: http://dev.rubyonrails.org/changeset/4690

I was using cascading save operations to update multiple objects on a single action because on my profile page I have properties that belong to 3 different objects, user, contact_info, and company. Now one could argue that I could either have separate pages for each or only provide an AJAX style update to the properties of these objects where you can only update one at a time. I may do that in the future but for now I'm not going to refactor my application for the sake of upgrading rails.

What does all this mean? Basically when designing your rails apps don't rely on cascading save operations anywhere. Explicitly save each object whose properties you have updated in the process of a request. And since it's not really desirable to update and save multiple objects per request the best approach is to keep the data in your request confined to a single object if you can which means using fine grained AJAX style field updates using inline editors everywhere or only showing a form that edits fields for a single object.

Unfortunately this doesn't always map very well to the business logic of your app so you may have to do something like:

User.transaction do 
  @user.save
  @user.contact_info.save
  @user.contact_info.company.save
end

Where as if you can rely on saves being cascaded down through a relationship hierarchy you can just write:

@user.save

I can't say I'm psyched to either patch rails back to the "broken" way or revamp my app to undo anywhere I'm relying on cascading saves but I'm a little scared at what else I might stumble across. Hopefully my tests are comprehensive enough to catch anything else, I'm glad they caught this one.

Be careful out there.

Many geeks have descended upon an unsuspecting Portland for a few days of geekery about some software thingy most have never heard of. Many of these poor luddites have never even heard of Twitter and appear uncertain of the herds of geeks wearing shirts that say "[Skip Intro]", "He broke the build", and "Powerset". I think I overheard some guy asking one of the crew if Powerset was that new gym over on 13th.

So today was Day 1 of RailsConf. I saw another blogger posting an entry titled "RailsConf Day 2". I don't know if he was working on tomorrow's posting or what but any way you slice it today is Day 1. Today was the first full day of the conference and yesterday was just all day tutorial sessions, so that means day 1. Or if you want to be a geek and make the argument that yesterday was the first day, well then you have to be a geek all the way and realize that yesterday was Day 0.

This morning started of with the big keynote. Folks were ready with their buzzword bingo cards, giddy like school kids waiting for the dirty words. Fresh off my "Geek retreat" last weekend in Vegas I went to the sports book down in the vendor area with odds on swear word usage. The over/under was 5. Counting on a kindler/gentler talk I bet $100 on the under.

Turns out DHH was a big teddy bear today and I'm $30 richer. There was no real big Earth shattering news that was revealed today, though he did lay out 9 things that are interesting that are going into Rails 2.0.

1. Breakpoints are back - ruby-debug is integrated into rails 2. The big win here is being able to put a simple statement into your action, stop the request in flight, inspect and change things and then continue the action.
   
2. HTTP perf and asset packaging -

   javascript_include_tag :all, :cache => true

   will package all js into one file for delivery. The same can also be done for stylesheets. You can also setup an environment configuration to create virtual asset servers to overcome the browser behavior of serially requesting images 2 at a time from the same host. Read more at Chad's blog.
   
3. query cache - AR has query caching that you get for free and works by matching exact SQL calls. You don't need to turn this on it just works. It will be interesting to see how big mongrel processes are going to get as a result of this.
   
4. action.mimetype.renderer - separate the mime type from the template type so that you can request a file type and it can be rendered easily by any template type without a special naming convention. Examples he gave: people/index.html.erb people/index.xml.builder index.rss.erb index.atom.builder
   
5. config/initializers - To slim down the environment configuration you can put ruby files into the "initializers" dir that get run automatically on init
   
6. sexy migrations - flip name -> type declaration to type -> name which can allow you to declare more than one column on a line and simpler timestamp declarations. Check out Chris' post here
   
7. HTTP auth - Basic auth plugin is moved to core.
   
8. The MIT assumption - default license file for plugin generation
   
9. Spring cleaning - removal of deprecated stuff
   

Of all the things he mentioned I think breakpoint being back is probably the best news for rails devs, especially those that are new and trying to learn the framework. The query cache is interesting as well. It's appears simple and will likely help perf for many apps which do the same queries again and again, especially for those of us that load the user data on each request from the id stored in the session. He didn't show how to expire entries in the cache though.

The keynote demo displayed the restful stuff and he mentioned that you get search for free with the stuff in edge. The convention that is put forth with the REST approach will make writing and reading existing app code that much easier to deal with.

The talks have been very good and the ones that are popular are ridiculously packed. I approached the "Standing on the shoulders of giants" talk which was a guided tour through the code of several ruby/rails projects that were considered to be good code examples.

At the entrance a couple of middle aged ladies were guarding the doors and fending off hordes of geeks salivating at the code porn being shown just beyond those doors. I made my way through the crowd to the front and one of the lady's was like "Sorry it's full". Hmm.... but I'm here as press damnit, I have an obligation to the geek community. I figured "what the hell?" and I asked "Does this get me in?" as I lift up my press pass. "Nope". OK, time to move on to "bouncer" number 2. "Hi, I'm press and I'd like to get in". "OK go on". Sweet! I'm in! Holy crap there's a ton of people in here! Luckily I found a seat and we proceeded to be taken on a tour of Rick Olson's handywork for the most part. Adam did a good job of walking through the code and picking out good examples. He also did a good job of describing his process of "code spelunking" as he put it. I don't think I need to tell all you SouthPark fans out there why that's funny.

Alright I'm off to see about a BOF or if those Powerset guys are wandering the streets of Portland helping Rocko find his gym. If only there were something out there that Rocko could use, in his own natural language to tell him where to find his gym. Oh well, I'm sure something's bound to come out soon, like this year. In the meantime it's very gracious of them to perform user searches with such a personal touch.

I still remember the first time I was exposed to groovy. James Strachan stood on stage (I think it was 2003 or 2004) and with his very smooth and soothing voice dazzled everyone in the room with this new language that allowed you to get more done with less. In the period since that time groovy has had its ups and downs but lately it really seems to be coming of age since there's finally a release (1.0 was released in January of this year), there's the grails project, and the popularity of ruby and rails has at least raised the awareness of dynamic scripting languages to new levels. For java developers that haven't made the venture outside the confines of Java it appears now is as good a time as any to put a new weapon to your arsenal. Groovy is certainly a natural choice for many java developers that don't wish to learn a new API with ruby though there's quite a bit of the groovy API that extends java that appears to be at least partially inspired by ruby.

Today I attended Rod Cope's "Advanced Groovy" talk. So while I'd never written a line of groovy until today (I copied Rod's numerous examples for today's post) I felt up to the task of attending an "advanced" talk based on the level of most talks at JavaOne. It turned out I was right, as Rod spewed out code example after code example I felt quite at home with a few minor syntax differences that I will say are less "beautiful" than ruby but that's probably because I'm used to ruby now. But you could take any ruby developer, give them groovy and they would have no problems working with it based on the examples I saw in today's talk.

Let's look at some of the advanced features from Rod's examples:

Safe navigation - Allows safe traversal without getting NPEs and avoid doing null checks. When you call the property with "?" if anything in the chain is null the result is null.

people = ['rod' : ['age' : 36, 'height' : "5'9"]]
people.rod.age
-> 36
people?.joe?.age
-> null

Expando - a dynamic etch-a-sketch object that you can add methods and properties to on the fly

import groovy.util.Expando

rod = new Expando(name: 'Rod', age: 36)
rod.drinkWater = {num -> num.times { println "yummy" }}

print rod.age
-> 36
rod.drinkwater(2)
-> yummy
-> yummy

Template engines - A closure as template engine

t = {p -> "${p.name} is ${p.age()}" }
rod = new Person(name: 'Rod', age: 36)
println t(rod)

He also mentioned the existence of a GStringTemplateEngine. groovy.text.GStringTemplateEngine

Default parameters - just like ruby it supports default param values in method signatures

Single object iteration/identity. You can iterate over any object whether a collection or not. If it's not a collection you'll get back the object itself once in the block.

currentCustomer.employees['joe'].manager.secretary.each { it.bonus = 1000 }

With identity it's a way to have an implicit object set in the block. I'm not really clear on why this is such a big win at the moment.

currentCustomer.employees['joe'].manager.secretary.identity { 
println "salary = $salary"  } <- same as saying secretary.salary

Currying - This is the process of using functions to manufacture other functions.
First a groovy example:

c1 = {a, b -> a + b}
c2 = c1.curry("Hi ")  result = c2 = {b -> "Hi " + b}

Now let's see the same thing in ruby for folks that have never heard of currying before. The following code isn't mine it's taken from the the ruby quiz site.

require "curry"

scale = lambda { |size, object| object * size }
puts "3 scaleded to a size of 10 is #{scale[10, 3]}." # 30
puts

# curry some functions
double = scale.curry(2)
triple = scale.curry(3)
halve = scale.curry(0.5)

puts "4 doubled is #{double[4]}." # 8
puts "1 tripled is #{triple[1]}." # 3
puts "Half of 10 is #{halve[10]}." # 5.0

Dynamic language extensions - mixins. Of all the things that were shown the implementation of mixins made me cringe because you have to create a new class and then pollute your implementation code that uses the mixin with the "use" method. The ruby way is seamless and much more elegant to me but the groovy way is arguably a little safer because it's so explicit and you'd always know that you're dealing with mixed in implementation. Still, I don't like it :-)

class PropertiesHelper 
{
  public static List getProperyNames(Object bean)
  {
    def methodNames = bean.class.methods.name.findAll { it.startsWith('get') }
    def goodNames = methodNames - ['getMetaClass', 'getClass', 'getProperty']
    def propertyNames = goodNames.sort().collect {
      // "getName" -> "n" + "ame" -> "name"
      it[3].toLowerCase() + it[4..-1]
    }
    return propertyNames
  }
}

rod = new Person(firstName: 'Rod', lastName: 'Cope')
use(PropertiesHelper) {
  for (prop in rod.propertyNames) {
    println "${prop} = ${rod[prop]}"
  }
}
-> firstName = Rod
-> lastName = Cope

Method aliasing

def p = System.out.&println
p('hi')

def doSomething(method) { method('dog') }
doSomething(p)
-> dog

After these simple examples Rod broke into a couple more complex examples that really showed the power of groovy and to me rivals DHH's video that made RoR so attractive to so many people.

Demo of xml-rpc
First start up a server

import groovy.net.xmlrpc.*

server = new XMLRPCServer()
server.testme = {name -> "$name is cool"}
server.startServer(new ServerSocket(9047))
go

Then create and run the client code

import groovy.net.xmlrpc.*

server = new XMLRPCServerProxy('http://localhost:9047)
println server.testme("Groovy")
go
Groovy is cool

Going back to the server let's add a new method to our service:

server.multiply = {it * 7}
go

Back to client, let's call it:

println server.multiply(3)
go
21

No reboots were needed in the above example code which is impressive

Next up was the ActiveX demo using http://danadler.com/jacob (Java API to COM)

import org.codehaus.groovy.scriptom.ActiveXProxy

excel = new ActiveXProxy("Excel.Application")
excel.visible = true
go

This opens excel on your machine

book = excel.workbooks.add()
go

Creates a new workbook in your opened excel instance

sheet = book.activesheet
go

Selects the first sheet in the workbook

He then proceeded to add data to the sheet programmatically, create charts and add formulas (this code isn't complete as I wasn't able to keep up with this section)

a1 = sheet.range('a1'); a2 = sheet.range('a2')
co = sheet.chartobjects.add(50,50,400,200)
go

After that he demonstrated the SwingBuilder and updated the excel spreadsheet via that UI

swing = new groovy.swing.SwingBuilder()
mybutton = swing.button(text: 'click me')
mybutton.actionPerformed = { 
  a1.value = new Random().nextFloat() * 500
  chart.export('graphname.gif)
  mybutton.icon = new javax.swing.ImageIcon('graphname.gif') 
}

frame = swing.Frame(title: "The frame")
frame.show()

The above code shows the swing GUI and when you click the button the chart is updated both in excel and in the button that you've just clicked. The speed was fast with no noticeable slowness.

A couple final examples that were thrown in at the end

"cmd /c dir".execute().text -> execute a system process
Thread.start { any code } -> firing off a thread, this is just like ruby

It's great to see groovy finally coming into its own. Performance isn't bad and will only get better with optimization and for java devs that want to get into scripting or a ruby dev that needs to work in java and can't use jruby groovy's a really comfortable choice for all but the most religious ruby zealots :-)

Day 2 of Javathehutt's ruby coverage at JavaOne. Huh? Yeah that's right. Next week at Railsconf I may just have to talk about all the java related discussions that are occurring there. So what happened on day 2 that would be of interest to the ruby faithful? Well today there was "Ruby on Rails Meets the World of Enterprise Applications" given by Daniel McWeeney from Colgate-Palmolive. I walked into another pretty good sized room that was full, perhaps about half the size of the room for the JRuby session.

Daniel was a very charismatic speaker who would lift his arm in the air and bark out "demo" or "slides" to switch from one context to another. He spoke about his company's need to create a more user friendly interface to an SAP system. The system is a project/resource management system where they create projects and then assign resources (people, for those in the non-enterprisey world). The current interface is pretty unfriendly to the untrained user and as a result they weren't seeing a satisfactory level of usage of their SAP system.

So it was the job of our hero Daniel and his crack team of young energetic developers (actually I think it was just him) to figure out a way to come up with an interface on this system that would be easy to use, do it on zero budget, and with minimal time. He said that when he started he had zero knowledge of ruby and rails, and while it by no means a knock on him it actually was pretty easy to see that it was his first rails app from looking at his code as well as some of the UI that he'd built. He did a pretty good job of walking through his code and explaining exactly what was happening at all times. He made heavy use of RJS and drag and drop, and while there is certainly room for improvement he generally did a pretty good job.

One thing he made use of that was new to me was a gem called SAP4rails that he used to make calls into SAP to read/write the project data. It looked pretty straightforward to use and I hope I never have to use it :-)

In non-ruby related news I attended a press/analyst lunch panel on international developer perspectives where I learned that the entire country of Brazil loves java and that the guy from the Philippines was surprised at how much a developer in Singapore makes and was thinking of flying there to work every week.

I wanted to attend the prototype, scriptaculous, and rico talk but the line was too long and well, I'm likely to already know much of what they're talking about so I skipped out on that and headed to the press room to linger and talk to some journalists and espouse the great benefits of PitchWire to them. I actually ended up meeting a couple of coders in there including Tim O'Brien who blogs for O'Reilly and is into rails development. I gave him a quick rundown of my sortable table plugin and he immediately identified need in his current apps. Always nice to build stuff that people use.

Later in the day Tim and I headed over to the blogger meetup and then the Google party. I'm going to defer to Hani's coverage which is pretty on target :-)

I will say that it was great to run into Josh Bloch again after meeting him at last year's JavaOne. We chatted about his API design talk and he also disclosed to Cedric and myself that he had now written his first line of ruby code this week :-) Can "Effective Ruby" be far away?

Oh and top quote overheard during a press event from a member of the press "I was a college professor and I graduated students that could neither read nor write"

Last year Sun latched on to JRuby and hired the developers that work on the project mostly because of the astuteness of Tim Bray. It's been amazing to watch this project grow from relative obscurity to headliner in such a short amount of time. Yesterday as I made my way through the halls at Moscone to attend the JRuby talk I was thinking to myself "It shouldn't be too bad finding a seat here, I mean this is "Java" one, not ruby or jruby one". While it wasn't difficult to find a seat I was surprised when I walked through the doors of Gateway 104 to find a large room that was full for this talk. A quick estimate put it somewhere in the neighborhood of at least 1,000 people which is way more than I would've thought going in.

The talk itself was really very introductory. Charles and Tom assumed no prior ruby knowledge and even asked the question "Who here doesn't know what ruby is?" As I thought to myself "there's no way... ", I looked around and saw at least 5-6 hands raised in just my visible radius. I didn't stand up to see the aggregate total of folks who wandered in seemingly off the street with no idea of what they were attending. Later in the afternoon I mentioned this to Hani and we were theorizing about how that could happen. We proposed various reasons as to how someone could be attending a talk about a different programming language and yet have zero idea of that fact.

Were they plants in the audience by Sun? Charles' and Tom's relatives? Though I didn't see anyone with a "We're Nutter about you Charles" sign or anything like that. Though I did see one poor soul holding up a "Vote for Sanjaya" sign. I'm pretty sure he had his hand raised during the "Does anyone not know what ruby is?" question. We finally settled on the notion that those people had lost a bet. "OK if you don't get her number you have to go to the JRuby talk."

Charles and Tom did a great job breaking down the features in ruby that would be new to java developers and what makes them interesting, modules, blocks, duck typing, etc. They went over rails and its pieces, calling ActiveRecord an "ORM on steroids", which I can't say I agree with. I'd call EJB3 or Hibernate or Kodo an ORM on steroids. ActiveRecord is just an ORM.

I'm really pleased with the great lengths the JRuby team has gone to make it easy to get working. Simply download the package and run jruby from the command line and pass it your ruby script.

During the talk they provided a simple example of a JRuby script that was something like the following:

require 'java'

a = java.util.ArrayList.new
a << 'all your code belong to jruby'
a.each do |el|
  puts el
end

Pretty neat stuff. As a ruby developer that's as seamless as one could expect, and for java developers it's great insurance that they don't have to leave their world behind. While I didn't really have a look at perf on the simple example above appeared to be on par with the regular ruby VM.

During the presentation they created a sample rails app and demonstrated migrations and scaffolding. First they showed the sample app running using webrick, and then they created a WAR file and deployed the same app to glassfish. Unfortunately they only showed the app load once and didn't actually use the app at all as I would've liked to see if there was any noticeable performance change even for a single user in development mode, but still an impressive nonetheless.

They demonstrated portions of the "Goldspike" project. This is the project that contains the tools required to build a WAR file containing your rails app and all dependent gems. Basically you create a ruby file that describes your dependencies and you run the tool over your app and it builds a deployable WAR file that you just drop into glassfish, tomcat, etc.

The only caveats that I can see right now are probably still performance and usage of native libraries, like RMagik. Charles indicated that you'd run into problems with such a library and that there are plans to port them.

Roaming around the pavilion floor I had a chat with the guys over at Appistry about their application "fabric". In a nutshell it's a deployment framework that allows you to scale your applications while you kick back and work on learning rails/grails/sails/snails or level your rogue in WoW. I also stopped by and spoke with Layer 7 Technologies about their XML appliance. I have to admit before I went over and talked with them I didn't think I'd find any of it interesting because I just lumped it in as part of the WS deathstar debacle but they were really pragmatic and intelligent guys. They clearly position their appliance as something for the enterprise and for folks that have fairly complex systems of web services that they need to secure and have a single point of management. Their product basically serves as an entry point and central managing system for an enterprise's services.

After the day's activities I met up with some friends over on the Embarcadero for dinner. The conversation was lively, geeky, and engaging as always. One topic of note was the revelation of companies in London that look to outsource their development work to San Francisco because of how "cheap" it is compared to hiring developers in London. I sure hope they don't forget to account for the language barrier in their cost analysis ;-)

After dinner we strolled over to the Tangosol party where there was an open bar. Congrats Cam and his team for a well deserved exit and thanks for the great party! Wednesday night is the blogger party at Thirsty Bear and the Google party. I'm on my way to a session to be named right now. Hopefully it'll be about something completely foreign to me so that someone else can shake their head in disbelief at my ignorance.

After yesterday's post about fine grained column updates with update_attribute I was discussing the matter with Sam Pullara and Patrick Linskey with regards to how Hibernate and Kodo deal with these kinds of updates.

First off as a reminder of the data and scenarios I'm talking about:

class Reservation < ActiveRecord::Base
end


# end user action
def edit_reservation
  reservation = Reservation.find(params[:id])
  reservation.update_attributes(params[:reservation])
end


# admin only action
def confirm_reservation
  reservation = Reservation.find(params[:id])
  reservation.update_attribute('confirmed', true)
end
  

The desire is to be able to update the "confirmed" field without saving the rest of the record. The update_attribute that is part of ActiveRecord::Base updates the field and saves the whole record which seems wrong. I wanted to be able to "cheat" and not have to add either optimistic concurrency or a new table to support the notion of confirming a reservation and only allowing an admin user to do so.

Speaking with Patrick he said that Kodo doesn't allow for explicit partial object flushing as it is fraught with peril. He asserted that optimistic concurrency is the only way to go and it's really cheap. He actually went as far as to suggest always having an optimistic lock column on all tables.

Yesterday I said that I felt it was too heavy handed but as I was lying in bed last night at my new apartment, kept awake by the unknown man living above me clearing his throat with great vigor every 20 minutes or so, just long enough intervals to doze off and be abruptly awoken at the next "clearing", fun... What was I saying? Oh right, heavy handed optimistic concurrency.

After thinking about it more I realized that Patrick was right (Wow imagine that, the guy who's been focused on O/R mapping for the past 6 years or so :-). Even doing fine grained updates like I talked about yesterday isn't a good solution to my particular problem and is probably something that you shouldn't be doing at all unless you know that you're never going to have concurrent editing (and we know how often application requirements change so this probably isn't a good assumption to make and bake into your app). I was so concerned with not forcing the end user to deal with OC errors and thinking "well I just need to update this flag that's not edited by the user at all", that I failed to remember that because AR always saves all values that I still have a race condition, only the other way now, where a user could potentially change the "confirmed" value that I've so carefully set with my tricked out update_attribute method.

So my choices really are:

1) Add an optimistic lock to the table which has the undesired effect of possibly forcing a user to retry their edit, or
2) Create a new table for the "confirmed" flag which would only be editable by the admin users.

I think that for my case I'm going to add a new table because I really don't think it's appropriate to either have end users retry an operation that should always work, nor do I think it's a good policy to just try again on their behalf if it fails the first time. The "confirmed" flag is really an admin function and only used internally to the company so while it feels like a bit much to create a table for just that flag I may have to, but will reevaluate if there's an appropriate design that's a little more practical and not so "wasteful".

Anyway, for the folks that are doing single field updates, don't believe that the patch I created is something that's going to safeguard you against concurrency because it won't. It's the wrong solution. The update_attribute ability doesn't even exist on other O/R mappers and I can see why more clearly now. They get around a problem like this by offering users the ability to define lock groups, where only certain fields are used for optimistic concurrency, and more.

ActiveRecord isn't as sophisticated so if you're going to use it know your limitations in terms of what you should or shouldn't do when designing your schema as it relates to your app's requirements. So in the end it really doesn't matter that update_attribute saves the entire record by default because if you're desiring a fine grained update you're likely asking for trouble at some point. Time to unhack my hack :-)