www.flickr.com
Michael Kovacs' photos More of Michael Kovacs' photos
Recommend Me Cable Car Software logo

Wednesday, January 18, 2006

Rails realities part 6 (/dev/null)

For anyone running a production site on rails today's reality tip is for you. I was in the process of cleaning up disk space and backing up production log files. I haven't setup a script to perform this task for me as of yet so every so often I'll copy off the production.log file to a historical log directory. Normally what I do is copy the file, pray nobody's hit the site, run "rake clear_logs" which truncates the log file. Well it's been awhile since I've done this and last night I forgot this procedure and just did a straight "mv production.log ../backup_dir".

The result of this folly is that today when I went to check the production.log file I saw that it wasn't there. Questioning the likelihood that the site's had zero hits since yesterday I hit the site and took a look again. Still no file. I figured that rails must not have liked my move so I needed it to realize that the file is gone and make a new one for me. I tried SIGHUP on my fastcgi processes but that didn't work either. I ended up having to kill my app and restart it to get my app logging to the log file again.

So in conclusion don't be stupid like me, always run "rake clear_logs" or better yet setup something to clear them for you daily and back them up. That's on my list of things to automate.

Comments:
The reason is because on Linux you can move or delete an open file and the process that has it open can still write to it. You should find all your logs from the missing day in the file that you moved it to.
 
Cool.. I did not know that but I have to say that the thought did enter my mind and I wish I'd have checked the file. Good to know that I didn't lose any info though.
 
I use a log roller to automatically roll the log files on a cron job every night. I recall Rails also has built in support for doing this.
 
I remember reading that rails did have something but wasn't aware that it was built in. I need to do my homework here as I've been lax :-)
 
Most of the larger production sites use Syslog. This allows you to join logs from multiple FCGIs as well. And it does automatic and clean roll overs.
 

Post a Comment





<< Home

This page is powered by Blogger. Isn't yours?