Summary of My Rails/Ackbar/KirbyBase Experiment

I’ve had several days to reflect on my experience, which was to work through the Depot application in Agile Web Development with Rails, using Ackbar/KirbyBase as a database backend instead of MySQL.

I’m very happy with the fact that it was a success. I was able to build the complete application, with all of the same functionality of the book’s app. Along, the way, there were a couple of bugs, but these were quickly fixed by Assaph.

The success of this experiment encouraged me to begin porting an existing client/server database app, which I originally wrote using FXRuby and MySQL, to Rails/Ackbar/KirbyBase. I am approximately half-way through the re-write and have not run into any problems so far.

So, this experiment did answer one important question for me: Is it possible to write Rails applications using Ackbar/KirbyBase? Yes, it is.

Granted, the application I’m porting will run on an intranet, have a total of approximately 40 users, and will probably never have more than 5-10 concurrent users at any one time.

Would I use Ackbar/KirbyBase to develop a Rails application that would have tens of thousands of hits every day? Probably not.

One of the reasons I enjoyed developing a Rails app with Ackbar/KirbyBase is that it is so easy to view the data while writing/testing code. Simply do a “cat tablename.tbl”, or open it up in your editor.

Another advantage was the ability, if I wanted to, to use one language, Ruby, for everything: write database creation scripts, write #find queries, etc. No switching back and forth between Ruby and SQL.

And probably the biggest advantage of using Ackbar/KirbyBase instead of MySQL for this experiment was the incredibly easy installation and configuration. Since Ackbar and KirbyBase are both pure-Ruby and available as gems, installing them on your box is a simple “gem install ackbar”. Setting up Rails to use Ackbar involves adding one line to your Rails environment.rb file and modifying your database.yml file to set the adapter to kirbybase. That’s it!

One thing did worry me when performing the experiment. I was using KirbyBase in single-user, embedded mode while writing the Depot application. This wasn’t an issue, since I was the only user. Additionally, since (I think) Webrick only processes one client request at a time, there would be no concurrency issues.

However, in a production setting, multiple users would be hitting the Depot app at the same time. So, I fired up a server instance of KirbyBase using the kbserver.rb script that is included in the distribution. Next, I modified database.yml to point my app at the KirbyBase server. I opened up Firefox and pointed it at the Depot app. It worked like a charm.

This is important for a couple of reasons:

  1. When you run KirbyBase in client/server mode, it takes care of all of the messy details for you like concurrent access and locking.
  2. KirbyBase has the ability to create and use indexes that can dramatically speed-up queries. When you first start KirbyBase in client/server mode, it immediately creates these indexes in memory so that they are available for use. User’s don’t have to wait for their client-side process to build the indexes upon startup. Ackbar does not yet take advantage of KirbyBase indexes (except for the built-in recno index), but it is something that Assaph and I think can be added in the future.

The Future

So, where does that leave us? Well, I know that Assaph is hard at work on the next version of Pimki, which, I think, will support both SQLite and KirbyBase. He also has a lot of ideas for improvements to Ackbar.

Some of the improvements he has mentioned would be:

  1. Have Ackbar/KirbyBase handle even more SQL syntax (for example, the ‘LIKE’ operator).
  2. Have Ackbar automatically use KirbyBase indexes if they are available.
  3. Possibly getting Rails has_one and has_many to map directly to KirbyBase’s Lookup and Link_many types, respectively.

I’ve got homework, too. There are some things I can improve in KirbyBase, like having each KBTable manage an array of KBColumn instances, instead of the kludgy way KirbyBase currently handles columns. Additionally, I am going to try to give Assaph a hand getting Ackbar to use KirbyBase indexes.

One of the other ways I plan on helping is to keep developing/testing using Rails with Ackbar/KirbyBase. Hopefully, this will help uncover and squash more bugs, and give more ideas for ways to improve both Ackbar and KirbyBase.

If you get the chance to play with Ackbar/KirbyBase, or better yet, use it in developing an actual application, please drop either Assaph or myself a line. I think I can speak for him when I say that we would love to hear about your experiences, both positive and negative.

In conclusion, I would like to mention that Assaph has been great to work with on all of this. I think his development of Ackbar is brilliant. He’s always been quick to respond to my questions and bug reports. I’m very much looking forward to working with him to make both Ackbar and KirbyBase better.

March 5, 2006. Ruby. 2 comments.

Chapter 11

Page 125. I need to create a users table. You’ve seen me do this enough, before. I don’t think I need to show it again.

Page 126. I generate the model for User. I generate all of the controllers for Login and User.

Ok, I keep following through th chapter. It doesn’t get interesting until…

Page 131. I need to create a User.login method. In it, I need to pass the name and password that the user entered, to the find method. Here’s how the book shows that method call:

find(:first,
     :conditions => ["name = ? and hashed_password = ?",
                      name, hashed_password])

Now, Ackbar has the ability to do simple SQL fragments like this one. So, I could code the method exactly this way and it would work. Or, to make the conditions look a little more KirbyBase-esque, here is an alternate implementation:

find(:first) { |rec| rec.name == name and
 rec.hashed_password == hashed_password }

I am probably a little biased, but I like the second way a lot better. :)

One more change on page 131. I need to add an Order.count_pending method. The book’s version looks like this:

def self.count_pending
  count("shipped_at is null")
end

But, as mentioned earlier, KirbyBase uses Ruby’s nil object instead. So, my implementation of this method will look like this:

def self.count_pending
  count { |rec| rec.shipped_at.nil? }
end

That’s all there is too it. The rest of the chapter, while interesting to finish, holds no code that KirbyBase could not handle.

So, I made it through Chapter 11. And, I’m left with a finished Depot Application that performs identically to the one in the book, only it is using Ackbar/KirbyBase as it’s database management system.

This exercise actually went smoother than I had hoped. A big reason is the obviously well thought out design of ActiveRecord, making it possible and practicable to substitue non-SQL database management systems. The other big reason this went so smoothly is due to Assaph Mehr’s care and dedication in crafting Ackbar. Thanks, Assaph!

Even though I accomplished my goal, I would like to post one more entry in the next few days. In it, I will try to summarize some of the lessons I learned while working through this exercise, some of reasons you might consider using Ackbar/KirbyBase in your Rails development, and some of the areas where using Ackbar/KirbyBase probably doesn’t make sense.

February 22, 2006. Ruby. 2 comments.

Chapter 10

Looks like I’m tackling shipping in this chapter.

Page 116. I need to add a “shipped_at” column to the orders table. So I open up create_db.rb and modify the create orders table code:

db.create_table(:orders,
 :name, :String,
 :email, :String,
 :address, :String,
 :pay_type, :String,
 :shipped_at, :Time)

Notice that the MySQL datatype datetime maps to the KirbyBase datatype :Time. I learned this lesson in an earlier chapter.

I now rerun create_db.rb to recreate the tables. As the book mentions, I’m tired of having to re-enter product info into the products table every time I re-create it. So, I’m going to follow the book’s lead and create a file called product_data.rb that I can run to repopulate the products table. The big difference is that my file will have Ruby code in it instead of SQL statements, since KirbyBase is pure-Ruby. Here’s what the product_data.rb file looks like:

require 'kirbybase'
require 'time'

db = KirbyBase.new
products_tbl = db.get_table(:products)

products_tbl.insert('Pragmatic Project Automation',
                    'A really great read!',
                    '/images/sk_auto_small.jpg',
                    29.95,
                    Time.parse('2004-12-25 05:00:00'))

products_tbl.insert('Pragmatic Version Control',
                    'A really controlled read!',
                    '/images/sk_svn_small.jpg',
                    29.95,
                    Time.parse('2004-12-25 05:00:00'))

In line 2, I require ‘time’, so that I can use the Time.parse method to easily create Time objects. In line 4, I grab a handle to the products table so that I can easily reference it’s insert method. Notice, also, that since KirbyBase uses Ruby data types, I am passing it an actual Ruby Float object and a Ruby Time object.

I run product_data.rb and my products table is populated!

Page 117. The book creates a pending_shipping method in the Order model that will return all orders that have not been shipped yet. Here’s what the method in the book looks like:

def self.pending_shipping
  find(:all, :conditions => "shipped_at is null")
end

Instead of “null” KirbyBase uses Ruby’s nil object to signify what “null” means in other database management systems. Therefore, to rewrite the pending_shipping method so that KirbyBase can understand it, I create the method like so:

def self.pending_shipping
  find(:all) {|rec| rec.shipped_at.nil?}
end

Notice how we use Ruby’s nil? method and enclose the Ruby expression in a block.

Next, I create ship.rhtml.

Page 118. I create the partial template, _order_line.rhtml. I pretty up admin.rhtml.
Page 119. I refresh my browser and the “Orders to be Shipped” screen appears in all it’s glory.

Page 122 I implement the ship, do_shipping, and pluralize methods in admin_controller.rb.  I add #mark_as_shipped to order.rb.  I don’t need to change anything in these method.  They work exactly the way they do in the book.  I go back to my browser and mark one of the orders as shipped. The order disappears and the flash message says “One order marked as shipped”.

Just to be sure, I go open up orders.tbl. There, in the first record’s :shipped_at field, is “Wed Feb 22 15:25:44 EST 2006″. Ackbar’s still going strong!

On to the final chapter.

February 22, 2006. Ruby. 3 comments.

Chapter 9

This chapter introduces the has_many concept, so I’m hoping Ackbar/KirbyBase don’t let me down. :)

Page 102. I need to create an “orders” table and add a new column to the “line_items” table. So, I edit create_db.rb. Here’s the code for the new “orders” table:

db.drop_table(:orders) if db.table_exists?(:orders)
db.create_table(:orders,
 :name, :String,
 :email, :String,
 :address, :String,
 :pay_type, :String)

And here’s the modified create statement for “line_items”:

db.create_table(:line_items,
 :product_id, :Integer,
 :order_id, :Integer,
 :quantity, {:DataType => :Integer, :Default => 0},
 :unit_price, :Float)

I rerun create_db.rb to make the necessary database changes. I modify the Order class to let it know that it has_many :line_items. I modify the LineItem class to let it know that it belongs_to both :product and :order.

I create the Checkout form. I create the save_order method in the store controller. I put all the little touches in like validating the presence of the Order fields, showing errors, etc.

Page 109. Now for the moment of truth. I add some books to my cart and go to checkout. I enter the order info on the checkout screen, cross my fingers, and hit the “Checkout” button. My browse goes back to the store index with a message saying “Thank you for your order.”. Great! But did it really put the order and line_items in the database? It’s easy to find out.

I change directory over to db/development and open up orders.tbl in a text editor. Here’s what I see:

000001|000000|Struct|recno:Integer|name:String|...
1|John Doe|jdoe@email.com|123 Robin Lane|po

Look’s good! Now, how about the line_items table:

000002|000000|Struct|recno:Integer|product_id:Integ...
1|2|1|2|39.99
2|3|1|1|25.99

Fantastic!

The rest of the chapter involves making things a little prettier, so it doesn’t have any bearing on our experiment.

I’m done with Chapter 9 and I’m very happy with Ackbar. Assaph has done a great job. I have two more chapters to get through. I can see the finish line up ahead!

February 22, 2006. Ruby. 2 comments.

Chapter 8

There is not a lot to blog about this chapter, since there isn’t very much database interaction in it. It gets a little more interesting from an Ackbar/KirbyBase perspective in Chapter 9. But, here goes anyway…

Page 81. In order to create the line_items table, I’m going to open up the create_db.rb script and add the following lines:

db.drop_table(:line_items) if db.table_exists?(:line_items)
db.create_table(:line_items,
 :product_id, :Integer,
 :quantity, {:DataType => :Integer, :Default => 0},
 :unit_price, :Float)

Now, there are a couple of interesting things here. One is that, unlike the book, I don’t need to specify an “id” field for the line_items table. KirbyBase automatically creates an auto-incrementing, primary key field for each table called :recno. Assaph, in Ackbar, maps the :recno field to a field called “id”, so that Rails is happy.

The second thing to point out is that you can specify default values for KirbyBase columns, as we do here for the :quantity field. Because we are specifying additional info for this field, we have to put the info, including the field type in a hash, so that KirbyBase knows that all of that info applies to the :quantity field.

Now I simply execute create_db.rb again to recreate all of the tables.

The rest of the chapter deals primarily with creating the cart model, which isn’t an actual database table.

The good news is I am able to complete the chapter using Ackbar/KirbyBase and everything works. I’m on to Chapter 9.

February 22, 2006. Ruby. 1 comment.

Chapter 7

It looks like we are now going to work on what the customer sees in our app.

Page 71. I generate the Store controller. I fire up my browser, and the default index screen shows up.

Page 72. I add:

@products = Product.salable_items

to the index method in store_controller.rb.

Page 73. We now come to the first example of how KirbyBase differs significantly from other Rails backends. The book wants me to add the following method to the product.rb model:

def self.salable_items
  find(:all,
       :conditions => "date_available <= now()",
       :order => "date_available desc")
end

The above expression uses the MySQL now() function to get the current date and time. Since KirbyBase allows you to use any Ruby expression in your query, I can easily rewrite the above method to work with KirbyBase like this:

def self.salable_items
  find(:all,
       :conditions => "date_available <= Time.now",
       :order => "date_available desc")
end

Notice that all I had to change was MySQL’s now() function to a call to Ruby’s Time.now method.
An alternative implementation of this method would be to make it more KirbyBase-esque. Since KirbyBase uses actual Ruby expressions as it’s query language, you can pass a block to KirbyBase as your query. Assaph has worked hard on Ackbar to make it easy to use KirbyBase from within Rails. Because of this, I could have rewritten the salable_items method like so:

def self.salable_items
  find(:all, :order => "date_available desc") { |rec|
   rec.date_available <= Time.now }
end

Either way, it works!
I add the code from the book to views/store/index.rhtml. I switch back to my browser. Looks good. The records I have added show up in descending date order. I add another record with a future date, go back to the index screen and the future dated record does not show up. Looks like the salable_items method is working.
Page 75. Time to clean up the look of the store index. I add the code for views/layouts/store.rhtml. I tidy up views/store/index.rhtml.
Page 76. I add depot.css to public/stylesheets. Hit refresh on my browser and now my online store looks a lot better.
That’s it for Chapter 7. This was an easy one.

February 20, 2006. Ruby. 1 comment.

Conclusion of Chapter 6

Ok, I’m going to cheat a little bit. :)

I’m going to go to RubyForge and grab the latest version of Ackbar in CVS. The reason for this is because Assaph already has put in a fix for the DateTime issue I had in my last post. Also, he has a fix in for an error that I haven’t mentioned yet: when I would click on “edit” for an existing record on my “Listing products” page, I would get a full screen error stating, “Expecting a list of IDs!”.

Anyway, both of these errors, and the error I posted about a couple of days ago have all been fixed in CVS head, so grab it if you are following along.

Back to Chapter 6. Page 64. I add the validates_presence_of code to the Product class. Switch over to my browser and try to add a new, empty record. I get the three validation errors. Great! I enter data into the fields and click “Create” again. This time Rails adds the new record.

Page 65. I open up the Product class again to add the validates_numericality_of code for :price. I test this and do indeed get an error message when trying to save alphabetic characters in the :price field.

Page 66. I add the validate method in class Product to check for a positive number in :price. I test this and it gives me an error when I attempt to save -123 in :price.

Page 66. I add in the last two validations: uniqueness of :title and format of :image_url. I test for both and they throw errors as expected.

Were making good progress!

Page 67. I make all of the changes to list.rhtml and scaffold.css. I put some images in public images. I switch over to my browser and reload admin/list. Everything looks great.

I click on “Destroy” for one of the records, and the record is deleted. I edit the description of a record, and do a “cat products.tbl” to make sure it is reflected in the database.

By the way, here’s another reason why Ackbar/KirbyBase might be attractive to you. Since KirbyBase tables are simply text files, it’s very easy to open them up and make sure changes you make in a Rails app are properly reflected in the database.

Alright. I’m done with Chapter 6 and, besides a few minor glitches that Assaph has already fixed, Ackbar/KirbyBase passed the test.

On to Chapter 7!

February 20, 2006. Ruby. 9 comments.

Chapter 6 (continued)

Ok. Back to work.

Page 61. I need to add another column to my table. I’m going to do something similar to what the book does: alter the script I have that creates the table, although KirbyBase does have an #add_column method that would work also.

So, I open up db/development/create_db.rb. I need to add a column called “date_available”. The column type is datetime. No problem, KirbyBase has a DateTime column type. Here’s what the modified line in the script now looks like:

db.create_table(:products, :title, :String,
 :description, :String, :image_url, :String,
 :price, :Float, :date_available, :DateTime)

Now, I just execute the create_db.rb script again to recreate the table.

Page 62. I run the command to regenerate the Admin scaffolding.

Page 63. I refresh the browser and find myself back at the “Listing Products” screen. I click on “New product”. I notice that the “Date Available” field shows up a the bottom of the screen. So far so good. I enter a new record, leaving the default date of today, and click “Create”. Uh, oh. Another full-screen error. It is telling me that there is invalid data in the “date_available” field. The trace goes back to the #validate_input method in KirbyBase. Obviously, the data that Rails is passing back to KirbyBase is not a DateTime. I throw a couple of debugging “puts” in kirbybase.rb and rerun. It turns out that Rails is trying to save a Time object in the date_available field, and, since KirbyBase has it defined as a DateTime, it throws up.

Well, I will need to send this off to Assaph to see if Rails is supposed to be interpreting a KirbyBase DateTime field type as a Time field type. In the meantime, a quick fix is to simply change the field type in my KirbyBase table to Time. It will work just as well for this exercise. So, I open up create_db.rb again and change DateTime to Time. Re-execute create_db.rb. Rerun the Admin scaffolding. Click “New product” again. Oops! Where’s the “Date Available” field? The “Date Available” label shows on the form, but there is no form field. This looks like another issue. It appears that Rails does not know how to render an input field for a KirbyBase :Time field. So, I send this info off to Assaph in an email.

Now, how do I temporarily work around this issue so I can keep going. I’ve got it. First of all, I change the “Date Available” field type in my table back to DateTime. Next I regenerate the scaffolding. This will make sure that Rails creates a form field for “Date Available”. But, Rails is going to pass back a Time field to KirbyBase and KirbyBase is expecting a DateTime. Ok, I open up my KirbyBase table in a text editor and change the field type for “Date Available” back to :Time. Now, I make sure not to regenerate the scaffolding. Now, go back to the “Listing Products” screen. Click on “New product”. Fill in the info. Click on “Create”. Bingo! It works. My new record is staring me in the face, including the “Date Available” data.

Whew! I’m tired again. Blogging is tough work. I’m going to bed.

February 20, 2006. Ruby. 14 comments.

Chapter 6

Ok, here we go. As I said in my previous entry, I’m working through chapters 6 through 11 of the book, Agile Web Development with Rails, building the Depot application using Ackbar/KirbyBase instead of MySQL.

Alright, page 53. I change to my work directory and type:

rails depot

And Rails creates the application.

Page 54. Instead of logging into MySQL and creating the development, test, and production databases, all I need to do is create three directories below the db directory, since KirbyBase considers a directory as a database. To create the products table, I will write a Ruby script to create it in KirbyBase. Here’s what that script looks like:

require 'kirbybase' 
db = KirbyBase.new 
db.drop_table(:products) if db.table_exists?(:products)
db.create_table(:products, :title, :String,
 :description, :String, :image_url, :String,
 :price, :Float)

I will place this script in the db/development directory and execute it to create the table.

Pages 55-56. Next, I’m going to edit the config/database.yml file to tell it to use KirbyBase:

development:
  adapter: kirbybase
  database: db/development

test:
  adapter: kirbybase
  database: db/test

production:
  adapter: kirbybase
  database: db/production

I have to do one more thing to get Rails to use KirbyBase. I need to edit config/environment.rb and add

require 'kirbybase_adapter'

to the top of the file.

Page 57. Generate the scaffold for the Admin function:

ruby script/generate scaffold Product Admin

Now, start the server:

ruby script/server

I switch over to Firefox and go to localhost:3000/admin and, BAM my first error. The error message tells me that there is a NoMethodError in Admin#index. Further, it says, “You have a nil object when you didn’t expect it!”. The trace points to line 649 in kirbybase_adapter.rb, in method #translate_sql_to_code.

I end up backtracking through three or four methods and I find the problem in the overridden count method (line 656 in kirbybase_adapter.rb). It is expecting an empty args array if we want to count all records, but Rails is actually passing in an args array of [nil, nil]. So, I modified the line to look like this:

if args.empty? or args == [nil, nil]

I restart the server, click reload on my browser, and up pops the “Listing Products” page!

I click on “New product”, fill in the info, and click on “Create”. The browser goes back to the “Listing products” screen with my newly entered record showing up. Ackbar works!

Ok, I think that’s enough for tonight. Hopefully I’ll get through the rest of Chapter 6 in my next post.

February 19, 2006. Ruby. 1 comment.

Ackbar, KirbyBase, and Rails

Hi! Jamey Cribbs, here. First time blogger. I’m the developer of KirbyBase, a small, pure-Ruby, database management system.

Recently, Assaph Mehr, a fellow Ruby developer, released version 0.1.0 of Ackbar, an ActiveRecord adapter for KirbyBase. Now, Assaph wrote Ackbar primarily so he can use it in Pimki, which he also developed.

Although I’m pretty experienced with Ruby, I am a Rails newbie. I recently bought Agile Development With Rails and had made it through Chapter 6, following along and building the Depot application using MySQL. Then, work got in the way and several weeks passed without being able to continue reading the book.

Then, a couple of days ago, as I started to play with Ackbar, I had a thought: Why not try to build the Depot application in the Rails book using Ackbar/KirbyBase as the back-end instead of MySQL?

This exercise could serve several purposes:

  1. Motivate me to learn Rails.
  2. Identify bugs-to-be-fixed/changes-necessary for Ackbar/KirbyBase to function as a useful Rails back-end.
  3. Identify potential areas where an Ackbar/KirbyBase back-end would be an attractive alternative for developers (i.e. ease of installation, small-footprint, etc.).
  4. Document configuration settings, workarounds, pitfalls encountered when using Ackbar/KirbyBase as a Rails back-end (i.e. limited used of SQL).

After convincing myself that this would be a good idea, I also decided that blogging the whole process also made sense. Rather than try to take notes and then post an email on Ruby-Talk, putting this exercise on a blog would make it more accessible and easier for others to comment on it.

My plan is to create a new blog entry for each chapter I finish in the Rails book. So far, I have completed Chapter 6 (the first chapter where you actually start to code the Depot application) using Ackbar and, other than a couple of small fixes, it was smooth sailing. This blog should be considered part tutorial, part travelogue, and part mystery story, because I don’t know how it’s going to end. Will I be able to finish the Depot application using Ackbar or will I run into a show-stoper, something that KirbyBase just can’t handle? It’s going to be fun finding out!

So, keep your eye on this space. I hope to post a new entry by the end of the weekend detailing my journey through Chapter 6 of the Rails book.

February 18, 2006. Ruby. 7 comments.

Follow

Get every new post delivered to your Inbox.