Thursday, December 28, 2006

Rails Tests With Fixtures Suddenly Stopped Working From Command Line?

Now, this is a weird one. I'm hoping someone might have a clue as to why this started.

All of my Rails tests work fine in RadRails. However, if I try to run them from the command line via `rake`, they fail. After much searching, turns out that only the tests with fixtures were failing. And after even more investigation, it turns out that the method `setup_with_fixtures` was never being called. This method is defined in `fixtures.rb`, which also aliases the `setup` method to `setup_with_fixtures`.

To get my tests which use fixtures to work again, I had to place a call to `super` inside my test's `setup` method. This kicked off `setup_with_fixtures` just fine.

Now, here's the weird part. This all works without the explicit call to `super` in RadRails. Also, apparently I don't need this call because the builtin generators don't include that call.

So, something is happening in the load order on the command line with is altering the behavior of `setup` in tests.

Great Plugin For Eclipse Finds and Opens Any File With a Hotkey

The GotoFile Eclipse Plugin is great for opening a file with a simple hotkey. I often need to open a file for which I already know the name and I don't want to browse around my project for it. It's often easier just to hit `Ctl-Alt-N` and then start typing. The plugin even supports fuzzy matching.

This plugin works in RadRails, too!

Wednesday, December 20, 2006

validates_date_time Not Ready for Prime Time

Just a warning to those that might be looking to use `validates_date_time` for Rails.

It turns out that it implements its own date and time parsing. This is separate from the current date and time parsing in Rails, which is currently handled by the DB driver. This dual reality is very confusing, and creates two different parsing logics and semantics in your application. I fought for an hour trying to figure out why '2006-1-1' was not a valid date. It was working before this plugin.

Reason? My current database, MySQL knows what '2006-1-1' is and thus was able to parse it. But the plugin doesn't know this format.

I'm not saying that I approve of the fact that the date/time conversion is handled by the database driver. In fact, I know it's not very portable. But this plugin should rely on that behavior, or be much more liberal and understand 2006-1-1, or use any existing Date or Time parsing.

Another odd thing about this plugin is that `validates_date_time` can't handle just '1/1/2006', as it is expecting a time part in there. It doesn't default to midnight.

For the time being, I'd recommend writing your own `validates_datetime` to match what you are expecting. Don't reinvent the wheel. If you need to do date parsing, there are existing options to rely on.

Tuesday, December 19, 2006

Testing Rails ActionMailers

The Exciter has a great post for testing Rails ActionMailer fixtures. It tells you to throw out the generated fixtures and tests that come from the generators, and to write very simple tests that check bits and pieces of the email's content.

Wednesday, December 13, 2006

Sometimes, You Just Don’t Need a Plugin

I loved this blog post by Obie Fernandez because it illustrates how we, as developers,
sometimes choose the more technically challenging way to do things
instead of just the simple way.

In blog entry, this developer needed to solve a problem, and ended up
writing a whole rails plugin for it. If you read through the
comments, you'll realize he took the difficult route. His problem
could have been solved by some simple OOP.

Read the blog entry, from top to bottom including the comments. I love it
because it shows how sometimes you just need to write a few lines of
code. :)

Friday, December 8, 2006

Locking Rows in SQL Server 2005

There turns out to be a few different ways to lock rows in SQL Server 2005.

The `SERIALIZABLE` tag is one of the options. Here are a few more:

SQL Server Row Locking Strategies

As an aside, Rails (and Hibernate) support object locking with database locking semantics. In Rails 1.1.x, only optimistic locking was supported. Rails 1.2 added support for pesimistic locking.

Hibernate accomplishes the row lock with:

return tableName + " with (updlock, rowlock)";

AFAICT, Rails punts on this. I can't find the locking syntax in the connection adapter. There is an add_lock! But it expects the locking syntax string unless you can use the default (`FOR UPDATE`, which of course SQL Server doesn't support). That's pretty lame on Rails' part. But anyway...

Friday, December 1, 2006

When Local is Remote in Rails, and Other Tales of Eccentric Error Enforcement

(*Note*: This applies to Rails 1.1.6)

Yesterday I was adding custom error pages that can be rendered like any other page in our application, and came across some very odd (and imho broken) logic deep inside Rails. In today's "What's Really Reird with Rails", I'll show when a local connection is not a local connection and hope you can't always believe the documentation.

To set the stage, I first need to explain the theoretical difference between a local request and a remote (or public) request. In Rails, a local request gets special treatment on error. When your request is local, you will see handy dandy debug messages full of stack trace goodness. The idea here is that if you're a local request, you must be a developer and you would want to see just what caused the error.

In contrast, a public request is considered not coming from a developer, and a user friendly error page will be displayed instead of the debug stack traces.

Just what is considered a local request? Stay tuned, for the answer may surprise you.

Let's talk about the error pages themselves for a moment. In a local request, the debug error message is generated from a template found inside the Rails code itself. Specifically, the templates can be found in `vendor/rails/actionpack/lib/action_controller/templates/rescues`. When the controller encounters an error, and it considers the request a local request, it will reach into that directory for the template to render.

For those following along at home, this logic is in rescue_action_locally in `vendor/rails/actionpack/lib/action_controller/rescue.rb`.

However, when the request is considered public (not local), Rails by default will render the file `public/404.html` (when encountering a RoutingError or UnknownAction) or the message "<h1>Application error (Rails)</h1>" on any other error. These are static files, and Rails does not attempt to render them through the normal layout process or process them in any way. The assumption here is that if you have encountered an error, you should do the most simple thing and show a static file.

You can see this for yourself at `rescue_action_public` in `vendor/rails/actionpack/lib/action_controller/rescue.rb`.

Let's review before we get to the weirdness.

1. Rails makes a distinction between local request and public requests.
1. On error, two different things happen depending on if the request is local or public.
1. With a local request, Rails internal templates are rendered optimized for developers and displaying exception details.
1. With a public request, Rails renders either public/404.html or a static error message.

Now to the Business Problem:

The default logic for displaying errors to public requests is weak at best. Display rich error pages in production mode, rendered through the layout system so that error pages look like regular DSES pages.

OK, no problem. If you've been following along at home, you'll remember that there are two error handlers in rescue.rb: `rescue_action_locally` and `rescue_action_public`. It's the later method that we want to override in order to display rich error messages.

Why not override both? We still want to have local requests to display those handy debug error messages, so we want to leave rescue_action_locally alone.

So we've go ahead and implemented our own rescue_action_public in our application's application.rb. We restart the application, generate an error, but we continue to get the debug page instead of our pretty 404 page. What they diddly??!?!

Here's where it starts to get interesting, and the concept of a local request gets a bit fuzzy.

As I mentioned above, rails makes a distinction between local and public requests. If you are testing on your own machine, you are without a doubt generating a *local* request. Not many people will dispute that. But what if you want to see your public error pages even if you are testing on your local machine? Turns out you can do this, but you have to override more methods in your application.rb. More on this soon.

Some more background on how Rails tests if a connection is local or public:

There is a configuration option called `config.action_controller.consider_all_requests_local` which should be set to false in production mode. When set to false, it forces Rails to consider the actual IP address of the remote user when determining if a request is local or public. In development mode, this configuration option is set to true, which always forces all requests to be considered local (no matter what IP address the connection actually comes from).

Our production mode, camber_production, does have this config option set to false. I thought all I have to do to test my new error pages is use another other machine to connect to my instance and generate the error. Because they weren't local (that is, 127.0.0.1) they should see the public error messages. So I bounced my server into production mode and walked over to Wenyi's computer. When I generated the same error, I saw the developer local debug error message!?!!??!!!

So how could a connection from an external machine be treated as local, even when running in production mode with `config.action_controller.consider_all_requests_local` set to false? To find out, we need to dive deeper into Rails.

We've now reached the true mystery of this adventure: When is a Remote Request a Local Request?

To explain, let's begin by looking at rescue_action in rescue.rb. This method handles all exceptions from controllers and determines if `rescue_action_locally` or `rescue_action_public` should be called. Here's what it looks like:

def rescue_action(exception)
log_error(exception) if logger
erase_results if performed?

if consider_all_requests_local || local_request?
rescue_action_locally(exception)
else
rescue_action_in_public(exception)
end
end

Notice the check `consider_all_requests_local || local_request?`. We know we've set `consider_all_requests_local` to false, so we turn our attention to `local_request?` This method attempts to determine if the request is a local one. For some reason, this is returning true even when I use Wenyi's computer (which is clearly not local). So what's going on? Here's the logic:

def local_request? #:doc:
[@request.remote_addr, @request.remote_ip] == ["127.0.0.1"] * 2
end

Hmmm, OK. It delegates to `@request.remote_addr` and `@request.remote_ip`. Those can't possibly be equal to 127.0.0.1 when connecting from Wenyi's computer, so how come this method returns true?

We have to move to `@request.remote_ip`, which is found in `vendor/rails/actionpack/lib/action_controller/request.rb`. Here's the logic:

def remote_ip
return @env['HTTP_CLIENT_IP'] if @env.include? 'HTTP_CLIENT_IP'
if @env.include? 'HTTP_X_FORWARDED_FOR' then
remote_ips = @env['HTTP_X_FORWARDED_FOR'].split(',').reject do |ip|
ip =~ /^unknown$|^(10|172\.(1[6-9]|2[0-9]|30|31)|192\.168)\./i
end

return remote_ips.first.strip unless remote_ips.empty?
end

@env['REMOTE_ADDR']
end

So what's going on here? Well, correctly, this method attempts to look for the HTTP Header `HTTP_X_FORWARDED_FOR`, which is set by a forwarding proxy such as our Apache 2.2 proxy balancer. This header is set to the IP address of the original requesting computer (in this case, Wenyi's computer). If we didn't have this header, then all requests would have a remote IP address of the proxy server and that would muck things up. So, OK, +1 to Rails for checking the right header.

But look at what it does with that header. It rejects values that are equal to the known internal IP address ranges for internal, non-public addresses. Specifically, that's the 10.0.0.0, 172.16.0.0, and 192.168.0.0 networks. If the IP address of the remote connection is first hitting a forwarding proxy (as it is on my machine) and the remote connection is coming from one of those IP ranges (which it is here at Camber Hawaii) then the address is dropped!!! So it falls back to `@env['REMOTE_ADDR']` which, (wait for it......) because we are running behind a proxy on the same machine, will always be 127.0.0.1.

And because 127.0.0.1 is considered local (and rightly so), even when I run in production mode, and even when I use Wenyi's computer, my requests are considered local.

Holy Hack Batman.

As a kicker, let's look at the docs for the remote_ip method:

# Determine originating IP address. REMOTE_ADDR is the standard
# but will fail if the user is behind a proxy. HTTP_CLIENT_IP and/or
# HTTP_X_FORWARDED_FOR are set by proxies so check for these before
# falling back to REMOTE_ADDR. HTTP_X_FORWARDED_FOR may be a comma-
# delimited list in the case of multiple chained proxies; the first is
# the originating IP.

There's NOTHING about ripping out IPs that match non-public IP address ranges.

So, how do we force showing the public clean error page? We jump back to rescue.rb and we have to override `local_request?` in application.rb to always return `false`. The nice thing about this is if you are in development mode you'll still see the helpful debug error messages. However, once in production mode, you'll always see the pretty error message.

I definitely think that removing those IPs from `remote_ip` method is wrong.

I hope you've enjoyed this mini-tour of some Rails internals. Back to development!

Disclaimer

I'm probably required to say that the views expressed in this blog are my own, and do not necessarily reflect those of my employer. Also, except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the BSD License.