Debugging Ruby

Debugging Ruby Debugging Ruby code & Rails applications by example Ryan Bigg This book is for sale at http://leanpub.com/debuggingruby This version was published on 2014-10-16

This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to get reader feedback, pivot until you have the right book and build traction once you do. ©2013 - 2014 Ryan Bigg

Tweet This Book! Please help Ryan Bigg by spreading the word about this book on Twitter! The suggested hashtag for this book is #debuggingruby. Find out what other people are saying about the book by clicking on this link to search for this hashtag on Twitter: https://twitter.com/search?q=#debuggingruby

Contents General Debugging Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Debugging frame of mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1

Debugging Ruby . . Basic Example #1 Basic Example #2 Basic Example #2

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

3 3 3 5

Debugging Rails . . . . . . . . . . . . . . . . . . . Workflow . . . . . . . . . . . . . . . . . . . . . Rails Example #1 - form_for . . . . . . . . . . . Rails Example #2 - Routing HTTP verbs/methods Debugging slow code . . . . . . . . . . . . . . . TODO . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

9 9 9 20 27 30

Handling Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

31

Advanced Rails Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Advanced Rails Example #1 - Broken Devise Application . . . . . . . . . . . . . . . . . . .

32 32

CONTENTS

i

Thank you for reading Debugging Ruby. If you find any misteaks while reading this book, please email them to [email protected]. If email’s not your thing, I have a review tool that you can use too. I’ll need your email address to add you to that though. Tweet me (@ryanbigg) or email is cool. If it’s a problem to do with your code, then please put the code on GitHub and link me to it so I can clone it and attempt to reproduce the problem myself. If you don’t understand something, then it’s more likely that I’m the idiot and rushed it when I wrote it. Let me know! If you want to see some code examples and compare them to what you have, you can go view them over on GitHub¹. For now, they’re the absolute latest code from the book. This far into the book and there’s already one error. You should expect that it is not alone. It has friends and their ways are devious. They are coming after your perception of reality. Beware.

I’ve been helping out in the #rubyonrails channel on Freenode now for about 7 years and I’ve seen a wide variety of skill levels come through. I also help out frequently on Stack Overflow and my day job requires doing some of that as well. More often than not though, the visitors of the channel or people on Stack Overflow are just lacking the basic skills for debugging their code. The ability to debug code is arguably more important than the ability to write code. Writing code is easy! Just throw some things together and bash them a couple of times until it works. What could possibly go wrong? Anything and everything. Code is not perfect the first time it is written, or even the 10th time it is rewritten. The ability to debug code written by ourselves, strangers – or versions of us from the past that can feel like strangers sometimes – is a valuable ability to have. When things go wrong – and they will go wrong – having that ability allows you to dive in without fear into fixing whatever code is causing the fault quickly and efficiently so you can get back to doing more productive things. Debugging can be fun because it provides a technical challenge; a puzzle. Deciphering that puzzle is usually just a matter of walking through the flow of execution within the piece of code which is breaking, but it is sometimes hard to know where the right place to start is. Once you’ve found the place to start, knowing where to go from there can also be difficult. The solution is its own reward and if you’re anything like me you should get a nice little Dopamine hit every time you solve a bug. Or even one of those from just helping out other people by “squashing” bugs they’ve reported. Within this book are common examples things that can go wrong in Ruby and Rails code. Ranging anywhere from a typo in the code, to performance issues and nasty exceptions. This book will cover a wide gamut of how code can break, and exactly what we can do about it to fix those breakages and ensure that we know about them as soon as possible. Let’s go! ¹https://github.com/radar/debug_book_examples

General Debugging Tips The ability to debug code is arguably more important than the ability to write code. Written code is sometimes not completely perfect the first time it’s written, especially when humans are involved in the writing of that code. Humans make mistakes, and those mistakes can cost time and money. Being able to fix those mistakes in an efficient manner is a core competency to being a good programmer. These mistakes are sometimes extremely simple; a typo here, a missing bracket there, defining code outside of the correct scope. Other times they involve interconnected bits of code and tracking down the misbehaving cog (or cogs) in that wonderful machine can take some time. In this book, we’re going to go through some common coding mistakes that people make and we will look at how to solve them. The deeper and deeper you get into this book, the more complicated the examples will get. All the examples in this book will be written in Ruby. Before we get to see any code, let’s talk about getting in the right frame of mind for debugging.

Debugging frame of mind Have you ever come across a bug that’s been too hard to fix, only to have the answer come to you after you distract yourself from the problem for an extended period of time? I most certainly have. I think that just by sitting there for a bit longer that I can magically know the answer. Most of the time that this is not true. There are many other ways to solve the problem, but I focus on my current line of thinking and get caught up in a frustrating loop. The loop is thinking that I can solve the problem, attempting to solve it and then not being able to solve it. I’ve solved these kinds of problems by sleeping, having lunch², exercising or by talking people in the community and asking them about the problem. These are all common sense, but it’s worth covering them even if it is for a short while. Sleep has the best effect on my ability to debug. A good night’s sleep is easy to come by as I’m a newly married man who lives in a first-world country and I’m thankful for all of those things. Sleeping well makes me less grumpy, more rational in my thinking and generally improves my mood and outlook on things. Debugging on low sleep leads only to frustration. Similarly, eating (and eating healithly) also helps. Having to debug something when my stomach is rumbling is hard work. Staying the course and attempting to fix the problem is not the best course of action, but I still do it anyway. Grabbing something to eat helps a lot with that. If it’s some lollies, then the initial sugar hit boosts me up, but then comes the inevitable crash. Eating healthily improves my long-term ability to debug well, and so I try to do that as often as I can. ²Hence the book’s subtitle “How to find the answer to your problems in a sandwich”. Realistically for me though it would be an Indian curry, Thai dish, salad, burger or yiros. Sandwich just has a better ring to it than all those other things.

General Debugging Tips

2

Exercising has been scientifically proven ³ ⁴ ⁵ to enhance brain power (and general happiness!), and along with that comes the ability to reason better about code. Going for a walk, riding a bike or a session at the gym are all ways that I think have boosted my ability to debug code. Lastly, talking to other people is absolutely worthwhile. I’ve found the answer to many a problem by just simply explaining what’s going wrong to someone else. This technique is called the Rubber Ducky Technique. You talk to the person, they say nothing (or very little), and suddenly an epiphany happens. The answer is known. It’s a funny thing. If the answer doesn’t come by way of explanation, other people have their own brains to reason about things with and their own takes on things. The way that you debug something might be different to the way that they debug it. Getting someone else to look over the code and work with me in reasoning about a problem is, by far, the most productive debugging technique that I know of. So in summary: make sure you’ve caught up on your sleep, have eaten well, have exercised well, and don’t be afraid to speak to other people in the Ruby community about any issues you have. Let’s now dive into some practical examples of debugging in Ruby. ³http://www.psychologytoday.com/blog/the-athletes-way/201401/what-is-the-best-way-improve-your-brain-power-life ⁴http://www.apa.org/gradpsych/2013/09/exercise.aspx ⁵http://www.nytimes.com/2012/04/22/magazine/how-exercise-could-lead-to-a-better-brain.html?pagewanted=all

Debugging Ruby In this chapter we’ll cover some basic Ruby code that has some small problems with it, and figure out how to fix it so that the code works once again.

Basic Example #1 Our first problem we can cause ourselves in the irb console. Let’s start this up now by using irb and put this in it: 1

"test".join

When we type this and hit enter, we’ll see a NoMethodError exception: 1 2 3 4

irb(main):001:0> "test".join NoMethodError: undefined method `join' for "test":String from (irb):1 from .../ruby-2.1.3/bin/irb:11:in `'

This exception shows us that something exceptional happened with our code – namely a NoMethodError. The text after NoMethodError shows more detail about what happened, and then the lines underneath it show us how the script got to that point, starting from the bottom up, and that’s called a stack trace. Why are we getting this error? Well, the "test" object is an object of type String, as is indicated on the first line of our exception output with the words "test":String. If we look at the documentation for Ruby’s String class⁶, we can see that there is indeed no method called join. This error is happening because we’re calling a method that doesn’t exist. In this example we’ve seen what a stack trace is and how to read it. The example is slightly contrived, but nevertheless demonstrates how we can find what’s happening. Let’s take a look at a proper Ruby program which has another, similar bug and talk more about stack traces.

Basic Example #2 Let’s take a look at our first very simple Ruby script with an error: ⁶http://www.ruby-doc.org/core-2.1.3/String.html

Debugging Ruby

4

class Car attr_accesssor :on end

This is some fairly simple Ruby code. All we’re doing is defining a class called Car and then calling the attr_accesssor method inside that class which should define a setter method called on= where we can store a value, and a getter method called on where we can retrieve that value. We don’t necessarily care about the setting and getting just yet. All we care about is that our code works. Put this code into a new file called car.rb and try running it with ruby car.rb. Ideally, nothing should be output to the terminal, because we have not told Ruby to output anything. Instead of nothing, you should see this: car.rb:2:in `': undefined method `attr_accesssor' for Car:Class (NoMethodError) from car.rb:1:in `'

The output here is modified slightly to fit into the book. The first two lines in the above output should be one line. This is another stack trace, just like the one we saw in the earlier example. These are output by any Ruby program that encounters an exceptional circumstance and it shows us the execution flow of the program in reverse order. Let’s start from the final line of the stacktrace: from car.rb:1:in `'

This line tells us that the execution flow of the program started on line 1 of the car.rb file. The here indicates the main namespace that all code within Ruby operates in. This line of the stacktrace is not too important. All it’s indicating is that on line 1 of our program, something is executing and altering what our program is going to do. The first line of the stacktrace begins with this: car.rb:2:in `'

This line tells us that on line 2 of car.rb, we’re operating within the scope of a class called Car (). Because this is the top line of the stacktrace, this is probably where our error has occurred. In most cases, the top line of the stacktrace will point to the exact line where an error occurs. The remainder of this line is this: undefined method `attr_accesssor' for Car:Class (NoMethodError)

Debugging Ruby

5

This part of it tells us the error that’s happened. In this case, we’re trying to call a method that Ruby has no knowledge of, the attr_accesssor method. We’re trying to call that method on the Car class, but that method does not exist there. When we try to call a method that Ruby does not know about, it raises a NoMethodError exception, which is shown in parentheses at the end of the line. Our program is 3 lines long, but the third line is just an end, and those are never included in stacktraces because they’re just there to demarcate the end of a scope within the code, rather than perform a function. From this stacktrace, we can gather that the error is happening on line 2. All we’re doing on line 2 is this: attr_accesssor :on

Do you see the mistake we’ve made yet? It might be glaringly obvious or it might not be. Look at it for a moment longer. Do you see it now? The mistake we’ve made is dead simple: we’ve typed three “s”es rather than two. We’ve done attr_accesssor, rather than attr_accessor. The code should be this: attr_accessor :on

Change the code on line 2 to that now and re-run the program. It should run without showing anything. This is the standard for Ruby programs when they execute successfully. Unless we have specifically told it to output anything – i.e. with a call to the puts method – nothing will happen. Verify program success You can verify that a program has run successfully by running this command (assuming you’re using Bash or a similar shell): 1

echo $?

If that outputs 0, then everything has worked correctly. TODO: Bash error codes mention goes here.

Now that we’ve fixed that error, let’s cover another.

Basic Example #2 Look at this code:

Debugging Ruby

6

class Car attr_accessor :on def start on = true end end car = Car.new car.start puts car.on

Can you tell what it’s doing just by looking at it? It defines a new class called Car, and a virtual attribute inside that class by way of attr_accessor called on. This time, attr_accessor is spelled correctly. Remember that the attr_accessor call defines a getter method called on and a setter method called on=. After the attr_accessor, the code defines a method called start which sets on to true. It then ends the method and class definition. On the final two lines, the code creates a new instance of this class with Car.new and assigns it to a local variable, calls start which should do what it’s told, and finally outputs the value of car.on using puts. Just by looking at this code, you may expect that when it is run that it will output true. You’d be wrong. Put this code into car.rb and try running it. This is what will happen: $ ruby car.rb $

Rather than nothing being output this time, a blank line has been output. This is because the program has done what we’ve told it to do. It does not output true, even though the start method is setting on to true. This time we don’t have a stacktrace to tell us what line the error is probably on. We have to step through the program’s execution ourselves. Since there are no exceptions raised within the code, we can assume that the code itself is valid. Let’s step through the final three lines of the code: car = Car.new car.start puts car.on

The first line here creates a new instance of the Car class. That’s Ruby’s code, so that’s probably not the source of our frustration. The second line calls the start method which sets on to true. The final line just outputs the value of on, which isn’t too special a task. The problem probably lies within our second line. Let’s look at how that method is defined:

Debugging Ruby

7

def start on = true end

This code inside the method is not dissimilar to the first line of the previous example: car = Car.new

When we call car = Car.new we’re setting a local variable to the value of whatever Car.new returns. In the start method, we’re setting an on local variable to true. This is not our intention! Our intention is to set the virtual attribute for the instance to true. Beware this very fine difference! There’s more than one way to set this virtual attribute. We can go through the setter method call, like this: def start self.on = true end

The caveat of this is that the on= method may define other behaviour. This is a rare occurrence in Ruby code, but it can happen. In this case, it’s obvious that the on= method isn’t redefined to add extra behaviour, so it’s OK to do this. Another way of doing it would be to set the instance variable of the same name, like this: def start @on = true end

Doing it this way has the benefit of the code being shorter and there being no unintended sideeffects with the on= method. Using either the setter method or setting the instance variable are both legitimate ways of solving the issue with our code. Go ahead and try both ways now. They should both cause the program to output this when it’s run: $ ruby car.rb true

Great, so we’ve solved this problem. The issue here was that we were trying to set a local variable inside the method, rather than referencing the virtual attribute setter method or instance variable. With that out of the way, let’s look at the possibility of a setter method behaving badly as we talked about before. Let’s change our code to this:

Debugging Ruby

8

class Car attr_accessor :on def start self.on = true end def on=(value) @on = !value end end car = Car.new car.start puts car.on

The start method is now using the setter method, on=, to set the value of the virtual attribute called on. The setter method is overriden from the default here directly underneath that. It takes the value and negates it, and then stores it on that @on instance variable. This means that when we call on, we’ll get the opposite of what it should be: setting it to true will make it false and vice versa. Try running this code yourself now and you’ll see: $ ruby car.rb false

The on= method is maliciously negating our value, giving us an unexpected outcome. If we set the instance variable within the start method directly, this would bypass the on= method and we would see the right value. The start method would look like this: def start @on = true end

And the output of the program would look like this: $ ruby car.rb true

It’s extremely rare that a setter method will perform strange functions like this, but it still is worthwhile knowing that it can happen, just in case we come across a situation like this in our debugging experiences.

Debugging Rails Rails applications are generally more complex than one or two files. More often than not, you’ll have many different pieces of the application all working together: the routes, the controllers, the models, the views and the helpers. In this chapter, we’ll cover some examples of errors that people make within Rails applications and how to fix them.

Workflow Each section below works through one particular problem in a Rails application. These Rails applications are kept on GitHub at https://github.co m/radar/debugging_book_examples⁷ and are numbered the same way as their examples; i.e. Example #1’s code is within the directory called “1” underneath the “rails” directory. To work through the examples of these Rails applications, you’ll need to clone that repository to your computer: git clone git://github.com/radar/debugging_book_examples

Each Rails application comes fitted with RSpec and Capybara tests which will pass once the code has been fixed. We can run these tests to execute some code which wil check if the application is working. Running automated tests is just an easier way compared with manually viewing the application ourselves and saves time in the long run as our applications get more and more complex. Let’s begin with the first example now.

Rails Example #1 - form_for In this example we’ll cover: • The debug helper • How to read log output • How constant lookup in Ruby works In this first application example, we have a very basic Rails application. The thing that we want this application to do right now is for the users to be able to go to the “New Post” page without any issues, but unfortunately there’s a problem that’s preventing them from doing that which we’ll see shortly. Before we do anything with this application, let’s install all the dependencies: ⁷https://github.com/radar/debugging_book_examples

Debugging Rails

10

bundle install

Now let’s look at the automated test we have inside spec/features/posts_spec.rb inside the application, written using RSpec and Capybara’s DSL: require 'spec_helper' feature "Posts" do it "can begin to create a new post" do visit root_path click_link 'New Post' find_field("Title") find_field("Body") end end

This test will pass if it can navigate to the root page, click ‘New Post’ and see two fields, one for “Title” and one for “Body”. If we run the test now, we’ll see this error: 1) Posts can begin to create a new post Failure/Error: click_link 'New Post' ActionView::Template::Error: First argument in form cannot contain nil or be empty # ./app/views/posts/new.html.erb:1:in `...' # ./spec/features/posts_spec.rb:6:in `block (2 levels) in '

This is a typical failure output from RSpec, shown when code raises an exception. The first line of the message shows us which test is failing. We should have a pretty good idea which test that is out of the one test that we have so far. The particular exception that we’re seeing with this test is ActionView::Template::Error, and the message for that exception is “First argument in form cannot contain nil or be empty”. We’ll get to that in a moment. The two remaining lines in this output show some of the stacktrace for the error. It shows us that posts_spec.rb line 6 does something that triggers some code in app/views/posts/new.html.erb to run, and that’s where the error is (probably) occurring. According to the stacktrace, that’s line 1 of app/views/posts/new.html.erb. All this application has inside it is a PostsController which does nothing other than inherit from ApplicationController. This controller is routed to by the two routes within config/routes.rb:

Debugging Rails

11

Bug::Application.routes.draw do root "posts#index" resources :posts end

We have a view at app/views/posts/index.html.erb which just contains a link to the new action:

When we go to that new_post_path, it will render app/views/posts/new.html.erb which contains this content: Writing a new post

So that’s all we have: some routes, a controller that does nothing special, a template at app/views/posts/index.html. and another at app/views/posts/new.html.erb. Just four interconnected pieces in this application, nothing too special. Let’s jump back to our test now and see what it said when we ran it with bundle exec rspec spec/features/posts_spec.rb. 1) Posts can begin to create a new post Failure/Error: click_link 'New Post' ActionView::Template::Error: First argument in form cannot contain nil or be empty # ./app/views/posts/new.html.erb:1:in `...' # ./spec/features/posts_spec.rb:6:in `block (2 levels) in '

The first line of the error is pointing directly to the first line in app/views/posts/new.html.erb:

Debugging Rails

12



All we’re doing on this line is calling the form_for method and passing it the @post instance variable and then starting a block. Later on in that file, we’re calling some methods on the block argument, but that doesn’t really matter. The error is fair-and-square happening on this line. The most useful part of the error message is the message for the exception: First argument in form cannot contain nil or be empty

The argument that it’s talking about here is the @post variable that we’re passing. It’s claiming that the variable is nil or empty, and it probably is.

The debug helper We can see for ourselves if this variable is actually nil by removing all the code in app/views/posts/new.html.erb and replacing it with this single line:

The debug method here is provided by Rails and will output a YAML’ized version of whatever it is passed. We’re passing to it the output of the @post.inspect call. The inspect method is a method provided by Ruby which outputs a human-friendly version of the object. If you’ve ever written Ruby in irb, you’ve seen the inspected versions of objects perhaps without even realising it. Start a new irb session now and try entering these things: • • • • • • • • • • • •

1 1.inspect puts 1.inspect "1" "1".inspect puts "1".inspect [1,2,3] [1,2,3].inspect puts [1,2,3].inspect nil nil.inspect puts nil.inspect

Debugging Rails

13

We’ll see that the non-inspect versions are almost identical to the inspect versions. The inspect versions just have more quotes around them. This is because the inspect call always returns a String object, as those are easiest for humans to read. Whatever the final thing is on the line for some code entered into IRB is what will be returned in the IRB prompt. In the case of the puts calls, IRB returns nil because puts returns nil. IRB automatically uses inspect to return a humanfriendly representation of whatever is entered into the prompt. The debug method in our view will not automatically call inspect on whatever it is passed. Instead, it calls another method: to_s. In some cases this method gives back similar output to inspect. Try the above examples with to_s rather than inspect and see what happens. For everything including nil we see a string representation of that object. For the number 1 we see "1". For the string "1", we see the object again because it’s already a string. Converting an Array to a String gives us "[1,2,3]", which clearly shows us that the object is an array consisting of the elements 1, 2 and 3. Calling nil on the other hand, produces nothing; just an empty string (“”). That’s because nil is nothing. This whole explanation was to demonstrate that calling this code is not enough:

Due to the debug method calling to_s rather than inspect on the objects it is passed. We must call inspect ourselves:

This way, the debug method receives a String object already and will just output that. Now let’s see this in action by firing up a new server process with this application: rails server

Navigating to http://localhost:3000 will show just the “New Post” button. We know that the root action works because our test is not failing on that line. Clicking “New Post” now will show us the output of the debug call, which will be this: --- nil ...

We can see here that our @post variable is indeed nil, just like the error message said: “First argument in form cannot contain nil or be empty”. Now why is this? We know that the route is working correctly because we’re currently on the page at /posts/new looking at the debug message we put there. The route routes to the controller, and the controller is

Debugging Rails

14

empty. After the controller runs, the view template is rendered. Nowhere along the chain is the @post variable defined for our form_for call in app/views/posts/new.html.erb and that is the cause of this bug. Where should we be defining this variable? Not in the view itself, because it is never the responsibility of the view to collect data. That is the controller’s job, and so the code to define the @post variable should go in the controller. But where in the controller? A clue lies in the server output over in our console.

Reading log output When we made a request to /posts/new, it shows this: Started GET "/posts/new" for 127.0.0.1 at 2013-11-09 11:29:09 +1100 Processing by PostsController#new as HTML Rendered posts/new.html.erb within layouts/application (0.6ms) Completed 200 OK in 4ms (Views: 3.6ms | ActiveRecord: 0.0ms)

This text has a whole lot of information compressed into a little bit of space. Knowing how to read and interpret logs from Rails is an important skill, so let’s go through the details of this now. On the first line we have the details about the request: Started GET "/posts/new" for 127.0.0.1 at 2013-11-09 11:29:09 +1100

This shows us that we have made a GET request to the application, requesting the /posts/new path. The next two bits of information is the IP address of our local computer and the timestamp for the request. On the second line we have this: Processing by PostsController#new as HTML

This indicates to us that the route has been matched by Rails and has routed to the PostsController’s new action. The request is a standard HTML format, meaning that HTML output will be returned by this request. The third line is this: Rendered posts/new.html.erb within layouts/application (0.6ms)

This tells us that the posts/new.html.erb template was rendered within layouts/application. This means that the controller has automatically chosen to display this template for this action, and has used the default layout for the application to wrap around that template. All the rendering took 0.6ms in this case. The fourth and final line shows this:

Debugging Rails

15

Completed 200 OK in 4ms (Views: 3.6ms | ActiveRecord: 0.0ms)

This line tells us that the response was completed successfully and returned a 200 OK response. This is the HTTP status part of the response which indicates to browsers the final status of their request. We can see that the request completed in 4ms total, with the views taking 3.6ms of that time and ActiveRecord taking no time at all. The remaining 0.4ms were taken up by unknown things. Logs are written to files If you want to go back and see what happened after you’ve shut down the server, we’ll be able to do that by viewing logs/development.log, which stores the exact same data that is displayed when the server is running. The logs directory is where Rails writes its log data to, and the filename is simply the environment that the Rails application is running within. By default, that environment is “development”, so the log file that will be written to will be logs/development.log.

We know that we need to define the @post variable within the controller to make it available to the view, but where exactly? The logs tell us where exactly: Processing by PostsController#new as HTML

This line from the logs is telling us that the new action within the PostsController is being run before the view is rendered. This would be a perfect place to set up the variable, so let’s do that now by defining this code: app/controllers/posts_controller.rb

1 2 3 4 5

class PostsController < ApplicationController def new @post = Post.new end end

Is this enough to fix our test? Let’s find out by running bundle exec rspec spec/features/posts_spec.rb.

Debugging Rails

16

1) Posts can begin to create a new post Failure/Error: click_link 'New Post' NameError: uninitialized constant PostsController::Post # ./app/controllers/posts_controller.rb:3:in `new' # ./spec/features/posts_spec.rb:6:in `block (2 levels) in '

Not quite! We’re now seeing a new error which is a NameError exception. This time it’s happening from line 3 of the PostsController, which is this line: @post = Post.new

The error says “uninitialized constant PostsController::Post”, but on this line we’re not looking up the PostsController::Post constant, we just want Post! So what’s happening here? Why does it say PostsController::Post and not just Post?

Constant lookups in Ruby When Ruby attempts to look up a constant, it will first attempt to look it up within the current constant context. Because we’re within the PostsController, it will attempt to look it up there. It will then travel up the hierarchy looking for that constant until it reaches the top-level namespace. If it can’t find it there, then it gives up and shows an error message saying that it couldn’t find it in the current context. We can demonstrate this constant lookup using a very basic Ruby program: FOO = "foo" module Foo class Putter def self.put puts FOO end end end Foo::Putter.put

With this code, we’ve defined a FOO constant at the very top level. After that, we’ve defined a module called Foo and a class called Putter, and that class has a method called put which calls puts FOO. This code will search up the hierarchy, looking for the constant in the Putter class, then the Foo module and then finally the main namespace. Go ahead and put this code in a new file and try to run it. We’ll see it outputs “foo”. The constant lookup is working correctly. Now comment out the FOO constant and try running it again. We’ll see this happen:

Debugging Rails

17

foo-putter.rb:5:in `put': uninitialized constant Foo::Putter::FOO (NameError) from foo-putter.rb:10:in `'

Ruby is telling us here that it cannot find the constant any more, which is true because we commented it out! The most important part of this error message is that it can’t find the constant within the Foo::Putter namespace. Try now uncommenting FOO and moving the constant to inside the module Foo, like this: module Foo FOO = "foo" class Putter def self.put puts FOO end end end Foo::Putter.put

When we run this code again, we’ll see that it works just the same as if the constant was defined in the main namespace. The code will work also if the FOO constant is inside the class: module Foo class Putter FOO = "foo" def self.put puts FOO end end end Foo::Putter.put

This should demonstrate quite well how constant lookup works within Ruby. Let’s go back to solving our new problem, the “uninitialized constant PostsController::Post” message, armed now with our new knowledge of constant lookup.

Creating the Post model We need to define a Post constant within the application for our test to be happy. The best way to do this would be to generate a new Post model, which we can do with this command:

Debugging Rails

18

rails g model post

Along with this model comes a migration to create the table. If we attempt to view our application without running this migration, we’ll see this error when we make a request to http://localhost:3000. Migrations are pending; run 'bin/rake db:migrate RAILS_ENV=development' to resolve this issue.

We can fix this by running the command that it tells us to: bin/rake db:migrate RAILS_ENV=development

We’ll see a similar error when we try to run our automated test, bundle exec rspec spec/features/posts_spec.rb: ... Migrations are pending; run 'bin/rake db:migrate RAILS_ENV=test' to resolve this issue.

We can run the recommended command here as well to fix the problem. bin/rake db:migrate RAILS_ENV=test

When we run our test again, we’ll see a different error: Failure/Error: click_link 'New Post' ActionView::Template::Error: undefined method `title' for # # ./app/views/posts/new.html.erb:4:... # ./app/views/posts/new.html.erb:1:... # ./spec/features/posts_spec.rb:6:...

We’ve now gotten past the error in our controller and now we’re back to an error within app/views/posts/new.html.erb. While line 1 of this file is mentioned in the stacktrace, it is not the final line and therefore the error is probably not occurring on that line. The very first line of the stacktrace points to line 4, which is this line:

Debugging Rails

19



The error we’re seeing is happening because somehow title is being called on an instance of the Post class. This is happening because the text_field helper, along with many other form helpers in Rails, will attempt to populate the form with the value from the attribute. It does this by trying to call the attribute’s method. If the attribute’s method is not there, then we see this error happening. What’s happening here is that we didn’t define any columns in the posts table, which means that there will be no title or body attribute defined for the form to use. We can fix this by altering the migration for the posts table, which is the only migration in the db/migrate folder. Before we alter it, we need to undo the migration, which we can do by running this command: bin/rake db:rollback

Now the database is back to its pristine state, we can alter that migration: class CreatePosts < ActiveRecord::Migration def change create_table :posts do |t| t.string :title t.text :body t.timestamps end end end

Now that the migration is correct, we can run this command to create the table with the correct columns: bin/rake db:migrate

We need to do the same thing with the test database, and that can be done with this command: bin/rake db:test:prepare

Let’s run that test again and see if everything’s running smoothly: 1 example, 0 failures

Yay! Our test is now passing. The router is receiving the request and passing it to the PostsController. The new action in that controller is defining a @post instance variable set to a new instance of the Post model. The app/views/posts/new.html.erb template is being run and rendering the form. The form is attempting to fetch the attributes from the new Post model instance, but since there are none then the fields will be left blank. All the parts are working in unity and we’ve debugged this bug.

Debugging Rails

20

Rails Example #2 - Routing HTTP verbs/methods In this example we’ll cover: • Debugging routing issues • The rake routes command This example is very similar to the first example. The changes are that our tests in spec/features/posts_spec.rb have now changed to this: require 'spec_helper' feature "Posts" do it "can create a new post" do visit "/" click_link "New Post" fill_in "Title", :with => "Hello world" fill_in "Body", :with => "This is the first post." click_button "Create Post" page.should have_content("Created post.") end it "can update an existing post" do Post.create({ :title => "Hello world", :body => "This is the first post." }) visit "/" click_link "Hello world" fill_in "Title", :with => "Hello World" click_button "Update Post" page.should have_content("Updated post.") end end

When we run the tests using bundle exec rspec spec/features/posts_spec.rb, we’ll see this error:

Debugging Rails

21

1) Posts can update an existing post Failure/Error: click_button "Update Post" ActionController::RoutingError: No route matches [POST] "/posts/1" # ./spec/features/posts_spec.rb:21 ...

Let’s mull over this error while we look through the code in the application. Rather than having a test to ensure that we can see the new post form correctly, the first test now goes through all the motions of creating a new post. It does this by going to the root path of the application and then clicking “New Post”. This routes to the new action within the PostsController: def new @post = Post.new end

This then renders the app/views/posts/new.html.erb view which has changed since the last time we saw it: Writing a new post

The form has been moved into a partial so that it can be used within the edit action’s template as well. The partial is located at app/views/posts/_form.html.erb and contains this content: { :method => :post } do |f| %> "btn btn-primary" %>

For the new action, this form will send a request to the create action. For the edit action, we can expect it to send the data to the update action instead because the record represented by @post is persisted within the database. When this form is submitted from the new template, it will go to the create action within PostsController:

Debugging Rails

22

def create @post = Post.create(post_params) flash[:notice] = "Created post." redirect_to root_path end

This simply creates the post from the data posted in and sets a flash notice. We know that all of this is working because the first test within spec/features/posts_spec.rb is working. It’s the second test which is failing. Let’s look at that failing test again: it "can update an existing post" do post = Post.create({ :title => "Hello world", :body => "This is the first post." }) visit "/" click_link post.title fill_in "Title", :with => "Hello World" click_button "Update Post" page.should have_content("Updated post.") end

This test is failing like this: 1) Posts can update an existing post Failure/Error: click_button "Update Post" ActionController::RoutingError: No route matches [POST] "/posts/1" # ./spec/features/posts_spec.rb:21 ...

This test first creates a new Post instance. It then visits the root path, which is a listing of posts. The index action within PostsController is responsible for collecting all the posts: def index @posts = Post.all end

The template at app/views/posts/index.html.erb renders all these posts within a table, showing a link with the post’s title for each post:

Debugging Rails

23

Posts "btn btn-primary" %> Title

The test then clicks on the newly created post’s title which takes us to the edit action with PostsController: def edit @post = Post.find(params[:id]) end

This action renders the template at app/views/posts/edit.html.erb: Editing

This renders the same form partial that we saw earlier used by the template in app/views/posts/new.html.erb: { :method => :post } do |f| %> "btn btn-primary" %>

After the edit template has been rendered, the test edits the title of the post to capitalize the word “world” within the title. It then clicks “Update Post” and that’s where our test is failing. The main part of the error, one more time:

24

Debugging Rails

ActionController::RoutingError: No route matches [POST] "/posts/1" # ./spec/features/posts_spec.rb:21 ...

The stacktrace that follows this isn’t really that helpful because it only shows us the line in the spec which is triggering this problem and doesn’t point to the error in our code so far. We could get a bigger stacktrace by running this command: bundle exec rspec spec/features/posts_spec.rb -b

But this shows us a lot of Rails and Capybara internal code, and it’s more likely that our code is at fault than Rails’s or Capybara’s. This time, we’re not being pointed in any particular direction and so we need to walk through the steps by ourselves to figure this out. We know in the test that we’re about to navigate to the root page, and then from there navigate to the edit page and fill in the ‘Title’ field. It’s the clicking the “Update Post” button that is failing, claiming that there is no route defined. We can check the routes defined for the application by running the rake routes command which will show us this: Prefix Verb URI Pattern Controller#Action root GET / posts#index posts GET /posts(.:format) posts#index POST /posts(.:format) posts#create new_post GET /posts/new(.:format) posts#new edit_post GET /posts/:id/edit(.:format) posts#edit post GET /posts/:id(.:format) posts#show PATCH /posts/:id(.:format) posts#update PUT /posts/:id(.:format) posts#update DELETE /posts/:id(.:format) posts#destroy

This output shows us all routes which are defined within our small application. We want a route that goes to the update action. There are two of these within the rake routes output: PATCH PUT

/posts/:id(.:format) /posts/:id(.:format)

posts#update posts#update

When making a request within Rails, we need to be careful that we use the correct HTTP verb/method. The routes defined for the update action use either a PATCH or a PUT request. From the feedback in our test, we can quite clearly see we’re not doing that:

Debugging Rails

25

ActionController::RoutingError: No route matches [POST] "/posts/1" # ./spec/features/posts_spec.rb:21 ...

We’re using POST rather than PATCH or PUT. Since we know that we can get to the edit action template just fine, let’s start our debugging from there. What we’re looking for is somewhere that is telling the code to make a POST request rather than a PUT or PATCH. The app/views/posts/edit.html.erb template contains this code: Editing

This is fairly standard and nothing is standing out here. Let’s look at the partial within app/views/posts/_form.html.erb: { :method => :post } do |f| %> "btn btn-primary" %>

On the very first line here is where we’re explictly telling the code to use the POST verb rather than PUT or PATCH. If we change this to use put or patch, like this: { :method => :put } do |f| %>

Or: { :method => :patch } do |f| %>

Then run our tests with bundle exec rspec spec/features/posts_spec.rb, we’ll see our first test is now failing with a similar reason:

Debugging Rails

26

1) Posts can create a new post Failure/Error: click_button "Create Post" ActionController::RoutingError: No route matches [PATCH] "/posts" # ./spec/features/posts_spec.rb:9 ...

Doomed if you do, doomed if you don’t. What we’re failing to realise here at this point is that form_for automatically takes care of using the right HTTP verb/method. We don’t actually need to specify the method. Let’s remove that option from the form_for now, leaving that first line like this:

When we run our tests again, they will now pass: 2 examples, 0 failures

The issue all along was that we were explicitly specifying the HTTP verb/method for the form, but we didn’t need to do that because Rails deals with this automatically. Rails does this by checking the persisted? method’s return value. This method returns true if the object is represented in the database with a record in the table, or false if not. Let’s go into the rails console now and see what this does. When we try that method with a new Post object, we’ll get this: irb(main):001:0> post = Post.new => # irb(main):002:0> post.persisted? => false

If we try it with a Post object that’s fetched from the database, we’ll get this: irb(main):001:0> post = Post.create => # irb(main):002:0> post.persisted? => true

If the return value of this persisted? method is false, the form_for helper will route to the plural name of the resource using posts_path in this case. It will also use the POST HTTP verb/method. If the return value of the method is true, then it will route to the singular name of the resource using post_path(post), using the PATCH HTTP verb/method. Let’s leave the HTTP verb/method deciding up to form_for, unless we absolutely know better.

Debugging Rails

27

Debugging slow code Without proper care, applications can get slow over time. Pages that once loaded quickly can degrade into taking more than a second to load. This can be caused by a slow database lookup, too many database lookups or just even too much information displayed on the page at any particular time. Slowness within an application can almost be called a “bug” itself. It’s definitely not a feature! In this chapter, we’ll cover some common pitfalls that people can inadvertently stumble upon during Rails application development. You might even be surprised at how easy some of the fixes are.

Rails Example #4 - Reducing Repeated Queries In this section we’ll cover: • Includes queries • Pagination This example app is a slight modifications on the earlier blogging application. In this application, we have a Post model which has_many :comments, and that Comment model has_many :users. Before we start this application, we’re going to need to run this command to set up some test data: rake db:seed

All this task will do is run the code within the db/seeds.rb file within the context of our Rails application’s development environment. This code looks like this: post = Post.create(:title => "Hello World", :body => "This is the first post.") users = 5.times.map do User.create(:name => "User#{rand(9999)}") end 1000.times do Comment.create( :post => post, :user => users[rand(5)], :body => "This is just a comment." ) end

This code creates a single post, then 5 randomly numbered users, then 1000 comments with the same post and text, but a randomly chosen user for each comment. Let’s look at what this data makes our application do by firing up the Rails server:

28

Debugging Rails

rails s

Going to http://localhost:3000 will show us the familiar list of posts:

Posts

Clicking on this post’s link will now show us the post itself, along with a whole bunch of comments:

Posts

Your numbers will probably⁸ be different, but that’s alright. If we switch over to the terminal session where rails s is running, we’ll see a quite large amount of queries that are being run to load our data: Processing by PostsController#show as HTML Parameters: {"id"=>"1"} Post Load (0.1ms) SELECT "posts".* FROM "posts" ... Comment Load (2.8ms) SELECT "comments".* FROM "comments" ... User Load (0.2ms) SELECT "users".* FROM "users" ... CACHE (0.0ms) SELECT "users".* FROM "users" ... User Load (0.2ms) SELECT "users".* FROM "users" ... CACHE (0.0ms) SELECT "users".* FROM "users" ...

At the top of this mountain of queries is the query to load the post. This needs to happen, otherwise we would not be able to show the post’s data. The second query is one to load the comments, which runs for a similar reason: we need to show the comments. The 1000 queries (don’t count them) that follow are loading all the users for the comments. But why are there 1000 queries when there’s only 5 users to load? Because Rails doesn’t know any better. Luckily for us though, Active Record is caching the results of these queries and fetching the back from the cache when it goes to run that query again. These lines in the output begin with CACHE. At the end of the output, we’ll see this: Completed 200 OK in 681ms (Views: 655.3ms | ActiveRecord: 18.4ms)

The entire action is taking 681ms to render, 655ms of that is within the view and 18.4ms is happening within Active Record. A lot of this slowness is due to the number of queries and query cache hits that Rails is undergoing during this request. ⁸Interesting trivia fact: The probability of the numbers not being different is 1 in 10000 to the power of 5, or 1 in 100 quintrillion (1 in 100,000,000,000,000,000,000).

Debugging Rails

29

When any action within our application runs, we should try and minimize the database querying needed for that action. Even a single query takes time, and so ideally we would like to reduce it down to 0 queries. Let’s not go all the way to that extreme yet. We should focus our efforts on making Active Record stop performing 1000 lookups for these users. The way we can fix this problem is with something in Active Record called eager loading. Eager loading will load the required data in as few queries as possible. We can use this within the show action of the PostsController like this: def show @post = Post.includes(comments: :user).find(params[:id]) end

The includes statement here triggers eager loading for this query. The Hash object passed as the argument to this tells Active Record to eager load the comments association from the Post model, as well as the user association for each comment. When we reload the page in our browser and look at the queries again, there’s no longer over a thousand of them: Started GET "/posts/1" for 127.0.0.1 at 2013-11-24 11:52:14 +1100 Processing by PostsController#show as HTML Parameters: {"id"=>"1"} ... SELECT "posts".* FROM "posts" WHERE "posts"."id" = ? LIMIT 1 [["id", "1"]] ... SELECT "comments".* FROM "comments" WHERE "comments"."post_id" IN (1) ... SELECT "users".* FROM "users" WHERE "users"."id" IN (1, 3, 2, 5, 4)

There are now only 3 queries for the data that needs to be displayed on this page: one for the post, one for the comments and one for all the users for all the comments. The eager loading here has altered the query for the comments to load it using an IN query, rather than the query it ran before: ... SELECT "comments".* FROM "comments" WHERE "comments"."post_id" = ? [["post_id", 1]]

This doesn’t make too much of a different in the query speeds. The two queries are going to produce the same result. The major difference here is that the 1000 queries for the users for the comments has now been reduced to one simple query:

Debugging Rails

...

30

SELECT "users".* FROM "users" WHERE "users"."id" IN (1, 3, 2, 5, 4)

We can also see that the page which once took around 700ms to load (650ms in the view, 25ms in Active Record), is now loading much, much faster: Completed 200 OK in 108ms (Views: 51.0ms | ActiveRecord: 3.9ms)

The page is now loading seven times faster with this one easy trick. That’s great!

TODO • Pagination • Fragment caching • (?) Mention the Bullet gem

Handling Exceptions Not all exceptions known to humankind have been documented in this book so far. There will be more than these which pop up in the applications that you will write. By default, when users cause an exception to happen within a Rails application, this is what they’ll see:

500 Error Page

Exceptions will happen in your applications, because nobody writes perfect code. Sometimes these exceptions might happen under a set of very, very specific circumstances and only for one user. Unless that user contacts you specifically, you may never know that this exception is happening. That’s where exception notification services come into play. These range from the basic exception_notification gem (http://smartinez87.github.io/exception_notification/) which you can install and configure to email you whenever an exception happens, all the way up to the large scale New Relic which does much much more than exception notifications. There are others too, such as Airbrake (http://airbrake.io) and Honeybadger (honeybadger.io).

Advanced Rails Debugging Advanced Rails Example #1 - Broken Devise Application This example was from a debugging session I did with Steven Baker on November 25th, 2013. In this example we’ll cover: • bundle show • Debugging with the pry gem • How to find where methods are defined Not all applications that we debug are going to be on the latest and greatest versions of everything. Sometimes we’ll be asked to debug code that is years old and more often than not written by somebody else. Unfamiliarity with an older version of Rails or another gem may seem like a huge hurdle at first, but on closer inspection that hurdle is a lot smaller than it seems. Debugging sometimes requires a large amount of patience and concentration, as this (almost 5,000 word) example shows. The pay off at the end of such a long debugging session is worth it, in my opinion. I get a little kick out of it. Let’s start debugging. Here in this application, we are having a problem where (imaginary) users are unable to sign in. The (imaginary) user is reportedly seeing a 500 error page when they try to sign in, which is indicative of a bigger problem. Thankfully another (imaginary) developer within this aplication has written a test to cover this breakage using RSpec + Capybara. This test lives at spec/features/sign_in_spec.rb: require 'spec_helper' describe "sign in" do before do User.create!(:email => "[email protected]", :password => "password") end it "can sign in successfully" do visit "/" click_link "Sign in" fill_in "Email", :with => "[email protected]" fill_in "Password", :with => "password" click_button "Sign in" page.should have_content("Signed in successfully.") end end

33

Advanced Rails Debugging

When we run this test using bundle exec rspec spec/features/sign_in_spec.rb, we’ll see that it is indeed breaking: 1) sign in can sign in successfully Failure/Error: click_button "Sign in" ActionController::RoutingError: uninitialized constant SessionsController # ./spec/features/sign_in_spec.rb:13...

It says “uninitialized constant SessionsController”, but Devise should be handling the authentication, shouldn’t it? Devise has a controller within its own code called Devise::SessionsController which should be handling this request, but for some reason it is not. It’s trying to use the SessionsController without the Devise namespace. Running the test with bundle exec rspec spec/features/sign_in_spec.rb -b yields no better information, showing us only stacktrace from within Rails or Rack. Let’s investigate this problem ourselves by starting up this application now: rails server

If we now go to http://localhost:3000 and click “Sign in”, we’ll see the standard Devise form consisting of an email and password field.

Sign in form

If we attempt to sign in with “[email protected]” and “password”, it will fail and show the same error that we saw from our tests:

34

Advanced Rails Debugging

Routing error for SessionsController

Why is this happening? The path for this request looks weird. It shows as “localhost:3000/session.user”. Where is that coming from? The best first place to look would be back on the sign in page itself to see if the form tag is sending us there. If we go back to that page, right click on an element of the form and click “Inspect Element”, we’ll see that the form’s HTML makeup begins with this:

This explains how we’re being taken to /session.user, but not the why. Let’s investigate how Devise generates that form now. The template that displays this form lives inside the Devise gem rather than our application. To get to the template, we’re going to need to open the current version of Devise that the application is using. To get there, we’re going to need to find where Devise is installed on our system. Lucky for us, there’s a command to do that: bundle show devise

This command will show the path to the gem that the application’s Gemfile will use when it sources its dependencies. It will show something like this: /Users/user/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/devise-1.1.9

We can open this path in our editor by running this command: $EDITOR `bundle show devise`

If that doesn’t work, then you will need to open the path manually in your editor. Now that we have Devise open, we need to figure out what template is being shown that renders this form. We can find that out by looking at the log output from our rails server session when a request to the sign in page is made:

35

Advanced Rails Debugging

Started GET "/users/sign_in" for 127.0.0.1 ... Processing by Devise::SessionsController#new as HTML Rendered .../devise-1.1.9/app/views/devise/shared/_links.erb (0.9ms) Rendered .../devise-1.1.9/app/views/devise/sessions/new.html.erb within layouts/application (5.6ms)

In this output, we can see that we’re making a request to /users/sign_in and that request is being routed to the new action within Devise::SessionsController. This action then goes on to render a couple of templates, namely devise/shared/_links.erb and devise/sessions/new.html.erb. The code for the form is probably going to be in the sessions/new.html.erb template. Inside that template we’ll see this code to generate the form: resource_name, :url => session_path(resource_name)) do |f| %>

For the URL of the form, this form_for call is calling the session_path helper, which is probably a routing helper. We can find out where this is defined within our application by running this command: rake routes

This command will show this output for this application: root

/(.:format) ... new_user_session GET /users/sign_in(.:format) user_session POST /users/sign_in(.:format) destroy_user_session GET /users/sign_out(.:format) user_password POST /users/password(.:format) new_user_password GET /users/password/new(.:format) edit_user_password GET /users/password/edit(.:format) PUT /users/password(.:format) ... user_registration POST /users(.:format) new_user_registration GET /users/sign_up(.:format) edit_user_registration GET /users/edit(.:format) PUT /users(.:format) ... DELETE /users(.:format) ... session POST /session(.:format) new_session GET /session/new(.:format) edit_session GET /session/edit(.:format) GET /session(.:format) ... PUT /session(.:format) ... DELETE /session(.:format) ...

... ... ... ... ... ... ... ... ...

... ... ...

36

Advanced Rails Debugging

On the left-hand side of this output, we can see the path helpers defined for the routes if they are present. We can see that there is one line in particular that defines a session helper: session POST /session(.:format) {:action=>"create", :controller=>"sessions"}

This is the helper that is being used by Devise to generate the route for the form incorrectly. Where is this coming from though? Let’s look in config/routes.rb and see if anything there sticks out: Bug::Application.routes.draw do root :to => "welcome#index" devise_for :users resource :session end

At the very bottom of this file there is a resource :session line, which is what is defining the session_path helper. Since there is no SessionsController for this route to use within the application, it probably got left over in this application from a previous developer. Let’s remove this line from config/routes.rb now: resource :session

For this route change to take full-effect, we’ll need to restart our rails server session so that the session_path helper is completely forgotten about. Once we’ve restarted rails server, we can try to sign in again with “[email protected]” and “password”. This time we’ll see a completely different error message:

Invalid Email or Password

37

Advanced Rails Debugging

Ok, a different error message is progress because it means something new is happening! First thing to check: is there actually a user with the email address “[email protected]” within the database? The easiest way to check that would be to launch a new console session: rails c

Within this console session, we want to find if a user with the email “[email protected]” exists, which we can do like this: User.exists?(:email => "[email protected]")

When we run this code, we’ll see that it doesn’t exist: irb(main):001:0> User.exists?(:email => "[email protected]") => false

This is happening because the only place where we have been creating this user is within the test environment, and not the development environment. These two environments use separate databases and after a test is run within the test environment, that database is emptied anyway. Therefore this problem may be caused by a simple case of a missing user. Let’s create this new user in the database now: User.create!(:email => "[email protected]", :password => "password")

With this user created, we can try to sign in again.

Invalid Email or Password

Advanced Rails Debugging

38

Unfortunately, that did not fix the problem yet. The error message is still telling us invalid email and password, although we’re pretty certain that it’s the right email address and the right password. We can double-check this by first finding the user in the console: >> user = User.find_by_email("[email protected]") => #

With this, we can see that the user does actually exist. Now let’s check to see if the password is valid using Devise’s valid_password? method: >> user.valid_password?("password") => true

The user definitely exists and their password is definitely valid, even though the error message is telling us that this is not true. The error message is lying to us. Let’s take a look at the logs which may give us more leads as to where to look to figure this one out. Here’s what happening for this request: Started POST "/users/sign_in" for 127.0.0.1 at 2013-11-27 18:03:40 +1100 Processing by Devise::SessionsController#create as HTML Parameters: ... Completed in 4ms Processing by Devise::SessionsController#new as HTML Parameters: ... Rendered .../app/views/devise/shared/_links.erb (0.8ms) Rendered .../app/views/devise/sessions/new.html.erb within layouts/application (\ 5.0ms) Completed 200 OK in 97ms ...

We can see here that Rails is receiving the request from the sign in form and that the request is being handled by the create action within Devise::SessionsController. Somehow, the action does not respond in any particular fashion, indicated with the “Completed in 4ms” message. Typically we would expect to see a “200 OK” or “302 Redirected” message there, but we’re not seeing that. After this action complese, the new action within the same controller gets the same parameters and then re-renders the app/views/devise/sessions/new.html.erb template. To begin to discover what’s happening here, let’s check out the code within Devise::SessionsController. If the gem is not open any more, re-open it using this command: $EDITOR `bundle show devise`

Within Devise, this controller is located at app/controllers/devise/sessions_controller.rb and its create action looks like this:

Advanced Rails Debugging

39

devise/app/controllers/devise/sessions_controller.rb

1 2 3 4 5

def create resource = warden.authenticate!(:scope => resource_name, :recall => "new") set_flash_message :notice, :signed_in sign_in_and_redirect(resource_name, resource) end

The create action first calls out to the warden method which is provided by the Warden gem which Devise depends on. The warden method is a proxy object which keeps track of the current session state. We can find out where the authenticate! method for this proxy object is defined by firstly adding the pry gem to our ‘Gemfile: gem 'pry'

We can install this gem using bundle install. This gem provides us with some useful debugging tools which we’ll see used as we go along. The first of these is the breakpoint helper called binding.pry. This stops code execution right in its tracks and will bring up an IRB-esque prompt which we can then use to debug code. We’re going to use this within the create action in Devise::SessionsController like this: devise/app/controllers/devise/sessions_controller.rb

1 2 3 4 5 6

def create binding.pry resource = warden.authenticate!(:scope => resource_name, :recall => "new") set_flash_message :notice, :signed_in sign_in_and_redirect(resource_name, resource) end

To ensure that our server has this latest and greatest code, we’ll need to restart it. Once it’s been restarted, if we try signing in again the browser should hang and then in the server window we’ll see the Pry prompt: 12: def create => 13: binding.pry 14: resource = warden.authenticate!(:scope => resource_name, :recall => "new") 15: set_flash_message :notice, :signed_in 16: sign_in_and_redirect(resource_name, resource) 17: end

We can treat this prompt just like any other Ruby prompt. We can call whatever Ruby code we wish within it. Right now though, all we want to know is where the authenticate! method is defined. Let’s find that out now by using this code:

Advanced Rails Debugging

40

warden.method(:authenticate!).source_location

The method method returns a Method object that represents the method. Calling source_location on that object tells us this: => [".../gems/warden-1.0.6/lib/warden/proxy.rb", 112]

This indicates to us that the method is defined within the Warden gem, and so we should go look there to see what’s happening. Now that we know where that method is defined, we no longer need the binding.pry within Devise::SesionsController. We should remove this call now before we continue. Let’s open the warden gem using this command in our terminal: $EDITOR `bundle show warden`

Once we’ve got the gem open, we’ll then open lib/warden/proxy.rb and navigate to line 112 within that file. The authenticate! method will be here: warden/lib/warden/proxy.rb:112

1 2 3 4 5

def authenticate!(*args) user, opts = _perform_authentication(*args) throw(:warden, opts) unless user user end

This method takes any number of arguments and stashes them in the args variable. These arguments are from Devise::SessionsController’s create action: resource = warden.authenticate!(:scope => resource_name, :recall => "new")

What resource_name is isn’t too important right now. What is important is tracking down why this method is sending us off to the new action in Devise::SessionsController rather than processing our request properly. It would seem here that the _perform_authentication method returns two things: a user and a set of options. If our authentication is failing, then it would make sense that it would probably be failing within this _perform_authentication method. A quick search for this method within this file will show us where it is: down on line 272.

Advanced Rails Debugging

41

warden/lib/warden/proxy.rb:272

1 2 3 4 5 6 7 8 9 10 11 12 13

def _perform_authentication(*args) scope, opts = _retrieve_scope_and_opts(args) user = nil # Look for an existing user in the session for this scope. # If there was no user in the session. See if we can get one from the request. return user, opts if user = user(scope) _run_strategies_for(scope, args) if winning_strategy && winning_strategy.user opts[:store] = opts.fetch(:store, winning_strategy.store?) set_user(winning_strategy.user, opts.merge!(:event => :authentication)) end [@users[scope], opts] end

The first thing this method does is call out the _retrieve_scope_and_opts method, passing it the same args that this method takes. This method is the next method defined in this file: warden/lib/warden/proxy.rb:289

1 2 3 4 5 6

def _retrieve_scope_and_opts(args) #:nodoc: opts = args.last.is_a?(Hash) ? args.pop : {} scope = opts[:scope] || @config.default_scope opts = (@config[:scope_defaults][scope] || {}).merge(opts) [scope, opts] end

We can see what this method returns by altering _perform_authentication to have a binding.pry right after the _retrieve_scope_and_opts call: warden/lib/warden/proxy.rb:272

1 2 3

def _perform_authentication(*args) scope, opts = _retrieve_scope_and_opts(args) binding.pry

To trigger this breakpoint, we’ll need to restart the server so that the code for all the gems, including Warden, is reloaded. After that, we’ll try signing in again which will bring up a new Pry prompt:

Advanced Rails Debugging

1 2 3 4

42

272: def _perform_authentication(*args) 273: scope, opts = _retrieve_scope_and_opts(args) => 274: binding.pry 275: user = nil

We can inspect the values for scope and opts easily in the prompt: [1] pry(#)> scope => :user [2] pry(#)> opts => {:scope=>:user, :recall=>"new"}

Now that we know what the value of scope and opts is, we can go through the rest of the _perform_authentication method’s flow. Let’s make sure to remove this binding.pry first, now that we’ve found the information we wanted. The next part of the _perform_authentication method is this: warden/lib/warden/proxy.rb:277

1 2 3

# Look for an existing user in the session for this scope. # If there was no user in the session. See if we can get one from the request. return user, opts if user = user(scope)

This part of the code is pretty well-documented. It attempts to fetch a user from the session, but there probably isn’t one because we’re attempting to sign in as a user now. If there was a user, then this method would return the user and the options at this point. There isn’t a user, and therefore it’s going to fall to the next code within that method: warden/lib/warden/proxy.rb:280

1 2 3 4 5

_run_strategies_for(scope, args) if winning_strategy && winning_strategy.user opts[:store] = opts.fetch(:store, winning_strategy.store?) set_user(winning_strategy.user, opts.merge!(:event => :authentication)) end

The _run_strategies_for method is called, passing in the scope and args arguments. Let’s see what this method does:

Advanced Rails Debugging

43

warden/lib/warden/proxy.rb:298

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

# Run the strategies for a given scope def _run_strategies_for(scope, args) #:nodoc: self.winning_strategy = @winning_strategies[scope] return if winning_strategy && winning_strategy.halted? if args.empty? defaults = @config[:default_strategies] strategies = defaults[scope] || defaults[:_all] end (strategies || args).each do |name| strategy = _fetch_strategy(name, scope) next unless strategy && !strategy.performed? && strategy.valid? self.winning_strategy = @winning_strategies[scope] = strategy strategy._run! break if strategy.halted? end end

There’s quite a lot going on inside this method. At the top, it sets the winning_strategy virtual attribute (defined on line 9 of this file), to whatever the output of @winning_strategies[scope] is. If @winning_strategies[scope] is not nil and is halted?, then the method stops executing and _perform_authentication continues. Is @winning_strategies[scope] returning nil? Let’s use Pry to find out by sticking a binding.pry as the first line in that method: warden/lib/warden/proxy.rb:298

1 2 3

def _run_strategies_for(scope, args) #:nodoc: binding.pry self.winning_strategy = @winning_strategies[scope]

We will need to restart our rails server process here for the changes to take effect. Let’s try to sign in again. This will bring up the Pry prompt again: 298: def _run_strategies_for(scope, args) #:nodoc: => 299: binding.pry 300: self.winning_strategy = @winning_strategies[scope]

With the Pry prompt, we can see what @winning_strategies[scope] is:

Advanced Rails Debugging

44

[1] pry(#)> @winning_strategies[scope] => nil

Since this call returns nil, the method will continue executing past the line that checks for the object’s presence. Let’s keep reading this method. warden/lib/warden/proxy.rb:303

1 2 3 4

if args.empty? defaults = @config[:default_strategies] strategies = defaults[scope] || defaults[:_all] end

This part of the method is just setting up some defaults for the strategies to use within this method, unless some arguments explicitly telling us what strategies to use are passed in. With our Pry prompt still open, we can see if args is empty by simply asking: [2] pry(#)> args => []

Since args is empty, this will set defaults to the return value of @config[:default_strategies]. What’s that? Let’s ask Pry again: [3] pry(#)> @config[:default_strategies] => {:user=>[:rememberable, :database_authenticatable]}

This method returns the default configured strategies from Devise, namely rememberable and database_authenticatable. We know that scope is :user, and therefore we can work out from this that strategies is going to be set to [:rememberable, :database_authenticatable]. We can see this in practice if we run that code in Pry: [4] pry(#)> defaults = @config[:default_strategies] => {:user=>[:rememberable, :database_authenticatable]} [5] pry(#)> strategies = defaults[scope] || defaults[:_all] => [:rememberable, :database_authenticatable]

Now that we know what that part of the method does, let’s remove that binding.pry and move on to the next couple of lines:

Advanced Rails Debugging

45

warden/lib/warden/proxy.rb:308

1 2 3

(strategies || args).each do |name| strategy = _fetch_strategy(name, scope) next unless strategy && !strategy.performed? && strategy.valid?

This method iterates through either strategies or args. We know that args is empty, and because of this the strategies variable is being set, therefore this will iterate through strategies. For each strategy, the _run_strategies_for method calls _fetch_strategy and passes it the name of the strategy, as well as the scope variable. Let’s see what this _fetch_strategy method does now; it’s the last method within this file. warden/lib/warden/proxy.rb:319

1 2 3 4 5 6 7 8 9 10

# Fetchs strategies and keep them in a hash cache. def _fetch_strategy(name, scope) @strategies[scope][name] ||= if klass = Warden::Strategies[name] klass.new(@env, scope) elsif @config.silence_missing_strategies? nil else raise "Invalid strategy #{name}" end end

Let’s stick a binding.pry at the top of this method now, so that we can investigate what it’s doing: warden/lib/warden/proxy.rb:319

1 2 3

# Fetchs strategies and keep them in a hash cache. def _fetch_strategy(name, scope) binding.pry

Restarting the server and attempting to sign in again will bring up a new Pry prompt: 319: def _fetch_strategy(name, scope) => 320: binding.pry 321: @strategies[scope][name] ||= if klass = Warden::Strategies[name]

The first part of this code that will execute is the Warden::Strategies[name] part. Let’s see what that returns now:

Advanced Rails Debugging

46

[1] pry(#)> Warden::Strategies[name] => Devise::Strategies::Rememberable

This call is going to return Devise::Strategies::Rememberable, so that means the if part of the method will be executed: warden/lib/warden/proxy.rb:319

1 2

@strategies[scope][name] ||= if klass = Warden::Strategies[name] klass.new(@env, scope)

This code calls the initialize method located within Devise::Strategies::Rememberable, passsing it the @env object (which is the current request’s environment), as well as the scope. This call returns a new object of the Devise::Strategies::Rememberable class, which is then sent back to the _run_strategies_for method. Now that the _run_strategies_for method actually has a strategy to run, it does that with this line: warden/lib/warden/proxy.rb:308

1

next unless strategy && !strategy.performed? && strategy.valid?

We know that strategy here is going to be an instance of the Devise::Strategies::Rememberable class, and that’s definitely not nil, so the first check here completes. We don’t know however what performed? or valid? return. To find that out, let’s remove the binding.pry from _fetch_strategy and add a new one directly above line 308 in warden/proxy.rb: warden/lib/warden/proxy.rb:308

1 2

binding.pry next unless strategy && !strategy.performed? && strategy.valid?

Restarting the server and attempting a sign in again will bring up the Pry prompt at the new location: 307: (strategies || args).each do |name| 308: strategy = _fetch_strategy(name, scope) => 309: binding.pry

With this prompt, we can now check to see what performed? and valid? return.

Advanced Rails Debugging

47

[1] pry(#)> strategy.performed? => false [2] pry(#)> strategy.valid? => false

It makes sense here that performed? is returning false, since this is the first time (as far as we have seen) that the strategy is being fetched and checked. The reason why valid? is returning false is unclear. So let’s investigate where that’s coming from so that we can rule it out as a cause of our problem. We can find out where the valid? method is defined by using source_location: >> strategy.method(:valid?).source_location => ["...gems/devise-1.1.9/lib/devise/strategies/rememberable.rb", 11]

Let’s go back to the open Devise project and find that file. On line 11, it has this code: devise/lib/strategies/rememberable.rb:11

1 2 3

def valid? remember_cookie.present? end

Where is remember_cookie? It’s a little further down, near the bottom of the file: devise/lib/strategies/rememberable.rb:44

1 2 3

def remember_cookie @remember_cookie ||= cookies.signed[remember_key] end

This method calls cookies.signed[remember_key]. What is remember_key? It’s defined a little further up: devise/lib/strategies/rememberable.rb:35

1 2 3

def remember_key "remember_#{scope}_token" end

The scope variable here is going to be “user”, so therefore remember_key is going to be “remember_user_token”. Does remember_cookie return anything? Let’s check back in our Pry console:

Advanced Rails Debugging

48

[4] pry(#)> strategy.send(:remember_cookie) => nil

So no, this doesn’t return anything. Therefore this particular strategy is not valid. This means that Warden will now move onto the next strategy. Let’s type exit to allow Warden’s code to continue to run. It will stop on the next strategy. We can find out that strategy’s class by running this: [1] pry(#)> strategy.class => Devise::Strategies::DatabaseAuthenticatable

This strategy is used by Devise to authenticate user sessions using the database. It finds a user with an email address and checks that their password is valid. This sounds like exactly the right place to be looking to find out what’s going wrong. Let’s look at this very closely. We can do the same checks as we did with the previous strategy. The strategy does exist, because we were able to find out the class of it, and that class wasn’t NilClass. Next, let’s see if this strategy hasn’t been performed yet: [2] pry(#)> strategy.performed? => false

Ok, so that’s the same as the first strategy. Now let’s see if this strategy is valid: [3] pry(#)> strategy.valid? => false

This strategy isn’t valid either. Again, there’s not much information to go on and so let’s investigate what that valid? method is doing. [4] pry(#)> strategy.method(:valid?).source_location => [".../gems/devise-1.1.9/lib/devise/strategies/authenticatable.rb", 11]

The valid? method in this strategy is defined like this: devise/lib/strategies/authenticatable.rb:11

1 2 3

def valid? valid_for_http_auth? || valid_for_params_auth? end

What do each of these methods return? Let’s see in our Pry console:

Advanced Rails Debugging

49

[5] pry(#)> strategy.send(:valid_for_http_auth?) => false [6] pry(#)> strategy.send(:valid_for_params_auth?) => false

We should investigate the valid_for_http_auth? method first. It’s defined like this: devise/lib/strategies/authenticatable.rb:36

1 2 3 4 5

def valid_for_http_auth? http_authenticatable? && request.authorization && with_authentication_hash(http_auth_hash) end

Let’s see what the first method is doing here: [7] pry(#)> strategy.send(:http_authenticatable?) => false

The http_authenticatable? method returns false for this strategy, which means that valid_for_http_auth? is going to return false as well. Therefore valid_for_http_auth? is a dead-end. Let’s look at valid_for_params_auth?: devise/lib/strategies/authenticatable.rb:47

1 2 3 4

def valid_for_params_auth? params_authenticatable? && valid_request? && valid_params? && with_authentication_hash(params_auth_hash) end

Let’s see what these methods do.

Advanced Rails Debugging

50

[8] pry(#)> strategy.send(:params_authenticatable?) => true [9] pry(#)> strategy.send(:valid_request?) => true [10] pry(#)> strategy.send(:valid_params?) => true [11] pry(#)> strategy.send(:with_authentication_hash, strategy.send(:params_auth_hash)) => false

It’s that final method, with_authentication_hash, that is failing. Let’s look at what this method does: devise/lib/strategies/authenticatable.rb:104

1 2 3 4 5 6 7

# Sets the authentication hash and the password from params_auth_hash or http_au\ th_hash. def with_authentication_hash(hash) self.authentication_hash = hash.slice(*authentication_keys) self.password = hash[:password] authentication_keys.all?{ |k| authentication_hash[k].present? } end

This method takes a hash, namely the one defined by params_auth_hash: devise/lib/strategies/authenticatable.rb:62

1 2 3 4

# Extract the appropriate subhash for authentication from params. def params_auth_hash params[scope] end

What do the params look like? {"utf8"=>"✓", "authenticity_token"=>"8Lreg3wifRd81HlP0Spo3oQSNrgOhq5odrEAekHqnYQ=", "user"=> {"email"=>"[email protected]", "password"=>"password", "remember_me"=>"0"}, "commit"=>"Sign in", "action"=>"create", "controller"=>"devise/sessions"}

We know from before that scope is “user”, so the only parameters used in with_authentication_hash are params[:user]:

Advanced Rails Debugging

51

{"email"=>"[email protected]", "password"=>"password", "remember_me"=>"0"}

Now that we know what hash with_authentication_hash has, let’s see what the method does next: devise/lib/strategies/authenticatable.rb:106

1

self.authentication_hash = hash.slice(*authentication_keys)

The hash gets sliced using authentication_keys. What is authentication_keys? [12] pry(#)> strategy.send(:authentication_keys) => [:login]

The authentication_keys method returns a 1-element array with the symbol :login inside it. This means that when hash.slice is called within with_authentication_hash, it will pick out any key called :login from our params[:user] hash. Is there any key with that name? {"email"=>"[email protected]", "password"=>"password", "remember_me"=>"0"}

No, there isn’t! We only have the keys of "email", "password", and "remember_me". It would seem strange that authentication_keys is returning :login when our hash doesn’t contain that key at all. This is most likely where our bug is happening. If we look at the authentication_keys method, we can find out where this is coming from: devise/lib/strategies/authenticatable.rb:111

1 2 3 4

# Holds the authentication keys. def authentication_keys @authentication_keys ||= mapping.to.authentication_keys end

Let’s see where that authentication_keys method is coming from: [13] pry(#)> strategy.mapping.to.method(:authentication_keys).source_location => ["/Users/ryanbigg/.gem/ruby/2.1.3/gems/devise-1.1.9/lib/devise/models.rb", 22]

The method is defined like this:

Advanced Rails Debugging

52

devise/lib/devise/models.rb:22

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

def self.config(mod, *accessors) #:nodoc: accessors.each do |accessor| mod.class_eval nil

Nope, that’s not defined at all. Let’s move on to the next statement: [2] pry(User)> superclass.respond_to?(:authentication_keys) => false

That’s also not defined. Therefore it must be in Devise.authentication_keys: [3] pry(User)> Devise.authentication_keys => [:login]

Yes! That’s where this value is coming from. Now that we know we can remove the binding.pry that we just added. Since this method is defined on the Devise class itself, we can expect it to be defined in lib/devise.rb within Devise, because that’s where the Devise module itself is defined.⁹ Let’s check in that file for authentication_keys. We’ll come across this code: lib/devise.rb:64

1 2 3

# Keys used when authenticating an user. mattr_accessor :authentication_keys @@authentication_keys = [ :email ]

If we look through this whole file, it looks like the mattr_accessor methods are defining (quite a lot of) configuration for Devise. It would seem that our authentication_keys value of [:login] doesn’t match the default value, specified here in lib/devise.rb, as [:email]. Something in our code is changing this. Where could that place be? Well, we have one big block of Devise configuration in our application at config/initializers/devise.rb. On line 23 of that file, we have this: config/initializers/devise.rb:23

1

config.authentication_keys = [:login] ⁹Methods defined on classes and modules in Ruby are typically found in the same files as those classes and modules; it’s just really best practice to do so.

54

Advanced Rails Debugging

The culprit has been here all along! Let’s try commenting out that line and see what happens when we restart our application and sign in once again.

Signed in successfully

Great! So now we’ve been able to sign in successfully. Why was the authentication_keys setting failing us in the first place though? It was Devise’s with_authentication_hash method in Devise::Strategies::Authenticatable which was returning false: devise/lib/strategies/authenticatable.rb:104

1 2 3 4 5 6 7

# Sets the authentication hash and the password from params_auth_hash or http_au\ th_hash. def with_authentication_hash(hash) self.authentication_hash = hash.slice(*authentication_keys) self.password = hash[:password] authentication_keys.all?{ |k| authentication_hash[k].present? } end

The authentication_keys method returned [:login], but authentication_hash didn’t contain that key. It contained this: {"email"=>"[email protected]", "password"=>"password", "remember_me"=>"0"}

So when the final line of that method is called, it returns false because the authentication_hash does not contain the :login key. This bubbles up the stack, back to the valid? method for this strategy:

Advanced Rails Debugging

55

devise/lib/strategies/authenticatable.rb:11

1 2 3

def valid? valid_for_http_auth? || valid_for_params_auth? end

That method returns false, which makes Warden not use that strategy at all. When Warden finds no valid strategy to authenticate against, it ultimately assumes that the authentication attempt was unsuccessful. This bubbles up and up the stack, back to the authenticate! method: warden/lib/warden/proxy.rb:112

1 2 3 4 5

def authenticate!(*args) user, opts = _perform_authentication(*args) throw(:warden, opts) unless user user end

The user variable here is going to be nil because no authentication was successful, so the throw will happen. The throw method in Ruby requires there to be a catch somewhere, and sure enough if we look for catch(:warden in Warden’s source, we’ll come across one: warden/lib/warden/manager.rb:30

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

def call(env) # :nodoc: return @app.call(env) if env['warden'] && env['warden'].manager != self env['warden'] = Proxy.new(env, self) result = catch(:warden) do @app.call(env) end result ||= {} case result when Array if result.first == 401 && intercept_401?(env) process_unauthenticated(env) else result end when Hash process_unauthenticated(env, result) end end

Advanced Rails Debugging

56

The catch is there on line 34. The throw is throwing out a Hash object (opts). An Array would be returned if the @app.call method completed successfully without a throw(:warden) happening. Therefore the call method will fall to the last couple of lines of this method which calls the process_unauthenticated method, and we know where it goes from there. All of this was due to a single line of configuration in config/initializers/devise.rb. A lot of debugging issues (as we’ve seen) come down to a single line of code that is wrong or, in this case, misconfigured. It’s rare to come across several lines working together in tandem to make our lives miserable, but it can happen to. Thanks to Pry and the method and source_location methods, this bug has been extremely easy to track down, even if the process was a bit laborious.