A topic that always crops on when speaking to Ruby on Rails developers is fixtures, or factories. A common dividing line or fence that people typically fall on one side or another of.

Having worked on a large number of projects I’m confident to say that the majority lean into two things in particular that are not the default: RSpec and FactoryBot.

I can talk about RSpec another day, but today I want to talk about factories (in the context of a test suite).

Whether you use fixtures or factories, what we typically use them for is to create a version of our system that is ready to test. Usage of either happens in a phase of testing that we call “test setup”. You want to define the state of the world that the application is going to run within so you can test various aspects of it.

For example, you might want to test how the application behaves for a free vs. paid user to make sure a free user doesn’t accidentally get access to paid features.

To make sure we’re on the same page, fixtures are a set of database records that you can populate the database with when running a test. They’re static (mostly) and are available to every test when needed. There’s really no magic behind them as when you invoke the fixture it takes the values and pops them in the database ready for you to use.

And factories are a description of how an object in your system is created (often a database record). There’s nothing created until you actually tell it to do so, and you can customise it when you create one.

If a factory says a user has a username, when you use that factory you can pass any username in you like.

Factories alone do nothing, you need to invoke them manually to create the set of records you wish to place under test. You are often starting with a completely empty database each time you run your test suite and you “define the world” at the start of each test by using your factories.

Factories themselves aren’t problematic: the factory pattern is actually perfectly acceptable. It’s just how it’s used and what it encourages that I think is flawed.

Think about this: you start with an empty database and then use your factories to create the records needed to test a certain piece of functionality.

You need to do this for every single test, probably changing which set of records get created in the database each time, just to make sure you’ve covered every possibility.

We’re often using tests to make sure that we aren’t breaking the application, so we set up test cases to cover those. Now, a lot of the times our application can break due to missing data. Or we get something that fails in production but the tests still pass because the data setup in the tests didn’t fully cover every scenario available.

As soon as your application has gone live and has a single user, you should not be testing your application by starting from an empty database. It’s not representing your application’s state at all anymore! Not even close. You need to test in a real world scenario, where you have existing data to manage with existing edge cases. This is why fixtures shine and why I think the factory approach is wrong. It’s not that the pattern itself is wrong, just the approach it encourages.

Populate your database using fixtures with known data flaws, whether that’s legacy models or setups, or just having a bunch of well known setups that you can easily test with, without having to invoke a bunch of factories up front to get to the same place.

This just gives you increased confidence that the thing you changed that might affect 10 different types of accounts is actually working still, because your other tests in the test suite are using the same source of data: your fixtures!

Maintaining fixtures is difficult, I won’t argue that. And in a large codebase it can be onerous. Having said that, maintaining factories isn’t a walk in the park either at a certain scale.

The upside of factories is that just by looking at the test in question you can see all the records that are going to be created.

Whereas for fixtures, you really need an understanding of the whole system and the fixtures that have already been provided to you in order to write your tests effectively.

Even though fixtures are the actual Rails default, factories have been the community’s default for a while now. But hopefully we’ll see a bit of a transition back to the Rails defaults as we’re starting to see with everything else.