Generating the Gemfile.next.lock

At FastRuby.io, we recommend using the Dual Boot opens a new window technique for upgrades. This requires us to generate a Gemfile.next.lock file that will be used to boot the app with the next version or Rails. In this article we’ll share 2 techniques to generate this file: the faster one and the safer one.

The Faster

This is the easiest one. The steps are:

1. Add the next? conditionals to the Gemfile
2. Create the Gemfile.next symlink
3. Run next bundle install (or next bundle update)

If you are not using next_rails, you can run BUNDLE_GEMFILE=Gemfile.next bundle install instead

This process will generate a new Gemfile.next.lock file from scratch. It will let Bundler resolve any dependency needed for the next version of Rails with the restrictions defined in the Gemfile and it will use the latest version of each gem that matches the requirements.

There is a problem with this approach, a lot of extra gems will be updated along with Rails and its dependencies, so you are not only upgrading Rails, you are also upgrading any other gems unrelated to the upgrade itself. Upgrading more things than the minimum needed can lead to more failures if the upgraded gems happen to introduce changes that require extra work.

Pros:

• It’s simple and fast, one command and the new file is created

Cons:

• Other gems get updated, not only the ones needed for the upgrade, and more changes == more things can break

The Safer

This may be a bit harder to do (not always, though), but it ensures that only the gems related to the upgrade are updated and nothing else. Those other gems can be updated at another time. The steps are:

1. Add the next? conditionals to the Gemfile
2. Create the Gemfile.next symlink
3. Copy the original Gemfile.lock as Gemfile.next.lock (note this is a copy, not a symlink)
4. Run next bundle update rails (note we only tell Bundler to update rails first)
5. At this point, the command may fail due to some problem trying to resolve a dependency version.
6. Check the reason for the failure, update your Gemfile if needed, and append the name of the dependency that needs to be updated to the bundle update command
7. Repeat steps 5 and 6 until all the dependencies are updated.

This process will generate a Gemfile.next.lock file keeping as many gems as possible in the original version and updating only the ones that are strictly needed for the Rails upgrade. We have to help Bundler to decide which gems should be updated.

This approach can take more time and requires more manual input if Bundler can’t resolve the dependencies, but the result is more conservative. We’ve seen that many failures during upgrades are introduced by unwanted gem updates when using the faster approach. This safer technique will take some extra time at the beginning but it can potentially save hours of debugging.

Pros:

• Prevents introducing failures by unwanted gem updates

Cons:

• It may require manual inspection of errors and manual input to assist Bundler

An example

Let’s see an example generating a Gemfile.next.lock file for Points opens a new window using a really old commit of the code, starting with Rails 5.2 and dual booting with Rails 6.0.

This was the Gemfile back then opens a new window .

Now let’s roughly compare the changes introduced for the Gemfile.next.lock with the two approaches.

Diff between original Gemfile.lock and Gemfile.next.lock using the conservative approach:

Diff between original Gemfile.lock and Gemfile.next.lock using the fast approach:

We can see that, even in a really small section of the file, the fast approach includes upgrades for a lot more gems than the safer approach, even a major version jump for database_cleaner for example. We can compare the diff that shows at the right side of each image to see how much more red and green there is for the second diff.

Fixing Bundler Conflicts

Depending on the dependencies listed in your Gemfile, trying to upgrade only the rails gem can generate conflicts, because Bundler will only update rails and it’s dependencies when possible but it won’t update other gems that may depend on different versions of the same dependencies.

When we encounter these errors, we have to go from top to bottom analyzing the reason for each conflict Bundler reports.

For example, in a different application upgrading from Rails 3.2 to 4.0, when running the next bundle update rails command, Bundler shows this error message:

You have requested:
  capistrano ~> 2.15.4

The bundle currently has capistrano locked at 2.13.5.
Try running `bundle update capistrano`

If you are updating multiple gems in your Gemfile at once,
try passing them all to `bundle update`

In this application, the Gemfile has a conditional to use different capistrano versions:

if next?
  gem 'capistrano', '~> 2.15.4'
else
  gem 'capistrano', '~> 2.13.5'
end

So we have to include capistrano in the list of gems to upgrade:

next bundle update rails capistrano

After fixing that issue, new conflicts may show up. The reason for the failure must be analyzed to know which gem update will fix it. In some cases it may require an update of the Gemfile to use different versions of a gem (for example, if we have a version restriction that is too strict for the new Rails version, so we need a conditional to loosen the restriction for the next version). Then, with the identified gem, we run the bundle update rails capistrano ... command.

Each gem added to the list should reduce the number of conflicts until the bundle update command completes successfully.

Conclusion

We recommend using the conservative approach, especially in applications with a large number of dependencies. But you can see in the example using Points that, even for fairly small projects with not many dependencies, the number of updated gems is something to consider.

It’s hard to share a metric for how many problems updating all possible gems can create and how that compares to the time doing the conservative update of gems, since it depends on too many things (size of project, number of dependencies and its dependencies, how complex the code is, etc). But, in our experience, it takes a considerably longer amount of time to debug failures than to take the time to help Bundler resolve the dependencies.

Also, the process of solving dependencies is straightforward (analyze the Bundler error message, update the Gemfile, and append a new gem to the bundle update command), while the process of solving failures will be different for each failure.