Logically Negating an ActiveRecord scope

If you simply want to know how to negate an ActiveRecord scope, and you don’t care how it works, here’s the TL;DR:

scope(:not), ->(scope) { where(scope.where_values.reduce(:and).not) }

Arel is the powerful library that powers the Rails ActiveRecord library. It’s also black magic (and largely undocumented black magic at that). It was created in isolation from Rails, and later retrofitted into ActiveRecord. Prepare yourself for a crash course in the dark arts of Arel.

Imagine you’re creating a blog, and you needed a scope that would show you all of the posts that were recently tagged:

class Post < ActiveRecord::Base   scope :recently_tagged, -> {
    where(
      arel_table[:tagged_at].gt(10.days.ago)
    ).
    where(arel_table[:tags].not_eq(nil))
  }
end

We’ve chained two `where` clauses together with ActiveRecord, which will join them together with `AND`. In other words, a post that has tags and has had its tags changed within the last 10 days is “recently tagged”. Freed of the syntactic trappings of ActiveRecord and Arel, it can be simply represented with the following boolean expression:

tagged_at > 10.days.ago && tags != nil

Now suppose you’d like to find the opposite: the list of all posts that were not recently tagged (either because their tags haven’t changed in the last 10 days, or because all of their tags were removed). In boolean logic, this is simply the logical negation of the “recently_tagged” scope:

!(tagged_at > 10.days.ago && tags != nil)

which can be simplified to:

tagged_at <= 10.days.ago || tags == nil

The question is, how do we tell ActiveRecord / Arel to negate a scope? The answer is more difficult than you might expect.

Let’s start with looking at what a scope is:

> Post.recently_tagged.class
 => ActiveRecord::Relation::ActiveRecord_Relation_Post

That seems slightly odd. Apparently ActiveRecord generates a class for a scope. Let’s look at what this class is made of:

ActiveRecord::Relation::ActiveRecord_Relation_Post.ancestors
 => [ActiveRecord::Relation::ActiveRecord_Relation_Post, ActiveRecord::Delegation::ClassSpecificRelation, ActiveRecord::Relation, ActiveRecord::Relation::DeprecatedMethods, …...........]

Not suprisingly, it looks like ActiveRecord_Relation_Post is a type of ActiveRecord::Relation. If you peek inside ActiveRecord::Relation, you’ll see it’s a sort of wrapper around Arel (along with a few other ActiveRecord classes and modules). At the top of the ActiveRecord::Relation class file, there are several constants defined, including the following:

MULTI_VALUE_METHODS  = [
  :includes,
  :eager_load,
  :preload,
  :select,
  :group,
  :order,
  :joins,
  :where,
  :having,
  :bind,
  :references,
  :extending
]

This constant, along with the SINGLE_VALUE_METHODS constant, describes all of the possible constituents of an ActiveRecord scope. In our case, we want to peel off the where values of a scope so that we can logically negate them. Searching for usages of the ‘MULTI_VALUE_METHODS’ will lead you to this juicy bit of metaprogramming in ActiveRecord::QueryMethods:

Relation::MULTI_VALUE_METHODS.each do |name|
  class_eval <<-CODE, __FILE__, __LINE__ + 1
    def #{name}_values                   # def select_values
      @values[:#{name}] || []            #   @values[:select] || []
    end                                  # end
                                         #
    def #{name}_values=(values)          # def select_values=(values)
      raise ImmutableRelation if @loaded #   raise ImmutableRelation if @loaded
      @values[:#{name}] = values         #   @values[:select] = values
    end                                  # end
  CODE
end

ActiveRecord creates accessor methods for each of the scope’s constituent parts; since we’re interested in the where values, let’s see what `where_values` returns (note that you could also call values[:where]):

> Post.recently_tagged.where_values => [#<Arel::Nodes::GreaterThan:0x007fa759f263b0 @left=#<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x007fa75d011ba0 @name="posts", @engine=Post(id: integer, tagged_at: datetime, tags: string, created_at: datetime, updated_at: datetime), @columns=nil, @aliases=[], @table_alias=nil, @primary_key=nil>, name=:tagged_at>, @right=Fri, 28 Jun 2013 05:17:39 UTC +00:00>, #<Arel::Nodes::NotEqual:0x007fa759f26018 @left=#<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x007fa75d011ba0 @name="posts", @engine=Post(id: integer, tagged_at: datetime, tags: string, created_at: datetime, updated_at: datetime), @columns=nil, @aliases=[], @table_alias=nil, @primary_key=nil>, name=:tags>, @right=nil>]

Huzzah! We’ve now found the mysterious Arel. Although these class inspections don’t make it entirely obvious, these Arel nodes actually form the nodes of a tree representing boolean expressions in SQL.

The first node is:

         >
        /
       /
      /
     /
:tagged_at  10.days.ago

And the second is simply:

        !=
        /
       /
    :tags  nil

Now that we have these where values, what can we do with them? Well, we’d like to AND them back together, and then negate them. Fortunately, there are methods for both of these things on any Arel::Nodes::Node:

> Post.recently_tagged.values[:where].first.method(:not)
 #<Method: Arel::Nodes::GreaterThan(Arel::Nodes::Node)#not>
> Post.recently_tagged.values[:where].first.method(:and)
  #<Method: Arel::Nodes::GreaterThan(Arel::Nodes::Node)#and>

As you would expect, calling either of these methods will return a new Arel::Nodes::Node. In our specific case, we could call these methods directly:

> Post.recently_tagged.values[:where].first.and(Post.recently_tagged.values[:where].last).not

The `not` at the end of this expression is what we’ve been searching for. However, this returns an Arel::Nodes::Node (or more specifically, an Arel::Nodes::Not), but we’re using ActiveRecord – we need an ActiveRecord::Relation. Luckily, that’s exactly what `where` returns, and it can take, as an argument, an Arel::Nodes::Node.

> Post.where(Post.recently_tagged.values[:where].first.and(Post.recently_tagged.values[:where].last).not)

And this will, in fact, return what we’ve been searching for: all posts not recently tagged. It’s of course, incredibly ugly and not very reusable; luckily, we can create a generic negation scope with a little help from Ruby’s Enumerable#reduce method:

class Post
  #....
  scope :not, ->(scope_name) { where(send(scope_name).where_values.reduce(:and).not) }
end

And now, you could use it whenever you like:

Post.not(:recently_tagged)

About the Author

Matt Parker is Head of Engineering for Pivotal Labs