instance method `find_in_batches`

Ruby on Rails 7.0.10

Since v3.1.12

Available in: v3.1.12 v3.2.22.5 v4.0.13 v4.1.16 v4.2.9 v5.2.8.1 v6.0.6 v6.1.7.10 v7.0.10 v7.1.6 v7.2.3 v8.0.4 v8.1.2

Compare with:

find_in_batches(start: nil, finish: nil, batch_size: 1000, error_on_ignore: nil, order: :asc)

Yields each batch of records that was found by the find options as an array.

Person.where("age > 21").find_in_batches do |group|
  sleep(50) # Make sure it doesn't get too crowded in there!
  group.each { |person| person.party_all_night! }
end

If you do not provide a block to #find_in_batches, it will return an Enumerator for chaining with other methods:

Person.find_in_batches.with_index do |group, batch|
  puts "Processing group ##{batch}"
  group.each(&:recover_from_last_night!)
end

To be yielded each record one by one, use #find_each instead.

Options

:batch_size - Specifies the size of the batch. Defaults to 1000.
:start - Specifies the primary key value to start from, inclusive of the value.
:finish - Specifies the primary key value to end at, inclusive of the value.
:error_on_ignore - Overrides the application config to specify if an error should be raised when an order is present in the relation.
:order - Specifies the primary key order (can be :asc or :desc). Defaults to :asc.

Limits are honored, and if present there is no requirement for the batch size: it can be less than, equal to, or greater than the limit.

The options start and finish are especially useful if you want multiple workers dealing with the same processing queue. You can make worker 1 handle all the records between id 1 and 9999 and worker 2 handle from 10000 and beyond by setting the :start and :finish option on each worker.

# Let's process from record 10_000 on.
Person.find_in_batches(start: 10_000) do |group|
  group.each { |person| person.party_all_night! }
end

NOTE: Order can be ascending (:asc) or descending (:desc). It is automatically set to ascending on the primary key (“id ASC”). This also means that this method only works when the primary key is orderable (e.g. an integer or string).

NOTE: By its nature, batch processing is subject to race conditions if other processes are modifying the database.

Parameters

start key = nil
finish key = nil
batch_size key = 1000
error_on_ignore key = nil
order key = :asc

Source

# File activerecord/lib/active_record/relation/batches.rb, line 128
    def find_in_batches(start: nil, finish: nil, batch_size: 1000, error_on_ignore: nil, order: :asc)
      relation = self
      unless block_given?
        return to_enum(:find_in_batches, start: start, finish: finish, batch_size: batch_size, error_on_ignore: error_on_ignore, order: order) do
          total = apply_limits(relation, start, finish, order).size
          (total - 1).div(batch_size) + 1
        end
      end

      in_batches(of: batch_size, start: start, finish: finish, load: true, error_on_ignore: error_on_ignore, order: order) do |batch|
        yield batch.to_a
      end
    end

Defined in activerecord/lib/active_record/relation/batches.rb line 128 · View on GitHub · Improve this page · Find usages on GitHub

Defined in ActiveRecord::Batches

Signature

Options

Parameters