二分查找

RSpec 的 --order random 和 --seed 选项有助于找出只有在其他一个或多个示例在先执行时才会失败的闪烁示例。隔离触发失败的确切示例组合可能非常困难。--bisect 标志有助于解决这个问题。

传递 --bisect 选项（除了 --seed 和任何其他选项），RSpec 将重复运行套件的子集以隔离最小的示例集，这些示例集会复现相同的失败。

在二分查找运行过程中的任何时候，您可以按 Ctrl-C 中断，它将为您提供到目前为止发现的最小复现命令。

要获取更详细的输出（特别是在您想报告二分查找的错误时很有用），请使用 --bisect=verbose。

背景

假设一个名为 “lib/calculator.rb” 的文件，内容如下

class Calculator
  def self.add(x, y)
    x + y
  end
end

以及一个名为 “spec/calculator1spec.rb” 的文件，内容如下

require 'calculator'

RSpec.describe "Calculator" do
  it 'adds numbers' do
    expect(Calculator.add(1, 2)).to eq(3)
  end
end

以及文件 “spec/calculator2spec.rb” 到 “spec/calculator9spec.rb”，每个文件都有一个无关的通过规范

以及一个名为 “spec/calculator10spec.rb” 的文件，内容如下

require 'calculator'

RSpec.describe "Monkey patched Calculator" do
  it 'does screwy math' do
    # monkey patching `Calculator` affects examples that are
    # executed after this one!
    def Calculator.add(x, y)
      x - y
    end

    expect(Calculator.add(5, 10)).to eq(-5)
  end
end

使用 --bisect 标志为排序依赖关系创建一个最小的重现案例

当我运行 rspec --seed 1234 时

那么输出应该包含 “10 个示例，1 个失败”

当我运行 rspec --seed 1234 --bisect 时

那么二分查找应该成功，输出类似

Bisect started using options: "--seed 1234"
Running suite to find failures... (0.16755 seconds)
Starting bisect with 1 failing example and 9 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent

Round 1: bisecting over non-failing examples 1-9 .. ignoring examples 6-9 (0.30166 seconds)
Round 2: bisecting over non-failing examples 1-5 .. ignoring examples 4-5 (0.30306 seconds)
Round 3: bisecting over non-failing examples 1-3 .. ignoring example 3 (0.33292 seconds)
Round 4: bisecting over non-failing examples 1-2 . ignoring example 1 (0.16476 seconds)
Bisect complete! Reduced necessary non-failing examples from 9 to 1 in 1.26 seconds.

The minimal reproduction command is:
  rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] --seed 1234

当我运行 rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] --seed 1234 时

那么输出应该包含 “2 个示例，1 个失败”。

Ctrl-C 可用于提前中止二分查找并获取到目前为止发现的最小的命令

当我运行 rspec --seed 1234 --bisect 并在中间使用 Ctrl-C 中止时

那么二分查找应该失败，输出类似

Bisect started using options: "--seed 1234"
Running suite to find failures... (0.17102 seconds)
Starting bisect with 1 failing example and 9 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent

Round 1: bisecting over non-failing examples 1-9 .. ignoring examples 6-9 (0.32943 seconds)
Round 2: bisecting over non-failing examples 1-5 .. ignoring examples 4-5 (0.3154 seconds)
Round 3: bisecting over non-failing examples 1-3 .. ignoring example 3 (0.2175 seconds)

Bisect aborted!

The most minimal reproduction command discovered so far is:
  rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_3_spec.rb[1:1] --seed 1234

当我运行 rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_3_spec.rb[1:1] --seed 1234 时

那么输出应该包含 “3 个示例，1 个失败”。

使用 --bisect=verbose 启用详细的调试模式以获取更多细节

当我运行 rspec --seed 1234 --bisect=verbose 时

那么二分查找应该成功，输出类似

Bisect started using options: "--seed 1234" and bisect runner: :fork
Running suite to find failures... (0.16528 seconds)
 - Failing examples (1):
    - ./spec/calculator_1_spec.rb[1:1]
 - Non-failing examples (9):
    - ./spec/calculator_10_spec.rb[1:1]
    - ./spec/calculator_2_spec.rb[1:1]
    - ./spec/calculator_3_spec.rb[1:1]
    - ./spec/calculator_4_spec.rb[1:1]
    - ./spec/calculator_5_spec.rb[1:1]
    - ./spec/calculator_6_spec.rb[1:1]
    - ./spec/calculator_7_spec.rb[1:1]
    - ./spec/calculator_8_spec.rb[1:1]
    - ./spec/calculator_9_spec.rb[1:1]
Checking that failure(s) are order-dependent..
 - Running: rspec ./spec/calculator_1_spec.rb[1:1] --seed 1234 (n.nnnn seconds)
 - Failure appears to be order-dependent
Round 1: bisecting over non-failing examples 1-9
 - Running: rspec ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_6_spec.rb[1:1] ./spec/calculator_7_spec.rb[1:1] ./spec/calculator_8_spec.rb[1:1] ./spec/calculator_9_spec.rb[1:1] --seed 1234 (0.15302 seconds)
 - Running: rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_2_spec.rb[1:1] ./spec/calculator_3_spec.rb[1:1] ./spec/calculator_4_spec.rb[1:1] ./spec/calculator_5_spec.rb[1:1] --seed 1234 (0.19708 seconds)
 - Examples we can safely ignore (4):
    - ./spec/calculator_6_spec.rb[1:1]
    - ./spec/calculator_7_spec.rb[1:1]
    - ./spec/calculator_8_spec.rb[1:1]
    - ./spec/calculator_9_spec.rb[1:1]
 - Remaining non-failing examples (5):
    - ./spec/calculator_10_spec.rb[1:1]
    - ./spec/calculator_2_spec.rb[1:1]
    - ./spec/calculator_3_spec.rb[1:1]
    - ./spec/calculator_4_spec.rb[1:1]
    - ./spec/calculator_5_spec.rb[1:1]
Round 2: bisecting over non-failing examples 1-5
 - Running: rspec ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_4_spec.rb[1:1] ./spec/calculator_5_spec.rb[1:1] --seed 1234 (0.15836 seconds)
 - Running: rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_2_spec.rb[1:1] ./spec/calculator_3_spec.rb[1:1] --seed 1234 (0.19065 seconds)
 - Examples we can safely ignore (2):
    - ./spec/calculator_4_spec.rb[1:1]
    - ./spec/calculator_5_spec.rb[1:1]
 - Remaining non-failing examples (3):
    - ./spec/calculator_10_spec.rb[1:1]
    - ./spec/calculator_2_spec.rb[1:1]
    - ./spec/calculator_3_spec.rb[1:1]
Round 3: bisecting over non-failing examples 1-3
 - Running: rspec ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_2_spec.rb[1:1] --seed 1234 (0.21028 seconds)
 - Running: rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] ./spec/calculator_3_spec.rb[1:1] --seed 1234 (0.1975 seconds)
 - Examples we can safely ignore (1):
    - ./spec/calculator_2_spec.rb[1:1]
 - Remaining non-failing examples (2):
    - ./spec/calculator_10_spec.rb[1:1]
    - ./spec/calculator_3_spec.rb[1:1]
Round 4: bisecting over non-failing examples 1-2
 - Running: rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] --seed 1234 (0.17173 seconds)
 - Examples we can safely ignore (1):
    - ./spec/calculator_3_spec.rb[1:1]
 - Remaining non-failing examples (1):
    - ./spec/calculator_10_spec.rb[1:1]
Bisect complete! Reduced necessary non-failing examples from 9 to 1 in 1.47 seconds.

The minimal reproduction command is:
  rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] --seed 1234

当我运行 rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] --seed 1234 时

那么输出应该包含 “2 个示例，1 个失败”。

通过配置选项选择二分查找运行器

假设一个名为 “spec/spec_helper.rb” 的文件，内容如下

RSpec.configure do |c|
  c.bisect_runner = :shell
end

以及一个名为 “.rspec” 的文件，内容如下

--require spec_helper

当我运行 rspec --seed 1234 --bisect=verbose 时

那么二分查找应该成功，输出类似

Bisect started using options: "--seed 1234" and bisect runner: :shell
# ...
The minimal reproduction command is:
  rspec ./spec/calculator_10_spec.rb[1:1] ./spec/calculator_1_spec.rb[1:1] --seed 1234