KEMBAR78
Use test-queue for parallel testing by koic · Pull Request #1570 · octokit/octokit.rb · GitHub
Skip to content

Conversation

@koic
Copy link
Contributor

@koic koic commented May 7, 2023

This PR makes developer tests parallel using test-queue: https://github.com/tmm1/test-queue

Behavior

Before

Around 55 seconds:

$ bundle exec rake spec
(snip)

Finished in 55.03 seconds (files took 0.77671 seconds to load)
865 examples, 0 failures

Randomized with seed 6253

After

Around 11 seconds (e.g. 8 CPUs with hyper threading) :

$ bundle exec rake spec
rspec-queue spec
Starting test-queue master (/tmp/test_queue_12223_3620.sock)

==> Summary (16 workers in 11.6051s)

    [ 1]                                      65 examples, 0 failures         1 suites in 7.1969s      (pid 12226 exit 0 )
    [ 2]                                      19 examples, 0 failures         1 suites in 7.3070s      (pid 12227 exit 0 )
    [ 3]                                      20 examples, 0 failures         1 suites in 11.5977s      (pid 12228 exit 0 )
    [ 4]                                      80 examples, 0 failures         3 suites in 11.5978s      (pid 12229 exit 0 )
    [ 5]                                     101 examples, 0 failures         3 suites in 11.5977s      (pid 12230 exit 0 )
    [ 6]                                      55 examples, 0 failures         5 suites in 11.5976s      (pid 12231 exit 0 )
    [ 7]                                      49 examples, 0 failures         4 suites in 11.5975s      (pid 12232 exit 0 )
    [ 8]                                      57 examples, 0 failures         6 suites in 11.5974s      (pid 12233 exit 0 )
    [ 9]                                      31 examples, 0 failures         6 suites in 11.5973s      (pid 12234 exit 0 )
    [10]                                      46 examples, 0 failures         4 suites in 11.5972s      (pid 12235 exit 0 )
    [11]                                      55 examples, 0 failures         5 suites in 11.5971s      (pid 12236 exit 0 )
    [12]                                      57 examples, 0 failures         6 suites in 11.5968s      (pid 12237 exit 0 )
    [13]                                      70 examples, 0 failures         6 suites in 11.5967s      (pid 12238 exit 0 )
    [14]                                      57 examples, 0 failures         5 suites in 11.5962s      (pid 12239 exit 0 )
    [15]                                      47 examples, 0 failures         6 suites in 11.5951s      (pid 12240 exit 0 )
    [16]                                      56 examples, 0 failures         6 suites in 11.5933s      (pid 12241 exit 0 )

It can be up to 5x faster depending on the environment.

Other information

If fork is not supported, parallelization is not possible.


Additional info

Pull request checklist

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been reviewed and added / updated if needed (for bug fixes / features)
  • Added the appropriate label for the given change

Does this introduce a breaking change?

Please see our docs on breaking changes to help!

  • Yes (Please add the Type: Breaking change label)
  • No

If Yes, what's the impact:

  • N/A

Pull request type

Please add the corresponding label for change this PR introduces:

  • Bugfix: Type: Bug
  • Feature/model/API additions: Type: Feature
  • Updates to docs or samples: Type: Documentation
  • Dependencies/code cleanup: Type: Maintenance

This PR makes developer tests parallel using test-queue:
https://github.com/tmm1/test-queue

## Before

Around 55 seconds:

```console
$ bundle exec rake spec
(snip)

Finished in 55.03 seconds (files took 0.77671 seconds to load)
865 examples, 0 failures

Randomized with seed 6253
```

## After

Around 11 seconds (e.g. 8 CPUs with hyper threading) :

```console
$ bundle exec rake spec
rspec-queue spec
Starting test-queue master (/tmp/test_queue_12223_3620.sock)

==> Summary (16 workers in 11.6051s)

    [ 1]                                      65 examples, 0 failures         1 suites in 7.1969s      (pid 12226 exit 0 )
    [ 2]                                      19 examples, 0 failures         1 suites in 7.3070s      (pid 12227 exit 0 )
    [ 3]                                      20 examples, 0 failures         1 suites in 11.5977s      (pid 12228 exit 0 )
    [ 4]                                      80 examples, 0 failures         3 suites in 11.5978s      (pid 12229 exit 0 )
    [ 5]                                     101 examples, 0 failures         3 suites in 11.5977s      (pid 12230 exit 0 )
    [ 6]                                      55 examples, 0 failures         5 suites in 11.5976s      (pid 12231 exit 0 )
    [ 7]                                      49 examples, 0 failures         4 suites in 11.5975s      (pid 12232 exit 0 )
    [ 8]                                      57 examples, 0 failures         6 suites in 11.5974s      (pid 12233 exit 0 )
    [ 9]                                      31 examples, 0 failures         6 suites in 11.5973s      (pid 12234 exit 0 )
    [10]                                      46 examples, 0 failures         4 suites in 11.5972s      (pid 12235 exit 0 )
    [11]                                      55 examples, 0 failures         5 suites in 11.5971s      (pid 12236 exit 0 )
    [12]                                      57 examples, 0 failures         6 suites in 11.5968s      (pid 12237 exit 0 )
    [13]                                      70 examples, 0 failures         6 suites in 11.5967s      (pid 12238 exit 0 )
    [14]                                      57 examples, 0 failures         5 suites in 11.5962s      (pid 12239 exit 0 )
    [15]                                      47 examples, 0 failures         6 suites in 11.5951s      (pid 12240 exit 0 )
    [16]                                      56 examples, 0 failures         6 suites in 11.5933s      (pid 12241 exit 0 )
```

It can be up to 5x faster depending on the environment.
@koic
Copy link
Contributor Author

koic commented May 7, 2023

Here is the result from CI:

Before (27.87 seconds)

Run bundle exec rspec -w
(snip)

Finished in 27.87 seconds (files took 0.63688 seconds to load)
875 examples, 0 failures

https://github.com/octokit/octokit.rb/actions/runs/4725471255/jobs/8383924167

After (12.2976 seconds)

Run bundle exec rspec-queue -w
(snip)

==> Summary (2 workers in 12.2976s)

    [ 1]                                     634 examples, 0 failures        53 suites in 12.2939s      (pid 2000 exit 0 )
    [ 2]                                     241 examples, 0 failures        17 suites in 12.2940s      (pid 2001 exit 0 )

https://github.com/octokit/octokit.rb/actions/runs/4906312889/jobs/8760680945?pr=1570

On 2 CPUs GitHub Actions, it's around 2x faster.

@nickfloyd nickfloyd added the Type: Feature New feature or request label May 12, 2023
Copy link
Contributor

@nickfloyd nickfloyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid change @koic, thanks for doing this ❤️

@nickfloyd nickfloyd merged commit d5eec6e into octokit:main May 12, 2023
@koic koic deleted the use_test_queue branch May 13, 2023 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Type: Feature New feature or request

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants