KEMBAR78
Fix flaky test: prevent asymmetric utilization IO/CPU utilization by timvisee · Pull Request #6502 · qdrant/qdrant · GitHub
Skip to content

Conversation

@timvisee
Copy link
Member

@timvisee timvisee commented May 7, 2025

Our test_cancel_optimization test is flaky. It fails on high CPU systems.

$ QDRANT_NUM_CPUS=24 cargo test -p collection -- test_cancel_optimization

---- tests::test_cancel_optimization stdout ----
actual_optimization_duration = 114 ms

thread 'tests::test_cancel_optimization' panicked at lib/collection/src/tests/mod.rs:210:9:
assertion `left == right` failed
  left: 2
 right: 3

The reason is asymmetric utilization of IO and CPU budget (implemented in #6015). On 24 CPUs, the previous implementation allowed 2 concurrent IO tasks while it allowed 3 concurrent CPU tasks.

This adjusts the permit selection to make it symmetric.

If we don't like this, the alternative would be to adjust the test. Change this to:

// We skip optimizations that use less than half of the preferred CPU budget
let expected_optimization_count = {
    let cpus = common::cpu::get_cpu_budget(0);
    let hnsw_threads = num_rayon_threads(0);
    (cpus / hnsw_threads).clamp(1, 3)
};

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you formatted your code locally using cargo +nightly fmt --all command prior to submission?
  3. Have you checked your code using cargo clippy --all --all-features command?

Copilot

This comment was marked as resolved.

@coderabbitai

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@generall generall merged commit 4a94691 into dev May 9, 2025
17 checks passed
@generall generall deleted the fix-flaky-test-cancel-optimization branch May 9, 2025 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants