KEMBAR78
fix: table iterator should not use bqstorage when page_size is not None by chelsea-lin · Pull Request #2154 · googleapis/python-bigquery · GitHub
Skip to content

fix: table iterator should not use bqstorage when page_size is not None #2154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 3, 2025
Merged

fix: table iterator should not use bqstorage when page_size is not None #2154

merged 2 commits into from
Apr 3, 2025

Conversation

chelsea-lin
Copy link
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@chelsea-lin chelsea-lin requested a review from tswast April 2, 2025 23:16
@chelsea-lin chelsea-lin requested review from a team as code owners April 2, 2025 23:16
@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery API. labels Apr 2, 2025
@chelsea-lin chelsea-lin force-pushed the main_chelsealin_page_size branch from 0cc6e45 to 966cd28 Compare April 2, 2025 23:47
tswast
tswast approved these changes Apr 3, 2025
View reviewed changes
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tswast
Copy link
Contributor

tswast commented Apr 3, 2025

Looks like two tests are failing:

FAILED tests/unit/test_dbapi_cursor.py::TestCursor::test_fetchall_w_bqstorage_client_fetch_error_no_fallback
FAILED tests/unit/test_dbapi_cursor.py::TestCursor::test_fetchall_w_bqstorage_client_no_arrow_compression

I think we need to remove this logic:

python-bigquery/google/cloud/bigquery/dbapi/cursor.py

Lines 380 to 384 in 77d7173

if size is None:
# Since self.arraysize can be None (a deviation from PEP 249),
# use an actual PEP 249 default of 1 in such case (*some* number
# is needed here).
size = self.arraysize if self.arraysize else 1

Page size of 1 is going to be super slow, so I don't think we should be setting that if people explicitly request "None" as the page size.

@chelsea-lin chelsea-lin force-pushed the main_chelsealin_page_size branch from 966cd28 to 33709b1 Compare April 3, 2025 18:22
@chelsea-lin chelsea-lin merged commit e89a707 into main Apr 3, 2025
18 checks passed
@chelsea-lin chelsea-lin deleted the main_chelsealin_page_size branch April 3, 2025 18:45
@release-please release-please bot mentioned this pull request Apr 3, 2025
Merged
chelsea-lin added a commit that referenced this pull request May 19, 2025
@chelsea-lin
Revert "fix: table iterator should not use bqstorage when page_size i…
453d7ec 
…s not None (#2154)"

This reverts commit e89a707.
@shollyman shollyman mentioned this pull request May 19, 2025
Merged
Linchin pushed a commit that referenced this pull request May 19, 2025
@chelsea-lin
Revert "fix: table iterator should not use bqstorage when page_size i…
06ee3df 
…s not None (#2154)" (#2191)

This reverts commit e89a707.
This was referenced May 20, 2025
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tswast tswast tswast approved these changes

Assignees

@leahecole leahecole

Labels

api: bigquery Issues related to the googleapis/python-bigquery API. size: s Pull request size is small.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@chelsea-lin @tswast @leahecole