feat: better default stream support in client library #750

yirutang · 2020-12-18T22:57:00Z

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
Ensure the tests and linter pass
Code coverage does not decrease (if any source code was changed)
Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

generated-files-bot · 2020-12-18T22:57:04Z

Warning: This pull request is touching the following templated files:

.github/generated-files-bot.yml

stephaniewang526 · 2020-12-21T17:38:52Z

PTAL below error:

Error:  /home/runner/work/java-bigquerystorage/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:1: Line does not match expected header line of '^/\*$'. [RegexpHeader]

stephaniewang526 · 2020-12-21T19:27:44Z

One unit test failed:

Error:  Failures: 
Error:    JsonStreamWriterTest.testTwoParamNewBuilder_nullSchema:220 expected:<Stream[OrTable]Name is null.> but was:<Stream[]Name is null.>

Depends on googleapis/repo-automation-bots#1254 Fixes googleapis/repo-automation-bots#1096 Source-Author: Jeff Ching <chingor@google.com> Source-Date: Tue Dec 15 16:16:07 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: 3f67ceece7e797a5736a25488aae35405649b90b Source-Link: googleapis/synthtool@3f67cee

@miraleung

* changes without context autosynth cannot find the source of changes triggered by earlier changes in this repository, or by version upgrades to tools such as linters. * chore: migrate java-bigquerystorage to the Java microgenerator Committer: @miraleung PiperOrigin-RevId: 345311069 Source-Author: Google APIs <noreply@google.com> Source-Date: Wed Dec 2 14:17:15 2020 -0800 Source-Repo: googleapis/googleapis Source-Sha: e39e42f368d236203a774ee994fcb4d730c33a83 Source-Link: googleapis/googleapis@e39e42f * feat!: Updates to BigQuery Write API V1Beta2 public interface. This includes breaking changes to the API, it is fine because the API is not officially launched yet. PiperOrigin-RevId: 345469340 Source-Author: Google APIs <noreply@google.com> Source-Date: Thu Dec 3 09:33:11 2020 -0800 Source-Repo: googleapis/googleapis Source-Sha: b53c4d98aab1eae3dac90b37019dede686782f13 Source-Link: googleapis/googleapis@b53c4d9 * fix: Update gapic-generator-java to 0.0.7 Committer: @miraleung PiperOrigin-RevId: 345476969 Source-Author: Google APIs <noreply@google.com> Source-Date: Thu Dec 3 10:07:32 2020 -0800 Source-Repo: googleapis/googleapis Source-Sha: 7be2c821dd88109038c55c89f7dd48f092eeab9d Source-Link: googleapis/googleapis@7be2c82 * chore: rollback migrating java-bigquerystorage to the Java microgenerator Committer: @miraleung PiperOrigin-RevId: 345522380 Source-Author: Google APIs <noreply@google.com> Source-Date: Thu Dec 3 13:28:07 2020 -0800 Source-Repo: googleapis/googleapis Source-Sha: f8f975c7d43904e90d6c5f1684fdb6804400e641 Source-Link: googleapis/googleapis@f8f975c * chore: migrate java-bigquerystorage to the Java microgenerator Committer: @miraleung PiperOrigin-RevId: 346405446 Source-Author: Google APIs <noreply@google.com> Source-Date: Tue Dec 8 14:03:11 2020 -0800 Source-Repo: googleapis/googleapis Source-Sha: abc43060f136ce77124754a48f367102e646844a Source-Link: googleapis/googleapis@abc4306 * chore: update gapic-generator-java to 0.0.11 Committer: @miraleung PiperOrigin-RevId: 347036369 Source-Author: Google APIs <noreply@google.com> Source-Date: Fri Dec 11 11:13:47 2020 -0800 Source-Repo: googleapis/googleapis Source-Sha: 6d65640b1fcbdf26ea76cb720de0ac138cae9bed Source-Link: googleapis/googleapis@6d65640 * chore: update gapic-generator-java to 0.0.13 Committer: @miraleung PiperOrigin-RevId: 347849179 Source-Author: Google APIs <noreply@google.com> Source-Date: Wed Dec 16 10:28:38 2020 -0800 Source-Repo: googleapis/googleapis Source-Sha: 360a0e177316b7e9811f2ccbbef11e5f83377f3f Source-Link: googleapis/googleapis@360a0e1

stephaniewang526 · 2020-12-23T16:42:27Z

Getting NPE in ITBigQueryWriteManualClientTest.testJsonStreamWriterBatchWriteWithDefaultStream

java.lang.NullPointerException
	at com.google.cloud.bigquery.storage.v1beta2.TableFieldSchema$Builder.setMode(TableFieldSchema.java:1647)
	at com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter.ConvertFieldSchema(BQV2ToBQStorageConverter.java:69)
	at com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter.ConvertTableSchema(BQV2ToBQStorageConverter.java:56)
	at com.google.cloud.bigquery.storage.v1beta2.JsonStreamWriter.newBuilder(JsonStreamWriter.java:253)
	at com.google.cloud.bigquery.storage.v1beta2.it.ITBigQueryWriteManualClientTest.testJsonStreamWriterBatchWriteWithDefaultStream(ITBigQueryWriteManualClientTest.java:320)
...

full trace

yoshi-code-bot · 2020-12-28T03:59:57Z

...torage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java

+import com.google.cloud.bigquery.StandardSQLTypeName;
+import com.google.cloud.bigquery.storage.v1alpha2.Table;


Suggested change

import com.google.cloud.bigquery.StandardSQLTypeName;

import com.google.cloud.bigquery.storage.v1alpha2.Table;

import com.google.cloud.bigquery.StandardSQLTypeName;

codecov · 2020-12-28T04:06:44Z

Codecov Report

Merging #750 (0466702) into master (f3c897f) will increase coverage by 0.08%.
The diff coverage is 82.50%.

@@             Coverage Diff              @@
##             master     #750      +/-   ##
============================================
+ Coverage     80.80%   80.89%   +0.08%     
- Complexity      985      994       +9     
============================================
  Files            73       74       +1     
  Lines          5289     5339      +50     
  Branches        406      414       +8     
============================================
+ Hits           4274     4319      +45     
- Misses          845      848       +3     
- Partials        170      172       +2

Impacted Files	Coverage Δ	Complexity Δ
...oud/bigquery/storage/v1beta2/JsonStreamWriter.java	`78.33% <79.16%> (+1.14%)`	`14.00 <2.00> (+2.00)`
...e/cloud/bigquery/storage/v1beta2/StreamWriter.java	`85.01% <82.60%> (-0.08%)`	`38.00 <2.00> (+1.00)`	⬇️
...uery/storage/v1beta2/BQV2ToBQStorageConverter.java	`84.84% <84.84%> (ø)`	`6.00 <6.00> (?)`
.../cloud/bigquery/storage/v1alpha2/StreamWriter.java	`84.31% <0.00%> (+0.43%)`	`37.00% <0.00%> (ø%)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3c897f...0466702. Read the comment docs.

…#750)

yirutang requested a review from a team December 18, 2020 22:57

yirutang requested a review from a team as a code owner December 18, 2020 22:57

yirutang requested a review from stephaniewang526 December 18, 2020 22:57

product-auto-label bot added the api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. label Dec 18, 2020

yirutang changed the title ~~feat: better default stream support in client library.~~ feat: better default stream support in client library Dec 18, 2020

google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 18, 2020

yirutang and others added 16 commits December 22, 2020 13:33

.

01e46df

.

d205dd1

.

70d9a1e

feat: Support default stream on streamWriter and jsonStreamWriter

55b878f

add integration test

14a4444

.

017a543

.

5e9888b

.

489bdbd

.

72f0191

.

2297a30

.

940a43e

.

b9d53e6

.

97edec4

.

a24ae87

.

cf92c02

yoshi-code-bot reviewed Dec 28, 2020

View reviewed changes

.

e68f2d1

yirutang and others added 2 commits December 28, 2020 10:22

fix flushall test

287c393

Merge branch 'master' into default-stream-1

0466702

stephaniewang526 approved these changes Dec 30, 2020

View reviewed changes

stephaniewang526 merged commit 488f258 into googleapis:master Dec 30, 2020

yirutang deleted the default-stream-1 branch February 18, 2021 00:10

shubhwip pushed a commit to shubhwip/java-bigquerystorage that referenced this pull request Oct 7, 2023

fix(java): java 17 dependency arguments (googleapis#1266) (googleapis…

3f866da

…#750)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: better default stream support in client library #750

feat: better default stream support in client library #750

Uh oh!

yirutang commented Dec 18, 2020

Uh oh!

generated-files-bot bot commented Dec 18, 2020 •

edited

Loading

Uh oh!

stephaniewang526 commented Dec 21, 2020

Uh oh!

stephaniewang526 commented Dec 21, 2020

Uh oh!

stephaniewang526 commented Dec 23, 2020

Uh oh!

yoshi-code-bot Dec 28, 2020

Uh oh!

codecov bot commented Dec 28, 2020 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		import com.google.cloud.bigquery.StandardSQLTypeName;
		import com.google.cloud.bigquery.storage.v1alpha2.Table;

feat: better default stream support in client library #750

feat: better default stream support in client library #750

Uh oh!

Conversation

yirutang commented Dec 18, 2020

Uh oh!

generated-files-bot bot commented Dec 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephaniewang526 commented Dec 21, 2020

Uh oh!

stephaniewang526 commented Dec 21, 2020

Uh oh!

stephaniewang526 commented Dec 23, 2020

Uh oh!

yoshi-code-bot Dec 28, 2020

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Dec 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

generated-files-bot bot commented Dec 18, 2020 •

edited

Loading

codecov bot commented Dec 28, 2020 •

edited

Loading