KEMBAR78
BigQuery storage: errors while trying to insert bytes array · Issue #1356 · googleapis/java-bigquerystorage · GitHub
Skip to content

BigQuery storage: errors while trying to insert bytes array #1356

@ramasLTU

Description

@ramasLTU

I've encountered this, while trying to insert BigDecimal value into BQ (NUMERIC column). I converted my decimal value to byte array and tried to insert it, but was greeted with exception:

IllegalArgumentException: JSONObject does not have a bytes field at root.col_numeric.

I've dug into the source and discovered that JsonToProtoMessage is missing return statement in cases when fieldDescriptor.getType has BYTES and value comes as instance of JSONArray.

Note: I could not use string representation of BigDecimal and let BigDecimalByteStringEncoder do it's work, cause I am forced to use lower version of guava. The class mentioned uses a Bytes.reverse method, which is not present in guava 20.0.

Environment details

  1. BigQuery storage library
  2. OS type and version: macOs Big Sur Version 11.4
  3. Java version: 1.8.0_232
  4. bigquerystorage version(s): 2.3.3

Steps to reproduce

  1. Have a table in BQ with column type NUMERIC
  2. Run adapted example code (i'm sorry, it's in Scala)

Code example

val obj = new JSONObject()
val bytes = BigDecimal(1.9).bigDecimal.setScale(9).unscaledValue().toByteArray.reverse
// or just: val bytes = Seq(0, -77, 63, 113).toArray

obj.put("col_numeric", new JSONArray(bytes))
val objects = Seq(obj).asJava.toArray

val array: JSONArray = new JSONArray(objects)
val future = writer.append(array)
future.get()

Stack trace

Exception in thread "main" java.lang.IllegalArgumentException: JSONObject does not have a bytes field at root.col_numeric.
	at com.google.cloud.bigquery.storage.v1beta2.JsonToProtoMessage.fillField(JsonToProtoMessage.java:307)
	at com.google.cloud.bigquery.storage.v1beta2.JsonToProtoMessage.convertJsonToProtoMessageImpl(JsonToProtoMessage.java:139)
	at com.google.cloud.bigquery.storage.v1beta2.JsonToProtoMessage.convertJsonToProtoMessage(JsonToProtoMessage.java:87)
	at com.google.cloud.bigquery.storage.v1beta2.JsonStreamWriter.append(JsonStreamWriter.java:110)
	at com.google.cloud.bigquery.storage.v1beta2.JsonStreamWriter.append(JsonStreamWriter.java:90)

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: bigquerystorageIssues related to the googleapis/java-bigquerystorage API.priority: p2Moderately-important priority. Fix may not be included in next release.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions