-
Notifications
You must be signed in to change notification settings - Fork 163
Closed
Labels
api: storageIssues related to the googleapis/python-storage API.Issues related to the googleapis/python-storage API.priority: p3Desirable enhancement or fix. May not be included in next release.Desirable enhancement or fix. May not be included in next release.type: questionRequest for information or clarification. Not an issue.Request for information or clarification. Not an issue.
Description
Environment details
- OS type and version:
- Python version:
Python 3.11.4 - pip version:
pip 23.1.2 from /usr/local/lib/python3.11/site-packages/pip (python 3.11) google-cloud-storageversion:2.14.0
Steps to reproduce
- Usage of
Blob.upload_from_filename()seems to sometimes encounter a 503 response - This seems to result in an immediate failure, despite indications in that
python-storage/google/cloud/storage/retry.py
Lines 28 to 38 in ae9a53b
_RETRYABLE_TYPES = ( api_exceptions.TooManyRequests, # 429 api_exceptions.InternalServerError, # 500 api_exceptions.BadGateway, # 502 api_exceptions.ServiceUnavailable, # 503 api_exceptions.GatewayTimeout, # 504 ConnectionError, requests.ConnectionError, requests_exceptions.ChunkedEncodingError, requests_exceptions.Timeout, ) api_exceptions.ServiceUnavailableis a 503 and should be retried.
Code example
def upload_single(self, bucket, source_path, target_path):
"""Upload a single file to a bucket."""
logging.info('Uploading %s', target_path)
try:
blob = bucket.blob(target_path)
blob.upload_from_filename(source_path)
except Exception as e:
logging.error('Failed to export: %s', e)Stack trace
Failed to export: 503 POST https://storage.googleapis.com/upload/storage/v1/b/osv-test-vulnerabilities/o?uploadType=multipart: {
"error": {
"code": 503,
"message": "We encountered an internal error. Please try again.",
"errors": [
{
"message": "We encountered an internal error. Please try again.",
"domain": "global",
"reason": "backendError"
}
]
}
}
: ('Request failed with status code', 503, 'Expected one of', <HTTPStatus.OK: 200>)
Perhaps 503 needs to be added to _ADDITIONAL_RETRYABLE_STATUS_CODES also?
Metadata
Metadata
Assignees
Labels
api: storageIssues related to the googleapis/python-storage API.Issues related to the googleapis/python-storage API.priority: p3Desirable enhancement or fix. May not be included in next release.Desirable enhancement or fix. May not be included in next release.type: questionRequest for information or clarification. Not an issue.Request for information or clarification. Not an issue.