KEMBAR78
Add http.client.RemoteDisconnected to list of retryable exceptions · Issue #1231 · googleapis/python-storage · GitHub
Skip to content

Add http.client.RemoteDisconnected to list of retryable exceptions #1231

@andrewpollock

Description

@andrewpollock

Environment details

  • OS type and version:
  • Python version: Python 3.11.4
  • pip version: pip 23.1.2 from /usr/local/lib/python3.11/site-packages/pip (python 3.11)
  • google-cloud-storage version: 2.14.0

Steps to reproduce

  1. Usage of Blob.upload_from_filename() seems to sometimes raise a RemoteDisconnected exception
  2. https://docs.python.org/3/library/http.client.html#http.client.RemoteDisconnected get caught by our exception handler (see below)
  3. It doesn't look like
    _RETRYABLE_TYPES = (
    api_exceptions.TooManyRequests, # 429
    api_exceptions.InternalServerError, # 500
    api_exceptions.BadGateway, # 502
    api_exceptions.ServiceUnavailable, # 503
    api_exceptions.GatewayTimeout, # 504
    ConnectionError,
    requests.ConnectionError,
    requests_exceptions.ChunkedEncodingError,
    requests_exceptions.Timeout,
    )
    will treat this as retryable?

I think this is in the same vein as #414 and #426

Code example

https://github.com/google/osv.dev/blob/b705d0d0b7450ce94137624118a2b54a7f719147/docker/exporter/exporter.py#L57-L64

  def upload_single(self, bucket, source_path, target_path):
    """Upload a single file to a bucket."""
    logging.info('Uploading %s', target_path)
    try:
      blob = bucket.blob(target_path)
      blob.upload_from_filename(source_path)
    except Exception as e:
      logging.error('Failed to export: %s', e)

Stack trace

Failed to export: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the googleapis/python-storage API.priority: p3Desirable enhancement or fix. May not be included in next release.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions