KEMBAR78
After reloading `tensorboard` module, public APIs are broken · Issue #1989 · tensorflow/tensorboard · GitHub
Skip to content

After reloading tensorboard module, public APIs are broken #1989

@wchargin

Description

@wchargin

The public APIs notebook, program, and summary are exposed from
the tensorboard module as LazyModules. If the tensorboard module
is reloaded after one of these modules has been dereferenced, then
attempting to dereference that module again will blow the stack on a
recursion error.

In a Python 2 virtualenv:

$ python2
Python 2.7.14+ (default, Dec  5 2017, 15:17:02) 
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorboard
>>> tensorboard.program.TensorBoard
<class 'tensorboard.program.TensorBoard'>
>>> import imp  # Python 2
>>> imp.reload(tensorboard)
<module 'tensorboard' from '<snip>'>
>>> tensorboard.program.TensorBoard
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<snip>/tensorboard/lazy.py", line 65, in __getattr__
    return getattr(load_once(self), attr_name)
  <snip>
  File "<snip>/tensorboard/lazy.py", line 65, in __getattr__
    return getattr(load_once(self), attr_name)
RuntimeError: maximum recursion depth exceeded

Or in a Python 3 virtualenv:

$ python3
Python 3.6.5 (default, Mar 31 2018, 05:34:57) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorboard
>>> tensorboard.program.TensorBoard
<class 'tensorboard.program.TensorBoard'>
>>> import importlib  # Python 3
>>> importlib.reload(tensorboard)
<module 'tensorboard' from '<snip>'>
>>> tensorboard.program.TensorBoard
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<snip>/tensorboard/lazy.py", line 65, in __getattr__
    return getattr(load_once(self), attr_name)
  <snip>
  [Previous line repeated 329 more times]
RecursionError: maximum recursion depth exceeded

Note that using the IPython directive %reload_ext EXTENSION will
reload the module specified by EXTENSION, so this precludes us from
exposing %load_ext tensorboard. (Reloading a module directly via
imp/importlib may not be super common, but using %reload_ext is.)

As I understand it, the execution flow is:

  • tensorboard is initialized. tensorboard.program is a LazyModule.
  • tensorboard.program is forced, importing the program module for
    the first time, which effects the following:
    • sys.modules["tensorboard.program"] is initialized to an empty
      module object.
    • The program module is initialized.
    • The program attribute on the tensorboard module object is
      set to the program module object.
  • tensorboard is reloaded. tensorboard.program is reset to a lazy
    module.
  • tensorboard.program is forced, importing the program module.
    Because this module already exists in sys.modules, it is not
    reinitialized, and the program attribute on the tensorboard
    module object is not affected. Thus, the LazyModule initializer
    resolves to the lazy module itself, hitting a recursion error.

I believe that it should suffice to change

  import tensorboard.program as module  # pylint: disable=g-import-not-at-top
  return module

to

  return importlib.import_module("tensorboard.program")

with the caveat that this would require changes in the Google-internal
sync process (and I’m not exactly thrilled about these changes:
rewriting import syntax forms is one thing, but rewriting function
calls that happen to be of the form importlib.import_module is in
principle neither sound nor complete…—but, if it’s the only thing that
works, then so be it).

cc resident module spelunking expert @nfelt for confirmation

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions