docs/get_started.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SB93Ge748VQs"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "0sK8X2O9bTlz"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HEYuO5NFwDK9"
      },
      "source": [
        "# Get started with TensorBoard\n",
        "\n",
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/tensorboard/get_started\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/tensorboard/blob/master/docs/get_started.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/tensorboard/blob/master/docs/get_started.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/tensorboard/docs/get_started.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "56V5oun18ZdZ"
      },
      "source": [
        "In machine learning, to improve something you often need to be able to measure it. TensorBoard is a tool for providing the measurements and visualizations needed during the machine learning workflow. It enables tracking experiment metrics like loss and accuracy, visualizing the model graph, projecting embeddings to a lower dimensional space, and much more.\n",
        "\n",
        "This quickstart will show how to quickly get started with TensorBoard. The remaining guides in this website provide more details on specific capabilities, many of which are not included here. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6B95Hb6YVgPZ"
      },
      "outputs": [],
      "source": [
        "# Load the TensorBoard notebook extension\n",
        "%load_ext tensorboard"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_wqSAZExy6xV"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "import datetime"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ao7fJW1Pyiza"
      },
      "outputs": [],
      "source": [
        "# Clear any logs from previous runs\n",
        "!rm -rf ./logs/ "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z5pr9vuHVgXY"
      },
      "source": [
        "Using the [MNIST](https://en.wikipedia.org/wiki/MNIST_database) dataset as the example, normalize the data and write a function that creates a simple Keras model for classifying the images into 10 classes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j-DHsby18cot"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11493376/11490434 [==============================] - 0s 0us/step\n"
          ]
        }
      ],
      "source": [
        "mnist = tf.keras.datasets.mnist\n",
        "\n",
        "(x_train, y_train),(x_test, y_test) = mnist.load_data()\n",
        "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
        "\n",
        "def create_model():\n",
        "  return tf.keras.models.Sequential([\n",
        "    tf.keras.layers.Input(shape=(28, 28), name='layers_input'),\n",
        "    tf.keras.layers.Flatten(name='layers_flatten'),\n",
        "    tf.keras.layers.Dense(512, activation='relu', name='layers_dense'),\n",
        "    tf.keras.layers.Dropout(0.2, name='layers_dropout'),\n",
        "    tf.keras.layers.Dense(10, activation='softmax', name='layers_dense_2')\n",
        "  ])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XKUjdIoV87um"
      },
      "source": [
        "## Using TensorBoard with Keras Model.fit()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8CL_lxdn8-Sv"
      },
      "source": [
        "When training with Keras's [Model.fit()](https://www.tensorflow.org/api_docs/python/tf/keras/models/Model#fit), adding the `tf.keras.callbacks.TensorBoard` callback ensures that logs are created and stored. Additionally, enable histogram computation every epoch with `histogram_freq=1` (this is off by default)\n",
        "\n",
        "Place the logs in a timestamped subdirectory to allow easy selection of different training runs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WAQThq539CEJ"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train on 60000 samples, validate on 10000 samples\n",
            "Epoch 1/5\n",
            "60000/60000 [==============================] - 15s 246us/sample - loss: 0.2217 - accuracy: 0.9343 - val_loss: 0.1019 - val_accuracy: 0.9685\n",
            "Epoch 2/5\n",
            "60000/60000 [==============================] - 14s 229us/sample - loss: 0.0975 - accuracy: 0.9698 - val_loss: 0.0787 - val_accuracy: 0.9758\n",
            "Epoch 3/5\n",
            "60000/60000 [==============================] - 14s 231us/sample - loss: 0.0718 - accuracy: 0.9771 - val_loss: 0.0698 - val_accuracy: 0.9781\n",
            "Epoch 4/5\n",
            "60000/60000 [==============================] - 14s 227us/sample - loss: 0.0540 - accuracy: 0.9820 - val_loss: 0.0685 - val_accuracy: 0.9795\n",
            "Epoch 5/5\n",
            "60000/60000 [==============================] - 14s 228us/sample - loss: 0.0433 - accuracy: 0.9862 - val_loss: 0.0623 - val_accuracy: 0.9823\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fc8a5ee02e8>"
            ]
          },
          "execution_count": 6,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model = create_model()\n",
        "model.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "log_dir = \"logs/fit/\" + datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n",
        "tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)\n",
        "\n",
        "model.fit(x=x_train, \n",
        "          y=y_train, \n",
        "          epochs=5, \n",
        "          validation_data=(x_test, y_test), \n",
        "          callbacks=[tensorboard_callback])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "asjGpmD09dRl"
      },
      "source": [
        "Start TensorBoard through the command line or within a notebook experience. The two interfaces are generally the same. In notebooks, use the `%tensorboard` line magic. On the command line, run the same command without \"%\"."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "A4UKgTLb9fKI"
      },
      "outputs": [],
      "source": [
        "%tensorboard --logdir logs/fit"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MCsoUNb6YhGc"
      },
      "source": [
        "<!-- <img class=\"tfo-display-only-on-site\" src=\"https://github.com/tensorflow/tensorboard/blob/master/docs/images/quickstart_model_fit.png?raw=1\"/> -->"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gi4PaRm39of2"
      },
      "source": [
        "A brief overview of the visualizations created in this example and the dashboards (tabs in top navigation bar) where they can be found:\n",
        "\n",
        "* **Scalars** show how the loss and metrics change with every epoch. You can use them to also track training speed, learning rate, and other scalar values. Scalars can be found in the **Time Series** or **Scalars** dashboards.\n",
        "* **Graphs** help you visualize your model. In this case, the Keras graph of layers is shown which can help you ensure it is built correctly. Graphs can be found in the **Graphs** dashboard.\n",
        "* **Histograms** and **Distributions** show the distribution of a Tensor over time. This can be useful to visualize weights and biases and verify that they are changing in an expected way. Histograms can be found in the **Time Series** or **Histograms** dashboards. Distributions can be found in the **Distributions** dashboard.\n",
        "\n",
        "Additional TensorBoard dashboards are automatically enabled when you log other types of data. For example, the Keras TensorBoard callback lets you log images and embeddings as well. You can see what other dashboards are available in TensorBoard by clicking on the \"inactive\" dropdown towards the top right."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nB718NOH95yG"
      },
      "source": [
        "## Using TensorBoard with other methods\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IKNt0nWs-Ekt"
      },
      "source": [
        "When training with methods such as [`tf.GradientTape()`](https://www.tensorflow.org/api_docs/python/tf/GradientTape), use `tf.summary` to log the required information.\n",
        "\n",
        "Use the same dataset as above, but convert it to `tf.data.Dataset` to take advantage of batching capabilities:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nnHx4DsMezy1"
      },
      "outputs": [],
      "source": [
        "train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))\n",
        "test_dataset = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n",
        "\n",
        "train_dataset = train_dataset.shuffle(60000).batch(64)\n",
        "test_dataset = test_dataset.batch(64)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SzpmTmJafJ10"
      },
      "source": [
        "The training code follows the [advanced quickstart](https://www.tensorflow.org/tutorials/quickstart/advanced) tutorial, but shows how to log metrics to TensorBoard. Choose loss and optimizer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "H2Y5-aPbAANs"
      },
      "outputs": [],
      "source": [
        "loss_object = tf.keras.losses.SparseCategoricalCrossentropy()\n",
        "optimizer = tf.keras.optimizers.Adam()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cKhIIDj9Hbfy"
      },
      "source": [
        "Create stateful metrics that can be used to accumulate values during training and logged at any point:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jD0tEWrgH0TL"
      },
      "outputs": [],
      "source": [
        "# Define our metrics\n",
        "train_loss = tf.keras.metrics.Mean('train_loss', dtype=tf.float32)\n",
        "train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy('train_accuracy')\n",
        "test_loss = tf.keras.metrics.Mean('test_loss', dtype=tf.float32)\n",
        "test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy('test_accuracy')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "szw_KrgOg-OT"
      },
      "source": [
        "Define the training and test functions:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TTWcJO35IJgK"
      },
      "outputs": [],
      "source": [
        "def train_step(model, optimizer, x_train, y_train):\n",
        "  with tf.GradientTape() as tape:\n",
        "    predictions = model(x_train, training=True)\n",
        "    loss = loss_object(y_train, predictions)\n",
        "  grads = tape.gradient(loss, model.trainable_variables)\n",
        "  optimizer.apply_gradients(zip(grads, model.trainable_variables))\n",
        "\n",
        "  train_loss(loss)\n",
        "  train_accuracy(y_train, predictions)\n",
        "\n",
        "def test_step(model, x_test, y_test):\n",
        "  predictions = model(x_test)\n",
        "  loss = loss_object(y_test, predictions)\n",
        "\n",
        "  test_loss(loss)\n",
        "  test_accuracy(y_test, predictions)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nucPZBKPJR3A"
      },
      "source": [
        "Set up summary writers to write the summaries to disk in a different logs directory:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3Qp-exmbWf4w"
      },
      "outputs": [],
      "source": [
        "current_time = datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n",
        "train_log_dir = 'logs/gradient_tape/' + current_time + '/train'\n",
        "test_log_dir = 'logs/gradient_tape/' + current_time + '/test'\n",
        "train_summary_writer = tf.summary.create_file_writer(train_log_dir)\n",
        "test_summary_writer = tf.summary.create_file_writer(test_log_dir)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qgUJgDdKWUKF"
      },
      "source": [
        "Start training. Use `tf.summary.scalar()` to log metrics (loss and accuracy) during training/testing within the scope of the summary writers to write the summaries to disk. You have control over which metrics to log and how often to do it. Other `tf.summary` functions enable logging other types of data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "odWvHPpKJvb_"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1, Loss: 0.24321186542510986, Accuracy: 92.84333801269531, Test Loss: 0.13006582856178284, Test Accuracy: 95.9000015258789\n",
            "Epoch 2, Loss: 0.10446818172931671, Accuracy: 96.84833526611328, Test Loss: 0.08867532759904861, Test Accuracy: 97.1199951171875\n",
            "Epoch 3, Loss: 0.07096975296735764, Accuracy: 97.80166625976562, Test Loss: 0.07875105738639832, Test Accuracy: 97.48999786376953\n",
            "Epoch 4, Loss: 0.05380449816584587, Accuracy: 98.34166717529297, Test Loss: 0.07712937891483307, Test Accuracy: 97.56999969482422\n",
            "Epoch 5, Loss: 0.041443776339292526, Accuracy: 98.71833038330078, Test Loss: 0.07514958828687668, Test Accuracy: 97.5\n"
          ]
        }
      ],
      "source": [
        "model = create_model() # reset our model\n",
        "\n",
        "EPOCHS = 5\n",
        "\n",
        "for epoch in range(EPOCHS):\n",
        "  for (x_train, y_train) in train_dataset:\n",
        "    train_step(model, optimizer, x_train, y_train)\n",
        "  with train_summary_writer.as_default():\n",
        "    tf.summary.scalar('loss', train_loss.result(), step=epoch)\n",
        "    tf.summary.scalar('accuracy', train_accuracy.result(), step=epoch)\n",
        "\n",
        "  for (x_test, y_test) in test_dataset:\n",
        "    test_step(model, x_test, y_test)\n",
        "  with test_summary_writer.as_default():\n",
        "    tf.summary.scalar('loss', test_loss.result(), step=epoch)\n",
        "    tf.summary.scalar('accuracy', test_accuracy.result(), step=epoch)\n",
        "  \n",
        "  template = 'Epoch {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}'\n",
        "  print (template.format(epoch+1,\n",
        "                         train_loss.result(), \n",
        "                         train_accuracy.result()*100,\n",
        "                         test_loss.result(), \n",
        "                         test_accuracy.result()*100))\n",
        "\n",
        "  # Reset metrics every epoch\n",
        "  train_loss.reset_state()\n",
        "  test_loss.reset_state()\n",
        "  train_accuracy.reset_state()\n",
        "  test_accuracy.reset_state()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JikosQ84fzcA"
      },
      "source": [
        "Open TensorBoard again, this time pointing it at the new log directory. We could have also started TensorBoard to monitor training while it progresses."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-Iue509kgOyE"
      },
      "outputs": [],
      "source": [
        "%tensorboard --logdir logs/gradient_tape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NVpnilhEgQXk"
      },
      "source": [
        "<!-- <img class=\"tfo-display-only-on-site\" src=\"https://github.com/tensorflow/tensorboard/blob/master/docs/images/quickstart_gradient_tape.png?raw=1\"/> -->"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ozbwXgPIkCKV"
      },
      "source": [
        "That's it! You have now seen how to use TensorBoard both through the Keras callback and through `tf.summary` for more custom scenarios. "
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "get_started.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}