notebooks/dino-gpt4v-autodistill.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c_nGQ9Ps842E"
      },
      "source": [
        "[![Roboflow Notebooks](https://media.roboflow.com/notebooks/template/bannertest2-2.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672932710194)](https://github.com/roboflow/notebooks)\n",
        "\n",
        "# DINO-GPT4-V: Use GPT-4V in a Two-Stage Detection Model\n",
        "\n",
        "[Autodistill](https://github.com/autodistill/autodistill) uses big, slower foundation models to train small, faster supervised models. Using `autodistill`, you can go from unlabeled images to inference on a custom model running at the edge with no human intervention in between.\n",
        "\n",
        "With Autodistill, you can use DINO-GPT4-V, a model that uses GPT-4V and Grounding DINO to label data for use in training fine-tuned object detection models.\n",
        "\n",
        "[Read the guide that accompanies this blog post](https://blog.roboflow.com/dino-gpt-4v/).\n",
        "\n",
        "By the end of this guide, we will label car brands in images. Here is an example:\n",
        "\n",
        "<div>\n",
        "<img src=\"https://blog.roboflow.com/content/images/2023/11/Screenshot-2023-11-07-at-12.55.18.png\" width=\"500\"/>\n",
        "</div>\n",
        "\n",
        "## Steps in this Tutorial\n",
        "\n",
        "In this tutorial, we are going to cover:\n",
        "\n",
        "- Preparing a dataset\n",
        "- Label a dataset with DINO-GPT4-V\n",
        "- Train a YOLOv8 model with the labeled dataset\n",
        "- (Optional): Deploy the model with Roboflow Inference\n",
        "\n",
        "Let's begin!\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "anyZ7uFz_HNQ"
      },
      "source": [
        "## ⚡ Before you start\n",
        "\n",
        "Let's make sure that we have access to a GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0nFmBOnd8vFv",
        "outputId": "f62fdc30-f883-4afe-a058-3d45cb1767a2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Thu Jun  8 06:38:56 2023       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 525.85.12    Driver Version: 525.85.12    CUDA Version: 12.0     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   69C    P8    13W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j8ai--6D_7rG"
      },
      "source": [
        "## 🧪 Install autodistill and Dependencies\n",
        "\n",
        "Run the following code cell to install the dependencies required for this project:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "byokHef__Jfd",
        "outputId": "f46f49af-20a9-43ad-c155-3a4f3f84b415"
      },
      "outputs": [],
      "source": [
        "!pip install autodistill autodistill-grounding-dino autodistill-gpt-4v -q"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KI8XZLnxA0D6"
      },
      "source": [
        "To make it easier for us to manage datasets, images and models we create a `HOME` constant."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "udhh5dN_Ai54",
        "outputId": "b9fb472b-9ace-4a97-8685-0a1dfb798f3b"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "HOME = os.getcwd()\n",
        "print(HOME)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lNBxRPlW-12t"
      },
      "source": [
        "## 🖼️ Image dataset preperation\n",
        "\n",
        "All you need to have is a folder of images that you want to automatically annotate, and use for target model training, to get started."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "czIeRkMXr8KE"
      },
      "outputs": [],
      "source": [
        "!mkdir {HOME}/images"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Jx7ZPiBfwTd"
      },
      "source": [
        "## Download Images\n",
        "\n",
        "Below, we provide three methods of retrieving images for use with Autodistill:\n",
        "\n",
        "1. Upload images into Colab (exclusive to Colab);\n",
        "2. Download a dataset with images from Roboflow, and;\n",
        "3. Download a raw video and split it up into images.\n",
        "\n",
        "For this guide, we will use a zipped dataset of raw images and upload it into Colab, but instructions for all options are provided below."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Upload images into Colab\n",
        "\n",
        "We have an example dataset of vehicles in a parking lot that we will use for this tutorial. You can download the dataset [here](https://media.roboflow.com/dino-gpt-4v-example.zip). You can also zip up a folder of your own images.\n",
        "\n",
        "Open the file browser from the left sidebar of Colab and upload the ZIP file. Uncomment the command below to unzip the folder, replacing `dataset.zip` with the name of the dataset with which you are working:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# !unzip dataset.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VaFttCcbfwTd"
      },
      "source": [
        "### Download Images from Roboflow\n",
        "\n",
        "You can download datasets from Roboflow using the below lines of code. You can download datasets associated with your account, or any of the 200,000+ public datasets on [Roboflow Universe](https://universe.roboflow.com/). To download a dataset from Roboflow, you will need a [free Roboflow account](https://app.roboflow.com).\n",
        "\n",
        "You need to paste in a URL to a full model, like so:\n",
        "\n",
        "```\n",
        "https://universe.roboflow.com/mohamed-traore-2ekkp/taco-trash-annotations-in-context/model/16\n",
        "```\n",
        "\n",
        "To find a model URL, go to a dataset page on Universe (i.e. the [TACO dataset](https://universe.roboflow.com/mohamed-traore-2ekkp/taco-trash-annotations-in-context/)), then click \"Model\" in the sidebar to view the latest model version. Copy the page URL, then paste it into the `download_dataset()` function below.\n",
        "\n",
        "![Universe model link](https://media.roboflow.com/universe_deploy_tab.png)\n",
        "\n",
        "Uncomment the lines of code below to use the Roboflow dataset upload."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ztCii0dSfwTd"
      },
      "outputs": [],
      "source": [
        "# import roboflow\n",
        "\n",
        "# roboflow.login()\n",
        "\n",
        "# dataset = roboflow.download_dataset(dataset_url=\"https://universe.roboflow.com/mohamed-traore-2ekkp/taco-trash-annotations-in-context/model/16\", model_format=\"yolov8\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S7xzFhi6fwTd"
      },
      "source": [
        "Once you have downloaded your dataset, move all of the images from the `train` set in your downloaded dataset into the `images/` directory we created earlier:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QvJmLyA5fwTd"
      },
      "outputs": [],
      "source": [
        "# %mv {HOME}/<dataset-name>/train/* {HOME}/images"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bhXomuTyfwTd"
      },
      "source": [
        "Now we are ready to start using Autodistill!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IF16BQIWWwfb"
      },
      "source": [
        "### Download raw videos\n",
        "\n",
        "**NOTE:** In this tutorial, we will start with a directory containing video files and I will show you how to turn it into a ready-to-use collection of images. If you are working with your images, you can skip this part."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_C04w3vS-FFJ"
      },
      "outputs": [],
      "source": [
        "!mkdir {HOME}/videos\n",
        "%cd {HOME}/videos\n",
        "\n",
        "# download zip file containing videos\n",
        "!wget https://media.roboflow.com/milk.zip\n",
        "\n",
        "# unzip videos\n",
        "!unzip milk.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AAI8n81PO08E"
      },
      "source": [
        "### Convert videos into images\n",
        "\n",
        "**NOTE:** Now, let's convert videos into images. By default, the code below saves every `10th` frame from each video. You can change this by manipulating the value of the `FRAME_STRIDE` parameter."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1_9OwJw0T1l2"
      },
      "outputs": [],
      "source": [
        "VIDEO_DIR_PATH = f\"{HOME}/videos\"\n",
        "IMAGE_DIR_PATH = f\"{HOME}/images\"\n",
        "FRAME_STRIDE = 10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gGrfasVvyomx"
      },
      "source": [
        "**NOTE:** Notice that we put two of our videos aside so that we can use them at the end of the notebook to evaluate our model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "b1f33b8580474febb5ffe5be8442e090",
            "ec38cea329c64274a2747e7387b8ae3c",
            "c3cff42073644b91863056fe6a2d8f34",
            "177f20cb500d4dc7ad338e844897c735",
            "23636106a532455c8146205e4f7be2fe",
            "6250697a80024656af350a4c05d456bc",
            "58be2b02981b4aedad5294cecbccc3be",
            "000ec4a942dc4fceac816501fe6436a7",
            "90924baeb3644d45bbfdfb9f7b0520dc",
            "43df62c36cec4dfaba5203164756f692",
            "f19fe218fc5940b490dbd9d4dfae89f9"
          ]
        },
        "id": "8zS9Tb8hO9sX",
        "outputId": "ba4edb35-ed0c-4360-91c7-f09b5295bff5"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "b1f33b8580474febb5ffe5be8442e090",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/6 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import supervision as sv\n",
        "from tqdm.notebook import tqdm\n",
        "\n",
        "video_paths = sv.list_files_with_extensions(\n",
        "    directory=VIDEO_DIR_PATH,\n",
        "    extensions=[\"mov\", \"mp4\"])\n",
        "\n",
        "TEST_VIDEO_PATHS, TRAIN_VIDEO_PATHS = video_paths[:2], video_paths[2:]\n",
        "\n",
        "for video_path in tqdm(TRAIN_VIDEO_PATHS):\n",
        "    video_name = video_path.stem\n",
        "    image_name_pattern = video_name + \"-{:05d}.png\"\n",
        "    with sv.ImageSink(target_dir_path=IMAGE_DIR_PATH, image_name_pattern=image_name_pattern) as sink:\n",
        "        for image in sv.get_video_frames_generator(source_path=str(video_path), stride=FRAME_STRIDE):\n",
        "            sink.save_image(image=image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yt0DJTpvUIm5"
      },
      "source": [
        "### Display image sample\n",
        "\n",
        "**NOTE:** Before we start building a model with autodistill, let's make sure we have everything we need."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-hKxoZuw2Mze",
        "outputId": "3b946c4a-a871-4c01-abdf-23769a468719"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "image count: 165\n"
          ]
        }
      ],
      "source": [
        "import supervision as sv\n",
        "\n",
        "image_paths = sv.list_files_with_extensions(\n",
        "    directory=IMAGE_DIR_PATH,\n",
        "    extensions=[\"png\", \"jpg\", \"jpg\"])\n",
        "\n",
        "print('image count:', len(image_paths))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z3mlA1Xw2ZdV"
      },
      "source": [
        "**NOTE:** We can also plot sample of our image dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eaBaWn-DUaec"
      },
      "outputs": [],
      "source": [
        "IMAGE_DIR_PATH = f\"{HOME}/images\"\n",
        "SAMPLE_SIZE = 16\n",
        "SAMPLE_GRID_SIZE = (4, 4)\n",
        "SAMPLE_PLOT_SIZE = (16, 16)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "QQEvtmvnN_ih",
        "outputId": "a4256fde-356a-48fe-cd42-d76171e0517a"
      },
      "outputs": [],
      "source": [
        "import cv2\n",
        "import supervision as sv\n",
        "\n",
        "titles = [\n",
        "    image_path.stem\n",
        "    for image_path\n",
        "    in image_paths[:SAMPLE_SIZE]]\n",
        "images = [\n",
        "    cv2.imread(str(image_path))\n",
        "    for image_path\n",
        "    in image_paths[:SAMPLE_SIZE]]\n",
        "\n",
        "sv.plot_images_grid(images=images, titles=titles, grid_size=SAMPLE_GRID_SIZE, size=SAMPLE_PLOT_SIZE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FCMpPz3wVb_M"
      },
      "source": [
        "## 🏷️ Autolabel Dataset\n",
        "\n",
        "DINO-GP4V is a combination of two models:\n",
        "\n",
        "- Grounding DINO, which detects objects, and;\n",
        "- GPT-4V, which can classify objects and provide specific information about an object.\n",
        "\n",
        "In this example, we will detect car brands. Grounding DINO can detect cars but doesn't know about car brands. GPT-4V can't detect objects accurately, but can classify objects. With Autodistill, we can detect cars with Grounding DINO, then pass each car through GPT-4V for classification.\n",
        "\n",
        "We need to define two ontologies, which refer to what we want to detect and what we want to classify. In this example, we want to detect cars and classify car brands.\n",
        "\n",
        "What you detect should be a common object, then GPT-4V can refine the results."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "24qFSVyhUV8C"
      },
      "outputs": [],
      "source": [
        "from autodistill_gpt_4v import GPT4V\n",
        "from autodistill.detection import CaptionOntology\n",
        "from autodistill_grounding_dino import GroundingDINO\n",
        "from autodistill.utils import plot\n",
        "\n",
        "from autodistill.core.custom_detection_model import CustomDetectionModel\n",
        "import cv2\n",
        "\n",
        "classes = [\"mercedes\", \"toyota\"]\n",
        "\n",
        "DINOGPT = CustomDetectionModel(\n",
        "    detection_model=GroundingDINO(\n",
        "        CaptionOntology({\"car\": \"car\"})\n",
        "    ),\n",
        "    classification_model=GPT4V(\n",
        "        CaptionOntology({k: k for k in classes}),\n",
        "    api_key = \"YOUR_OPENAI_API_KEY\"\n",
        "    )\n",
        ")\n",
        "\n",
        "IMAGE = \"mercedes.jpeg\"\n",
        "\n",
        "results = DINOGPT.predict(IMAGE)\n",
        "\n",
        "plot(\n",
        "    image=cv2.imread(IMAGE),\n",
        "    detections=results,\n",
        "    classes=classes\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Above, replace:\n",
        "\n",
        "- `YOUR_OPENAI_API_KEY` with your OpenAI API key. You will need this API key to use GPT-4V.\n",
        "- `IMAGE` with the path to an image.\n",
        "- \n",
        "\n",
        "`classes` is the list of classes you want to use in classification.\n",
        "\n",
        "When you run the code, an image will be labeled. You can use the code above for testing DINO-GPT-4V on images in your dataset. When you are ready, run the code cell below to label your dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "DINOGPT.label(f\"{HOME}/images\", extension=\".jpeg\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Above, replace `.jpeg` with the file extension used by images in your dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OM9zH9BGZGp0"
      },
      "source": [
        "### Display dataset sample\n",
        "\n",
        "Run the code cells below to create a dataset, which can then be saved, and to visualize your dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XsTHheE-bMbY"
      },
      "outputs": [],
      "source": [
        "ANNOTATIONS_DIRECTORY_PATH = f\"{HOME}/dataset/train/labels\"\n",
        "IMAGES_DIRECTORY_PATH = f\"{HOME}/dataset/train/images\"\n",
        "DATA_YAML_PATH = f\"{HOME}/dataset/data.yaml\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Uhgy9rbdYSCZ",
        "outputId": "afc61564-c368-4532-c610-2c3bec1abc72"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "132"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import supervision as sv\n",
        "\n",
        "dataset = sv.DetectionDataset.from_yolo(\n",
        "    images_directory_path=IMAGES_DIRECTORY_PATH,\n",
        "    annotations_directory_path=ANNOTATIONS_DIRECTORY_PATH,\n",
        "    data_yaml_path=DATA_YAML_PATH)\n",
        "\n",
        "len(dataset)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "krNCDdDZcm7H",
        "outputId": "51b26cc1-7884-4bd3-e118-d69106cab0a1"
      },
      "outputs": [],
      "source": [
        "import supervision as sv\n",
        "\n",
        "image_names = list(dataset.images.keys())[:SAMPLE_SIZE]\n",
        "\n",
        "mask_annotator = sv.MaskAnnotator()\n",
        "box_annotator = sv.BoxAnnotator()\n",
        "\n",
        "images = []\n",
        "for image_name in image_names:\n",
        "    image = dataset.images[image_name]\n",
        "    annotations = dataset.annotations[image_name]\n",
        "    labels = [\n",
        "        dataset.classes[class_id]\n",
        "        for class_id\n",
        "        in annotations.class_id]\n",
        "    annotates_image = mask_annotator.annotate(\n",
        "        scene=image.copy(),\n",
        "        detections=annotations)\n",
        "    annotates_image = box_annotator.annotate(\n",
        "        scene=annotates_image,\n",
        "        detections=annotations,\n",
        "        labels=labels)\n",
        "    images.append(annotates_image)\n",
        "\n",
        "sv.plot_images_grid(\n",
        "    images=images,\n",
        "    titles=image_names,\n",
        "    grid_size=SAMPLE_GRID_SIZE,\n",
        "    size=SAMPLE_PLOT_SIZE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MQ9ESdTCagkm"
      },
      "source": [
        "## 🔥 Train a Model\n",
        "\n",
        "A Target Model is a supervised model that consumes a Dataset and outputs a distilled model that is ready for deployment. Target Models are usually small, fast, and fine-tuned to perform a specific task very well (but they don't generalize well beyond the information described in their Dataset). Examples of Target Models are YOLOv8 and DETR."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9sLzt1MdalB9"
      },
      "outputs": [],
      "source": [
        "%cd {HOME}\n",
        "\n",
        "from autodistill_yolov8 import YOLOv8\n",
        "\n",
        "target_model = YOLOv8(\"yolov8n.pt\")\n",
        "target_model.train(DATA_YAML_PATH, epochs=50)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zlv5QA3Vg06Z",
        "outputId": "1767c48d-64b8-4da3-e1f4-ef1a7d68bf8f"
      },
      "outputs": [],
      "source": [
        "!ls {HOME}/runs/detect/train/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G8Tn0fgNgoQl"
      },
      "source": [
        "## ⚖️ Evaluate target model\n",
        "\n",
        "**NOTE:** As with the regular YOLOv8 training, we can now take a look at artifacts stored in `runs` directory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 484
        },
        "id": "z8EW50NAgSdU",
        "outputId": "336bf84d-1e38-49de-d001-3de817897078"
      },
      "outputs": [],
      "source": [
        "%cd {HOME}\n",
        "\n",
        "from IPython.display import Image\n",
        "\n",
        "Image(filename=f'{HOME}/runs/detect/train/confusion_matrix.png', width=600)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 334
        },
        "id": "jwCnrPUYhIrE",
        "outputId": "771018e2-efbd-4b01-e605-931b5702e0bd"
      },
      "outputs": [],
      "source": [
        "%cd {HOME}\n",
        "\n",
        "from IPython.display import Image\n",
        "\n",
        "Image(filename=f'{HOME}/runs/detect/train/results.png', width=600)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 378
        },
        "id": "Xy3lqKL7hRTD",
        "outputId": "eeb97b37-50f7-4d5d-9751-c6ffce121570"
      },
      "outputs": [],
      "source": [
        "%cd {HOME}\n",
        "\n",
        "from IPython.display import Image\n",
        "\n",
        "Image(filename=f'{HOME}/runs/detect/train/val_batch0_pred.jpg', width=600)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gWpUXXkyhaq3"
      },
      "source": [
        "# Deploy Your Model to the Edge\n",
        "\n",
        "In addition to using the Roboflow hosted API for deployment, you can use [Roboflow Inference](https://inference.roboflow.com), an open source inference solution that has powered millions of API calls in production environments. Inference works with CPU and GPU, giving you immediate access to a range of devices, from the NVIDIA Jetson to TRT-compatible devices to ARM CPU devices.\n",
        "\n",
        "With Roboflow Inference, you can self-host and deploy your model on-device. You can deploy applications using the [Inference Docker containers](https://inference.roboflow.com/quickstart/docker/) or the pip package.\n",
        "\n",
        "For example, to install Inference on a device with an NVIDIA GPU, we can use:\n",
        "\n",
        "```\n",
        "docker pull roboflow/roboflow-inference-server-gpu\n",
        "```\n",
        "\n",
        "Then we can run inference via HTTP:\n",
        "\n",
        "```python\n",
        "import requests\n",
        "\n",
        "workspace_id = \"\"\n",
        "model_id = \"\"\n",
        "image_url = \"\"\n",
        "confidence = 0.75\n",
        "api_key = \"\"\n",
        "\n",
        "infer_payload = {\n",
        "    \"image\": {\n",
        "        \"type\": \"url\",\n",
        "        \"value\": image_url,\n",
        "    },\n",
        "    \"confidence\": confidence,\n",
        "    \"iou_threshold\": iou_thresh,\n",
        "    \"api_key\": api_key,\n",
        "}\n",
        "res = requests.post(\n",
        "    f\"http://localhost:9001/{workspace_id}/{model_id}\",\n",
        "    json=infer_object_detection_payload,\n",
        ")\n",
        "\n",
        "predictions = res.json()\n",
        "```\n",
        "\n",
        "Above, set your Roboflow workspace ID, model ID, and API key.\n",
        "\n",
        "- [Find your workspace and model ID](https://docs.roboflow.com/api-reference/workspace-and-project-ids?ref=blog.roboflow.com)\n",
        "- [Find your API key](https://docs.roboflow.com/api-reference/authentication?ref=blog.roboflow.com#retrieve-an-api-key)\n",
        "\n",
        "Also, set the URL of an image on which you want to run inference. This can be a local file.\n",
        "\n",
        "# 🏆 Congratulations\n",
        "\n",
        "### Learning Resources\n",
        "\n",
        "Roboflow has produced many resources that you may find interesting as you advance your knowledge of computer vision:\n",
        "\n",
        "- [Roboflow Notebooks](https://github.com/roboflow/notebooks): A repository of over 20 notebooks that walk through how to train custom models with a range of model types, from YOLOv7 to SegFormer.\n",
        "- [Roboflow YouTube](https://www.youtube.com/c/Roboflow): Our library of videos featuring deep dives into the latest in computer vision, detailed tutorials that accompany our notebooks, and more.\n",
        "- [Roboflow Discuss](https://discuss.roboflow.com/): Have a question about how to do something on Roboflow? Ask your question on our discussion forum.\n",
        "- [Roboflow Models](https://roboflow.com): Learn about state-of-the-art models and their performance. Find links and tutorials to guide your learning.\n",
        "\n",
        "### Convert data formats\n",
        "\n",
        "Roboflow provides free utilities to convert data between dozens of popular computer vision formats. Check out [Roboflow Formats](https://roboflow.com/formats) to find tutorials on how to convert data between formats in a few clicks.\n",
        "\n",
        "### Connect computer vision to your project logic\n",
        "\n",
        "[Roboflow Templates](https://roboflow.com/templates) is a public gallery of code snippets that you can use to connect computer vision to your project logic. Code snippets range from sending emails after inference to measuring object distance between detections."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "machine_shape": "hm",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "000ec4a942dc4fceac816501fe6436a7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "177f20cb500d4dc7ad338e844897c735": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_43df62c36cec4dfaba5203164756f692",
            "placeholder": "​",
            "style": "IPY_MODEL_f19fe218fc5940b490dbd9d4dfae89f9",
            "value": " 6/6 [01:25&lt;00:00, 12.37s/it]"
          }
        },
        "23636106a532455c8146205e4f7be2fe": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "43df62c36cec4dfaba5203164756f692": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "58be2b02981b4aedad5294cecbccc3be": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6250697a80024656af350a4c05d456bc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "90924baeb3644d45bbfdfb9f7b0520dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b1f33b8580474febb5ffe5be8442e090": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ec38cea329c64274a2747e7387b8ae3c",
              "IPY_MODEL_c3cff42073644b91863056fe6a2d8f34",
              "IPY_MODEL_177f20cb500d4dc7ad338e844897c735"
            ],
            "layout": "IPY_MODEL_23636106a532455c8146205e4f7be2fe"
          }
        },
        "c3cff42073644b91863056fe6a2d8f34": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_000ec4a942dc4fceac816501fe6436a7",
            "max": 6,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_90924baeb3644d45bbfdfb9f7b0520dc",
            "value": 6
          }
        },
        "ec38cea329c64274a2747e7387b8ae3c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6250697a80024656af350a4c05d456bc",
            "placeholder": "​",
            "style": "IPY_MODEL_58be2b02981b4aedad5294cecbccc3be",
            "value": "100%"
          }
        },
        "f19fe218fc5940b490dbd9d4dfae89f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "state": {}
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}