notebooks/gradio_web_demo.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "8rclB0ccBp2X"
      },
      "source": [
        "![image.png](https://raw.githubusercontent.com/ymcui/Chinese-LLaMA-Alpaca/main/pics/small_banner.png)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "daB8KJRc5VAx"
      },
      "source": [
        "以ChatGPT、GPT-4等为代表的大语言模型（Large Language Model, LLM）掀起了新一轮自然语言处理领域的研究浪潮，展现出了类通用人工智能（AGI）的能力，受到业界广泛关注。然而，由于大语言模型的训练和部署都极为昂贵，为构建透明且开放的学术研究造成了一定的阻碍。\n",
        "\n",
        "为了促进大模型在中文NLP社区的开放研究，本项目开源了中文LLaMA模型和指令精调的Alpaca大模型。这些模型在原版LLaMA的基础上扩充了中文词表并使用了中文数据进行二次预训练，进一步提升了中文基础语义理解能力。同时，中文Alpaca模型进一步使用了中文指令数据进行精调，显著提升了模型对指令的理解和执行能力。详细内容请参考技术报告[(Cui, Yang, and Yao, 2023)](https://arxiv.org/abs/2304.08177)。"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "注意：\n",
        "\n",
        "1. 该demo中默认加载了由第三方转换的LLaMA权重。本项目不对其合规性和正确性负责。\n",
        "2. 该demo默认加载Chinese-Alpaca-7B模型。若想尝试其他模型，请先阅读[模型合并与转换](https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/模型合并与转换)。"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "8w6hgNddOTwH"
      },
      "source": [
        "克隆项目源码"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rcVpwUNCBX66"
      },
      "outputs": [],
      "source": [
        "!git clone https://github.com/ymcui/Chinese-LLaMA-Alpaca.git"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ifMwZ6aRByXr"
      },
      "source": [
        "安装依赖"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b3jLONDhB1JZ"
      },
      "outputs": [],
      "source": [
        "!pip install -r Chinese-LLaMA-Alpaca/requirements.txt\n",
        "!pip install peft==0.3.0\n",
        "!pip install gradio\n",
        "!pip install sentencepiece"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "t5IAVbuAOmWg"
      },
      "source": [
        "从Hugging Face下载llama权重和chinese llama lora权重"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FwhwVq0FHTQo"
      },
      "outputs": [],
      "source": [
        "!git clone https://huggingface.co/ziqingyang/chinese-alpaca-lora-7b\n",
        "!git clone https://huggingface.co/elinas/llama-7b-hf-transformers-4.29"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "iR6CkYAUCW2h"
      },
      "source": [
        "启动web demo，如果没有申请到T4 colab，则添加--only_cpu选项，如果是T4机器，则使用--load_in_8bit选项，如果是V100、A100机器则不需要，选用load_in_8bits需安装bitsandbytes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CuTPG8YDNMu-"
      },
      "outputs": [],
      "source": [
        "!pip install bitsandbytes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HVWjCNMjCp0p"
      },
      "outputs": [],
      "source": [
        "!python Chinese-LLaMA-Alpaca/scripts/inference/gradio_demo.py --base_model llama-7b-hf-transformers-4.29 --lora_model chinese-alpaca-lora-7b --load_in_8bit"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}