examples/configurable_chain/client.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Client\n",
    "\n",
    "Demo of client interacting with the simple chain server, which deploys a chain that tells jokes about a particular topic."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can interact with this via API directly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": ["import requests\n\ninputs = {\"input\": {\"topic\": \"sports\"}}\nresponse = requests.post(\"http://localhost:8000/configurable_temp/invoke\", json=inputs)\n\nresponse.json()"]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also interact with this via the RemoteRunnable interface (to use in other chains)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": ["from langserve import RemoteRunnable\n\nremote_runnable = RemoteRunnable(\"http://localhost:8000/configurable_temp\")"]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remote runnable has the same interface as local runnables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": ["response = await remote_runnable.ainvoke({\"topic\": \"sports\"})"]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The client can also execute langchain code synchronously, and pass in configs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": ["from langchain_core.runnables import RunnableConfig\n\nremote_runnable.batch([{\"topic\": \"sports\"}, {\"topic\": \"cars\"}])"]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The server supports streaming (using HTTP server-side events), which can help interact with long responses in real time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": ["async for chunk in remote_runnable.astream({\"topic\": \"bears, but a bit verbose\"}):\n    print(chunk, end=\"\", flush=True)"]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configurability\n",
    "\n",
    "The server chains have been exposed as configurable chains!\n",
    "\n",
    "```python \n",
    "\n",
    "model = ChatOpenAI(temperature=0.5).configurable_alternatives(\n",
    "    ConfigurableField(\n",
    "        id=\"llm\",\n",
    "        name=\"LLM\",\n",
    "        description=(\n",
    "            \"Decide whether to use a high or a low temperature parameter for the LLM.\"\n",
    "        ),\n",
    "    ),\n",
    "    high_temp=ChatOpenAI(temperature=0.9),\n",
    "    low_temp=ChatOpenAI(temperature=0.1),\n",
    "    default_key=\"medium_temp\",\n",
    ")\n",
    "prompt = PromptTemplate.from_template(\n",
    "    \"tell me a joke about {topic}.\"\n",
    ").configurable_fields(  # Example of a configurable field\n",
    "    template=ConfigurableField(\n",
    "        id=\"prompt\",\n",
    "        name=\"Prompt\",\n",
    "        description=(\"The prompt to use. Must contain {topic}.\"),\n",
    "    )\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now use the configurability of the runnable in the API!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": ["await remote_runnable.ainvoke(\n    {\"topic\": \"sports\"},\n    config={\n        \"configurable\": {\"prompt\": \"how to say {topic} in french\", \"llm\": \"low_temp\"}\n    },\n)"]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configurability Based on Request Properties\n",
    "\n",
    "If you want to change your chain invocation based on your request's properties,\n",
    "you can do so with `add_routes`'s `per_req_config_modifier` method as follows:\n",
    "\n",
    "```python \n",
    "\n",
    "# Add another example route where you can configure the model based\n",
    "# on properties of the request. This is useful for passing in API\n",
    "# keys from request headers (WITH CAUTION) or using other properties\n",
    "# of the request to configure the model.\n",
    "def fetch_api_key_from_header(config: Dict[str, Any], req: Request) -> Dict[str, Any]:\n",
    "    if \"x-api-key\" in req.headers:\n",
    "        config[\"configurable\"][\"openai_api_key\"] = req.headers[\"x-api-key\"]\n",
    "    return config\n",
    "\n",
    "dynamic_auth_model = ChatOpenAI(openai_api_key='placeholder').configurable_fields(\n",
    "    openai_api_key=ConfigurableField(\n",
    "        id=\"openai_api_key\",\n",
    "        name=\"OpenAI API Key\",\n",
    "        description=(\n",
    "            \"API Key for OpenAI interactions\"\n",
    "        ),\n",
    "    ),\n",
    ")\n",
    "\n",
    "dynamic_auth_chain = dynamic_auth_model | StrOutputParser()\n",
    "\n",
    "add_routes(\n",
    "    app, \n",
    "    dynamic_auth_chain, \n",
    "    path=\"/auth_from_header\",\n",
    "    config_keys=[\"configurable\"], \n",
    "    per_req_config_modifier=fetch_api_key_from_header\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can see that our request to the model will only work if we have a specific request\n",
    "header set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": ["# The model will fail with an auth error\nunauthenticated_response = requests.post(\n    \"http://localhost:8000/auth_from_header/invoke\", json={\"input\": \"hello\"}\n)\nunauthenticated_response.json()"]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, ensure that you have run the following locally on your shell\n",
    "```bash\n",
    "export TEST_API_KEY=<INSERT MY KEY HERE>\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": ["# The model will succeed as long as the above shell script is run previously\nimport os\n\ntest_key = os.environ[\"TEST_API_KEY\"]\nauthenticated_response = requests.post(\n    \"http://localhost:8000/auth_from_header/invoke\",\n    json={\"input\": \"hello\"},\n    headers={\"x-api-key\": test_key},\n)\nauthenticated_response.json()"]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [""]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}