"The new [Assistants API](https://platform.openai.com/docs/assistants/overview) is a stateful evolution of our [Chat Completions API](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) meant to simplify the creation of assistant-like experiences, and enable developer access to powerful tools like Code Interpreter and Retrieval.\n",
"\n",
"## Chat Completions API vs Assistants API\n",
"\n",
"The primitives of the **Chat Completions API** are `Messages`, on which you perform a `Completion` with a `Model` (`gpt-3.5-turbo`, `gpt-4`, etc). It is lightweight and powerful, but inherently stateless, which means you have to manage conversation state, tool definitions, retrieval documents, and code execution manually.\n",
"\n",
"The primitives of the **Assistants API** are\n",
"\n",
"- `Assistants`, which encapsulate a base model, instructions, tools, and (context) documents,\n",
"- `Threads`, which represent the state of a conversation, and\n",
"- `Runs`, which power the execution of an `Assistant` on a `Thread`, including textual responses and multi-step tool use.\n",
"\n",
"We'll take a look at how these can be used to create powerful, stateful experiences.\n"
"> We've updated our [Python SDK](https://github.com/openai/openai-python) to add support for the Assistants API, so you'll need to update it to the latest version (`1.2.3` at time of writing).\n"
" 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n",
" 'metadata': {},\n",
" 'model': 'gpt-4-1106-preview',\n",
" 'name': 'Math Tutor',\n",
" 'object': 'assistant',\n",
" 'tools': []}"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from openai import OpenAI\n",
"\n",
"client = OpenAI()\n",
"\n",
"assistant = client.beta.assistants.create(\n",
" name=\"Math Tutor\",\n",
" instructions=\"You are a personal math tutor. Answer questions briefly, in a sentence or less.\",\n",
" model=\"gpt-4-1106-preview\",\n",
")\n",
"show_json(assistant)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Regardless of whether you create your Assistant through the Dashboard or with the API, you'll want to keep track of the Assistant ID. This is how you'll refer to your Assistant throughout Threads and Runs.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we'll create a new Thread and add a message to it. This will hold the state of our conversation, so we don't have re-send the entire message history each time.\n"
" content=\"I need to solve the equation `3x + 11 = 14`. Can you help me?\",\n",
")\n",
"show_json(message)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Note**\n",
"> Even though you're no longer sending the entire history each time, you will still be charged for the tokens of the entire conversation history with each Run.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Runs\n",
"\n",
"Notice how the Thread we created is **not** associated with the Assistant we created earlier! Threads exist independently from Assistants, which may be different from what you'd expect if you've used ChatGPT (where a thread is tied to a model/GPT).\n",
"\n",
"To get a completion from an Assistant for a given Thread, we must create a Run. Creating a Run will indicate to an Assistant it should look at the messages in the Thread and take action: either by adding a single response, or using tools.\n",
"\n",
"> **Note**\n",
"> Runs are a key difference between the Assistants API and Chat Completions API. While in Chat Completions the model will only ever respond with a single message, in the Assistants API a Run may result in an Assistant using one or multiple tools, and potentially adding multiple messages to the Thread.\n",
"\n",
"To get our Assistant to respond to the user, let's create the Run. As mentioned earlier, you must specify _both_ the Assistant and the Thread.\n"
"Unlike creating a completion in the Chat Completions API, **creating a Run is an asynchronous operation**. It will return immediately with the Run's metadata, which includes a `status` that will initially be set to `queued`. The `status` will be updated as the Assistant performs operations (like using tools and adding messages).\n",
"To know when the Assistant has completed processing, we can poll the Run in a loop. (Support for streaming is coming soon!) While here we are only checking for a `queued` or `in_progress` status, in practice a Run may undergo a [variety of status changes](https://platform.openai.com/docs/api-reference/runs/object#runs/object-status) which you can choose to surface to the user. (These are called Steps, and will be covered later.)\n"
"As you can see, Messages are ordered in reverse-chronological order – this was done so the most recent results are always on the first `page` (since results can be paginated). Do keep a look out for this, since this is the opposite order to messages in the Chat Completions API.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's ask our Assistant to explain the result a bit further!\n"
" 'value': 'Certainly! The goal is to get x by itself on one side of the equation. \\n\\nStep 1: Subtract 11 from both sides to isolate the term with x: \\n`3x + 11 - 11 = 14 - 11`\\nThis simplifies to: \\n`3x = 3`\\n\\nStep 2: Now, divide both sides by 3, the coefficient of x, to solve for x:\\n`3x / 3 = 3 / 3`\\nThis results in:\\n`x = 1`'},\n",
"This may feel like a lot of steps to get a response back, especially for this simple example. However, you'll soon see how we can add very powerful functionality to our Assistant without changing much code at all!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Example\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's take a look at how we could potentially put all of this together. Below is all the code you need to use an Assistant you've created.\n",
"\n",
"Since we've already created our Math Assistant, I've saved its ID in `MATH_ASSISTANT_ID`. I then defined two functions:\n",
"\n",
"- `submit_message`: create a Message on a Thread, then start (and return) a new Run\n",
"- `get_response`: returns the list of Messages in a Thread\n"
"I've also defined a `create_thread_and_run` function that I can re-use (which is actually almost identical to the [`client.beta.threads.create_and_run`](https://platform.openai.com/docs/api-reference/runs/createThreadAndRun) compound function in our API ;) ). Finally, we can submit our mock user requests each to a new Thread.\n",
"\n",
"Notice how all of these API calls are asynchronous operations; this means we actually get async behavior in our code without the use of async libraries! (e.g. `asyncio`)\n"
"assistant: Linear algebra is the branch of mathematics concerning vector spaces and linear mappings between these spaces, which includes the study of lines, planes, and subspaces, but is also concerned with properties common to all vector spaces.\n",
"assistant: Find aspects of math that connect with your interests or everyday life to make it more relatable and engaging, and consider working with a tutor or using interactive tools that can make learning math more enjoyable.\n",
"assistant: Find aspects of math that connect with your interests or everyday life to make it more relatable and engaging, and consider working with a tutor or using interactive tools that can make learning math more enjoyable.\n",
"You may have noticed that this code is not actually specific to our math Assistant at all... this code will work for any new Assistant you create simply by changing the Assistant ID! That is the power of the Assistants API.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tools\n",
"\n",
"A key feature of the Assistants API is the ability to equip our Assistants with Tools, like Code Interpreter, Retrieval, and custom Functions. Let's take a look at each.\n",
"Let's equip our Math Tutor with the [Code Interpreter](https://platform.openai.com/docs/assistants/tools/code-interpreter) tool, which we can do from the Dashboard...\n"
" \"Generate the first 20 fibbonaci numbers with code.\"\n",
")\n",
"run = wait_on_run(run, thread)\n",
"pretty_print(get_response(thread))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And that's it! The Assistant used Code Interpreter in the background, and gave us a final response.\n",
"\n",
"For some use cases this may be enough –however, if we want more details on what precisely an Assistant is doing we can take a look at a Run's Steps.\n",
"\n",
"### Steps\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A Run is composed of one or more Steps. Like a Run, each Step has a `status` that you can query. This is useful for surfacing the progress of a Step to a user (e.g. a spinner while the Assistant is writing code or performing retrieval).\n"
" 'code_interpreter': {'input': '# Function to generate first 20 Fibonacci numbers\\ndef generate_fibonacci(n):\\n fib_series = [0, 1]\\n for _ in range(2, n):\\n fib_series.append(fib_series[-1] + fib_series[-2])\\n return fib_series\\n\\n# Generate and display the first 20 Fibonacci numbers\\nfirst_20_fibonacci = generate_fibonacci(20)\\nfirst_20_fibonacci',\n",
"Another powerful tool in the Assistants API is [Retrieval](https://platform.openai.com/docs/assistants/tools/knowledge-retrieval): the ability to upload files that the Assistant will use as a knowledge base when answering questions. This can also be enabled from the Dashboard or the API, where we can upload files we want to be used.\n"
"assistant: One cool math concept in this ML paper is the application of zero-shot learning where language models (LMs) demonstrate the ability to transfer well across different domains and datasets without task-specific training, improving the state of the art in seven out of eight datasets【11†source】. Specifically, it highlights the models' performance on complex tasks like LAMBADA and the Children’s Book Test, which are benchmarks created to measure long-term dependencies in text, and notes challenges in datasets with extensive pre-processing that disrupts long-range structure【11†source】.\n",
" \"What are some cool math concepts behind this ML paper pdf? Explain in two sentences.\"\n",
")\n",
"run = wait_on_run(run, thread)\n",
"pretty_print(get_response(thread))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Note**\n",
"> There are more intricacies in Retrieval, like [Annotations](https://platform.openai.com/docs/assistants/how-it-works/managing-threads-and-messages), which may be covered in another cookbook.\n"
"As a final powerful tool for your Assistant, you can specify custom [Functions](https://platform.openai.com/docs/assistants/tools/function-calling) (much like the [Function Calling](https://platform.openai.com/docs/guides/function-calling) in the Chat Completions API). During a Run, the Assistant can then indicate it wants to call one or more functions you specified. You are then responsible for calling the Function, and providing the output back to the Assistant.\n",
"Unfortunately I don't know how to get user input within a Python Notebook, so I'll be mocking out responses with `get_mock_response...`. This is where you'd get the user's actual input.\n"
"> Pasting the function JSON into the Dashboard was a bit finicky due to indentation, etc. I just asked ChatGPT to format my function the same as one of the examples on the Dashboard :).\n"
" 'function': {'arguments': '{\\n \"title\": \"Math Fundamentals Quiz\",\\n \"questions\": [\\n {\\n \"question_text\": \"Explain in your own words why a negative times a negative is a positive.\",\\n \"question_type\": \"FREE_RESPONSE\"\\n },\\n {\\n \"question_text\": \"What is the result of the expression 2^(3) * 2^(-4)?\",\\n \"question_type\": \"MULTIPLE_CHOICE\",\\n \"choices\": [\"1/2\", \"1/4\", \"1/8\", \"1/16\"]\\n }\\n ]\\n}',\n",
"The `required_action` field indicates a Tool is waiting for us to run it and submit its output back to the Assistant. Specifically, the `display_quiz` function! Let's start by parsing the `name` and `arguments`.\n",
"\n",
"> **Note**\n",
"> While in this case we know there is only one Tool call, in practice the Assistant may choose to call multiple tools.\n"
"Great! (Remember these responses are the one's we mocked earlier. In reality, we'd be getting input from the back from this function call.)\n",
"\n",
"Now that we have our responses, let's submit them back to the Assistant. We'll need the `tool_call` ID, found in the `tool_call` we parsed out earlier. We'll also need to encode our `list`of responses into a `str`.\n"
"assistant: Your responses show that there is some confusion that we need to address.\n",
"\n",
"For the open-ended question, saying \"I don't know\" indicates we need to explore the concept of multiplying negatives together. Remember, a negative number is essentially an opposite sign, so when you multiply two negatives, you are effectively doing the \"opposite of an opposite,\" which results in a positive. It's a fundamental rule in math that helps maintain consistency in mathematical operations.\n",
"Regarding the multiple choice question, the answer you chose, \"a,\" corresponds to the option \"1/2.\" However, the correct answer is actually \"1/16.\" To solve this, you can use the rule of exponents that states when multiplying powers with the same base, you can add the exponents: 2^(3 + (-4)) = 2^(-1) = 1/2^1 = 1/2.\n",
"Don't be discouraged; these concepts can be tricky, but with practice, you'll get the hang of it! Let's work through similar problems to build your understanding. Would you like to go over more examples together?\n",
"We covered a lot of ground in this notebook, give yourself a high-five! Hopefully you should now have a strong foundation to build powerful, stateful experiences with tools like Code Interpreter, Retrieval, and Functions!\n",
"\n",
"There's a few sections we didn't cover for the sake of brevity, so here's a few resources to explore further:\n",
"- [Files](https://platform.openai.com/docs/api-reference/assistants/file-object): Thread scoped vs Assistant scoped\n",
"- [Parallel Function Calls](https://platform.openai.com/docs/guides/function-calling/parallel-function-calling): calling multiple tools in a single Step\n",
"- Multi-Assistant Thread Runs: single Thread with Messages from multiple Assistants\n",
