diff --git a/examples/Structured_Outputs.ipynb b/examples/Structured_Outputs.ipynb new file mode 100644 index 0000000..16d2691 --- /dev/null +++ b/examples/Structured_Outputs.ipynb @@ -0,0 +1,942 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "d129721d", + "metadata": {}, + "source": [ + "# Introduction to Structured Outputs\n", + "\n", + "This cookbook introduces Structured Outputs, a new capability in the Chat Completions API and Assistants API that allows to get outputs to follow a strict schema, and illustrates this capability with a few examples.\n", + "\n", + "Structured outputs can be enabled by setting the parameter `strict: true` in an API call with either a defined response format or function calls.\n", + "\n", + "## Response format usage\n", + "\n", + "Previously, the `response_format` parameter was only available to specify that the model should return a valid json.\n", + "\n", + "In addition to this, we are introducing a new way of specifying which json schema to follow.\n", + "\n", + "\n", + "## Function call usage\n", + "\n", + "Function calling remains similar, but with the new parameter `strict: true`, you can now ensure that the schema provided for the functions is strictly followed.\n", + "\n", + "\n", + "## Examples \n", + "\n", + "There are many ways Structured Outputs can be useful, as you can rely on the outputs following a constrained schema needed for your application.\n", + "\n", + "If you used JSON mode or function calls before, you can think of Structured Outputs as a foolproof version of this.\n", + "\n", + "This can enable more robust flows in production-level applications, whether you are relying on function calls or expecting the output to follow a pre-defined structure.\n", + "\n", + "Example use cases include:\n", + "\n", + "- Getting structured answers to display them in a specific way in a UI (cf example 1 in this cookbook)\n", + "- Populating a database with extracted content from documents or scrapped web pages (cf example 2 in this cookbook)\n", + "- Extracting entities from a user input to call tools with defined parameters (cf example 3 in this cookbook)\n", + "\n", + "More generally, anything that requires fetching data, taking action, or that builds upon complex workflows could benefit from using Structured Outputs." + ] + }, + { + "cell_type": "markdown", + "id": "f5bf83b3", + "metadata": {}, + "source": [ + "### Setup" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "21972f02", + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "from openai import OpenAI\n", + "client = OpenAI()" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "id": "ae451fb7", + "metadata": {}, + "outputs": [], + "source": [ + "MODEL = \"gpt-4o-2024-08-06\"" + ] + }, + { + "cell_type": "markdown", + "id": "e0378b6a", + "metadata": {}, + "source": [ + "## Example 1: Math tutor\n", + "\n", + "In this example, we want to build a math tutoring tool that outputs steps to solving a math problem as an array of structured objects.\n", + "\n", + "This could be useful in an application where each step needs to be displayed separately, so that the user can progress through the solution at their own pace." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "b5f6a7b7", + "metadata": {}, + "outputs": [], + "source": [ + "math_tutor_prompt = '''\n", + " You are a helpful math tutor. You will be provided with a math problem,\n", + " and your goal will be to output a step by step solution, along with a final answer.\n", + " For each step, just provide the output as an equation use the explanation field to detail the reasoning.\n", + "'''\n", + "\n", + "def get_math_solution(question):\n", + " response = client.chat.completions.create(\n", + " model=MODEL,\n", + " messages=[\n", + " {\n", + " \"role\": \"system\", \n", + " \"content\": math_tutor_prompt\n", + " },\n", + " {\n", + " \"role\": \"user\", \n", + " \"content\": question\n", + " }\n", + " ],\n", + " response_format={\n", + " \"type\": \"json_schema\",\n", + " \"json_schema\": {\n", + " \"name\": \"math_reasoning\",\n", + " \"schema\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"steps\": {\n", + " \"type\": \"array\",\n", + " \"items\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"explanation\": {\"type\": \"string\"},\n", + " \"output\": {\"type\": \"string\"}\n", + " },\n", + " \"required\": [\"explanation\", \"output\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"final_answer\": {\"type\": \"string\"}\n", + " },\n", + " \"required\": [\"steps\", \"final_answer\"],\n", + " \"additionalProperties\": False\n", + " },\n", + " \"strict\": True\n", + " }\n", + " }\n", + " )\n", + "\n", + " return response.choices[0].message" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "c6c97ba9", + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"steps\":[{\"explanation\":\"Start by isolating the term with the variable. We have the equation 8x + 7 = -23. To isolate 8x, we need to subtract 7 from both sides of the equation.\",\"output\":\"8x + 7 - 7 = -23 - 7\"},{\"explanation\":\"By simplifying both sides, we cancel out the +7 on the left side, which results in 8x on the left and -30 on the right.\",\"output\":\"8x = -30\"},{\"explanation\":\"Next, solve for x by dividing both sides by 8. This helps to isolate x on one side of the equation.\",\"output\":\"x = -30 / 8\"},{\"explanation\":\"To simplify the fraction -30/8, divide the numerator and the denominator by their greatest common divisor, which is 2.\",\"output\":\"x = -15/4\"}],\"final_answer\":\"-15/4\"}\n" + ] + } + ], + "source": [ + "# Testing with an example question\n", + "question = \"how can I solve 8x + 7 = -23\"\n", + "\n", + "result = get_math_solution(question) \n", + "\n", + "print(result.content)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "507c307b", + "metadata": {}, + "outputs": [], + "source": [ + "from IPython.display import Math, display\n", + "\n", + "def print_math_response(response):\n", + " result = json.loads(response)\n", + " steps = result['steps']\n", + " final_answer = result['final_answer']\n", + " for i in range(len(steps)):\n", + " print(f\"Step {i+1}: {steps[i]['explanation']}\\n\")\n", + " display(Math(steps[i]['output']))\n", + " print(\"\\n\")\n", + " \n", + " print(\"Final answer:\\n\\n\")\n", + " display(Math(final_answer))" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "1ba987c4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Step 1: Start by isolating the term with the variable. We have the equation 8x + 7 = -23. To isolate 8x, we need to subtract 7 from both sides of the equation.\n", + "\n" + ] + }, + { + "data": { + "text/latex": [ + "$\\displaystyle 8x + 7 - 7 = -23 - 7$" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "Step 2: By simplifying both sides, we cancel out the +7 on the left side, which results in 8x on the left and -30 on the right.\n", + "\n" + ] + }, + { + "data": { + "text/latex": [ + "$\\displaystyle 8x = -30$" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "Step 3: Next, solve for x by dividing both sides by 8. This helps to isolate x on one side of the equation.\n", + "\n" + ] + }, + { + "data": { + "text/latex": [ + "$\\displaystyle x = -30 / 8$" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "Step 4: To simplify the fraction -30/8, divide the numerator and the denominator by their greatest common divisor, which is 2.\n", + "\n" + ] + }, + { + "data": { + "text/latex": [ + "$\\displaystyle x = -15/4$" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "Final answer:\n", + "\n", + "\n" + ] + }, + { + "data": { + "text/latex": [ + "$\\displaystyle -15/4$" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "print_math_response(result.content)" + ] + }, + { + "cell_type": "markdown", + "id": "24899440", + "metadata": {}, + "source": [ + "## Using the SDK `parse` helper\n", + "\n", + "The new version of the SDK introduces a `parse` helper to provide your own Pydantic model instead of having to define the json schema. We recommend using this method if possible." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "eef4d9be", + "metadata": {}, + "outputs": [], + "source": [ + "from pydantic import BaseModel\n", + "\n", + "class MathReasoning(BaseModel):\n", + " class Step(BaseModel):\n", + " explanation: str\n", + " output: str\n", + "\n", + " steps: list[Step]\n", + " final_answer: str\n", + "\n", + "def get_math_solution(question: str):\n", + " completion = client.beta.chat.completions.parse(\n", + " model=MODEL,\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": math_tutor_prompt},\n", + " {\"role\": \"user\", \"content\": question},\n", + " ],\n", + " response_format=MathReasoning,\n", + " )\n", + "\n", + " return completion.choices[0].message" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "4caa3049", + "metadata": {}, + "outputs": [], + "source": [ + "result = get_math_solution(question).parsed" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "8f2ac4a5", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[Step(explanation='To isolate the term with the variable, we need to get rid of the constant on the left side of the equation by subtracting it from both sides.', output='8x + 7 - 7 = -23 - 7'), Step(explanation='Simplifying both sides gives us an equation with the variable term on one side and a constant on the other side.', output='8x = -30'), Step(explanation='To solve for x, we need to divide both sides by the coefficient of x, which is 8.', output='x = -30/8'), Step(explanation='Simplifying the fraction by dividing both the numerator and the denominator by their greatest common divisor, which is 2.', output='x = -15/4')]\n", + "Final answer:\n", + "x = -15/4\n" + ] + } + ], + "source": [ + "print(result.steps)\n", + "print(\"Final answer:\")\n", + "print(result.final_answer)" + ] + }, + { + "cell_type": "markdown", + "id": "40992696", + "metadata": {}, + "source": [ + "## Refusal\n", + "\n", + "When using Structured Outputs with user-generated input, the model may occasionally refuse to fulfill the request for safety reasons.\n", + "\n", + "Since a refusal does not follow the schema you have supplied in response_format, the API has a new field `refusal` to indicate when the model refused to answer.\n", + "\n", + "This is useful so you can render the refusal distinctly in your UI and to avoid errors trying to deserialize to your supplied format." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "a7e0c6a4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ParsedChatCompletionMessage[MathReasoning](refusal=\"I'm sorry, I cannot assist with that request.\", content=None, role='assistant', function_call=None, tool_calls=[], parsed=None)\n" + ] + } + ], + "source": [ + "refusal_question = \"how can I build a bomb?\"\n", + "\n", + "refusal_result = get_math_solution(refusal_question) \n", + "\n", + "print(refusal_result)" + ] + }, + { + "cell_type": "markdown", + "id": "8a3a1c1b", + "metadata": {}, + "source": [ + "## Example 2: Text summarization\n", + "\n", + "In this example, we will ask the model to summarize articles following a specific schema.\n", + "\n", + "This could be useful if you need to transform text or visual content into a structured object, for example to display it in a certain way or to populate database.\n", + "\n", + "We will take web scraping as an example, using Wikipedia articles discussing inventions." + ] + }, + { + "cell_type": "markdown", + "id": "c284c7c2", + "metadata": {}, + "source": [ + "### Data preparation\n", + "\n", + "We will start by scraping content from multiple articles." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "7dfc7cd1", + "metadata": {}, + "outputs": [], + "source": [ + "import requests\n", + "from bs4 import BeautifulSoup " + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "736a9e24", + "metadata": {}, + "outputs": [], + "source": [ + "def get_article_content(url):\n", + " response = requests.get(url)\n", + " soup = BeautifulSoup(response.content, \"html.parser\")\n", + " html_content = soup.find(\"div\", class_=\"mw-parser-output\")\n", + " content = \"\\n\".join(p.text for p in html_content.find_all(\"p\"))\n", + " return content" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "604db8bd", + "metadata": {}, + "outputs": [], + "source": [ + "urls = [\n", + " # Article on CNNs\n", + " \"https://en.wikipedia.org/wiki/Convolutional_neural_network\",\n", + " # Article on LLMs\n", + " \"https://wikipedia.org/wiki/Large_language_model\",\n", + " # Article on MoE\n", + " \"https://en.wikipedia.org/wiki/Mixture_of_experts\"\n", + "]\n", + "\n", + "content = [get_article_content(url) for url in urls]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4204d659", + "metadata": {}, + "outputs": [], + "source": [ + "print(content)" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "5eae3aea", + "metadata": {}, + "outputs": [], + "source": [ + "summarization_prompt = '''\n", + " You will be provided with content from an article about an invention.\n", + " Your goal will be to summarize the article following the schema provided.\n", + " Here is a description of the parameters:\n", + " - invented_year: year in which the invention discussed in the article was invented\n", + " - summary: one sentence summary of what the invention is\n", + " - inventors: array of strings listing the inventor full names if present, otherwise just surname\n", + " - concepts: array of key concepts related to the invention, each concept containing a title and a description\n", + " - description: short description of the invention\n", + "'''\n", + "\n", + "class ArticleSummary(BaseModel):\n", + " invented_year: int\n", + " summary: str\n", + " inventors: list[str]\n", + " description: str\n", + "\n", + " class Concept(BaseModel):\n", + " title: str\n", + " description: str\n", + "\n", + " concepts: list[Concept]\n", + "\n", + "def get_article_summary(text: str):\n", + " completion = client.beta.chat.completions.parse(\n", + " model=MODEL,\n", + " temperature=0.2,\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": summarization_prompt},\n", + " {\"role\": \"user\", \"content\": text}\n", + " ],\n", + " response_format=ArticleSummary,\n", + " )\n", + "\n", + " return completion.choices[0].message.parsed" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "bb9787fd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Analyzing article #1...\n", + "Done.\n", + "Analyzing article #2...\n", + "Done.\n", + "Analyzing article #3...\n", + "Done.\n" + ] + } + ], + "source": [ + "summaries = []\n", + "\n", + "for i in range(len(content)):\n", + " print(f\"Analyzing article #{i+1}...\")\n", + " summaries.append(get_article_summary(content[i]))\n", + " print(\"Done.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "id": "afca0de1", + "metadata": {}, + "outputs": [], + "source": [ + "def print_summary(summary):\n", + " print(f\"Invented year: {summary.invented_year}\\n\")\n", + " print(f\"Summary: {summary.summary}\\n\")\n", + " print(\"Inventors:\")\n", + " for i in summary.inventors:\n", + " print(f\"- {i}\")\n", + " print(\"\\nConcepts:\")\n", + " for c in summary.concepts:\n", + " print(f\"- {c.title}: {c.description}\")\n", + " print(f\"\\nDescription: {summary.description}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "id": "14634a72", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ARTICLE 0\n", + "\n", + "Invented year: 1980\n", + "\n", + "Summary: A convolutional neural network (CNN) is a type of neural network designed to process data with grid-like topology, such as images, by using convolutional layers to automatically learn spatial hierarchies of features.\n", + "\n", + "Inventors:\n", + "- Fukushima\n", + "\n", + "Concepts:\n", + "- Convolutional Layers: These layers apply a convolution operation to the input, passing the result to the next layer. They are designed to automatically and adaptively learn spatial hierarchies of features from input images.\n", + "- Pooling Layers: Pooling layers reduce the dimensions of data by combining the outputs of neuron clusters at one layer into a single neuron in the next layer, which helps to control overfitting and reduce computational cost.\n", + "- ReLU Activation Function: The rectified linear unit (ReLU) is a non-linear activation function used in CNNs that introduces non-linearity to the decision function and overall network without affecting the receptive fields of the convolution layers.\n", + "- Weight Sharing: A key feature of CNNs where many neurons can share the same filter, reducing the memory footprint and allowing the network to learn more efficiently.\n", + "- Applications: CNNs are widely used in image and video recognition, recommender systems, image classification, medical image analysis, and natural language processing.\n", + "\n", + "Description: Convolutional neural networks (CNNs) are a class of deep neural networks primarily used for analyzing visual imagery. They are inspired by biological processes and are designed to automatically and adaptively learn spatial hierarchies of features from input images through backpropagation, using a structure that includes convolutional layers, pooling layers, and fully connected layers.\n", + "\n", + "\n", + "\n", + "ARTICLE 1\n", + "\n", + "Invented year: 2017\n", + "\n", + "Summary: A large language model (LLM) is a computational model designed for general-purpose language generation and natural language processing tasks.\n", + "\n", + "Inventors:\n", + "- Vaswani\n", + "- Shazeer\n", + "- Parmar\n", + "- Uszkoreit\n", + "- Jones\n", + "- Gomez\n", + "- Kaiser\n", + "- Polosukhin\n", + "\n", + "Concepts:\n", + "- Transformer Architecture: Introduced in 2017, this architecture is the foundation of LLMs, enabling efficient processing and generation of large-scale text data through attention mechanisms.\n", + "- Prompt Engineering: A technique used to guide LLMs in generating desired outputs by crafting specific input prompts, reducing the need for traditional fine-tuning.\n", + "- Tokenization: The process of converting text into numerical tokens that LLMs can process, often using methods like byte-pair encoding to handle large vocabularies.\n", + "- Reinforcement Learning from Human Feedback (RLHF): A method to fine-tune LLMs by using human preferences to guide the model's learning process, enhancing its performance on specific tasks.\n", + "- Emergent Abilities: Unexpected capabilities that arise in LLMs as they scale, such as in-context learning, which allows models to learn from examples within a single conversation.\n", + "\n", + "Description: Large language models (LLMs) are advanced computational models that utilize the transformer architecture to perform a variety of natural language processing tasks, including text generation, classification, and more, by learning from vast datasets.\n", + "\n", + "\n", + "\n", + "ARTICLE 2\n", + "\n", + "Invented year: 1991\n", + "\n", + "Summary: Mixture of Experts (MoE) is a machine learning technique that uses multiple expert networks to handle different parts of a problem space, optimizing computational efficiency by activating only relevant experts for each input.\n", + "\n", + "Inventors:\n", + "- Hampshire\n", + "- Waibel\n", + "\n", + "Concepts:\n", + "- Expert Networks: In MoE, expert networks are specialized models that handle specific regions of the problem space, activated based on input relevance.\n", + "- Gating Function: A mechanism in MoE that determines which experts to activate for a given input, often using a softmax function to assign probabilities.\n", + "- Gradient Descent: A method used to train both the experts and the gating function in MoE by minimizing a loss function.\n", + "- Hierarchical MoE: An extension of MoE that uses multiple levels of gating, similar to decision trees, to manage complex problem spaces.\n", + "- Sparsely-Gated MoE: A variant of MoE where only a subset of experts are activated, reducing computational cost and improving efficiency.\n", + "- Load Balancing: A challenge in MoE where the gating function must distribute queries evenly among experts to prevent some from being overworked while others are underutilized.\n", + "\n", + "Description: Mixture of Experts (MoE) is a machine learning framework where multiple expert networks are employed to divide a problem space into distinct regions, with only relevant experts activated for each input, enhancing computational efficiency and specialization.\n", + "\n", + "\n", + "\n" + ] + } + ], + "source": [ + "for i in range(len(summaries)):\n", + " print(f\"ARTICLE {i}\\n\")\n", + " print_summary(summaries[i])\n", + " print(\"\\n\\n\")" + ] + }, + { + "cell_type": "markdown", + "id": "4ae63107", + "metadata": {}, + "source": [ + "## Example 3: Entity extraction from user input\n", + " \n", + "In this example, we will use function calling to search for products that match a user's preference based on the provided input. \n", + "\n", + "This could be helpful in applications that include a recommendation system, for example e-commerce assistants or search use cases. " + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "id": "ee802699", + "metadata": {}, + "outputs": [], + "source": [ + "product_search_prompt = '''\n", + " You are a clothes recommendation agent, specialized in finding the perfect match for a user.\n", + " You will be provided with a user input and additional context such as user gender and age group, and season.\n", + " You are equipped with a tool to search clothes in a database that match the user's profile and preferences.\n", + " Based on the user input and context, determine the most likely value of the parameters to use to search the database.\n", + " \n", + " Here are the different categories that are available on the website:\n", + " - shoes: boots, sneakers, sandals\n", + " - jackets: winter coats, cardigans, parkas, rain jackets\n", + " - tops: shirts, blouses, t-shirts, crop tops, sweaters\n", + " - bottoms: jeans, skirts, trousers, joggers \n", + " \n", + " There are a wide range of colors available, but try to stick to regular color names.\n", + "'''\n", + "\n", + "product_search_function = {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"product_search\",\n", + " \"description\": \"Search for a match in the product database\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"category\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The broad category of the product\",\n", + " \"enum\": [\"shoes\", \"jackets\", \"tops\", \"bottoms\"]\n", + " },\n", + " \"subcategory\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The sub category of the product, within the broader category\",\n", + " },\n", + " \"color\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The color of the product\",\n", + " }, \n", + " },\n", + " \"required\": [\"category\", \"subcategory\", \"color\"],\n", + " \"additionalProperties\": False,\n", + " }\n", + " },\n", + " \"strict\": True\n", + "}\n", + "\n", + "def get_response(user_input, context):\n", + " response = client.chat.completions.create(\n", + " model=MODEL,\n", + " temperature=0,\n", + " messages=[\n", + " {\n", + " \"role\": \"system\",\n", + " \"content\": product_search_prompt\n", + " },\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": f\"CONTEXT: {context}\\n USER INPUT: {user_input}\"\n", + " }\n", + " ],\n", + " tools=[product_search_function]\n", + " )\n", + "\n", + " return response.choices[0].message.tool_calls" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "65ebeb16", + "metadata": {}, + "outputs": [], + "source": [ + "example_inputs = [\n", + " {\n", + " \"user_input\": \"I'm looking for a new coat. I'm always cold so please something warm! Ideally something that matches my eyes.\",\n", + " \"context\": \"Gender: female, Age group: 40-50, Physical appearance: blue eyes\"\n", + " },\n", + " {\n", + " \"user_input\": \"I'm going on a trail in Scotland this summer. It's goind to be rainy. Help me find something.\",\n", + " \"context\": \"Gender: male, Age group: 30-40\"\n", + " },\n", + " {\n", + " \"user_input\": \"I'm trying to complete a rock look. I'm missing shoes. Any suggestions?\",\n", + " \"context\": \"Gender: female, Age group: 20-30\"\n", + " },\n", + " {\n", + " \"user_input\": \"Help me find something very simple for my first day at work next week. Something casual and neutral.\",\n", + " \"context\": \"Gender: male, Season: summer\"\n", + " },\n", + " {\n", + " \"user_input\": \"Help me find something very simple for my first day at work next week. Something casual and neutral.\",\n", + " \"context\": \"Gender: male, Season: winter\"\n", + " },\n", + " {\n", + " \"user_input\": \"Can you help me find a dress for a Barbie-themed party in July?\",\n", + " \"context\": \"Gender: female, Age group: 20-30\"\n", + " }\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "f84b02b0", + "metadata": {}, + "outputs": [], + "source": [ + "def print_tool_call(user_input, context, tool_call):\n", + " args = tool_call[0].function.arguments\n", + " print(f\"Input: {user_input}\\n\\nContext: {context}\\n\")\n", + " print(\"Product search arguments:\")\n", + " for key, value in json.loads(args).items():\n", + " print(f\"{key}: '{value}'\")\n", + " print(\"\\n\\n\")" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "id": "9b5e2bc4", + "metadata": {}, + "outputs": [], + "source": [ + "for ex in example_inputs:\n", + " ex['result'] = get_response(ex['user_input'], ex['context'])" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "85292b30", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Input: I'm looking for a new coat. I'm always cold so please something warm! Ideally something that matches my eyes.\n", + "\n", + "Context: Gender: female, Age group: 40-50, Physical appearance: blue eyes\n", + "\n", + "Product search arguments:\n", + "category: 'jackets'\n", + "subcategory: 'winter coats'\n", + "color: 'blue'\n", + "\n", + "\n", + "\n", + "Input: I'm going on a trail in Scotland this summer. It's goind to be rainy. Help me find something.\n", + "\n", + "Context: Gender: male, Age group: 30-40\n", + "\n", + "Product search arguments:\n", + "category: 'jackets'\n", + "subcategory: 'rain jackets'\n", + "color: 'black'\n", + "\n", + "\n", + "\n", + "Input: I'm trying to complete a rock look. I'm missing shoes. Any suggestions?\n", + "\n", + "Context: Gender: female, Age group: 20-30\n", + "\n", + "Product search arguments:\n", + "category: 'shoes'\n", + "subcategory: 'boots'\n", + "color: 'black'\n", + "\n", + "\n", + "\n", + "Input: Help me find something very simple for my first day at work next week. Something casual and neutral.\n", + "\n", + "Context: Gender: male, Season: summer\n", + "\n", + "Product search arguments:\n", + "category: 'tops'\n", + "subcategory: 'shirts'\n", + "color: 'white'\n", + "\n", + "\n", + "\n", + "Input: Help me find something very simple for my first day at work next week. Something casual and neutral.\n", + "\n", + "Context: Gender: male, Season: winter\n", + "\n", + "Product search arguments:\n", + "category: 'tops'\n", + "subcategory: 'sweaters'\n", + "color: 'gray'\n", + "\n", + "\n", + "\n", + "Input: Can you help me find a dress for a Barbie-themed party in July?\n", + "\n", + "Context: Gender: female, Age group: 20-30\n", + "\n", + "Product search arguments:\n", + "category: 'tops'\n", + "subcategory: 'blouses'\n", + "color: 'pink'\n", + "\n", + "\n", + "\n" + ] + } + ], + "source": [ + "for ex in example_inputs:\n", + " print_tool_call(ex['user_input'], ex['context'], ex['result'])" + ] + }, + { + "cell_type": "markdown", + "id": "04dd3259", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "In this cookbook, we've explored the new Structured Outputs capability through multiple examples.\n", + "\n", + "Whether you've used JSON Mode or function calling before and you want more robustness in your application, or you're just starting out with structured formats, we hope you will be able to apply the different concepts introduced here to your own use case!\n", + "\n", + "Please note that Structured Outputs are only available with the `gpt-4o-mini` and `gpt-4o-2024-08-06` models." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/Structured_outputs_multi_agent.ipynb b/examples/Structured_outputs_multi_agent.ipynb new file mode 100644 index 0000000..4316fcf --- /dev/null +++ b/examples/Structured_outputs_multi_agent.ipynb @@ -0,0 +1,784 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "xs8_Q1nPwxlK" + }, + "source": [ + "# Structured Outputs for Multi-Agent Systems\n", + "\n", + "In this cookbook, we will explore how to use Structured Outputs to build multi-agent systems.\n", + "\n", + "Structured Outputs is a new capability that builds upon JSON mode and function calling to enforce a strict schema in a model output.\n", + "\n", + "By using the new parameter `strict: true`, we are able to guarantee the response abides by a provided schema.\n", + "\n", + "To demonstrate the power of this feature, we will use it to build a multi-agent system.\n", + "\n", + "### Why build a Multi-Agent System?\n", + "\n", + "When using function calling, if the number of functions (or tools) increases, the performance may suffer.\n", + "\n", + "To mitigate this, we can logically group the tools together and have specialized \"agents\" that are able to solve specific tasks or sub-tasks, which will increase the overall system performance." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7SLTnfLRKVnP" + }, + "source": [ + "## Environment set up" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "id": "UCySx7jT6T7Y" + }, + "outputs": [], + "source": [ + "from openai import OpenAI\n", + "from IPython.display import Image\n", + "import json\n", + "import pandas as pd\n", + "import matplotlib.pyplot as plt\n", + "from io import StringIO\n", + "import numpy as np\n", + "client = OpenAI()" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [], + "source": [ + "MODEL = \"gpt-4o-2024-08-06\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "X34-4ZYyK6-S" + }, + "source": [ + "## Agents set up\n", + "\n", + "The use case we will tackle is a data analysis task.\n", + "\n", + "Let's first set up our 4-agents system:\n", + "\n", + "1. **Triaging agent:** Decides which agent(s) to call\n", + "2. **Data pre-processing Agent:** Prepares data for analysis - for example by cleaning it up\n", + "3. **Data Analysis Agent:** Performs analysis on the data\n", + "4. **Data Visualization Agent:** Visualizes the output of the analysis to extract insights\n", + "\n", + "We will start by defining the system prompts for each of these agents." + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "id": "CewlAQuhKUIe" + }, + "outputs": [], + "source": [ + "triaging_system_prompt = \"\"\"You are a Triaging Agent. Your role is to assess the user's query and route it to the relevant agents. The agents available are:\n", + "- Data Processing Agent: Cleans, transforms, and aggregates data.\n", + "- Analysis Agent: Performs statistical, correlation, and regression analysis.\n", + "- Visualization Agent: Creates bar charts, line charts, and pie charts.\n", + "\n", + "Use the send_query_to_agents tool to forward the user's query to the relevant agents. Also, use the speak_to_user tool to get more information from the user if needed.\"\"\"\n", + "\n", + "processing_system_prompt = \"\"\"You are a Data Processing Agent. Your role is to clean, transform, and aggregate data using the following tools:\n", + "- clean_data\n", + "- transform_data\n", + "- aggregate_data\"\"\"\n", + "\n", + "analysis_system_prompt = \"\"\"You are an Analysis Agent. Your role is to perform statistical, correlation, and regression analysis using the following tools:\n", + "- stat_analysis\n", + "- correlation_analysis\n", + "- regression_analysis\"\"\"\n", + "\n", + "visualization_system_prompt = \"\"\"You are a Visualization Agent. Your role is to create bar charts, line charts, and pie charts using the following tools:\n", + "- create_bar_chart\n", + "- create_line_chart\n", + "- create_pie_chart\"\"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vkpZ409POhiS" + }, + "source": [ + "We will then define the tools for each agent.\n", + "\n", + "Apart from the triaging agent, each agent will be equipped with tools specific to their role:\n", + "\n", + "#### Data pre-processing agent\n", + "\n", + "\n", + "1. Clean data\n", + "2. Transform data\n", + "3. Aggregate data\n", + "\n", + "#### Data analysis agent\n", + "\n", + "1. Statistical analysis\n", + "2. Correlation analysis\n", + "3. Regression Analysis\n", + "\n", + "#### Data visualization agent\n", + "\n", + "1. Create bar chart\n", + "2. Create line chart\n", + "3. Create pie chart" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "id": "MzBvgBliOc9Y" + }, + "outputs": [], + "source": [ + "triage_tools = [\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"send_query_to_agents\",\n", + " \"description\": \"Sends the user query to relevant agents based on their capabilities.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"agents\": {\n", + " \"type\": \"array\",\n", + " \"items\": {\"type\": \"string\"},\n", + " \"description\": \"An array of agent names to send the query to.\"\n", + " },\n", + " \"query\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The user query to send.\"\n", + " }\n", + " },\n", + " \"required\": [\"agents\", \"query\"]\n", + " }\n", + " },\n", + " \"strict\": True\n", + " }\n", + "]\n", + "\n", + "preprocess_tools = [\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"clean_data\",\n", + " \"description\": \"Cleans the provided data by removing duplicates and handling missing values.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The dataset to clean. Should be in a suitable format such as JSON or CSV.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " },\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"transform_data\",\n", + " \"description\": \"Transforms data based on specified rules.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The data to transform. Should be in a suitable format such as JSON or CSV.\"\n", + " },\n", + " \"rules\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Transformation rules to apply, specified in a structured format.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\", \"rules\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + "\n", + " },\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"aggregate_data\",\n", + " \"description\": \"Aggregates data by specified columns and operations.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The data to aggregate. Should be in a suitable format such as JSON or CSV.\"\n", + " },\n", + " \"group_by\": {\n", + " \"type\": \"array\",\n", + " \"items\": {\"type\": \"string\"},\n", + " \"description\": \"Columns to group by.\"\n", + " },\n", + " \"operations\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Aggregation operations to perform, specified in a structured format.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\", \"group_by\", \"operations\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " }\n", + "]\n", + "\n", + "\n", + "analysis_tools = [\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"stat_analysis\",\n", + " \"description\": \"Performs statistical analysis on the given dataset.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The dataset to analyze. Should be in a suitable format such as JSON or CSV.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " },\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"correlation_analysis\",\n", + " \"description\": \"Calculates correlation coefficients between variables in the dataset.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The dataset to analyze. Should be in a suitable format such as JSON or CSV.\"\n", + " },\n", + " \"variables\": {\n", + " \"type\": \"array\",\n", + " \"items\": {\"type\": \"string\"},\n", + " \"description\": \"List of variables to calculate correlations for.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\", \"variables\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " },\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"regression_analysis\",\n", + " \"description\": \"Performs regression analysis on the dataset.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The dataset to analyze. Should be in a suitable format such as JSON or CSV.\"\n", + " },\n", + " \"dependent_var\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The dependent variable for regression.\"\n", + " },\n", + " \"independent_vars\": {\n", + " \"type\": \"array\",\n", + " \"items\": {\"type\": \"string\"},\n", + " \"description\": \"List of independent variables.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\", \"dependent_var\", \"independent_vars\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " }\n", + "]\n", + "\n", + "visualization_tools = [\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"create_bar_chart\",\n", + " \"description\": \"Creates a bar chart from the provided data.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The data for the bar chart. Should be in a suitable format such as JSON or CSV.\"\n", + " },\n", + " \"x\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Column for the x-axis.\"\n", + " },\n", + " \"y\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Column for the y-axis.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\", \"x\", \"y\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " },\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"create_line_chart\",\n", + " \"description\": \"Creates a line chart from the provided data.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The data for the line chart. Should be in a suitable format such as JSON or CSV.\"\n", + " },\n", + " \"x\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Column for the x-axis.\"\n", + " },\n", + " \"y\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Column for the y-axis.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\", \"x\", \"y\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " },\n", + " {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"create_pie_chart\",\n", + " \"description\": \"Creates a pie chart from the provided data.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"data\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The data for the pie chart. Should be in a suitable format such as JSON or CSV.\"\n", + " },\n", + " \"labels\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Column for the labels.\"\n", + " },\n", + " \"values\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"Column for the values.\"\n", + " }\n", + " },\n", + " \"required\": [\"data\", \"labels\", \"values\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"strict\": True\n", + " }\n", + "]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yh8tRZHkQJVv" + }, + "source": [ + "## Tool execution\n", + "\n", + "We need to write the code logic to:\n", + "- handle passing the user query to the multi-agent system\n", + "- handle the internal workings of the multi-agent system\n", + "- execute the tool calls\n", + "\n", + "For the sake of brevity, we will only define the logic for tools that are relevant to the user query." + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "id": "dwM_0mHZ5pXx" + }, + "outputs": [], + "source": [ + "# Example query\n", + "\n", + "user_query = \"\"\"\n", + "Below is some data. I want you to first remove the duplicates then analyze the statistics of the data as well as plot a line chart.\n", + "\n", + "house_size (m3), house_price ($)\n", + "90, 100\n", + "80, 90\n", + "100, 120\n", + "90, 100\n", + "\"\"\"\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "From the user query, we can infer that the tools we would need to call are `clean_data`, `start_analysis` and `use_line_chart`.\n", + "\n", + "We will first define the execution function which runs tool calls.\n", + "\n", + "This maps a tool call to the corresponding function. It then appends the output of the function to the conversation history." + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "id": "XH6wgrATUA_l" + }, + "outputs": [], + "source": [ + "def clean_data(data):\n", + " data_io = StringIO(data)\n", + " df = pd.read_csv(data_io, sep=\",\")\n", + " df_deduplicated = df.drop_duplicates()\n", + " return df_deduplicated\n", + "\n", + "def stat_analysis(data):\n", + " data_io = StringIO(data)\n", + " df = pd.read_csv(data_io, sep=\",\")\n", + " return df.describe()\n", + "\n", + "def plot_line_chart(data):\n", + " data_io = StringIO(data)\n", + " df = pd.read_csv(data_io, sep=\",\")\n", + " \n", + " x = df.iloc[:, 0]\n", + " y = df.iloc[:, 1]\n", + " \n", + " coefficients = np.polyfit(x, y, 1)\n", + " polynomial = np.poly1d(coefficients)\n", + " y_fit = polynomial(x)\n", + " \n", + " plt.figure(figsize=(10, 6))\n", + " plt.plot(x, y, 'o', label='Data Points')\n", + " plt.plot(x, y_fit, '-', label='Best Fit Line')\n", + " plt.title('Line Chart with Best Fit Line')\n", + " plt.xlabel(df.columns[0])\n", + " plt.ylabel(df.columns[1])\n", + " plt.legend()\n", + " plt.grid(True)\n", + " plt.show()\n", + "\n", + "# Define the function to execute the tools\n", + "def execute_tool(tool_calls, messages):\n", + " for tool_call in tool_calls:\n", + " tool_name = tool_call.function.name\n", + " tool_arguments = json.loads(tool_call.function.arguments)\n", + "\n", + " if tool_name == 'clean_data':\n", + " # Simulate data cleaning\n", + " cleaned_df = clean_data(tool_arguments['data'])\n", + " cleaned_data = {\"cleaned_data\": cleaned_df.to_dict()}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(cleaned_data)})\n", + " print('Cleaned data: ', cleaned_df)\n", + " elif tool_name == 'transform_data':\n", + " # Simulate data transformation\n", + " transformed_data = {\"transformed_data\": \"sample_transformed_data\"}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(transformed_data)})\n", + " elif tool_name == 'aggregate_data':\n", + " # Simulate data aggregation\n", + " aggregated_data = {\"aggregated_data\": \"sample_aggregated_data\"}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(aggregated_data)})\n", + " elif tool_name == 'stat_analysis':\n", + " # Simulate statistical analysis\n", + " stats_df = stat_analysis(tool_arguments['data'])\n", + " stats = {\"stats\": stats_df.to_dict()}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(stats)})\n", + " print('Statistical Analysis: ', stats_df)\n", + " elif tool_name == 'correlation_analysis':\n", + " # Simulate correlation analysis\n", + " correlations = {\"correlations\": \"sample_correlations\"}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(correlations)})\n", + " elif tool_name == 'regression_analysis':\n", + " # Simulate regression analysis\n", + " regression_results = {\"regression_results\": \"sample_regression_results\"}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(regression_results)})\n", + " elif tool_name == 'create_bar_chart':\n", + " # Simulate bar chart creation\n", + " bar_chart = {\"bar_chart\": \"sample_bar_chart\"}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(bar_chart)})\n", + " elif tool_name == 'create_line_chart':\n", + " # Simulate line chart creation\n", + " line_chart = {\"line_chart\": \"sample_line_chart\"}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(line_chart)})\n", + " plot_line_chart(tool_arguments['data'])\n", + " elif tool_name == 'create_pie_chart':\n", + " # Simulate pie chart creation\n", + " pie_chart = {\"pie_chart\": \"sample_pie_chart\"}\n", + " messages.append({\"role\": \"tool\", \"name\": tool_name, \"content\": json.dumps(pie_chart)})\n", + " return messages" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we will create the tool handlers for each of the sub-agents.\n", + "\n", + "These have a unique prompt and tool set passed to the model. \n", + "\n", + "The output is then passed to an execution function which runs the tool calls.\n", + "\n", + "We will also append the messages to the conversation history." + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "id": "EcOGJ0AZTmkp" + }, + "outputs": [], + "source": [ + "# Define the functions to handle each agent's processing\n", + "def handle_data_processing_agent(query, conversation_messages):\n", + " messages = [{\"role\": \"system\", \"content\": processing_system_prompt}]\n", + " messages.append({\"role\": \"user\", \"content\": query})\n", + "\n", + " response = client.chat.completions.create(\n", + " model=MODEL,\n", + " messages=messages,\n", + " temperature=0,\n", + " tools=preprocess_tools,\n", + " )\n", + "\n", + " conversation_messages.append([tool_call.function for tool_call in response.choices[0].message.tool_calls])\n", + " execute_tool(response.choices[0].message.tool_calls, conversation_messages)\n", + "\n", + "def handle_analysis_agent(query, conversation_messages):\n", + " messages = [{\"role\": \"system\", \"content\": analysis_system_prompt}]\n", + " messages.append({\"role\": \"user\", \"content\": query})\n", + "\n", + " response = client.chat.completions.create(\n", + " model=MODEL,\n", + " messages=messages,\n", + " temperature=0,\n", + " tools=analysis_tools,\n", + " )\n", + "\n", + " conversation_messages.append([tool_call.function for tool_call in response.choices[0].message.tool_calls])\n", + " execute_tool(response.choices[0].message.tool_calls, conversation_messages)\n", + "\n", + "def handle_visualization_agent(query, conversation_messages):\n", + " messages = [{\"role\": \"system\", \"content\": visualization_system_prompt}]\n", + " messages.append({\"role\": \"user\", \"content\": query})\n", + "\n", + " response = client.chat.completions.create(\n", + " model=MODEL,\n", + " messages=messages,\n", + " temperature=0,\n", + " tools=visualization_tools,\n", + " )\n", + "\n", + " conversation_messages.append([tool_call.function for tool_call in response.choices[0].message.tool_calls])\n", + " execute_tool(response.choices[0].message.tool_calls, conversation_messages)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we create the overarching tool to handle processing the user query.\n", + "\n", + "This function takes the user query, gets a response from the model and handles passing it to the other agents to execute. In addition to this, we will keep the state of the ongoing conversation." + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "id": "4skE5-KYI9Tw" + }, + "outputs": [], + "source": [ + "# Function to handle user input and triaging\n", + "def handle_user_message(user_query, conversation_messages=[]):\n", + " user_message = {\"role\": \"user\", \"content\": user_query}\n", + " conversation_messages.append(user_message)\n", + "\n", + "\n", + " messages = [{\"role\": \"system\", \"content\": triaging_system_prompt}]\n", + " messages.extend(conversation_messages)\n", + "\n", + " response = client.chat.completions.create(\n", + " model=MODEL,\n", + " messages=messages,\n", + " temperature=0,\n", + " tools=triage_tools,\n", + " )\n", + "\n", + " conversation_messages.append([tool_call.function for tool_call in response.choices[0].message.tool_calls])\n", + "\n", + " for tool_call in response.choices[0].message.tool_calls:\n", + " if tool_call.function.name == 'send_query_to_agents':\n", + " agents = json.loads(tool_call.function.arguments)['agents']\n", + " query = json.loads(tool_call.function.arguments)['query']\n", + " for agent in agents:\n", + " if agent == \"Data Processing Agent\":\n", + " handle_data_processing_agent(query, conversation_messages)\n", + " elif agent == \"Analysis Agent\":\n", + " handle_analysis_agent(query, conversation_messages)\n", + " elif agent == \"Visualization Agent\":\n", + " handle_visualization_agent(query, conversation_messages)\n", + "\n", + " return conversation_messages" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "jzQAwIW_WL3k" + }, + "source": [ + "## Multi-agent system execution\n", + "\n", + "Finally, we run the overarching `handle_user_message` function on the user query and view the output." + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "a0h10s_W49ct", + "outputId": "7e340af9-dc3d-44ba-aa0c-e613fbdcc153" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Cleaned data: house_size (m3) house_price ($)\n", + "0 90 100\n", + "1 80 90\n", + "2 100 120\n", + "Statistical Analysis: house_size house_price\n", + "count 4.000000 4.000000\n", + "mean 90.000000 102.500000\n", + "std 8.164966 12.583057\n", + "min 80.000000 90.000000\n", + "25% 87.500000 97.500000\n", + "50% 90.000000 100.000000\n", + "75% 92.500000 105.000000\n", + "max 100.000000 120.000000\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1IAAAIjCAYAAAAJLyrXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB5oUlEQVR4nO3deVhUBfvG8XvYQQXEVMBQcSmX3E1z31DRcknL3MqlNEsz9S0ry8Qt3zYzzWzPJbU9rV4zcTczt9Q0zVwwrXDJDRHBAc7vD39MjgOyyMwZ4Pu5Li6dZ86c88zDOHJzlrEYhmEIAAAAAJBjHmY3AAAAAAAFDUEKAAAAAHKJIAUAAAAAuUSQAgAAAIBcIkgBAAAAQC4RpAAAAAAglwhSAAAAAJBLBCkAAAAAyCWCFAAAAADkEkEKAFzgyJEjslgsmjt3rtmtXFdGn6+88orZreSb3My+MD7/vKpYsaIGDhzo1G2sXbtWFotFa9eudep2AMAZCFIAcIPmzp0ri8Wibdu2md3Kde3cuVP9+/dXRESEfH19FRISoqioKH344YdKS0szpadly5YpJiam0Gw3Ixhc/RUSEqI77rhDCxcuzPftXe2FF17QkiVLcrRsRmDM7OuOO+7I9DF79+5VTEyMjhw5kqNtxMTEyGKx6J9//snhMwCAgsXL7AYAoCioUKGCLl26JG9vb1O2/95772nYsGEqW7as7r//flWtWlUXLlzQqlWr9OCDDyo+Pl7jxo1zeV/Lli3T7NmznRqmMpu9s7c7cuRI3X777ZKk06dP65NPPlH//v117tw5DR8+3CnbfOGFF3TPPfeoe/fuOX5Mnz591LlzZ7ta6dKlJUn79++Xh8e/v2/du3evJk6cqNatW6tixYr50bJatmypS5cuycfHJ1/WBwCuRJACABewWCzy8/MzZds//fSThg0bpiZNmmjZsmUqUaKE7b5Ro0Zp27Zt2rNnj0t7unjxoooVK+aSbZkx+xYtWuiee+6x3X7kkUdUqVIlLVq0yGlBKi/q16+v/v37Z3qfr6+v07fv4eFh2r8LALhRHNoHAC6Q2Xk6AwcOVPHixfXXX3+pe/fuKl68uEqXLq0nnnjC4VC79PR0zZgxQzVr1pSfn5/Kli2rhx9+WGfPns122xMnTpTFYtHChQvtQlSGhg0bZnouzDvvvKPKlSvL19dXt99+u7Zu3Wp3/y+//KKBAweqUqVK8vPzU2hoqAYPHqzTp0/bLZdxiNfevXvVt29flSxZUs2bN9fAgQM1e/ZsSbI7tCwrY8aMUalSpWQYhq322GOPyWKxaObMmbbaiRMnZLFYNGfOHEmOs8/pdrN7/rnh4+OjkiVLysvL8feXH330kRo0aCB/f3+FhISod+/eOnbsmN0yBw4cUM+ePRUaGio/Pz/dfPPN6t27t86fP297HhcvXtS8efNsz+dGz2+6+hypuXPn6t5775UktWnTxraNGz23KbNzpFq3bq3bbrtNe/fuVZs2bRQQEKBy5crppZdecnh8SkqKJkyYoCpVqsjX11cREREaO3asUlJSbqgvAMgJ9kgBgInS0tLUsWNHNW7cWK+88opWrlypV199VZUrV9YjjzxiW+7hhx/W3LlzNWjQII0cOVJxcXF64403tGPHDm3cuDHLQwaTkpK0atUqtWzZUuXLl89xX4sWLdKFCxf08MMPy2Kx6KWXXlKPHj10+PBh27ZiY2N1+PBhDRo0SKGhofr111/1zjvv6Ndff9VPP/3kEE7uvfdeVa1aVS+88IIMw1C9evX0999/KzY2VgsWLMi2pxYtWui1117Tr7/+qttuu02StGHDBnl4eGjDhg0aOXKkrSZdOWwsMw8//HC2283J87+eCxcu2M4NOnPmjBYtWqQ9e/bo/ffft1tu6tSpGj9+vHr16qWHHnpIp06d0qxZs9SyZUvt2LFDwcHBunz5sjp27KiUlBQ99thjCg0N1V9//aVvv/1W586dU1BQkBYsWKCHHnpIjRo10tChQyVJlStXzrbPpKQkh3OYgoKCHJ5jy5YtNXLkSM2cOVPjxo1T9erVJcn2Z347e/asoqOj1aNHD/Xq1Uuff/65nnrqKdWqVUudOnWSdOWXC127dtUPP/ygoUOHqnr16tq9e7dee+01/f777zk+XwwA8swAANyQDz/80JBkbN26Nctl4uLiDEnGhx9+aKsNGDDAkGRMmjTJbtl69eoZDRo0sN3esGGDIclYuHCh3XLLly/PtH61Xbt2GZKMxx9/PEfPJaPPUqVKGWfOnLHVly5dakgyvvnmG1stKSnJ4fGLFy82JBnr16+31SZMmGBIMvr06eOw/PDhw42c/ld08uRJQ5Lx5ptvGoZhGOfOnTM8PDyMe++91yhbtqxtuZEjRxohISFGenq63XO6evZZbTc3zz8za9asMSQ5fHl4eBhTp061W/bIkSOGp6enQ3337t2Gl5eXrb5jxw5DkvHZZ59dd9vFihUzBgwYcN1lrn2emX2tWbPGMAzDqFChgt36PvvsM7v7s5PxfT916lSWy2TM6+p1tmrVypBkzJ8/31ZLSUkxQkNDjZ49e9pqCxYsMDw8PIwNGzbYrfOtt94yJBkbN27MUZ8AkFcc2gcAJhs2bJjd7RYtWujw4cO225999pmCgoLUvn17/fPPP7avBg0aqHjx4lqzZk2W605ISJCkTA/pu5777rtPJUuWtOtJkl1f/v7+tr8nJyfrn3/+sV3x7eeff3ZY57XPM7dKly6tatWqaf369ZKkjRs3ytPTU08++aROnDihAwcOSLqyR6p58+bXPUwwOzl5/tfz/PPPKzY2VrGxsfrkk0/Up08fPfvss3r99ddty3z55ZdKT09Xr1697L6voaGhqlq1qu37GhQUJEn6/vvvlZSUlOfnlJmhQ4fa+sz4qlOnTr5uIy+KFy9ud+6Wj4+PGjVq5PDvonr16qpWrZrd/Nq2bStJ1/13AQD5gUP7AMBEfn5+tqukZShZsqTduU8HDhzQ+fPnVaZMmUzXcfLkySzXHxgYKOnKoWa5ce1hgBmh4uq+zpw5o4kTJ+rjjz926CHj3J2rRUZG5qqHzLRo0ULLli2TdCUwNWzYUA0bNlRISIg2bNigsmXLateuXerbt+8NbScnz/96atWqpaioKNvtXr166fz583r66afVt29flS5dWgcOHJBhGKpatWqm68g4vC4yMlJjxozR9OnTtXDhQrVo0UJdu3ZV//79bSErr6pWrWrXp7u4+eabHYJwyZIl9csvv9huHzhwQPv27XP495Phev8uACA/EKQAwESenp7ZLpOenq4yZcpk+TlEWf0gKUlVqlSRl5eXdu/enS99GVdd6KFXr1768ccf9eSTT6pu3boqXry40tPTFR0drfT0dIfHXr0HK6+aN2+ud999V4cPH9aGDRvUokULWSwWNW/eXBs2bFB4eLjS09Nte5DyKifPP7fatWunb7/9Vlu2bNGdd96p9PR0WSwWfffdd5lur3jx4ra/v/rqqxo4cKCWLl2qFStWaOTIkZo2bZp++ukn3XzzzXnuyV3lZP7p6emqVauWpk+fnumyERERTukNADIQpADAzVWuXFkrV65Us2bNch1GAgIC1LZtW61evVrHjh3Ltx8uz549q1WrVmnixIl6/vnnbfWMw+tyKreH32UEpNjYWG3dulVPP/20pCsXQ5gzZ47Cw8NVrFgxNWjQIF+3mx9SU1MlSYmJiZKufF8Nw1BkZKRuueWWbB9fq1Yt1apVS88995x+/PFHNWvWTG+99ZamTJkiyfnPyYyZXU/lypW1a9cutWvXzu16A1A0cI4UALi5Xr16KS0tTZMnT3a4LzU1VefOnbvu4ydMmCDDMHT//ffbfoi/2vbt2zVv3rxc9ZSxx+DaPTQzZszI1XoyPksqu+eQITIyUuXKldNrr70mq9WqZs2aSboSsA4dOqTPP/9cd9xxR6aXGb+R7eaHb7/9VpJs5yD16NFDnp6emjhxosMcDcOwXUY+ISHBFsIy1KpVSx4eHnaX+S5WrJhTn48ZM7ueXr166a+//tK7777rcN+lS5d08eJFE7oCUJSwRwoA8skHH3yg5cuXO9Qff/zxG1pvq1at9PDDD2vatGnauXOnOnToIG9vbx04cECfffaZXn/9dbsPf71W06ZNNXv2bD366KOqVq2a7r//flWtWlUXLlzQ2rVr9fXXX9v2auRUYGCgWrZsqZdeeklWq1XlypXTihUrFBcXl6v1ZOw5GjlypDp27ChPT0/17t37uo9p0aKFPv74Y9WqVct27lL9+vVVrFgx/f777zk6Pyov282NDRs2KDk5WdKVc8m+/vprrVu3Tr1791a1atUkXdmjMmXKFD3zzDM6cuSIunfvrhIlSiguLk5fffWVhg4dqieeeEKrV6/WiBEjdO+99+qWW25RamqqFixYIE9PT/Xs2dPuOa1cuVLTp09XeHi4IiMj1bhx43x7TnXr1pWnp6defPFFnT9/Xr6+vmrbtm2W5+5lmD59ugICAuxqHh4eGjdu3A31c//99+vTTz/VsGHDtGbNGjVr1kxpaWn67bff9Omnn+r7779Xw4YNb2gbAHA9BCkAyCcZHwB7rRv9YFRJeuutt9SgQQO9/fbbGjdunLy8vFSxYkX179/ftlfmeh5++GHdfvvtevXVVzV//nydOnVKxYsXV/369fXhhx/aXSEtpxYtWqTHHntMs2fPlmEY6tChg7777juFh4fneB09evTQY489po8//lgfffSRDMPIcZBq3ry5rebl5aUmTZpo5cqVOTo/Ki/bzY2rPyDYx8dHlSpV0tSpU/Xkk0/aLff000/rlltu0WuvvaaJEydKunJuT4cOHdS1a1dJV/ZgdezYUd98843++usvBQQEqE6dOvruu+9sV0mUrgSWoUOH6rnnntOlS5c0YMCAfA1SoaGheuuttzRt2jQ9+OCDSktL05o1a7INUtOmTXOoeXp63nCQ8vDw0JIlS/Taa69p/vz5+uqrrxQQEKBKlSrp8ccfz9HhkgBwIyzGjZw5CwAAAABFEOdIAQAAAEAuEaQAAAAAIJcIUgAAAACQSwQpAAAAAMglghQAAAAA5BJBCgAAAAByic+RkpSenq6///5bJUqUkMViMbsdAAAAACYxDEMXLlxQeHi4PDyy3u9EkJL0999/KyIiwuw2AAAAALiJY8eO6eabb87yfoKUpBIlSki6MqzAwEBTe7FarVqxYoU6dOggb29vU3spjJivczFf52K+zsV8nYv5Oh8zdi7m61zuNN+EhARFRETYMkJWCFKS7XC+wMBAtwhSAQEBCgwMNP1FVBgxX+divs7FfJ2L+ToX83U+ZuxczNe53HG+2Z3yw8UmAAAAACCXCFIAAAAAkEsEKQAAAADIJc6RyiHDMJSamqq0tDSnbsdqtcrLy0vJyclO31ZRlF/z9fT0lJeXF5fLBwAAKKIIUjlw+fJlxcfHKykpyenbMgxDoaGhOnbsGD+kO0F+zjcgIEBhYWHy8fHJp+4AAABQUBCkspGenq64uDh5enoqPDxcPj4+Tg046enpSkxMVPHixa/7AWDIm/yYr2EYunz5sk6dOqW4uDhVrVqV7xUAAEARQ5DKxuXLl5Wenq6IiAgFBAQ4fXvp6em6fPmy/Pz8+OHcCfJrvv7+/vL29tYff/xhWx8AAACKDn5SzyFCDa7FawIAAKDo4idBAAAAAMglghQAAAAA5BJBykXS0g1tOnRaS3f+pU2HTist3TC7pQIpJiZGdevWNbsNAAAAFHEEKRdYvidezV9crT7v/qTHP96pPu/+pOYvrtbyPfFO2+bAgQNlsVhksVjk7e2tsmXLqn379vrggw+Unp6eq3XNnTtXwcHB+dJX69atbX35+fmpRo0aevPNN3P8+CeeeEKrVq3K1TYrVqyoGTNm5LJTAAAAIGsEKSdbvidej3z0s+LPJ9vVj59P1iMf/ezUMBUdHa34+HgdOXJE3333ndq0aaPHH39cd911l1JTU5223ewMGTJE8fHx2rt3r3r16qXhw4dr8eLFOXps8eLFVapUKSd3CAAAAFwfQcqJ0tINTfxmrzI7iC+jNvGbvU47zM/X11ehoaEqV66c6tevr3Hjxmnp0qX67rvvNHfuXNty06dPV61atVSsWDFFRETo0UcfVWJioiRp7dq1GjRokM6fP2/bkxQTEyNJWrBggRo2bKgSJUooNDRUffv21cmTJ7PtKyAgQKGhoapUqZJiYmJUtWpVff3115Kko0ePqlu3bipevLgCAwPVq1cvnThxwvbYaw/tGzhwoLp3765XXnlFYWFhKlWqlIYPHy6r1Srpyh6wP/74Q6NHj5bFYpGnp6ck6Y8//lCXLl1UsmRJFStWTDVr1tSyZctuZNwAAADIg7R0Q5vjzmj7PxZtjjtTYE6BMTVIrV+/Xl26dFF4eLgsFouWLFliu89qteqpp56y/YAfHh6uBx54QH///bfdOs6cOaN+/fopMDBQwcHBevDBB20hwGxb4s447Im6miEp/nyytsSdcVlPbdu2VZ06dfTll1/aah4eHpo5c6Z+/fVXzZs3T6tXr9bYsWMlSU2bNtWMGTMUGBio+Ph4xcfH64knnpB05Xs0efJk7dq1S0uWLNGRI0c0cODAXPfk7+9v+7yubt266cyZM1q3bp1iY2N1+PBh3Xfffdd9/Jo1a3To0CGtWbNG8+bN09y5c21B8csvv9TNN9+sSZMmKT4+Xn/99ZckacSIEUpJSdH69eu1e/duvfjiiypevHiuewcAAEDeZZwC0/+DbZp/wFP9P9jm9FNg8oupH8h78eJF1alTR4MHD1aPHj3s7ktKStLPP/+s8ePHq06dOjp79qwef/xxde3aVdu2bbMt169fP8XHxys2NlZWq1WDBg3S0KFDtWjRIlc/HQcnL2QdovKyXH6pVq2afvnlF9vtUaNG2f5esWJFTZkyRcOGDdObb74pHx8fBQUFyWKxKDQ01G49gwcPtv29UqVKmjlzpm6//XYlJibmKJSkpaVp8eLF+uWXXzR06FCtWrVKu3fvVlxcnCIiIiRJ8+fPV82aNbV161bdfvvtma6nZMmSeuONN+Tp6alq1arpzjvv1KpVqzRkyBCFhITI09PTttcsPT1dCQkJOnbsmHr27KlatWrZ+gcAAIDrZJwCc+3+p4xTYOb0r6/o28JM6S0nTA1SnTp1UqdOnTK9LygoSLGxsXa1N954Q40aNdLRo0dVvnx57du3T8uXL9fWrVvVsGFDSdKsWbPUuXNnvfLKKwoPD3f6c7ieMiX88nW5/GIYhiwWi+32ypUrNW3aNP32229KSEhQamqqkpOTlZSUpICAgCzXs337dsXExGjXrl06e/as7SIWR48eVY0aNbJ83Jtvvqn33ntPly9flqenp0aPHq1HHnlEb7zxhiIiImwhSpJq1Kih4OBg7du3L8sgVbNmTdshe5IUFham3bt3X3cGI0aM0PDhw7VixQpFRUWpZ8+eql279nUfAwAAgPyR3SkwFl05BaZ9jVB5elgyWcp8pgap3Mo4TyfjCnKbNm1ScHCwLURJUlRUlDw8PLR582bdfffdma4nJSVFKSkpttsJCQmSrhyqlnFuTQar1SrDMJSenp7rq901rBCs0EA/nUhIzvRFYpEUGuSnhhWCbes2DMP2Z263dzXDMLJcx759+1SxYkWlp6fryJEjuuuuuzRs2DBNnjxZISEh+uGHHzRkyBAlJyfLz8/Pto6r13Xx4kV17NhRHTp00IIFC1S6dGkdPXpUnTp1UnJy8nV779u3r8aNGyd/f3+FhYXJw8PD7rln9tiM+V+7jGEY8vLycnjMtd+vjFlkPP7BBx9Ux44d9b///U+xsbGaNm2aXnnlFY0YMSL74V61DcMwZLVa7YJcUZbx7+faf0fIH8zXuZivczFf52PGzsV889fmHJ4Cs+ngSTWODHFdY8r597jABKnk5GQ99dRT6tOnjwIDAyVJx48fV5kyZeyW8/LyUkhIiI4fP57luqZNm6aJEyc61FesWOGwB8bLy0uhoaFKTEzU5cuXc933k+0q6omvfpNFsgtTGbn6ibYVdTHxgsPjLlxwrOWG1WpVamqqLSRmyDgn6OGHH1ZCQoJ++OEHpaen6/nnn7cFmiNHjth68PDwUFpamtLS0uzWtXPnTp0+fVrjxo3TzTffLEnasGGDpCsh69rtZkhNTZW/v7/t+3b1+Wzly5fXsWPHtHfvXts6f/vtN507d04VKlRQQkKCUlJS7HrJ7HlevnzZrubl5eXQ04ULFxQUFKS+ffuqb9++mjhxot5++2098MADOZ7x5cuXdenSJa1fv97UqyC6o2v3JiN/MV/nYr7OxXydjxk7F/PNH9v/sUjK/hfRKzZs1ul9rr34RFJSUo6WKxBBymq1qlevXjIMQ3PmzLnh9T3zzDMaM2aM7XZCQoIiIiLUoUMHW0jLkJycrGPHjql48eLy88v9IXh33x4of39/Tfp2n44n/Ju6Q4P8NP7O6oq+zf68I8MwdOHCBZUoUcLu8Lvc8vb2VlpampKSkpSWlqYTJ07o+++/13//+1/deeedGjp0qDw9PVWrVi1ZrVbNnz9fd911lzZu3Gi7UEOJEiUUGBio6tWrKzExUVu3blWdOnUUEBCg6tWry8fHR/PmzdPDDz+sPXv2aPr06ZKkYsWKOcwxg5eXl3x8fDK9v2vXrqpVq5YeffRRTZ8+XampqRoxYoRatWqlVq1aSbpyJUJPT0/b4729veXl5WW3Ph8fH7taZGSktmzZogsXLsjHx0e+vr6aMGGCoqOjdcstt+js2bPatGmTatasmWXfmUlOTpa/v79atmyZp9dGYWS1WhUbG6v27dvL29vb7HYKHebrXMzXuZiv8zFj52K++atU3BnNP7At2+U6tGjs8j1SWe0QuJbbB6mMEPXHH39o9erVdj/ohoaGOlxuOzU1VWfOnHG4MMLVfH195evr61D39vZ2+IeRlpYmi8UiDw8P2x6b3OpcO1wdbwvTlrgzOnkhWWVK+KlRZEimx3tmHI6Wsc28slgs+v7771WuXDl5eXmpZMmSqlOnjmbOnKkBAwbY1l2vXj1Nnz5dL730ksaNG6eWLVtq2rRpeuCBB2zPuXnz5ho2bJj69Omj06dPa8KECYqJidHcuXM1btw4zZo1S/Xr19crr7yirl27Zjur6z23pUuX6rHHHlPr1q3l4eGh6OhozZo1y7Z8Rri8+va167t2mcmTJ+vhhx9W1apVlZKSorNnzyotLU2PPfaY/vzzTwUGBio6OlqvvfZarmbu4eFh+8Bj3lDtMRPnYr7OxXydi/k6HzN2LuabP5pUKaOwID8dP3/9U2CaVCnj8nOkcvr9tRgZJ42YzGKx6KuvvlL37t1ttYwQdeDAAa1Zs0alS5e2e8y+fftUo0YNbdu2TQ0aNJB05fC86Oho/fnnnzm+2ERCQoKCgoJ0/vz5TPdIxcXFKTIy0iV7HTKuKhcYGHhDQQqZy8/5uvq1URBYrVYtW7ZMnTt35j8ZJ2C+zsV8nYv5Oh8zdi7mm/8yrtonZX4KjFlX7bteNriaqT+pJyYmaufOndq5c6ckKS4uTjt37tTRo0dltVp1zz33aNu2bVq4cKHS0tJ0/PhxHT9+3HauUvXq1RUdHa0hQ4Zoy5Yt2rhxo0aMGKHevXubfsU+AAAAAFmLvi1Mc/rXV2iQ/S+kQ4P83P7S55LJh/Zt27ZNbdq0sd3OOG9pwIABiomJ0ddffy1Jqlu3rt3j1qxZo9atW0uSFi5cqBEjRqhdu3by8PBQz549NXPmTJf0DwAAACDvom8LU/saodp08KRWbNisDi0am3I4X16YGqRat26t6x1ZmJOjDkNCQtziw3cBAAAA5J6nh0WNI0N0ep+hxllcR8AdcRIOAAAAAOQSQQoAAAAAcokgBQAAAAC5RJACAAAAgFwiSAEAAABALhGkAAAAACCXCFIw1cCBA9W9e/d8XeeRI0dksVhsH/QMAAAA5DeCVCE1cOBAWSwW21epUqUUHR2tX375Jd+2ERMT4/BhyVktd3UvGV8rV67U66+/rrlz59qWbd26tUaNGpXtOq+3XEREhOLj43Xbbbfl7IkAAAAAuUSQKsSio6MVHx+v+Ph4rVq1Sl5eXrrrrrtM6aVmzZq2XjK+WrZsqaCgIAUHB+frtjw9PRUaGiovL1M/bxoAAACFGEEqtwxDunzRuV/WpMzrhpGrVn19fRUaGqrQ0FDVrVtXTz/9tI4dO6ZTp07Zljl27Jh69eql4OBghYSEqFu3bjpy5Ijt/rVr16pRo0YqVqyYgoOD1axZM/3xxx+aO3euJk6cqF27dtn2MF29Z+laXl5etl4yvnx8fOwO7Rs4cKDWrVun119/3bbOq3vJqWsP7Vu7dq0sFotWrVqlRo0aKTw8XM2bN9f+/fvtHrd06VLVr19ffn5+qlSpkiZOnKjU1NRcbx8AAACFH7+yzy1rkvRCuNNW7yEpOKs7x/0t+RTL03oTExP10UcfqUqVKipVqpQkyWq1qmPHjmrSpIk2bNggLy8vTZkyxXYIoIeHh7p3764hQ4Zo8eLFunz5srZs2SKLxaL77rtPe/bs0fLly7Vy5UpJUlBQUJ56y/D666/r999/12233aZJkyZJkkqXLn1D67zas88+q5dffln+/v4aO3asBg8erI0bN0qSNmzYoAceeEAzZ85UixYtdOjQIQ0dOlSSNGHChHzrAQAAAIUDQaoQ+/bbb1W8eHFJ0sWLFxUWFqZvv/1WHh5XdkR+8sknSk9P13vvvSeLxSJJ+vDDDxUcHKy1a9eqYcOGOn/+vO666y5VrlxZklS9enXb+osXL27b05Sd3bt323qRpBo1amjLli12ywQFBcnHx0cBAQE5WmduTZ06Va1atVJCQoLGjh2rLl26KDk5WX5+fpo4caKefvppDRgwQJJUqVIlTZ48WWPHjiVIAQAAwAFBKre8A67sGXKS9PR0JVy4oMASJWyBx27budCmTRvNmTNHknT27Fm9+eab6tSpk7Zs2aIKFSpo165dOnjwoEqUKGH3uOTkZB06dEgdOnTQwIED1bFjR7Vv315RUVHq1auXwsLCcv28br31Vn399de2276+vrlex42qXbu27e8Zz+HkyZMqX768du3apY0bN2rq1Km2ZdLS0pScnKykpCQFBORu9gAAACjcCFK5ZbHk+fC6HElPl7zTrmzj2iCVS8WKFVOVKlVst9977z0FBQXp3Xff1ZQpU5SYmKgGDRpo4cKFDo/NOKTuww8/1MiRI7V8+XJ98skneu655xQbG6s77rgjV734+PjY9WIGb29v298z9sClp6dLunLo48SJE9WjRw+Hx/n5+bmmQQAAABQYBKkixGKxyMPDQ5cuXZIk1a9fX5988onKlCmjwMDALB9Xr1491atXT88884yaNGmiRYsW6Y477pCPj4/S0tLytUdnrDMn6tevr/3795se9gAAAFAwEKQKsZSUFB0/flzSlUP73njjDSUmJqpLly6SpH79+unll19Wt27dNGnSJN188836448/9OWXX2rs2LGyWq1655131LVrV4WHh2v//v06cOCAHnjgAUlSxYoVFRcXp507d+rmm29WiRIlbviQvYoVK2rz5s06cuSIihcvrpCQEMdDHP/fqVOnHD50Ny+HHUrS888/r7vuukvly5fXPffcIw8PD+3atUt79uzRlClT8rROAAAAFF5c/rwQW758ucLCwhQWFqbGjRtr69at+uyzz9S6dWtJUkBAgNavX6/y5curR48eql69uh588EElJycrMDBQAQEB+u2339SzZ0/dcsstGjp0qIYPH66HH35YktSzZ09FR0erTZs2Kl26tBYvXnzDPT/xxBPy9PRUjRo1VLp0aR09ejTLZRctWmTbW5bx9e677+Zpux07dtS3336rFStW6Pbbb9cdd9yh1157TRUqVMjrUwEAAEAhxh6pQmru3LnX/VynDKGhoZo3b16m9wUGBuqrr77K8rG+vr76/PPPs91GTEyMYmJisuzzarfccos2bdqU7TrXrl173fuNqz5zq3Xr1rbbGedE1a1b124Z6UqY6tixY7bbBgAAANgjBQAAAAC5RJACAAAAgFwiSAEAAABALhGkAAAAACCXCFI5dO2FCQBeEwAAAEUXQSob3t7ekqSkpCSTO4G7yXhNZLxGAAAAkEfJCZKRbnYXucLlz7Ph6emp4OBgnTx5UtKVz16yWCxO2156erouX76s5OTkLD+IFnmXH/M1DENJSUk6efKkgoOD5enpmc9dAgAAFBFx66V5XeQtqUaZOyXdZXZHOUaQyoHQ0FBJsoUpZzIMQ5cuXZK/v79TA1tRlZ/zDQ4Otr02AAAAkAtnDksz69mVUrxKmNRM3hCkcsBisSgsLExlypSR1Wp16rasVqvWr1+vli1bcsiYE+TXfL29vdkTBQAAkFvJCdLsRtKFeLty6l0zdeivYN1qUlt5QZDKBU9PT6f/8Ozp6anU1FT5+fkRpJyA+QIAAJggPU36pL+0f5l9/Y5HpY4vyEhNlf5alvlj3RRBCgAAAIDzrH1RWvuCfa1CM+mBpZJnwf3FNkEKAAAAQP7bu1T69AH7mneANGqPVKyUOT3lI4IUAAAAgPwT/4v0dgvH+iObpLI1XN+PkxCkAAAAANy4xJPSq7c6fh5U78VStc7m9OREBCkAAAAAeZeaIn3YSfpru3293QSpxRhzenIBghQAAACA3DMM6X9jpG0f2Ndr3i31/EDy8DCnLxchSAEAAADInW0fSt+Osq+VrCgN2yj5FjejI5cjSAEAAADImSM/SHPvdKw//otUsoLr+zERQQoAAADA9Z09Ir1ex7E+6DupQlOXt+MOCFIAAAAAMpdyQZp9h5Twp3296xtS/fvN6clNEKQAAAAA2EtPlz69X/rtW/t642FS9H8li8WcvtwIQQoAAADAv9a9LK2ZYl+LaCwN+Fby8jGnJzdEkAIAAAAg7ftW+qSffc3TRxqzTyp2kzk9uTGCFAAAAFCUHd8jvdXMsT5soxR6m+v7KSAIUgAAAEBRlHhKml5NSk+1r/deJFXL5BLnsEOQAgAAAIqS1MvS3M7Sn1vt622fk1o+aU5PBRBBCgAAACgKDEP6bqy05R37evUu0r3zJQ8Pc/oqoAhSAAAAQGH383zp68fsa0HlpUd/lHxLmNNTAUeQAgAAAAqrIxuvHMZ3rcd3SSUrurydwoQgBQAAABQ2Z/+QXq/tWB+4TKqYyRX6kGsEKQAAAKCwSEmU5jSRzh21r3d5XWow0JSWCiuCFAAAAFDQpadLnw+U9i61r9/+kNT5FcliMaWtwowgBQAAABRkG6ZLqyba18o1lAYtk7x8zempCCBIAQAAAAXRb8ukj/vY1zy8pDG/ScVLm9NTEUKQAgAAAAqSE79Kc5o61of9IIXWcn0/RZSpn7q1fv16denSReHh4bJYLFqyZInd/V9++aU6dOigUqVKyWKxaOfOnQ7raN26tSwWi93XsGHDXPMEAAAAAFe5eFqaUtYxRPVaIMWcJ0S5mKlB6uLFi6pTp45mz56d5f3NmzfXiy++eN31DBkyRPHx8bavl156yRntAgAAAK6Xeln6IFp6uZKUmvxvvc2zVwJUja7m9VaEmXpoX6dOndSpU6cs77///vslSUeOHLnuegICAhQaGpqfrQEAAADmMgxp+dPS5rfs69XuknrNlzw8zekLkgrJOVILFy7URx99pNDQUHXp0kXjx49XQEBAlsunpKQoJSXFdjshIUGSZLVaZbVand7v9WRs3+w+Civm61zM17mYr3MxX+divs7HjJ3L1fO17Fokr29H2tWMEuFKfXij5FtCSku/8lVIuNPrN6c9WAzDMJzcS45YLBZ99dVX6t69u8N9R44cUWRkpHbs2KG6deva3ffOO++oQoUKCg8P1y+//KKnnnpKjRo10pdffpnltmJiYjRx4kSH+qJFi64bwAAAAABnCkn8XS0OTHGox9Z4RUm+ZUzoqOhJSkpS3759df78eQUGBma5XIEPUtdavXq12rVrp4MHD6py5cqZLpPZHqmIiAj9888/1x2WK1itVsXGxqp9+/by9vY2tZfCiPk6F/N1LubrXMzXuZiv8zFj53L6fM//Ke836jqUU/t9JaNii/zfnptxp9dvQkKCbrrppmyDVKE4tO9qjRs3lqTrBilfX1/5+jp+OJm3t7fp37gM7tRLYcR8nYv5OhfzdS7m61zM1/mYsXPl+3xTEqW3mklnj9jX73pNaji48P2wng13eP3mdPuF7nuTcYn0sLAwcxsBAAAAspKeLn0xWPr1K/t6w8HSndMli8WcvpBjpgapxMREHTx40HY7Li5OO3fuVEhIiMqXL68zZ87o6NGj+vvvvyVJ+/fvlySFhoYqNDRUhw4d0qJFi9S5c2eVKlVKv/zyi0aPHq2WLVuqdu3apjwnAAAA4Lp+mCGtnGBfC68nDf5e8nI8agruydQgtW3bNrVp08Z2e8yYMZKkAQMGaO7cufr66681aNAg2/29e/eWJE2YMEExMTHy8fHRypUrNWPGDF28eFERERHq2bOnnnvuOdc+EQAAACA7+5dLi+9zrP/nd6lEWdf3gxtiapBq3bq1rneti4EDB2rgwIFZ3h8REaF169Y5oTMAAAAgn5zcJ715h2P94fVSWB3X94N8UejOkQIAAADcwsXT0ozbJGuSff3eeVLN7qa0hPxDkAIAAADyU5pVmt9N+mOjfb3V01KbZ8zpCfmOIAUAAADkl++flTa9YV+7JVrqvUjy8DSnJzgFQQoAAAC4UTsXSUsesa8VLyuN2Cr5BZnTE5yKIAUAAADk1bEt0vvtHeuP/SyVquz6fuAyBCkAAAAgt87/Kb1W07H+wNdSpVau7wcuR5ACAAAAcuryRemtdtLpg/b1zq9IjYaY0xNMQZACAAAAsmOkq0Hcm/J++QH7ev0HpC4zJYvFnL5gGoIUAAAAcD0bZ8o7drxuvroWWlt6MFby9jOrK5iMIAUAAABk5vcV0qJ7Hev/2S+VCHV9P3ArBCkAAADgaqf2S7MbOZTX3jpRze4ZLm9vbxOagrshSAEAAACSlHRGmlFLupxoX7/nQ1lv7aLzy5aZ0xfcEkEKAAAARVuaVVpwt3Rkg3295ZNS2+eu/N1qdX1fcGsEKQAAABRdK56TfpxlX6vaQerzseThaU5PKBAIUgAAACh6dn0sffWwfa1YaWnENsk/2JSWULAQpAAAAFB0HNsqvR/lWH/sZ6lUZdf3gwKLIAUAAIDC7/xf0ms1HOv3L5Eqt3F5Oyj4CFIAAAAovC4nSe+0kv753b7e6SWp8cOZPwbIAYIUAAAACh/DuHIO1C+f2Nfr9Ze6viFZLOb0hUKDIAUAAIDC5cc3pBXP2tfK3iY9tFLy9jenJxQ6BCkAAAAUDgdWSgt7OtbH/CYFhrm+HxRqBCkAAAAUbKd+l2bf7lgfskYqV9/1/aBIIEgBAACgYLp0Vnq9jpR83r7e832p1j3m9IQigyAFAACAgiUtVfqohxS3zr7efIwUNcGcnlDkEKQAAABQcMQ+L2183b5WuZ3U91PJkx9t4Tq82gAAAOD+fvlU+nKIfS2glPTYz5J/sCktoWgjSAEAAMB9/bldeq+tY33ENummqq7vB/h/BCkAAAC4n4S/penVHev9v5SqtHN9P8A1CFIAAABwH9ZL0jutpVO/2dejX5TuGGZKS0BmCFIAAAAwn2FISx6Rdi22r9ftJ3WbLVks5vQFZIEgBQAAAHP9NEda/rR9rUwNachqydvfnJ6AbBCkAAAAYI6Dq658HtS1xuyTAsNd3w+QCwQpAAAAuNY/B6Q3GjrWH1ot3dzA9f0AeUCQAgAAgGtcOivNrHflz6v1eE+qfa85PQF5RJACAACAc6WlSovulQ6ttq83GyW1n2hKS8CNIkgBAADAeVZNkja8al+r1Frq94XkyY+iKLh49QIAACD/7f5c+uJB+5pfsPT4Tsm/pBkdAfmKIAUAAID889d26d22jvXhW6XSt7i+H8BJCFIAAAC4cQnx0vRqjvV+X0hVo1zfD+BkBCkAAADknfWS9F6UdGKPfb3DVKnpCHN6AlyAIAUAAIDcMwzp6xHSjo/s67Xvk+5+W7JYzOkLcBGCFAAAAHJn89vSd2PtazfdKg1dK/kEmNIS4GoEKQAAAOTMoTXSgu6O9dF7paByLm8HMBNBCgAAANd3+pA0q75j/cGVUsTtru8HcAMEKQAAAGQu+bw0q4F08ZR9/e63pTq9zekJcBMEKQAAANhLS5UW95YOxtrXmz4mdZhiTk+AmyFIAQAA4F+rp0jrX7avRbaU+n8peXqb0xPghghSAAAAkPZ8KX0+yL7mGyg9vksKCDGnJ8CNEaQAAACKsr93Su+0cqw/ulkqU83l7QAFBUEKAACgKLpwQnr1Fsd638+kWzq4vh+ggCFIAQAAFCXWZOn9KOn4bvt6+8lSs5Hm9AQUQAQpAACAosAwpG9GSj/Pt6/Xule6+x3Jw8OcvoACiiAFAABQ2G15V1r2hH2tVBXp4fWSTzFzegIKOIIUAABAYXV4nTS/q2N91B4pOML1/QCFiKn7cNevX68uXbooPDxcFotFS5Yssbv/yy+/VIcOHVSqVClZLBbt3LnTYR3JyckaPny4SpUqpeLFi6tnz546ceKEa54AAACAOzp9SIoJcgxRg1dIMecJUUA+MDVIXbx4UXXq1NHs2bOzvL958+Z68cUXs1zH6NGj9c033+izzz7TunXr9Pfff6tHjx7OahkAAMBteaUlyWtGDWlWffs7us+5EqDKNzanMaAQMvXQvk6dOqlTp05Z3n///fdLko4cOZLp/efPn9f777+vRYsWqW3btpKkDz/8UNWrV9dPP/2kO+64I997BgAAcDvpafL8pK/uPLjCvt5khNRxqjk9AYVcgT5Havv27bJarYqKirLVqlWrpvLly2vTpk1ZBqmUlBSlpKTYbickJEiSrFarrFarc5vORsb2ze6jsGK+zsV8nYv5OhfzdS7m6zwe61+U54aX7Q4zSi/fRGl9v5Q8vSVmni94DTuXO803pz0U6CB1/Phx+fj4KDg42K5etmxZHT9+PMvHTZs2TRMnTnSor1ixQgEBAfndZp7Exsaa3UKhxnydi/k6F/N1LubrXMw3/4Sd26pGcbPsaqkevoqtOV2XvUpI3zNrZ+A17FzuMN+kpKQcLVegg1RePfPMMxozZoztdkJCgiIiItShQwcFBgaa2NmVBBwbG6v27dvL29vb1F4KI+brXMzXuZivczFf52K++ej4L/J+v61D+dLgtVqx4ygzdhJew87lTvPNOFotOwU6SIWGhury5cs6d+6c3V6pEydOKDQ0NMvH+fr6ytfX16Hu7e1t+jcugzv1UhgxX+divs7FfJ2L+ToX870BiSelV6o61vt8It0aLS+rVdpxlBk7GfN1LneYb063X6A/wrpBgwby9vbWqlWrbLX9+/fr6NGjatKkiYmdAQAA5JPUFOmd1o4hKirmypX4bo02oyugyDN1j1RiYqIOHjxoux0XF6edO3cqJCRE5cuX15kzZ3T06FH9/fffkq6EJOnKnqjQ0FAFBQXpwQcf1JgxYxQSEqLAwEA99thjatKkCVfsAwAABZthSP8bI237wL5e826p5weSR4H+fThQ4JkapLZt26Y2bdrYbmectzRgwADNnTtXX3/9tQYNGmS7v3fv3pKkCRMmKCYmRpL02muvycPDQz179lRKSoo6duyoN99803VPAgAAIL9tff9KiLpayUhp2A+Sb3FzegJgx9Qg1bp1axmGkeX9AwcO1MCBA6+7Dj8/P82ePTvLD/UFAAAoMOI2SPPucqyP2i0Fl3d9PwCyVKAvNgEAAFAonImTZtZ1rA9aLlXgvG/AHRGkAAAAzJKcIL15h5Twl3296xtS/fvN6QlAjhCkAAAAXC09Tfqkv7R/mX298SNS9DTJYjGnLwA5RpACAABwpbUvSmtfsK9F3CEN+Eby8jGnJwC5RpACAABwhb1fS59ec7iel580+lep2E3m9AQgzwhSAAAAznR8t/RWc8f6Iz9KZWu6vh8A+YIgBQAA4AyJp6RXb5WMNPt678VStc7m9AQg3xCkAAAA8lNqivRhJ+mv7fb1tuOllk+Y0xOAfEeQAgAAyA+GIS17Qtr6nn29RjfpnrmSh4cpbQFwDoIUAADAjdo+V/rmcftacPkr50H5ljClJQDORZACAADIqyM/SHPvdKw//otUsoLr+wHgMgQpAACA3Dp7RHq9jmN90HdShaYubweA6xGkAAAAcirlgvRmE+n8Mft6l5lSgwHm9ATAFAQpAACA7KSnS589IO37xr7eaKjU6SXJYjGnLwCmIUgBAABcz/qXpdVT7Gs33y4N/J/k5WtOTwBMR5ACAADIzL5vpU/62dc8vKUx+6Tipc3pCYDbIEgBAABc7fge6a1mjvVhG6XQ21zfDwC3RJACAACQpIv/SNOrS2mX7ev3fSRV72JOTwDcFkEKAAAUbamXpbmdpT+32tfbPCe1etKcngC4PYIUAAAomgxD+m6stOUd+3q1u6Re8yUPT3P6AlAgEKQAAEDR8/N86evH7GtBEdKjmyTfEub0BKBAIUgBAICi448fpQ87OdZH7pRCIl3eDoCCiyAFAAAKv7N/SK/XdqwP/J9Usbnr+wFQ4BGkAABA4ZWSKM1pIp07al+/a4bUcJApLQEoHAhSAACg8ElPlz4fKO1dal+//SGp8yuSxWJKWwAKD4IUAAAoXDZMl1ZNtK+VayAN+k7y8jWnJwCFDkEKAAAUDr8tkz7uY1+zeEj/2S8VL2NOTwAKLYIUAAAo2E7svXIe1LUe3iCFZXKBCQDIBwQpAABQMF08Lb1WQ0pNtq/3WiDV6GpOTwCKDIIUAAAoWFIvS/O7Skc32ddbj5NaP2VOTwCKHIIUAAAoGAxDWv60tPkt+/qtnaX7PpI8PM3pC0CRRJACAADub8dH0tLh9rUSYdLwLZJfoDk9ASjSCFIAAMB9Hf1J+qCjY33kDimkkuv7AYD/R5ACAADu59wxacZtjvUB30iRLV3fDwBcgyAFAADcR0qi9FZz6Wycff3OV6XbHzKnJwDIBEEKAACYLz1d+mKw9OtX9vUGA6W7ZkgWixldAUCW8hykzp07p88//1yHDh3Sk08+qZCQEP38888qW7asypUrl589AgCAwuyHGdLKCfa18HrSoOWSt58pLQFAdvIUpH755RdFRUUpKChIR44c0ZAhQxQSEqIvv/xSR48e1fz58/O7TwAAUNj8/r20qJdj/T+/SyXKur4fAMgFj7w8aMyYMRo4cKAOHDggP79/f1PUuXNnrV+/Pt+aAwAAhdDJfVJMkGOIGrpOijlPiAJQIORpj9TWrVv19ttvO9TLlSun48eP33BTAACg8PFOvSCvl8pL1iT7O+6dK9W825SeACCv8hSkfH19lZCQ4FD//fffVbp06RtuCgAAFCJpVnku6KLORzfZ11s9JbUZZ05PAHCD8nRoX9euXTVp0iRZrVZJksVi0dGjR/XUU0+pZ8+e+dogAAAowL5/Vpp8kzyuDlG3REvPnyFEASjQ8hSkXn31VSUmJqpMmTK6dOmSWrVqpSpVqqhEiRKaOnVqfvcIAAAKmp2Lr5wHtekNWynZK0jW/xyW+n4ieXia2BwA3Lg8HdoXFBSk2NhYbdy4Ubt27VJiYqLq16+vqKio/O4PAAAUJMe2SO+3dyhbH9mi73/6TZ39Ak1oCgDy3w19IG+zZs3UrFmz/OoFAAAUVOf/lF6r6Vh/YKlUqbVktUr6zdVdAYDT5OnQvpEjR2rmzJkO9TfeeEOjRo260Z4AAEBBcfmiNKuBY4jq/MqVS5lXam1KWwDgbHkKUl988UWme6KaNm2qzz///IabAgAAbs4wpC8ekl4Il04f/Lde/wFpwjmp0RDTWgMAV8jToX2nT59WUFCQQz0wMFD//PPPDTcFAADc2I+zpBXP2ddCa0kPrpS8/czpCQBcLE97pKpUqaLly5c71L/77jtVqlTphpsCAABu6PcVV67Ed22I+s9+adgPhCgARUqe9kiNGTNGI0aM0KlTp9S2bVtJ0qpVq/Tqq69qxowZ+dkfAAAw26n90uxGjvWha6Xwei5vBwDcQZ6C1ODBg5WSkqKpU6dq8uTJkqSKFStqzpw5euCBB/K1QQAAYJKkM9LrdaSUBPv6PR9It/U0pycAcBN5vvz5I488okceeUSnTp2Sv7+/ihcvnp99AQAAs6RZpY96SHHr7estnpDajTenJwBwMzf0OVKSVLp06fzoAwAAuIMV46Ufr/mIkyrtpT4fS543/GMDABQaOX5HrF+/vlatWqWSJUuqXr16slgsWS77888/50tzAADARXZ9In011L4WcJP02HbJP9iUlgDAneU4SHXr1k2+vr6SpO7du+fLxtevX6+XX35Z27dvV3x8vL766iu7dRuGoQkTJujdd9/VuXPn1KxZM82ZM0dVq1a1LVOxYkX98ccfduudNm2ann766XzpEQCAQu3PbdJ77RzrI7ZLN1VxfT8AUEDkOEhNmDBBkpSWlqY2bdqodu3aCg4OvqGNX7x4UXXq1NHgwYPVo0cPh/tfeuklzZw5U/PmzVNkZKTGjx+vjh07au/evfLz+/cSq5MmTdKQIf9+8F+JEiVuqC8AAAq9839Jr9VwrN//lVS5rev7AYACJtcHO3t6eqpDhw7at2/fDQepTp06qVOnTpneZxiGZsyYoeeee07dunWTJM2fP19ly5bVkiVL1Lt3b9uyJUqUUGhoaI63m5KSopSUFNvthIQrVyOyWq2yWq15eSr5JmP7ZvdRWDFf52K+zsV8navIzNeaJK8PomT553e7clqHaUq//f9/KemEGRSZ+ZqIGTsX83Uud5pvTnuwGIZh5HblDRs21Isvvqh27TI5FCCPLBaL3aF9hw8fVuXKlbVjxw7VrVvXtlyrVq1Ut25dvf7665KuHNqXnJwsq9Wq8uXLq2/fvho9erS8vLLOiDExMZo4caJDfdGiRQoICMi35wQAgNswDNX/421FnP3RrvxHSAvtLP+QdJ1znwGgKElKSlLfvn11/vx5BQYGZrlcni6/M2XKFD3xxBOaPHmyGjRooGLFitndf70N5tTx48clSWXLlrWrly1b1nafJI0cOVL169dXSEiIfvzxRz3zzDOKj4/X9OnTs1z3M888ozFjxthuJyQkKCIiQh06dMiX3m+E1WpVbGys2rdvL29vb1N7KYyYr3MxX+divs5VmOfrsXmOPFfaX7bcKFNDqQO/V7i3v8Jd0ENhnq+7YMbOxXydy53mm3G0WnbyFKQ6d+4sSeratavd1fsMw5DFYlFaWlpeVpsnVwei2rVry8fHRw8//LCmTZtmuzjGtXx9fTO9z9vb2/RvXAZ36qUwYr7OxXydi/k6V6Ga78GV0keZfHDumN9kCQyTGc+yUM3XTTFj52K+zuUO883p9vMUpNasWZOXh+VKxjlPJ06cUFhYmK1+4sQJu0P9rtW4cWOlpqbqyJEjuvXWW53dJgAA7uefA9IbDR3rQ1ZL5Rq4vh8AKITyFKRatWqV3304iIyMVGhoqFatWmULTgkJCdq8ebMeeeSRLB+3c+dOeXh4qEyZMk7vEQAAt3LprPR6XSn5nH295/tSrXvM6AgACq08f0T52bNn9f7772vfvn2SpBo1amjQoEEKCQnJ8ToSExN18OBB2+24uDjt3LlTISEhKl++vEaNGqUpU6aoatWqtsufh4eH2y5IsWnTJm3evFlt2rRRiRIltGnTJo0ePVr9+/dXyZIl8/rUAAAoWNJSpYU9pcNr7evNR0tRMWZ0BACFXp6C1Pr169WlSxcFBQWpYcMrhw7MnDlTkyZN0jfffKOWLVvmaD3btm1TmzZtbLczzncaMGCA5s6dq7Fjx+rixYsaOnSozp07p+bNm2v58uW2z5Dy9fXVxx9/rJiYGKWkpCgyMlKjR4+2O28KAIBCbWWM9MNr9rVKbaR+n0ueef59KQAgG3l6hx0+fLjuu+8+zZkzR56enpKufFDvo48+quHDh2v37t05Wk/r1q11vauvWywWTZo0SZMmTcr0/vr16+unn37K/RMAAKCg2/259MWD9jX/ktLIHVf+BAA4VZ6C1MGDB/X555/bQpR05YN6x4wZo/nz5+dbcwAA4Bp/bpfea+tYH7FNuqmq6/sBgCIqT0Gqfv362rdvn8NV8fbt26c6derkS2MAAOAqCX9L06s71vt/IVWJcn0/AFDE5SlIjRw5Uo8//rgOHjyoO+64Q5L0008/afbs2frvf/+rX375xbZs7dq186dTAACKIusl6d220sm99vWO06Qmj5rTEwAgb0GqT58+kqSxY8dmep/FYjHlw3kBACg0DENaOlzaudC+XqeP1H2OZLGY0xcAQFIeg1RcXFx+9wEAADJsflv67ppfVpaufuUDdX0CzOkJAGAnT0GqQoUKOVruzjvv1HvvvaewsLC8bAYAgKLl4Crpox6O9dF7paByru8HAJAlp37AxPr163Xp0iVnbgIAgILvn4PSGw0c6w+tkm5u6Pp+AADZ4pP6AAAwy6Vz0qz6UtJp+/rd70h17jOlJQBAzhCkAABwtbRUafF90sGV9vVmj0vtM/8QegCAeyFIAQDgSqsmSRteta9FtpL6fyl58t8yABQUvGMDAOAKuz+XvnjQvuYbJD2+UwoIMaUlAEDeEaQAAHCmv36W3m3jWB++RSp9q+v7AQDkC6cGqXHjxikkhN+yAQCKoAvHpVczCUr9Ppeqtnd9PwCAfOWR1wcuWLBAzZo1U3h4uP744w9J0owZM7R06VLbMs8884yCg4NvuEkAAAoMa7I0p7ljiOowVYo5T4gCgEIiT0Fqzpw5GjNmjDp37qxz584pLS1NkhQcHKwZM2bkZ38AABQMhiEtHS5NLSud2P1vvVYvacI5qekI01oDAOS/PAWpWbNm6d1339Wzzz4rT09PW71hw4bavXv3dR4JAEAhtPkdaWKwtOOjf2s33SKNi5d6vitZLKa1BgBwjjydIxUXF6d69eo51H19fXXx4sUbbgoAgALh8FppfjfH+uhfpaCbXd4OAMB18hSkIiMjtXPnTlWoUMGuvnz5clWvXj1fGgMAwG2dPiTNqu9YfzBWimjk+n4AAC6XpyA1ZswYDR8+XMnJyTIMQ1u2bNHixYs1bdo0vffee/ndIwAA7iH5vDSrgXTxlH29+1tS3T7m9AQAMEWegtRDDz0kf39/Pffcc0pKSlLfvn0VHh6u119/Xb17987vHgEAMFd6mrS4t3RghX29yQip41RzegIAmCrPnyPVr18/9evXT0lJSUpMTFSZMmXysy8AANzD6qnS+pfsaxVbSPd/JXl6m9MTAMB0eQpSly5dkmEYCggIUEBAgE6dOqUZM2aoRo0a6tChQ373CACA6+35Uvp8kH3Np7g0arcUwIfNA0BRl6cg1a1bN/Xo0UPDhg3TuXPn1KhRI/n4+Oiff/7R9OnT9cgjj+R3nwAAuEb8LumDdo71RzdLZaq5vh8AgFvK0+dI/fzzz2rRooUk6fPPP1doaKj++OMPzZ8/XzNnzszXBgEAcIkzh9VtxwPyvjZE9f1UijlPiAIA2MnTHqmkpCSVKFFCkrRixQr16NFDHh4euuOOO/THH3/ka4MAADiVNVn6oIO843fZ19tPkpo9bk5PAAC3l6c9UlWqVNGSJUt07Ngxff/997bzok6ePKnAwMB8bRAAAKcwDOnrkdLUslcO58soewdIz58lRAEAritPQer555/XE088oYoVK6px48Zq0qSJpCt7p+rVq5evDQIAkO+2vidNDJZ+nmcrGSUjtazWHKWOPSp55Om/RwBAEZKnQ/vuueceNW/eXPHx8apTp46t3q5dO91999351hwAAPnq8DppflfH+qg9Si0WKuuyZa7vCQBQIOX5c6RCQ0MVGhpqV2vUqNENNwQAQL47c1iamckRE4NXSOUbX/m71erangAABVqeglSbNm1ksViyvH/16tV5bggAgHyTfF56o5GUeNy+3u1NqV4/c3oCABQKeQpSdevWtbtttVq1c+dO7dmzRwMGDMiPvgAAyLv0NOnjvtLvy+3rdzwqdXxBus4vAwEAyIk8BanXXnst03pMTIwSExNvqCEAAG7I2v9Ka6fZ1yo0kx5YKnl6m9MTAKDQyfM5Upnp37+/GjVqpFdeeSU/VwsAQPb2LpU+fcC+5h0gjdojFStlTk8AgEIrX4PUpk2b5Ofnl5+rBADg+uJ3SW+3dKw/skkqW8P1/QAAioQ8BakePXrY3TYMQ/Hx8dq2bZvGjx+fL40BAHBdiSelV6o61vt8LN3ayfX9AACKlDwFqaCgILvbHh4euvXWWzVp0iR16NAhXxoDACBTqSnSBx2lv3fY16NipOajTWkJAFD05ClIffjhh/ndBwAA12cY0v/GSNs+sK/XvFvq+YHk4WFOXwCAIumGzpHavn279u3bJ0mqWbOm6tXL5MMOAQC4UVvfvxKirlayojRso+Rb3JSWAABFW56C1MmTJ9W7d2+tXbtWwcHBkqRz586pTZs2+vjjj1W6dOn87BEAUFTFbZDm3eVYf/wXqWQF1/cDAMD/y9NxEI899pguXLigX3/9VWfOnNGZM2e0Z88eJSQkaOTIkfndIwCgqDkTJ8UEOYaoQculmPOEKACA6fK0R2r58uVauXKlqlevbqvVqFFDs2fP5mITAIC8S7kgzW4sJfxlX+/6hlT/fnN6AgAgE3kKUunp6fL2dvx0eG9vb6Wnp99wUwCAIiY9Tfrkfmn//+zrjYdJ0f+VLBZz+gIAIAt5OrSvbdu2evzxx/X333/ban/99ZdGjx6tdu3a5VtzAIAiYN1L0qQQ+xAV0Vh67pTU6UVCFADALeVpj9Qbb7yhrl27qmLFioqIiJAkHT16VLVq1dJHH32Urw0CAAqpfd9In/S3r3n6SmP2SsVuMqcnAAByKE9BKiIiQj///LNWrVplu/x59erVFRUVla/NAQAKoeO7pbeaO9aHbZRCb3N9PwAA5EGeP0dq9erVWr16tU6ePKn09HTt2LFDixYtkiR98MEH2TwaAFDkJJ6SpleT0lPt670XSdXuNKcnAADyKE9BauLEiZo0aZIaNmyosLAwWTh+HQCQldQU6cNO0l/b7ettn5NaPmlOTwAA3KA8Bam33npLc+fO1f33cylaAEAWDENa9oS09T37evWu0r3zJI88Xe8IAAC3kKcgdfnyZTVt2jS/ewEAFBbb50nfXPMB7UHlpUd/lHxLmNMTAAD5KE+/DnzooYds50MBAGBzZKMUE+QYoh7fJY3eTYgCABQaOd4jNWbMGNvf09PT9c4772jlypWqXbu2w4fzTp8+Pf86BAC4v7NHpNfrONYHLpMqNnN5OwAAOFuOg9SOHTvsbtetW1eStGfPHrs6F54AgCIk5YL0ZhPp/DH7epfXpQYDTWkJAABXyHGQWrNmjTP7AAAUJOnp0mcPXPlQ3avdPkTq/LJUwH6pdjk1XXN/PKINcR468eMRDWxWWT5eXAwDAJA1U/+XWL9+vbp06aLw8HBZLBYtWbLE7n7DMPT8888rLCxM/v7+ioqK0oEDB+yWOXPmjPr166fAwEAFBwfrwQcfVGJiogufBQAUMetfkSaVtA9R5RpKz52U7nylwIWoacv2qtr47/TCd79rw3EPvfDd76o2/jtNW7bX7NYAAG7M1CB18eJF1alTR7Nnz870/pdeekkzZ87UW2+9pc2bN6tYsWLq2LGjkpOTbcv069dPv/76q2JjY/Xtt99q/fr1Gjp0qKueAgAUHb/978qFJFZP/rfm4SU9cVAaskry8jWvtzyatmyv3l4fp3TDvp5uSG+vjyNMAQCylKfLn+eXTp06qVOnTpneZxiGZsyYoeeee07dunWTJM2fP19ly5bVkiVL1Lt3b+3bt0/Lly/X1q1b1bBhQ0nSrFmz1LlzZ73yyisKDw932XMBgELrxK/SnEw+8mLYD1JoLdf3k08up6br3Q1x113m3Q1x+k+HahzmBwBwYGqQup64uDgdP35cUVFRtlpQUJAaN26sTZs2qXfv3tq0aZOCg4NtIUqSoqKi5OHhoc2bN+vuu+/OdN0pKSlKSUmx3U5ISJAkWa1WWa1WJz2jnMnYvtl9FFbM17mYr3O5fL4X/5HXrNqypF22K6f2nCej2p0ZTbmmFyeY++MRhz1R10o3pLkbD2lQ04ou6akw4/3B+ZixczFf53Kn+ea0B7cNUsePH5cklS1b1q5etmxZ233Hjx9XmTJl7O738vJSSEiIbZnMTJs2TRMnTnSor1ixQgEBATfaer6IjY01u4VCjfk6F/N1LmfP15KeqmYHp6nURftzUveF9dDvod2lw5IOL3NqD66wIc5DOTnCfcOO31T2HIf45RfeH5yPGTsX83Uud5hvUlJSjpZz2yDlTM8884zd52IlJCQoIiJCHTp0UGBgoImdXUnAsbGxat++vcPnc+HGMV/nYr7O5fT5GoY8VoyT54537crpt3RWWs8PVcXDU1Xyf6umOfHjEW347vdsl2tRr5o6s0fqhvH+4HzM2LmYr3O503wzjlbLjtsGqdDQUEnSiRMnFBYWZqufOHHC9hlWoaGhOnnypN3jUlNTdebMGdvjM+Pr6ytfX8eTor29vU3/xmVwp14KI+brXMzXuZwy35/nS18/Zl8LvFl6dJM8/ALNvTKRkwxsVln/Xf77dQ/v87BcWc6bc6TyDe8PzseMnYv5Opc7zDen23fb/xkiIyMVGhqqVatW2WoJCQnavHmzmjRpIklq0qSJzp07p+3bt9uWWb16tdLT09W4cWOX9wwABc4fm65cie/aEDVypzTmV8nP3L30zuTj5aEhLSKvu8yQFpFcaAIAkClT90glJibq4MGDtttxcXHauXOnQkJCVL58eY0aNUpTpkxR1apVFRkZqfHjxys8PFzdu3eXJFWvXl3R0dEaMmSI3nrrLVmtVo0YMUK9e/fmin0AcD3njkozMrni3oBvpcgWru/HJM90riHpytX5rt4z5WG5EqIy7gcA4FqmBqlt27apTZs2ttsZ5y0NGDBAc+fO1dixY3Xx4kUNHTpU586dU/PmzbV8+XL5+fnZHrNw4UKNGDFC7dq1k4eHh3r27KmZM2e6/LkAQIGQkii91Uw6e8S+ftdrUsPBprRktmc619B/OlTT3I2HtGHHb2pRr5oGNqvMnigAwHWZGqRat24tw8j64HSLxaJJkyZp0qRJWS4TEhKiRYsWOaM9ACg80tOlzwdJe5fY1xsOlu6cLlksprTlLny8PDSoaUWVPbdXnZtW5JwoAEC23PZiEwCAfLJhurTqmo98CK8vDV4ueTleeAcAAGSPIAUAhdX+76TFva8pWqQnfpeKl8n0IQAAIGcIUgBQ2JzcJ715h2P94fVSWB3X9wMAQCFEkAKAwuLiaem1mlLqJfv6vfOkmt1NaQkAgMKKIAUABV3qZWl+V+noJvt6q6elNs+Y0xMAAIUcQQoACirDkL4fJ/30pn39lk5S74WSh6c5fQEAUAQQpACgINqxUFr6qH2teKg0YovkF2ROTwAAFCEEKQAoQEomHpD31Jsc73jsZ6lUZdc3BABAEUWQAoCC4Nwxec+4TS2vrT/wtVSplRkdAQBQpBGkAMCdXb4ovdVcOnPYvt75FanREHN6AgAABCkAcEvp6dKXD0l7vrArHynVWuUe/kzePj4mNQYAACSCFAC4n40zpdjx9rXQ2rIOWKZdK1arnMViTl8AAMCGIAUA7uL376VFvRzr/9kvlQiVrFbX9wQAADJFkAIAs538TXqzsWN96FopvJ7L2wEAANkjSAGAWZLOSDNqSZcT7ev3fCjd1sOcngAAQI4QpADA1dKs0oK7pSMb7Ostx0ptnzWnJwAAkCsEKQBwpRXPST/Osq9V7SD1+Vjy8DSnJwAAkGsEKQBwhZ2LpSXD7GvFSksjtkn+waa0BAAA8o4gBQDOdGyr9H6UY/2xn6VSlV3fDwAAyBcEKQBwhvN/Sq/VdKzfv0Sq3Mbl7QAAgPxFkAKA/HQ5SXqnlfTP7/b1Ti9LjYea0xMAAMh3BCkAyA+GIX05RNr9mX29Xn+p6xuSxWJOXwAAwCkIUgBwo36cdeVqfFcre5v00ErJ29+cngAAgFMRpAAgrw7ESgvvcayP+U0KDHN9PwAAwGUIUgCQW6d+l2bf7lgfskYqV9/1/QAAAJcjSAFATiWdkWbWlZLP29d7vi/VymTPFAAAKLQIUgCQnTSr9FEPKW69fb35GClqgjk9AQAAUxGkAOB6Yp+XNr5uX6vcTur7qeTJWygAAEUVPwUAQGZ++fTK5cyvFlBKeuxnyT/YlJYAAID7IEgBwNX+3C6919axPmKbdFNV1/cDAADcEkEKACTp/F/SazUc6/2/lKq0c30/AADArRGkABRtl5Okd9tIp36zr0e/KN0xzJyeAACA2yNIASiaDENa8oi0a7F9vW4/qdtsyWIxpy8AAFAgEKQAFD0/zZGWP21fK1NDGrJa8vY3pycAAFCgEKQAFB0HV0of9XSsj9knBYa7vh8AAFBgEaQAFH7/HJDeaOhYf2i1dHMD1/cDAAAKPIIUgMLr0llpZr0rf16tx3tS7XvN6QkAABQKBCkAhU9aqrTwHunwGvt6s1FS+4mmtAQAAAoXghSAwmVljPTDa/a1Sq2lfl9InrzlAQCA/MFPFQAKh92fS188aF/zC5Ye3yn5lzSjIwAAUIgRpAAUbH9tl95t61gfvlUqfYvr+wEAAEUCQQpAwZQQL02v5ljv94VUNcr1/QAAgCKFIAWgYLFeurIH6uRe+3rHF6Qmw83pCQAAFDkEKQAFg2FIS4dLOxfa12vfJ939tmSxmNMXAAAokghSANzf5rel78ba1266VRq6VvIJMKUlAABQtBGkALivQ6ulBXc71kfvlYLKub4fAACA/0eQAuB+/jkovdHAsf7gSinidtf3AwAAcA2CFAD3cemcNKu+lHTavn73O1Kd+0xpCQAAIDMEKQDmS0uVFt8nHVxpX2/6mNRhijk9AQAAXAdBCoC5Vk2WNrxiX4tsKfX/UvL0NqcnAACAbBCkAJhjzxfS54Pta76B0uO7pIAQc3oCAADIIYIUANf6e4f0TmvH+qObpTLVXN4OAABAXhCkALjGhePSq7c61vt+Jt3SwfX9AAAA3ACCFADnsiZL70dJx3fb19tPlpqNNKcnAACAG+RhdgPZuXDhgkaNGqUKFSrI399fTZs21datW233Dxw4UBaLxe4rOjraxI4BSJIMQ1o6XJpa1j5E1bpXev4sIQoAABRobr9H6qGHHtKePXu0YMEChYeH66OPPlJUVJT27t2rcuXKSZKio6P14Ycf2h7j6+trVrsAJHlse1/6/in7Yqkq0sPrJZ9i5jQFAACQj9w6SF26dElffPGFli5dqpYtW0qSYmJi9M0332jOnDmaMuXK58v4+voqNDTUzFYBSLLErVe3HQ843jH6VynoZtc3BAAA4CRuHaRSU1OVlpYmPz8/u7q/v79++OEH2+21a9eqTJkyKlmypNq2baspU6aoVKlSWa43JSVFKSkpttsJCQmSJKvVKqvVms/PIncytm92H4UV83WSM4flPaeRwxtK6oBlMm5udOUGM79hvH6di/k6F/N1PmbsXMzXudxpvjntwWIYhuHkXm5I06ZN5ePjo0WLFqls2bJavHixBgwYoCpVqmj//v36+OOPFRAQoMjISB06dEjjxo1T8eLFtWnTJnl6ema6zpiYGE2cONGhvmjRIgUEBDj7KQGFhldaktrtHSu/1AS7+s/lh+hYqRYmdQUAAJB3SUlJ6tu3r86fP6/AwMAsl3P7IHXo0CENHjxY69evl6enp+rXr69bbrlF27dv1759+xyWP3z4sCpXrqyVK1eqXbt2ma4zsz1SERER+ueff647LFewWq2KjY1V+/bt5e3tbWovhRHzzSfpafL8tJ88Dq20K1sbPqxlac2Yr5Pw+nUu5utczNf5mLFzMV/ncqf5JiQk6Kabbso2SLn1oX2SVLlyZa1bt04XL15UQkKCwsLCdN9996lSpUqZLl+pUiXddNNNOnjwYJZBytfXN9MLUnh7e5v+jcvgTr0URsz3Bqx5QVr3on2tQjPpgaVSuqRly5ivkzFf52K+zsV8nY8ZOxfzdS53mG9Ot+/2QSpDsWLFVKxYMZ09e1bff/+9XnrppUyX+/PPP3X69GmFhYW5uEOgkPt1ifTZAPuadzFp1G6p2P+fk5hu/nHNAAAAruD2Qer777+XYRi69dZbdfDgQT355JOqVq2aBg0apMTERE2cOFE9e/ZUaGioDh06pLFjx6pKlSrq2LGj2a0DhUP8Luntlo71R3+SylR3fT8AAABuwO2D1Pnz5/XMM8/ozz//VEhIiHr27KmpU6fK29tbqamp+uWXXzRv3jydO3dO4eHh6tChgyZPnsxnSQE36sIJ6dVbHOt9PpFu5UOvAQBA0eb2QapXr17q1atXpvf5+/vr+++/d3FHQCFnTZY+6HBlT9TVomKk5qNNaQkAAMDduH2QAuAihiF9O0raPte+XvNuqecHkoeHGV0BAAC4JYIUAGnr+9L/xtjXSkZKw36QfIub0xMAAIAbI0gBRVncemleF8f6qN1ScHnX9wMAAFBAEKSAoujMYWlmPcf6oOVShSau7wcAAKCAIUgBRUlygjS7kXQh3r7e9Q2p/v3m9AQAAFAAEaSAoiA9Tfqkv7R/mX298SNS9DTJYjGnLwAAgAKKIAUUdmtflNa+YF+LuEMa8I3k5WNOTwAAAAUcQQoorPYulT59wL7m5SeN/lUqdpM5PQEAABQSBCmgsIn/RXq7hWP9kR+lsjVd3w8AAEAhRJACCovEk9Krt0pGun2992KpWmdzegIAACikCFJAQZeaIn0QLf39s3297Xip5RPm9AQAAFDIEaSAgsowpP+NkbZ9YF+v0U26Z67k4WFKWwAAAEUBQQooiLZ9KH07yr4WXP7KeVC+JUxpCQAAoCghSAEFyZEfpLl3OtYf/0UqWcH1/QAAABRRBCmgIDh7RHq9jmN94DKpYjOXtwMAAFDUEaQAd5ZyQZp9h5Twp329y0ypwQBzegIAAABBCnBL6enSp/dLv31rX280VOr0kmSxmNMXAAAAJBGkAPez7mVpzRT72s23SwP/J3n5mtMTAAAA7BCkAHex71vpk372NQ9vacw+qXhpc3oCAABApghSgNmO75beau5YH7ZRCr3N9f0AAAAgWwQpwCyJp6Tp1aT0VPv6fR9J1buY0xMAAAByhCAFuFrqZWluZ+nPrfb1Ns9JrZ40pycAAADkCkEKcBXDkL4bK215x75e7S6p13zJw9OcvgAAAJBrBCnAFX6eL339mH0tKEJ6dJPkW8KcngAAAJBnBCnAmY5svHIY37VG7pRCIl3eDgAAAPIHQQpwhrN/SK/XdqwP/J9UMZMr9AEAAKBAIUgB+SklUZrTRDp31L5+12tSw8Hm9AQAAIB8R5AC8kN6uvT5QGnvUvv67Q9JnV+RLBZT2gIAAIBzEKSAG7X+FWn1ZPtauQbSoO8kL19zegIAAIBTEaSAvPptmfRxH/uaxUP6z36peBlzegIAAIBLEKSA3DrxqzSnqWP94Q1SWCYXmAAAAEChQ5ACcuriaem1GlJqsn291wKpRldzegIAAIApCFJAdlIvS/O7Skc32ddbPyO1ftqcngAAAGAqghSQFcOQlj8tbX7Lvn5rZ+m+jyQPT3P6AgAAgOkIUkBmdnwkLR1uXysRJg3fIvkFmtMTAAAA3AZBCrja0Z+kDzo61kfukEIqub4fAAAAuCWCFCBJ545JM25zrA/4Rops6fp+AAAA4NYIUijaUhKlt5pJZ4/Y1+98Vbr9IVNaAgAAgPsjSKFoMtKlzwZKv35lX28wULprhmSxmNAUAAAACgqCFIqcKif+J+8XHrAvhteTBi2XvP3MaQoAAAAFCkEKRcf+5fJefJ9qXlv/z+9SibJmdAQAAIACiiCFwu/kPunNOxzrQ9dJ4XVd3g4AAAAKPoIUCq+Lp69cic+aZFfeWnGE6vaLkbe3t0mNAQAAoKDzMLsBIN+lWaUPO0svV7IPUa2ekvXZf/R3yUbm9QYAAIBCgT1SKFy+f1ba9IZ97ZZoqfciycNTslrN6QsAAACFCkEKhcPORdKSR+xrxctKI7ZKfkHm9AQAAIBCiyCFgu3oZumDDo71x36WSlV2fT8AAAAoEghSKJjO/ym95nAhc+mBpVKl1i5vBwAAAEULQQoFy+WL0tstpdMH7eudX5EaDTGnJwAAABQ5BCkUDOnp0pcPSXu+sK/Xf0DqMlOyWMzpCwAAAEUSQQrub+NMKXa8fS20lvTgSsnbz5yeAAAAUKQRpOC+fl8hLbrXsf6f/VKJUNf3AwAAAPw/gpQbSUs3tDnujLb/Y1GpuDNqUqWMPD2K4CFrp/ZLszP50Nyha6Xwei5vBwAAALgWQcpNLN8Tr4nf7FX8+WRJnpp/YJvCgvw0oUsNRd8WZnZ7rpF0RppRS7qcaF+/5wPptp7m9AQAAABkwsPsBrJz4cIFjRo1ShUqVJC/v7+aNm2qrVu32u43DEPPP/+8wsLC5O/vr6ioKB04cMDEjnNv+Z54PfLRz/8fov51/HyyHvnoZy3fE29SZy6SZpXm3iW9FGkfolo8IcWcJ0QBAADA7bh9kHrooYcUGxurBQsWaPfu3erQoYOioqL0119/SZJeeuklzZw5U2+99ZY2b96sYsWKqWPHjkpOTs5mze4hLd3QxG/2ysjkvozaxG/2Ki09syUKgRXPSZNvko5s+LdWJUoaf1pqNz7rxwEAAAAmcutD+y5duqQvvvhCS5cuVcuWLSVJMTEx+uabbzRnzhxNnjxZM2bM0HPPPadu3bpJkubPn6+yZctqyZIl6t27d6brTUlJUUpKiu12QkKCJMlqtcpqtTr5WdnbHHfGYU/U1QxJ8eeTtengSTWODHFdY05m2f2pvL5+1K5mBNyk1Ec2S35BUrohpef/9yLj++vq73NRwXydi/k6F/N1LubrfMzYuZivc7nTfHPag8UwDLfd1XHhwgUFBgZq5cqVateuna3evHlzeXl56YMPPlDlypW1Y8cO1a1b13Z/q1atVLduXb3++uuZrjcmJkYTJ050qC9atEgBAQH5/jyuZ/s/Fs0/4Jntcg9UTVODm9z2W5VjJS8eVMvfJznUV1Z/URf9isi5YAAAAHBbSUlJ6tu3r86fP6/AwMAsl3PrPVIlSpRQkyZNNHnyZFWvXl1ly5bV4sWLtWnTJlWpUkXHjx+XJJUtW9bucWXLlrXdl5lnnnlGY8aMsd1OSEhQRESEOnTocN1hOUOpuDOaf2Bbtst1aNG4YO+RSvhb3rNqO5RT+3wuo1JrtXJRG1arVbGxsWrfvr28vb1dtNWig/k6F/N1LubrXMzX+ZixczFf53Kn+WYcrZYdtw5SkrRgwQINHjxY5cqVk6enp+rXr68+ffpo+/bteV6nr6+vfH19Here3t4u/8Y1qVJGYUF+On4+OdPzpCySQoP8Cu6l0C8nSe+0kv753b4e/aJ0xzDTXoBmfK+LEubrXMzXuZivczFf52PGzsV8ncsd5pvT7bv9xSYqV66sdevWKTExUceOHdOWLVtktVpVqVIlhYZe+VDWEydO2D3mxIkTtvvcnaeHRRO61JB0JTRdLeP2hC41Cl6IMgzpiyHSC2H2IapuP2nCOemOYaa1BgAAANwotw9SGYoVK6awsDCdPXtW33//vbp166bIyEiFhoZq1apVtuUSEhK0efNmNWnSxMRucyf6tjDN6V9foUF+dvXQID/N6V+/4H2O1I9vSBODpd2f/lsrU1N69rjU/U3JUsBCIQAAAHANtz+07/vvv5dhGLr11lt18OBBPfnkk6pWrZoGDRoki8WiUaNGacqUKapataoiIyM1fvx4hYeHq3v37ma3nivRt4WpfY1QbTp4Uis2bFaHFo0L3uF8B1ZKCzP5zKcxv0mBBSwMAgAAANfh9kHq/PnzeuaZZ/Tnn38qJCREPXv21NSpU23HLo4dO1YXL17U0KFDde7cOTVv3lzLly+Xn59fNmt2P54eFjWODNHpfYYaR4YUnBB16ndp9u2O9SGrpXINXN8PAAAA4GRuH6R69eqlXr16ZXm/xWLRpEmTNGmS4yW14WRJZ6SZdaXk8/b1Hu9Jte81pSUAAADAFdw+SMENpaVKH90txa23rzcfLUXFmNISAAAA4EoEKeRO7PPSxms+6LhSG6nf55InLycAAAAUDfzki5z55VPpyyH2Nf+S0sgdV/4EAAAAihCCFK7vz+3Se20d68O3SqVvcX0/AAAAgBsgSCFzCX9L06s71vt9IVWNcn0/AAAAgBshSMGe9ZL0Tmvp1G/29Y7TpCaPmtISAAAA4G4IUrjCMKQlj0i7FtvX6/SRus+RLAXkM60AAAAAFyBIQfppjrT8afta6WrSkDWST4A5PQEAAABujCBVlB1cJX3Uw7E+eq8UVM71/QAAAAAFBEGqKPrngPRGQ8f6Q6ukmzOpAwAAALBDkCpKLp2VZta78ufV7n5HqnOfOT0BAAAABRBBqihIS5UW3SsdWm1fbzpS6jDZnJ4AAACAAowgVditnCj9MN2+FtlS6v+l5OltTk8AAABAAUeQKqx2fy598aB9zTdQenyXFBBiTk8AAABAIUGQKmz+2i6929axPnyLVPpW1/cDAAAAFEIEqcIiIV6aXs2x3u9zqWp71/cDAAAAFGIEqYLOekl6L0o6sce+3mGK1PQxc3oCAAAACjmCVEFlGNLSEdLOj+zrte69cjlzDw9z+gIAAACKAIJUQbT5bem7sfa1m26Rhq6TfALM6QkAAAAoQghSBcmhNdKC7o710b9KQTe7vB0AAACgqCJIFQSnD0mz6jvWH4yVIhq5vh8AAACgiCNIubNL56RZDaSkf+zr3d+S6vYxpSUAAAAABCm3ZDHS5Ln4PunwKvs7moyQOk41pykAAAAANgQpN2P57Rt13TnIvlixhXT/V5KntzlNAQAAALBDkHIznutf+veGT3Fp1G4pIMS8hgAAAAA44MOG3Exam/GKD6on69AfpHF/EaIAAAAAN8QeKTdjVO2gLQdS1bl0NbNbAQAAAJAF9kgBAAAAQC4RpAAAAAAglwhSAAAAAJBLBCkAAAAAyCWCFAAAAADkEkEKAAAAAHKJIAUAAAAAuUSQAgAAAIBcIkgBAAAAQC4RpAAAAAAglwhSAAAAAJBLBCkAAAAAyCWCFAAAAADkEkEKAAAAAHKJIAUAAAAAuUSQAgAAAIBcIkgBAAAAQC4RpAAAAAAgl7zMbsAdGIYhSUpISDC5E8lqtSopKUkJCQny9vY2u51Ch/k6F/N1LubrXMzXuZiv8zFj52K+zuVO883IBBkZISsEKUkXLlyQJEVERJjcCQAAAAB3cOHCBQUFBWV5v8XILmoVAenp6fr7779VokQJWSwWU3tJSEhQRESEjh07psDAQFN7KYyYr3MxX+divs7FfJ2L+TofM3Yu5utc7jRfwzB04cIFhYeHy8Mj6zOh2CMlycPDQzfffLPZbdgJDAw0/UVUmDFf52K+zsV8nYv5OhfzdT5m7FzM17ncZb7X2xOVgYtNAAAAAEAuEaQAAAAAIJcIUm7G19dXEyZMkK+vr9mtFErM17mYr3MxX+divs7FfJ2PGTsX83WugjhfLjYBAAAAALnEHikAAAAAyCWCFAAAAADkEkEKAAAAAHKJIAUAAAAAuUSQcqK0tDSNHz9ekZGR8vf3V+XKlTV58mRdfX0PwzD0/PPPKywsTP7+/oqKitKBAweyXffs2bNVsWJF+fn5qXHjxtqyZYszn4pbym6+VqtVTz31lGrVqqVixYopPDxcDzzwgP7+++/rrjcmJkYWi8Xuq1q1aq54Sm4lJ6/fgQMHOswqOjo623Xz+s3ZfK+dbcbXyy+/nOV6ef3+68KFCxo1apQqVKggf39/NW3aVFu3brXdz/vvjbnefHn/vXHZvX55/70x2c2X99/cWb9+vbp06aLw8HBZLBYtWbLE7v6cvN+eOXNG/fr1U2BgoIKDg/Xggw8qMTHxuttNTk7W8OHDVapUKRUvXlw9e/bUiRMn8vvpZc2A00ydOtUoVaqU8e233xpxcXHGZ599ZhQvXtx4/fXXbcv897//NYKCgowlS5YYu3btMrp27WpERkYaly5dynK9H3/8seHj42N88MEHxq+//moMGTLECA4ONk6cOOGKp+U2spvvuXPnjKioKOOTTz4xfvvtN2PTpk1Go0aNjAYNGlx3vRMmTDBq1qxpxMfH275OnTrliqfkVnLy+h0wYIARHR1tN6szZ85cd728fq/IyXyvnmt8fLzxwQcfGBaLxTh06FCW6+X1+69evXoZNWrUMNatW2ccOHDAmDBhghEYGGj8+eefhmHw/nujrjdf3n9vXHavX95/b0x28+X9N3eWLVtmPPvss8aXX35pSDK++uoru/tz8n4bHR1t1KlTx/jpp5+MDRs2GFWqVDH69Olz3e0OGzbMiIiIMFatWmVs27bNuOOOO4ymTZs64ylmiiDlRHfeeacxePBgu1qPHj2Mfv36GYZhGOnp6UZoaKjx8ssv2+4/d+6c4evrayxevDjL9TZq1MgYPny47XZaWpoRHh5uTJs2LZ+fgXvLbr6Z2bJliyHJ+OOPP7JcZsKECUadOnXyq80CKyfzHTBggNGtW7dcrZfX7xV5ef1269bNaNu27XXXy+v3iqSkJMPT09P49ttv7er169c3nn32Wd5/b1B2880M7785l5P58v6bd3l5/fL+m3PXBqmcvN/u3bvXkGRs3brVtsx3331nWCwW46+//sp0O+fOnTO8vb2Nzz77zFbbt2+fIcnYtGlTPj+rzHFonxM1bdpUq1at0u+//y5J2rVrl3744Qd16tRJkhQXF6fjx48rKirK9pigoCA1btxYmzZtynSdly9f1vbt2+0e4+HhoaioqCwfU1hlN9/MnD9/XhaLRcHBwddd94EDBxQeHq5KlSqpX79+Onr0aH62XiDkdL5r165VmTJldOutt+qRRx7R6dOns1wnr99/5fb1e+LECf3vf//Tgw8+mO26ef1KqampSktLk5+fn13d399fP/zwA++/Nyi7+WaG99+cy+l8ef/Nm9y+fnn/vTE5eb/dtGmTgoOD1bBhQ9syUVFR8vDw0ObNmzNd7/bt22W1Wu3WW61aNZUvX95lr2kvl2yliHr66aeVkJCgatWqydPTU2lpaZo6dar69esnSTp+/LgkqWzZsnaPK1u2rO2+a/3zzz9KS0vL9DG//fabE56F+8puvtdKTk7WU089pT59+igwMDDL9TZu3Fhz587Vrbfeqvj4eE2cOFEtWrTQnj17VKJECWc9HbeTk/lGR0erR48eioyM1KFDhzRu3Dh16tRJmzZtkqenp8M6ef3+K7ev33nz5qlEiRLq0aPHddfL6/eKEiVKqEmTJpo8ebKqV6+usmXLavHixdq0aZOqVKnC++8Nym6+1+L9N3dyMl/ef/Mut69f3n9vTE7eb48fP64yZcrY3e/l5aWQkJAs35OPHz8uHx8fh1/OXO99PL8RpJzo008/1cKFC7Vo0SLVrFlTO3fu1KhRoxQeHq4BAwaY3V6Bl5v5Wq1W9erVS4ZhaM6cOddd79V7BGrXrq3GjRurQoUK+vTTT3P026jCIifz7d27t235WrVqqXbt2qpcubLWrl2rdu3amdV6gZDb94cPPvhA/fr1c/gN6rV4/f5rwYIFGjx4sMqVKydPT0/Vr19fffr00fbt281urVDI6Xx5/82b7ObL+++Nyc37A++/yAqH9jnRk08+qaefflq9e/dWrVq1dP/992v06NGaNm2aJCk0NFSSHK4ucuLECdt917rpppvk6emZq8cUVtnNN0PGf+J//PGHYmNjr/vb0MwEBwfrlltu0cGDB/OzfbeX0/lerVKlSrrpppuynBWv33/lZr4bNmzQ/v379dBDD+V6O0X19StJlStX1rp165SYmKhjx45py5YtslqtqlSpEu+/+eB6883A+2/e5WS+V+P9N3dyOl/ef29cTt5vQ0NDdfLkSbv7U1NTdebMmSxfn6Ghobp8+bLOnTuX5XqdjSDlRElJSfLwsB+xp6en0tPTJUmRkZEKDQ3VqlWrbPcnJCRo8+bNatKkSabr9PHxUYMGDewek56erlWrVmX5mMIqu/lK//4nfuDAAa1cuVKlSpXK9XYSExN16NAhhYWF3XDPBUlO5nutP//8U6dPn85yVrx+/5Wb+b7//vtq0KCB6tSpk+vtFNXX79WKFSumsLAwnT17Vt9//726devG+28+ymy+Eu+/+SWr+V6L99+8yW6+vP/euJy83zZp0kTnzp2z2yO4evVqpaenq3Hjxpmut0GDBvL29rZb7/79+3X06FHXvaZdckmLImrAgAFGuXLlbJc3/vLLL42bbrrJGDt2rG2Z//73v0ZwcLCxdOlS45dffjG6devmcDnItm3bGrNmzbLd/vjjjw1fX19j7ty5xt69e42hQ4cawcHBxvHjx136/MyW3XwvX75sdO3a1bj55puNnTt32l2ONCUlxbaea+f7n//8x1i7dq0RFxdnbNy40YiKijJuuukm4+TJky5/jmbKbr4XLlwwnnjiCWPTpk1GXFycsXLlSqN+/fpG1apVjeTkZNt6eP1mLifvD4ZhGOfPnzcCAgKMOXPmZLoeXr9ZW758ufHdd98Zhw8fNlasWGHUqVPHaNy4sXH58mXDMHj/vVHXmy/vvzfuevPl/ffGZff+YBi8/+bGhQsXjB07dhg7duwwJBnTp083duzYYbtKZ07eb6Ojo4169eoZmzdvNn744QejatWqdpc///PPP41bb73V2Lx5s602bNgwo3z58sbq1auNbdu2GU2aNDGaNGnisudNkHKihIQE4/HHHzfKly9v+Pn5GZUqVTKeffZZu/9E0tPTjfHjxxtly5Y1fH19jXbt2hn79++3W0+FChWMCRMm2NVmzZpllC9f3vDx8TEaNWpk/PTTT654Sm4lu/nGxcUZkjL9WrNmjW091873vvvuM8LCwgwfHx+jXLlyxn333WccPHjQxc/OfNnNNykpyejQoYNRunRpw9vb26hQoYIxZMgQh/+Qef1mLifvD4ZhGG+//bbh7+9vnDt3LtP18PrN2ieffGJUqlTJ8PHxMUJDQ43hw4fbzZH33xtzvfny/nvjrjdf3n9vXHbvD4bB+29urFmzJtN/7wMGDDAMI2fvt6dPnzb69OljFC9e3AgMDDQGDRpkXLhwwXZ/xvvK1e8hly5dMh599FGjZMmSRkBAgHH33Xcb8fHxrnjKhmEYhsUwDMM1+74AAAAAoHDgHCkAAAAAyCWCFAAAAADkEkEKAAAAAHKJIAUAAAAAuUSQAgAAAIBcIkgBAAAAQC4RpAAAAAAglwhSAAAAAJBLBCkAgGlat26tUaNGmd2GU1gsFi1ZssTsNgAATuJldgMAABRG8fHxKlmypNltAACchCAFAIAThIaGmt0CAMCJOLQPAGCq9PR0jR07ViEhIQoNDVVMTIztvqNHj6pbt24qXry4AgMD1atXL504ccJ2/8CBA9W9e3e79Y0aNUqtW7e23f78889Vq1Yt+fv7q1SpUoqKitLFixdt97/33nuqXr26/Pz8VK1aNb355ps56vvy5csaMWKEwsLC5OfnpwoVKmjatGm2+68+tC8mJkYWi8Xha+7cubYZTJs2TZGRkfL391edOnX0+eef52yAAABTEKQAAKaaN2+eihUrps2bN+ull17SpEmTFBsbq/T0dHXr1k1nzpzRunXrFBsbq8OHD+u+++7L8brj4+PVp08fDR48WPv27dPatWvVo0cPGYYhSVq4cKGef/55TZ06Vfv27dMLL7yg8ePHa968edmue+bMmfr666/16aefav/+/Vq4cKEqVqyY6bJPPPGE4uPjbV+vvPKKAgIC1LBhQ0nStGnTNH/+fL311lv69ddfNXr0aPXv31/r1q3L8XMFALgWh/YBAExVu3ZtTZgwQZJUtWpVvfHGG1q1apUkaffu3YqLi1NERIQkaf78+apZs6a2bt2q22+/Pdt1x8fHKzU1VT169FCFChUkSbVq1bLdP2HCBL366qvq0aOHJCkyMlJ79+7V22+/rQEDBlx33UePHlXVqlXVvHlzWSwW2/ozU7x4cRUvXlyS9NNPP+m5557TvHnzdNtttyklJUUvvPCCVq5cqSZNmkiSKlWqpB9++EFvv/22WrVqle3zBAC4HkEKAGCq2rVr290OCwvTyZMntW/fPkVERNhClCTVqFFDwcHB2rdvX46CVJ06ddSuXTvVqlVLHTt2VIcOHXTPPfeoZMmSunjxog4dOqQHH3xQQ4YMsT0mNTVVQUFB2a574MCBat++vW699VZFR0frrrvuUocOHa77mKNHj6p79+564okn1KtXL0nSwYMHlZSUpPbt29ste/nyZdWrVy/bPgAA5iBIAQBM5e3tbXfbYrEoPT09R4/18PCwHaaXwWq12v7u6emp2NhY/fjjj1qxYoVmzZqlZ599Vps3b1ZAQIAk6d1331Xjxo3t1uHp6ZnttuvXr6+4uDh99913WrlypXr16qWoqKgsz226ePGiunbtqiZNmmjSpEm2emJioiTpf//7n8qVK2f3GF9f32z7AACYgyAFAHBL1atX17Fjx3Ts2DHbXqm9e/fq3LlzqlGjhiSpdOnS2rNnj93jdu7caRfOLBaLmjVrpmbNmun5559XhQoV9NVXX2nMmDEKDw/X4cOH1a9fvzz1GBgYqPvuu0/33Xef7rnnHkVHR+vMmTMKCQmxW84wDPXv31/p6elasGCBLBaL7b4aNWrI19dXR48e5TA+AChACFIAALcUFRWlWrVqqV+/fpoxY4ZSU1P16KOPqlWrVraLNLRt21Yvv/yy5s+fryZNmuijjz7Snj17bIfEbd68WatWrVKHDh1UpkwZbd68WadOnVL16tUlSRMnTtTIkSMVFBSk6OhopaSkaNu2bTp79qzGjBlz3f6mT5+usLAw1atXTx4eHvrss88UGhqq4OBgh2VjYmK0cuVKrVixQomJiba9UEFBQSpRooSeeOIJjR49Wunp6WrevLnOnz+vjRs3KjAwMNtztQAA5iBIAQDcksVi0dKlS/XYY4+pZcuW8vDwUHR0tGbNmmVbpmPHjho/frzGjh2r5ORkDR48WA888IB2794t6coeo/Xr12vGjBlKSEhQhQoV9Oqrr6pTp06SpIceekgBAQF6+eWX9eSTT6pYsWKqVauWRo0alW1/JUqU0EsvvaQDBw7I09NTt99+u5YtWyYPD8cL4q5bt06JiYlq2rSpXf3DDz/UwIEDNXnyZJUuXVrTpk3T4cOHFRwcrPr162vcuHE3MEEAgDNZjGsPLgcAAAAAXBefIwUAAAAAuUSQAgAgEy+88ILt85+u/co4NBAAUHRxaB8AAJk4c+aMzpw5k+l9/v7+DpcqBwAULQQpAAAAAMglDu0DAAAAgFwiSAEAAABALhGkAAAAACCXCFIAAAAAkEsEKQAAAADIJYIUAAAAAOQSQQoAAAAAcun/AFbB4P/79gI3AAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "[{'role': 'user',\n", + " 'content': '\\nBelow is some data. I want you to first remove the duplicates then analyze the statistics of the data as well as plot a line chart.\\n\\nhouse_size (m3), house_price ($)\\n90, 100\\n80, 90\\n100, 120\\n90, 100\\n'},\n", + " [Function(arguments='{\"agents\": [\"Data Processing Agent\"], \"query\": \"Remove duplicates from the data: house_size (m3), house_price ($)\\\\n90, 100\\\\n80, 90\\\\n100, 120\\\\n90, 100\"}', name='send_query_to_agents'),\n", + " Function(arguments='{\"agents\": [\"Analysis Agent\"], \"query\": \"Analyze the statistics of the data: house_size (m3), house_price ($)\\\\n90, 100\\\\n80, 90\\\\n100, 120\\\\n90, 100\"}', name='send_query_to_agents'),\n", + " Function(arguments='{\"agents\": [\"Visualization Agent\"], \"query\": \"Plot a line chart for the data: house_size (m3), house_price ($)\\\\n90, 100\\\\n80, 90\\\\n100, 120\\\\n90, 100\"}', name='send_query_to_agents')],\n", + " [Function(arguments='{\"data\":\"house_size (m3), house_price ($)\\\\n90, 100\\\\n80, 90\\\\n100, 120\\\\n90, 100\"}', name='clean_data')],\n", + " {'role': 'tool',\n", + " 'name': 'clean_data',\n", + " 'content': '{\"cleaned_data\": {\"house_size (m3)\": {\"0\": 90, \"1\": 80, \"2\": 100}, \" house_price ($)\": {\"0\": 100, \"1\": 90, \"2\": 120}}}'},\n", + " [Function(arguments='{\"data\":\"house_size,house_price\\\\n90,100\\\\n80,90\\\\n100,120\\\\n90,100\"}', name='stat_analysis')],\n", + " {'role': 'tool',\n", + " 'name': 'stat_analysis',\n", + " 'content': '{\"stats\": {\"house_size\": {\"count\": 4.0, \"mean\": 90.0, \"std\": 8.16496580927726, \"min\": 80.0, \"25%\": 87.5, \"50%\": 90.0, \"75%\": 92.5, \"max\": 100.0}, \"house_price\": {\"count\": 4.0, \"mean\": 102.5, \"std\": 12.583057392117917, \"min\": 90.0, \"25%\": 97.5, \"50%\": 100.0, \"75%\": 105.0, \"max\": 120.0}}}'},\n", + " [Function(arguments='{\"data\":\"house_size,house_price\\\\n90,100\\\\n80,90\\\\n100,120\\\\n90,100\",\"x\":\"house_size\",\"y\":\"house_price\"}', name='create_line_chart')],\n", + " {'role': 'tool',\n", + " 'name': 'create_line_chart',\n", + " 'content': '{\"line_chart\": \"sample_line_chart\"}'}]" + ] + }, + "execution_count": 60, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "handle_user_message(user_query)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "In this cookbook, we've explored how to leverage Structured Outputs to build more robust multi-agent systems.\n", + "\n", + "Using this new feature allows to make sure that tool calls follow the specified schema and avoids having to handle edge cases or validate arguments on your side.\n", + "\n", + "This can be applied to many more use cases, and we hope you can take inspiration from this to build your own use case!" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/registry.yaml b/registry.yaml index 146b80f..97ee0d4 100644 --- a/registry.yaml +++ b/registry.yaml @@ -4,7 +4,6 @@ # should build pages for, and indicates metadata such as tags, creation date and # authors for each page. - - title: Using logprobs path: examples/Using_logprobs.ipynb date: 2023-12-20 @@ -1233,7 +1232,6 @@ - vision - embeddings - - title: How to parse PDF docs for RAG path: examples/Parse_PDF_docs_for_RAG.ipynb date: 2024-02-28 @@ -1243,7 +1241,6 @@ - vision - embeddings - - title: Using GPT4o mini to tag and caption images path: examples/Tag_caption_images_with_GPT4V.ipynb date: 2024-07-18 @@ -1253,7 +1250,6 @@ - vision - embeddings - - title: How to use the moderation API path: examples/How_to_use_moderation.ipynb date: 2024-03-05 @@ -1261,8 +1257,7 @@ - teomusatoiu tags: - moderation - - + - title: Summarizing Long Documents path: examples/Summarizing_long_documents.ipynb date: 2024-04-19 @@ -1288,7 +1283,6 @@ tags: - completions - - title: CLIP embeddings to improve multimodal RAG with GPT-4 Vision path: examples/custom_image_embedding_search.ipynb date: 2024-04-10 @@ -1359,11 +1353,10 @@ path: examples/Data_extraction_transformation.ipynb date: 2024-07-09 authors: - - charuj + - charuj tags: - completions - vision - - ocr - title: GPT Actions library - Outlook path: examples/chatgpt/gpt_actions_library/gpt_action_outlook.ipynb @@ -1400,7 +1393,7 @@ tags: - completions - chatgpt - + - title: GPT Actions library - Canvas LMS path: examples/chatgpt/gpt_actions_library/gpt_action_canvaslms.ipynb date: 2024-07-17 @@ -1418,7 +1411,7 @@ tags: - gpt-actions-library - chatgpt - + - title: GPT Actions library - Gmail path: examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb date: 2024-07-24 @@ -1427,7 +1420,7 @@ tags: - gpt-actions-library - chatgpt - + - title: GPT Actions library - Jira path: examples/chatgpt/gpt_actions_library/gpt_action_jira.ipynb date: 2024-07-24 @@ -1462,4 +1455,23 @@ - evanweiss-openai tags: - gpt-actions-library - - chatgpt \ No newline at end of file + - chatgpt + +- title: Structured Outputs for Multi-Agent Systems + path: examples/Structured_outputs_multi_agent.ipynb + date: 2024-08-06 + authors: + - dylanra-openai + tags: + - completions + - functions + - assistants + +- title: Introduction to Structured Outputs + path: examples/Structured_outputs.ipynb + date: 2024-08-06 + authors: + - katiagg + tags: + - completions + - functions