{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to use the DALL·E API\n",
    "\n",
    "This notebook shows how to use OpenAI's DALL·E image API endpoints.\n",
    "\n",
    "There are three API endpoints:\n",
    "- **Generations:** generates an image or images based on an input caption\n",
    "- **Edits:** edits or extends an existing image\n",
    "- **Variations:** generates variations of an input image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "- Import the packages you'll need\n",
    "- Import your OpenAI API key: You can do this by running \\``export OPENAI_API_KEY=\"your API key\"`\\` in your terminal.\n",
    "- Set a directory to save images to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "import openai  # OpenAI Python library to make API calls\n",
    "import requests  # used to download images\n",
    "import os  # used to access filepaths\n",
    "from PIL import Image  # used to print and edit images\n",
    "\n",
    "# set API key\n",
    "openai.api_key = os.environ.get(\"OPENAI_API_KEY\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "image_dir='./images'\n"
     ]
    }
   ],
   "source": [
    "# set a directory to save DALL·E images to\n",
    "image_dir_name = \"images\"\n",
    "image_dir = os.path.join(os.curdir, image_dir_name)\n",
    "\n",
    "# create the directory if it doesn't yet exist\n",
    "if not os.path.isdir(image_dir):\n",
    "    os.mkdir(image_dir)\n",
    "\n",
    "# print the directory to save to\n",
    "print(f\"{image_dir=}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generations\n",
    "\n",
    "The generation API endpoint creates an image based on a text prompt.\n",
    "\n",
    "**Required inputs:**\n",
    "- prompt (str): A text description of the desired image(s). The maximum length is 1000 characters.\n",
    "\n",
    "**Optional inputs:**\n",
    "- n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.\n",
    "- size (str): The size of the generated images. Must be one of \"256x256\", \"512x512\", or \"1024x1024\". Smaller images are faster. Defaults to \"1024x1024\".\n",
    "- response_format (str): The format in which the generated images are returned. Must be one of \"url\" or \"b64_json\". Defaults to \"url\".\n",
    "- user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse. [Learn more.](https://beta.openai.com/docs/usage-policies/end-user-ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"created\": 1667611641,\n",
      "  \"data\": [\n",
      "    {\n",
      "      \"url\": \"https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-SFJhix3AV4bmPFvqYRJDkssp.png?st=2022-11-05T00%3A27%3A21Z&se=2022-11-05T02%3A27%3A21Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-05T01%3A27%3A21Z&ske=2022-11-06T01%3A27%3A21Z&sks=b&skv=2021-08-06&sig=0ZHl38v5UTFjA7V5Oshu8M58uHI5itEfvo2PX0aO6kA%3D\"\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# create an image\n",
    "\n",
    "# set the prompt\n",
    "prompt = \"A cyberpunk monkey hacker dreaming of a beautiful bunch of bananas, digital art\"\n",
    "\n",
    "# call the OpenAI API\n",
    "generation_response = openai.Image.create(\n",
    "    prompt=prompt,\n",
    "    n=1,\n",
    "    size=\"1024x1024\",\n",
    "    response_format=\"url\",\n",
    ")\n",
    "\n",
    "# print response\n",
    "print(generation_response)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: If you get this error - `AttributeError: module 'openai' has no attribute 'Image'` - you'll need to upgrade your OpenAI package to the latest version. You can do this by running `pip install openai --upgrade` in your terminal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the image\n",
    "generated_image_name = \"generated_image.png\"  # any name you like; the filetype should be .png\n",
    "generated_image_filepath = os.path.join(image_dir, generated_image_name)\n",
    "generated_image_url = generation_response[\"data\"][0][\"url\"]  # extract image URL from response\n",
    "generated_image = requests.get(generated_image_url).content  # download the image\n",
    "\n",
    "with open(generated_image_filepath, \"wb\") as image_file:\n",
    "    image_file.write(generated_image)  # write the image to the file\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./images/generated_image.png\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 2202120 characters]",
      "text/plain": [
       "<PIL.PngImagePlugin.PngImageFile image mode=RGB size=1024x1024 at 0x114DE3640>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# print the image\n",
    "print(generated_image_filepath)\n",
    "display(Image.open(generated_image_filepath))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variations\n",
    "\n",
    "The variations endpoint generates new images (variations) similar to an input image.\n",
    "\n",
    "Here we'll generate variations of the image generated above.\n",
    "\n",
    "**Required inputs:**\n",
    "- image (str): The image to use as the basis for the variation(s). Must be a valid PNG file, less than 4MB, and square.\n",
    "\n",
    "**Optional inputs:**\n",
    "- n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.\n",
    "- size (str): The size of the generated images. Must be one of \"256x256\", \"512x512\", or \"1024x1024\". Smaller images are faster. Defaults to \"1024x1024\".\n",
    "- response_format (str): The format in which the generated images are returned. Must be one of \"url\" or \"b64_json\". Defaults to \"url\".\n",
    "- user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse. [Learn more.](https://beta.openai.com/docs/usage-policies/end-user-ids)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"created\": 1667611666,\n",
      "  \"data\": [\n",
      "    {\n",
      "      \"url\": \"https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-7HTTBl2k9l4Ir4BTHXnJvFz9.png?st=2022-11-05T00%3A27%3A46Z&se=2022-11-05T02%3A27%3A46Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-04T01%3A50%3A22Z&ske=2022-11-05T01%3A50%3A22Z&sks=b&skv=2021-08-06&sig=QlcKhn427bOAQobM8CmpEf3K90OiumP5jOQwkJpcH6Y%3D\"\n",
      "    },\n",
      "    {\n",
      "      \"url\": \"https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-KGKrKGzlsXN0INxaeII2t8XG.png?st=2022-11-05T00%3A27%3A46Z&se=2022-11-05T02%3A27%3A46Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-04T01%3A50%3A22Z&ske=2022-11-05T01%3A50%3A22Z&sks=b&skv=2021-08-06&sig=RbPoAwXMVfdPxKF40ZjVjlclrnzaQZS%2BxzhgkEcYhOk%3D\"\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# create variations\n",
    "\n",
    "# call the OpenAI API, using `create_variation` rather than `create`\n",
    "variation_response = openai.Image.create_variation(\n",
    "    image=generated_image,  # generated_image is the image generated above\n",
    "    n=2,\n",
    "    size=\"1024x1024\",\n",
    "    response_format=\"url\",\n",
    ")\n",
    "\n",
    "# print response\n",
    "print(variation_response)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the images\n",
    "variation_urls = [datum[\"url\"] for datum in variation_response[\"data\"]]  # extract URLs\n",
    "variation_images = [requests.get(url).content for url in variation_urls]  # download images\n",
    "variation_image_names = [f\"variation_image_{i}.png\" for i in range(len(variation_images))]  # create names\n",
    "variation_image_filepaths = [os.path.join(image_dir, name) for name in variation_image_names]  # create filepaths\n",
    "for image, filepath in zip(variation_images, variation_image_filepaths):  # loop through the variations\n",
    "    with open(filepath, \"wb\") as image_file:  # open the file\n",
    "        image_file.write(image)  # write the image to the file\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./images/generated_image.png\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 2202120 characters]",
      "text/plain": [
       "<PIL.PngImagePlugin.PngImageFile image mode=RGB size=1024x1024 at 0x119A22F40>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./images/variation_image_0.png\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 2053220 characters]",
      "text/plain": [
       "<PIL.PngImagePlugin.PngImageFile image mode=RGB size=1024x1024 at 0x119A0AF10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./images/variation_image_1.png\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 2061240 characters]",
      "text/plain": [
       "<PIL.PngImagePlugin.PngImageFile image mode=RGB size=1024x1024 at 0x114DF1DF0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# print the original image\n",
    "print(generated_image_filepath)\n",
    "display(Image.open(generated_image_filepath))\n",
    "\n",
    "# print the new variations\n",
    "for variation_image_filepaths in variation_image_filepaths:\n",
    "    print(variation_image_filepaths)\n",
    "    display(Image.open(variation_image_filepaths))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Edits\n",
    "\n",
    "The edit endpoint uses DALL·E to generate a specified portion of an existing image. Three inputs are needed: the image to edit, a mask specifying the portion to be regenerated, and a prompt describing the desired image.\n",
    "\n",
    "**Required inputs:** \n",
    "- image (str): The image to edit. Must be a valid PNG file, less than 4MB, and square.\n",
    "- mask (str): An additional image whose fully transparent areas (e.g. where alpha is zero) indicate where `image` should be edited. Must be a valid PNG file, less than 4MB, and have the same dimensions as `image`.\n",
    "- prompt (str): A text description of the desired image(s). The maximum length is 1000 characters.\n",
    "\n",
    "**Optional inputs:**\n",
    "- n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.\n",
    "- size (str): The size of the generated images. Must be one of \"256x256\", \"512x512\", or \"1024x1024\". Smaller images are faster. Defaults to \"1024x1024\".\n",
    "- response_format (str): The format in which the generated images are returned. Must be one of \"url\" or \"b64_json\". Defaults to \"url\".\n",
    "- user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse. [Learn more.](https://beta.openai.com/docs/usage-policies/end-user-ids)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set Edit Area\n",
    "\n",
    "An edit requires a \"mask\" to specify which portion of the image to regenerate. Any pixel with an alpha of 0 (transparent) will be regenerated. The code below creates a 1024x1024 mask where the bottom half is transparent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a mask\n",
    "width = 1024\n",
    "height = 1024\n",
    "mask = Image.new(\"RGBA\", (width, height), (0, 0, 0, 1))  # create an opaque image mask\n",
    "\n",
    "# set the bottom half to be transparent\n",
    "for x in range(width):\n",
    "    for y in range(height // 2, height):  # only loop over the bottom half of the mask\n",
    "        # set alpha (A) to zero to turn pixel transparent\n",
    "        alpha = 0\n",
    "        mask.putpixel((x, y), (0, 0, 0, alpha))\n",
    "\n",
    "# save the mask\n",
    "mask_name = \"bottom_half_mask.png\"\n",
    "mask_filepath = os.path.join(image_dir, mask_name)\n",
    "mask.save(mask_filepath)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perform Edit\n",
    "\n",
    "Now we supply our image, caption and mask to the API to get 5 examples of edits to our image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"created\": 1667611683,\n",
      "  \"data\": [\n",
      "    {\n",
      "      \"url\": \"https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-F5XQFFBLrN7LdXuG5CkQJpxr.png?st=2022-11-05T00%3A28%3A03Z&se=2022-11-05T02%3A28%3A03Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-04T02%3A06%3A29Z&ske=2022-11-05T02%3A06%3A29Z&sks=b&skv=2021-08-06&sig=2UhH%2BkKdvDVoRcgWJhmNFVzpvLzBAZpnA/tU80Zc8M0%3D\"\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# edit an image\n",
    "\n",
    "# call the OpenAI API\n",
    "edit_response = openai.Image.create_edit(\n",
    "    image=open(generated_image_filepath, \"rb\"),  # from the generation section\n",
    "    mask=open(mask_filepath, \"rb\"),  # from right above\n",
    "    prompt=prompt,  # from the generation section\n",
    "    n=1,\n",
    "    size=\"1024x1024\",\n",
    "    response_format=\"url\",\n",
    ")\n",
    "\n",
    "# print response\n",
    "print(edit_response)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the image\n",
    "edited_image_name = \"edited_image.png\"  # any name you like; the filetype should be .png\n",
    "edited_image_filepath = os.path.join(image_dir, edited_image_name)\n",
    "edited_image_url = edit_response[\"data\"][0][\"url\"]  # extract image URL from response\n",
    "edited_image = requests.get(edited_image_url).content  # download the image\n",
    "\n",
    "with open(edited_image_filepath, \"wb\") as image_file:\n",
    "    image_file.write(edited_image)  # write the image to the file\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./images/generated_image.png\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 2202120 characters]",
      "text/plain": [
       "<PIL.PngImagePlugin.PngImageFile image mode=RGB size=1024x1024 at 0x114C6F7C0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./images/edited_image.png\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 1963728 characters]",
      "text/plain": [
       "<PIL.PngImagePlugin.PngImageFile image mode=RGB size=1024x1024 at 0x119526A30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# print the original image\n",
    "print(generated_image_filepath)\n",
    "display(Image.open(generated_image_filepath))\n",
    "\n",
    "# print edited image\n",
    "print(edited_image_filepath)\n",
    "display(Image.open(edited_image_filepath))\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.9 ('openai')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "365536dcbde60510dc9073d6b991cd35db2d9bac356a11f5b64279a5e6708b97"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
