jim/openai-cookbook
1
0
Fork 0
mirror of https://github.com/james-m-jordan/openai-cookbook.git synced 2025-05-09 19:32:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix UTF-8 encoding in Chat_finetuning_data_prep.ipynb (#648)

Browse Source
This commit is contained in:
recordcrash 2023-08-29 03:12:30 +02:00 committed by GitHub
parent 63f95154b1
commit 1945bfe65c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
examples/Chat_finetuning_data_prep.ipynb
View File

@ -8,7 +8,6 @@
"outputs": [], "outputs": [],
"source": [ "source": [
"import json\n", "import json\n",
"import os\n",
"import tiktoken\n", "import tiktoken\n",
"import numpy as np\n", "import numpy as np\n",
"from collections import defaultdict" "from collections import defaultdict"
@ -32,7 +31,7 @@
"outputs": [], "outputs": [],
"source": [ "source": [
"# Load dataset\n", "# Load dataset\n",
"with open(data_path) as f:\n", "with open(data_path, 'r', encoding='utf-8') as f:\n",
" dataset = [json.loads(line) for line in f]" " dataset = [json.loads(line) for line in f]"
] ]
}, },