jim/openai-cookbook
1
0
Fork 0
mirror of https://github.com/james-m-jordan/openai-cookbook.git synced 2025-05-09 19:32:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix iteration error (#208)

Browse Source 
* fix iteration error

array iteration error: sentences up till max_len of token exceeds should be returned

* split list once instead of twice

split list once instead of twice and take parsed string up till max token length is exceeded
This commit is contained in:
Swastik Banerjee 2023-03-22 03:15:56 +05:30 committed by GitHub
parent feb53449e0
commit 3dde564948
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
examples/fine-tuned_qa/olympics-1-collect-data.ipynb
View File

@ -154,7 +154,7 @@
" for i, sentence in enumerate(sentences):\n",
" ntokens += 1 + count_tokens(sentence)\n",
" if ntokens > max_len:\n",
" return \". \".join(sentences[:i][:-1]) + \".\"\n",
" return \". \".join(sentences[:i]) + \".\"\n",
"\n",
" return long_text\n",
"\n",