Four Reasons to Use the GPT API, Three Reasons Not To, and Some Code to Get Started With
I use both the openai GPT graphical user interface and the API, depending on what I’m doing. Here are my thoughts on the advantages and disadvantages of the API, as well as some Python code to get you started, even if you’re really new to Python or coding in general.
Reasons to Use the API
If you’re currently using the openai GPT GUI, here are a few reasons why you might want to try the API.
No more copying and pasting. Let’s say you have a hundred Word documents you want summarized, and also you want to see whether GPT-4 performs better than GPT-3.5. That’s a lot of copying and pasting, and even more if you need to break each of the documents into pieces small enough for the API to accept. With not much programming, you can pull the text from all of your Word documents into Python, split it up into pieces which the API can accept, and then pass them to the API and get responses.
Responses in a searchable document. If you write the responses out into a document like a spreadsheet, which the code below does, they’ll be available for you to search later. By contrast, the GUI does not have good search capabilities, so you have to try to remember which conversation your message and response were part of in order to find them.
More control over your parameters. For instance, using the API, you can vary a parameter called “temperature”, which controls the randomness of the output. Other parameters you can modify include max_tokens, which controls the maximum length of the response you get.
Your data won’t get trained on. OpenAI has said that they won’t use data submitted via the API to train on, as opposed to data submitted via the GUI, so if you use the API this is less likely to happen to you.
Reasons to Not Use the API
There are also a few reasons why you might not want to use the API, or limit its usage to certain cases:
If you’re not logging your conversations yourself, they’ll be gone. The GPT graphical user interface isn’t the most helpful in terms of search capabilities or getting all of your prompts and responses out and into a document, but it does save them for you. If you do a good job of logging your API prompts and responses, you can do better than the GUI– but if you don’t, there’s no way to recover them.
It’s more difficult to have a conversation, as opposed to one-off prompts and responses. You can’t have a conversation using the API where you preserve ‘state’ - that is, where it automatically remembers all of the previous prompts and responses. There are ad hoc workarounds, where you pass it some or all of the previous prompts and responses with each call you make, but this isn’t optimal both because you’re being charged by the amount of text that you’re passing it each time and also because there are limits on how much text you can pass at once.
You have to get on the waitlist if you want to use the GPT-4 API. This is true even if you have access to it via the GUI. You can sign up here. (On the plus side, if you do get it, you don’t have the same hourly limits as the GUI.)
Repo and Code Notes
My repo for getting started using the API - which is intended to be beginner-friendly, and has a lot of comments and a detailed readme file - is here.
In order to make this work, you need an API key, which you can get here. You’ll also need to add billing information, if you haven’t already, and I recommend you monitor your activity to see how much you’re getting charged here.
To use the code, you'll need to:
Install the requirement.txt file. Or just pip install openai and pandas - but the requirements file will give you the versions that will continue to work with this code, even if there are new versions of those libraries.
Put your API key in a .txt file. You’ll also need to make sure the .txt file name and location correspond to where your Python file is looking for them. For instance, below: my “key.txt” file is in a subfolder called “key”, which is in a subfolder of the overall project folder; my code is running in a “code” subfolder that is also in that overall project folder. (The code folder is what’s tracked in my git repo so that I don’t accidentally put my key on GitHub.)
Change your your prompts, engines, iterations, and temperatures. You can do this using the dictMeta variable. For instance, below, I am making 16 API calls – four prompts * 1 engine * 2 temperatures * 2 iterations each.
If you want to pull the code and get started, you can do that from here.