Question 1

What is ChatGPT actually doing?

Accepted Answer

It predicts the next chunk of text, over and over, based on everything in the conversation so far. That is why context matters more than clever wording: give it the role, the audience, the format and an example, and the prediction lands much closer to what you wanted.

Question 2

Does it remember me between chats?

Accepted Answer

Only if memory is switched on in the app, and even then it keeps a short profile rather than a transcript. Through the API there is no memory at all — you resend the whole conversation on every call, which is also why long chats cost more.

Question 3

What is a token?

Accepted Answer

A token is roughly four characters of English, so about 750 words per 1,000 tokens. You pay for tokens in (your prompt plus the history) and tokens out (the reply), which is why trimming context is the cheapest optimisation there is.

Question 4

What is the difference between ChatGPT Plus and the API?

Accepted Answer

Plus is a monthly subscription for the chat app you use in a browser. The API is pay-per-token access for your own software, billed separately. Paying for Plus gives you no API credit, and an API key does not unlock Plus features.

Question 5

Which model should I pick?

Accepted Answer

Start on a mini model — it handles most drafting, summarising and rewriting for pennies. Move up only when you can point at a specific failure: long documents, tricky reasoning, or code that has to be right first time.

Question 6

Why does it make things up?

Accepted Answer

Confident wrong answers happen because fluent text is the goal, not truth. Ask it to cite what it used, tell it to say 'I don't know' when unsure, and paste the source material in yourself for anything factual.

Question 7

Is my data used for training?

Accepted Answer

API traffic is not used to train models by default, while consumer chat settings vary and can be turned off. Either way, do not paste secrets, keys or customer records you would not email to a stranger.

Question 8

401 — Incorrect API key provided

Accepted Answer

The key is wrong, revoked, or truncated when it was copied. Create a fresh key, store it as an environment variable on your server, and check you are not sending the literal word 'undefined' because the variable never loaded.

Question 9

429 — Rate limit reached

Accepted Answer

You sent too many requests or too many tokens per minute. Slow the pipeline down and retry with backoff — the full recipe is in the rate limits section below.

Question 10

429 — You exceeded your current quota

Accepted Answer

Same status code, different problem: this one is billing, not speed. Retrying will never help. Add a payment method or top up credit, then check your monthly usage limit has not been hit.

Question 11

400 — This model's maximum context length is…

Accepted Answer

Prompt plus history plus the reply cannot exceed the context window. Trim old turns, summarise the conversation so far into a short recap, or chunk long documents and process them one section at a time.

Question 12

400 — Invalid request / unknown parameter

Accepted Answer

Usually a typo in the JSON body, a field the model does not accept, or a model name that no longer exists. Read the message — it names the offending field — and never guess a model id from memory.

Question 13

500 / 503 — server error or overloaded

Accepted Answer

A blip on their side. Retry two or three times with a growing delay, then fail loudly in your UI instead of showing an empty answer. Log the request id so support can trace it.

Question 14

The reply gets cut off mid-sentence

Accepted Answer

Your max_tokens ceiling was reached. Raise it, ask for a shorter answer in the prompt, or stream the response so the user sees text arriving and you can tell truncation from a hang.

Question 15

The answer ignores half my instructions

Accepted Answer

Long prompts bury the important bits. Put the non-negotiables in a numbered list at the end, keep the system message short and firm, and split a mega-prompt into two calls when it is really two jobs.

Question 16

1. Know which limit you hit

Accepted Answer

Limits come in pairs: requests per minute and tokens per minute. The response headers tell you what remains of each and how long until they reset — read them instead of guessing.

Question 17

2. Retry with exponential backoff and jitter

Accepted Answer

On a 429, wait for the Retry-After value if it is present. Otherwise wait one second, then two, then four, plus a small random offset so all your users do not retry in lockstep. Cap it at four or five attempts.

Question 18

3. Never retry a quota error

Accepted Answer

If the message says quota or billing, stop the loop and surface it. Retrying a billing failure burns time and, when it is a paid endpoint, can still cost you for partial work.

Question 19

4. Queue instead of blasting

Accepted Answer

Push jobs onto a queue and drain it with a small number of workers. One steady stream of requests is faster end-to-end than a burst that trips the limit and forces every worker to sleep.

Question 20

5. Shrink what you send

Accepted Answer

Token limits hit hardest. Summarise old turns, drop boilerplate from system prompts, and cap max_tokens. Half the context is half the rate-limit pressure and half the bill.

Question 21

6. Cache the boring answers

Accepted Answer

Identical or near-identical prompts do not need a fresh call. A simple cache keyed on the prompt takes a surprising amount of traffic out of the system.

Question 22

7. Degrade, do not die

Accepted Answer

When you are still limited after the last retry, fall back: a cheaper model, a cached answer, or an honest 'we're busy, try again in a minute'. A clear message beats a spinner that never stops.

Every question
you were going to ask.

ChatGPT basics

What is ChatGPT actually doing?

Does it remember me between chats?

What is a token?

What is the difference between ChatGPT Plus and the API?

Which model should I pick?

Why does it make things up?

Is my data used for training?

Common errors, decoded

401 — Incorrect API key provided

429 — Rate limit reached

429 — You exceeded your current quota

400 — This model's maximum context length is…

400 — Invalid request / unknown parameter

500 / 503 — server error or overloaded

The reply gets cut off mid-sentence

The answer ignores half my instructions

How to handle rate limits

1. Know which limit you hit

2. Retry with exponential backoff and jitter

3. Never retry a quota error

4. Queue instead of blasting

5. Shrink what you send

6. Cache the boring answers

7. Degrade, do not die

Still curious?

Every questionyou were going to ask.

ChatGPT basics

What is ChatGPT actually doing?

Does it remember me between chats?

What is a token?

What is the difference between ChatGPT Plus and the API?

Which model should I pick?

Why does it make things up?

Is my data used for training?

Common errors, decoded

401 — Incorrect API key provided

429 — Rate limit reached

429 — You exceeded your current quota

400 — This model's maximum context length is…

400 — Invalid request / unknown parameter

500 / 503 — server error or overloaded

The reply gets cut off mid-sentence

The answer ignores half my instructions

How to handle rate limits

1. Know which limit you hit

2. Retry with exponential backoff and jitter

3. Never retry a quota error

4. Queue instead of blasting

5. Shrink what you send

6. Cache the boring answers

7. Degrade, do not die

Still curious?

Every question
you were going to ask.