A local API and the hosted endpoints and your first script

The concept

A local runner exposes an OpenAI-compatible endpoint on your own machine, and the same weights are served by many managed providers, so code written for one usually moves to the other. The idea is simple: the same model you have been chatting with also answers a web request, so you can put it inside a script, a spreadsheet or a page. A key identifies you; a request sends the text; a response comes back as data.

The reason to try it once, even if you never build anything: it makes the chat version less mysterious. You see that the whole conversation is text in and text out, that your instructions are literally lines of a request, and that "the model" is one parameter among several.

A first call looks like this:

import requests

r = requests.post(
    "http://localhost:11434/api/generate",
    json={"model": "llama3.2:3b", "prompt": "Explain a database index in two sentences.", "stream": False},
)

print(r.json()["response"])