On-chain AI

The documentation describes a canister that exposes model inference to other canisters. A caller sends a request describing the model, the prompt and the generation parameters, and receives a response containing the generated text. The interface is a canister interface, so a caller reaches it the same way it reaches any other canister: through a Candid call.

The documentation describes two styles of request. One is a completion-style interface, where the caller supplies a prompt and the service returns generated text. The other is a chat-style interface, where the caller supplies a sequence of messages with roles and the service returns the next message. Both are ordinary canister methods, and both are subject to the network's message limits.

The documentation notes the practical constraints. Generation is bounded in the number of tokens produced, and a request that asks for more than the bound allows is truncated. The service is a shared resource, so a canister that depends on it is depending on something it does not control. And the model behind the service is chosen by the service, not by the caller, which means a canister cannot pin itself to a particular model version.

The random beacon is the network's source of randomness that all nodes agree on. The documentation describes it as produced by the consensus protocol and available to canisters, and it is what makes a deterministic sampling procedure possible: if every node seeds its sampling from the same beacon value, every node draws the same sample.

The consequence is that the same request produces the same output across the subnet, which is what allows the result to be stored in the canister's state. It also means the output is not random in the sense a user might expect from a chat interface: the same prompt with the same parameters yields the same completion, because the randomness is fixed by the beacon rather than drawn fresh.

The documentation is careful about the security implication. A canister that uses model output to make a decision is trusting the model and the service, and the determinism does not make the output correct — it makes it agreed. A model that produces a wrong answer produces the same wrong answer on every node, and the network's consensus does not detect it.

The documentation presents on-chain inference as the piece that lets an application use a model without an off-chain service in the loop. A canister can classify an input, summarise a document, generate a response, or extract structured data, and the result can be stored in the canister's state as agreed data rather than as a cached response from a provider.

The applications that follow from that are the ones where the model output is part of the application's logic rather than a convenience. A governance system that summarises proposals, a marketplace that classifies listings, an assistant whose answers are part of a canister's state — these are cases where the determinism matters, because the output is something the network has to agree on.

The honest limits are the ones the documentation states. The service is shared and its capacity is finite; the output is bounded; the model is not chosen by the caller; and the cost is paid in cycles by the calling canister, which puts it under the reverse-gas model like everything else. A canister that calls a model on every request is a canister with a real and variable burn rate.