~/writing/math-foundations-for-reasoning-models $ ls
Reasoning Models as Representation Transformation
Token Sequence as Input
A reasoning model receives a token sequence, which becomes the structured input for all later transformations.
~/writing/math-foundations-for-reasoning-models $ cat topic.md
Token Sequence As Input
Definition
A token sequence is the ordered list of tokens given to the model.
Simple form:
prompt text -> tokens -> token sequence
The model does not receive the prompt as raw human meaning.
It receives a structured sequence of token IDs and embeddings.
The token sequence is the model's starting input structure.
Why Order Matters
The same tokens in a different order can produce a different task.
Example:
dog bites man
man bites dog
The tokens are similar.
The sequence structure is different.
Reasoning starts from ordered input, not isolated tokens.
What The Sequence Can Contain
A token sequence may include:
instructions
question
context
examples
constraints
tool results
previous conversation
All of these become part of the input state the model can use.
The model must transform this input into useful internal representations.
The prompt is a structured input environment.
Context Window
The model can only directly use tokens inside its available context.
Simple idea:
inside context -> available to the model
outside context -> not directly represented
This matters because reasoning depends on what information is present.
Missing context can produce weak or wrong behavior.
A reasoning model is constrained by the information available in its input context.
Input Is Not Understanding
Providing tokens does not mean the model understands them in a human way.
The tokens become representations.
Those representations are transformed by learned computation.
token sequence -> representation transformation
Input is the start of computation, not proof of understanding.
Final Summary
- A token sequence is the model's ordered input.
- Order changes structure.
- The sequence may contain instructions, context, constraints, and examples.
- The model is limited by what is inside its context.
- Reasoning begins by transforming token sequences into representations.
~/writing/math-foundations-for-reasoning-models $ open
Continue the series.
Intermediate Representation