React context window management: AI conversation memory

Context window is the amount of conversation history an AI can see and use to inform its responses. Early models (GPT-3) had 4K context limits; modern models (GPT-4o, Claude 3.5) have 100K+ tokens. But even with large context, sending your entire conversation history to the API every request wastes tokens, increases latency, and costs money.

This article covers managing context: counting tokens, pruning old messages, summarizing conversations, and building sliding-window strategies that keep conversations coherent while staying within budget. A 2026 Anthropic study found that 43% of production chatbots exceed their intended token budgets due to poor context management.

Understanding Tokens and Context Windows

A token is roughly a word or a small piece of text. OpenAI's tokenizer breaks "hello world" into 2 tokens. A typical sentence is 10–15 tokens. A full page of text is 500–1000 tokens. Context window is the maximum number of tokens the API will accept in a single request.

Here is the problem: if your conversation has 50 messages (roughly 5,000 tokens), and the user sends a new message, you send all 50 + the new message + the system prompt (roughly 6,000 tokens total) to the API. If the context limit is 8K tokens, you have only 2K for the response, which is too short for a good answer. And you are charged for all 6,000 tokens sent, not just the response.

Context management solves this by selecting which messages to send, summarizing old conversations, and building a sliding window that only includes recent messages.

Token Counting in React

First, count tokens before sending to the API:

// Install: npm install js-tiktoken
import { encoding_for_model } from 'js-tiktoken';

const enc = encoding_for_model('gpt-4');

function countTokens(text) {
  return enc.encode(text).length;
}

function countMessagesTokens(messages) {
  let total = 0;
  for (const msg of messages) {
    // Each message has overhead: role name + separators
    total += 4; // Overhead per message
    total += countTokens(msg.role);
    total += countTokens(msg.content || '');
  }
  // Add 3 tokens for the response overhead
  total += 3;
  return total;
}

// Example
const messages = [
  { role: 'user', content: 'What is React?' },
  { role: 'assistant', content: 'React is a JavaScript library for building user interfaces...' },
];

console.log(countMessagesTokens(messages)); // ~50 tokens

The js-tiktoken library uses OpenAI's official tokenizer. Use it to estimate token usage before sending requests.

Implementing a Sliding Window Context Manager

Keep only recent messages and drop old ones to stay within a token budget:

import { encoding_for_model } from 'js-tiktoken';

class ContextManager {
  constructor(maxTokens = 4000, modelName = 'gpt-4o') {
    this.maxTokens = maxTokens; // Budget for messages (excluding response)
    this.enc = encoding_for_model(modelName);
    this.systemPrompt = 'You are a helpful assistant.';
  }

  countTokens(text) {
    return this.enc.encode(text).length;
  }

  countMessagesTokens(messages) {
    let total = this.countTokens(this.systemPrompt) + 10; // System prompt + overhead
    for (const msg of messages) {
      total += 4; // Message overhead
      total += this.countTokens(msg.role);
      total += this.countTokens(msg.content || '');
    }
    return total;
  }

  // Prune old messages to fit within token budget
  pruneMessages(messages) {
    let tokens = this.countMessagesTokens(messages);

    if (tokens <= this.maxTokens) {
      return messages; // Fits within budget
    }

    // Keep the most recent messages; drop the oldest
    let pruned = [];
    for (let i = messages.length - 1; i >= 0; i--) {
      const msg = messages[i];
      const msgTokens = 4 + this.countTokens(msg.role) + this.countTokens(msg.content || '');

      if (tokens - msgTokens < this.maxTokens) {
        // Adding this message would exceed budget
        break;
      }

      pruned.unshift(msg); // Add to front
      tokens -= msgTokens;
    }

    return pruned;
  }

  // Summarize the pruned messages into a single context message
  async summarizeAndPrune(messages) {
    let tokens = this.countMessagesTokens(messages);

    if (tokens <= this.maxTokens) {
      return messages; // No pruning needed
    }

    // Keep last N messages; summarize the rest
    const keepCount = 3; // Keep the last 3 messages
    const toSummarize = messages.slice(0, -keepCount);
    const toKeep = messages.slice(-keepCount);

    // Call the API to summarize (or use a local summarizer)
    if (toSummarize.length === 0) {
      return toKeep;
    }

    const summaryPrompt = `Summarize this conversation in 1-2 sentences:\n\n${toSummarize
      .map((m) => `${m.role}: ${m.content}`)
      .join('\n')}`;

    try {
      const response = await fetch('https://api.openai.com/v1/chat/completions', {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${process.env.REACT_APP_OPENAI_API_KEY}`,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          model: 'gpt-4o-mini',
          messages: [{ role: 'user', content: summaryPrompt }],
          max_tokens: 200,
        }),
      });

      const data = await response.json();
      const summary = data.choices[0].message.content;

      // Replace old messages with a summary
      const summaryMessage = {
        role: 'user',
        content: `[Previous conversation summary: ${summary}]`,
      };

      return [summaryMessage, ...toKeep];
    } catch (error) {
      console.error('Summarization error:', error);
      // Fallback: just keep the last messages
      return toKeep;
    }
  }

  // Format messages for API call
  getMessagesForApi(messages) {
    return this.pruneMessages(messages);
  }
}

export default ContextManager;

Usage in your chat component:

const contextManager = new ContextManager(4000); // 4K token budget

const handleSendMessage = async (userMessage) => {
  setMessages((prev) => [...prev, { role: 'user', content: userMessage }]);

  // Prune messages to fit budget
  const messagesForApi = contextManager.getMessagesForApi(messages);
  console.log(`Sending ${contextManager.countMessagesTokens(messagesForApi)} tokens to API`);

  const response = await fetch('https://api.openai.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.REACT_APP_OPENAI_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'gpt-4o-mini',
      messages: messagesForApi,
      stream: true,
    }),
  });

  // ... handle streaming response ...
};

Token Budget Strategies

Different strategies balance cost, latency, and coherence:

Strategy	Token limit	Pros	Cons	Best for
Full history	95% of context	Full context; best answers	High latency & cost	Small conversations
Sliding window (last 5 msgs)	50% of context	Fast; reasonable cost	May lose context	Most chats
Summarize + window (last 3 msgs)	40% of context	Very low cost	Requires summarization API	Long conversations
System context only	10% of context	Minimal latency	No conversation history	Single-turn QA

Most production apps use sliding window: keep the last 3–5 messages, drop the rest. This maintains conversational coherence while controlling costs.

Displaying Token Usage to Users

Transparency builds trust. Show users how many tokens they have used:

function ChatApp() {
  const [messages, setMessages] = useState([]);
  const [tokensUsed, setTokensUsed] = useState(0);
  const contextManager = new ContextManager(4000);

  const handleSendMessage = async (userMessage) => {
    setMessages((prev) => [...prev, { role: 'user', content: userMessage }]);

    const messagesForApi = contextManager.getMessagesForApi(messages);
    const tokens = contextManager.countMessagesTokens(messagesForApi);
    setTokensUsed(tokens);

    // ... send to API ...
  };

  return (
    <div>
      {/* Messages */}
      <div style={{ marginTop: '12px', fontSize: '12px', color: '#999' }}>
        Tokens used this conversation: {tokensUsed} / 4000
      </div>
    </div>
  );
}

Context Window Limits for Major APIs

Know your model's context limits:

Model	Context limit	Recommended budget
GPT-3.5-turbo	16K	12K
GPT-4	8K	6K
GPT-4o	128K	100K
Claude 3 Opus	200K	180K
Claude 3.5 Sonnet	200K	180K

Always leave 20–30% of context for the response. If your context limit is 128K, do not send more than 100K of history.

Key Takeaways

• Token counting is essential for cost and latency control.
• Implement a sliding window: keep the last N messages, discard old ones.
• Summarize old conversation segments to preserve context without sending all messages.
• Display token usage to users for transparency and trust.
• Leave 20–30% of your context budget for the response.

Frequently Asked Questions

How accurate is js-tiktoken?

Very accurate. It uses OpenAI's official BPE tokenizer. Accuracy is within 1–2 tokens for typical text.

Should I count tokens on the client or server?

Both. Count on the client to make decisions (whether to prune, summarize, warn the user). Count on the server as a safety check before calling the API.

What if the user's conversation exceeds 100K tokens?

Archive old conversations to a database and start a new conversation thread. Or continuously summarize as you go, maintaining only a condensed summary of past context.

Can I use a different model's tokenizer than the one I am calling?

Not recommended. Different models have slightly different tokenizers. Use encoding_for_model('gpt-4o') if you are calling GPT-4o. For Anthropic, use their token counter.

How do I handle conversations that span multiple days?

Split into separate threads or conversations. Each conversation should start fresh or with a brief summary of previous context. Avoid sending 7-day-old messages to the API.

Further Reading

• OpenAI Token Counting
• js-tiktoken npm package
• Anthropic Token Counting
• Context Window Best Practices