Make your Credits go further with smarter context management.
Back
Have you ever found yourself deep in a productive, hours-long conversation with AI, only to notice your credit usage climbing faster than you expected? Long, continuous chats are powerful, but they come with a hidden cost. Today, we'd like to introduce a new feature designed to put you in control: the Smart Context Control. This allows you to view and manage your conversation's context window, empowering you to get the most out of every interaction while saving on credits.
In this post, we'll dive into what a "context window" is, why managing it is a game-changer, and how you can use our new feature to chat more efficiently than ever before.
Think of every AI chat as a conversation with a partner who has a very powerful, but limited, short-term memory. This "short-term memory" is what we call the context window.
Every time you send a new message, the AI doesn't just look at your latest question. To provide a relevant and coherent response, it re-reads the conversation history that fits within its context window — a limit measured in tokens.
Here’s why this matters to you:
We roll out the Smart Context Control feature—a simple, visual way to manage your conversation's context and optimize your credit usage. Here’s how it works:
You will now see a clear, intuitive meter in your chat interface that shows how much of the context window your current conversation is using.
Once the context size reaches a certain threshold where optimization would be beneficial, you'll be allowed to take two strategic options:
It gives you direct visibility and control over the conversation's memory, allowing you to balance detail with cost and performance. Knowing which to choose will make you an AI power user.
This is your go-to option when you're deep in a single, continuous task and want to reduce token load without losing the plot.
This is the best choice when you are switching to a completely new topic or unrelated task.
You may notice that the "Compact Chat" option is sometimes unavailable. This is by design — we’ve intentionally disabled it in certain situations to balance efficiency with output quality. The button is grayed out in two key scenarios:
The new Smart Context Control puts you in the driver’s seat—giving you the visibility and control to optimize every conversation for your specific needs.
By understanding when to compact, when to start fresh, you'll get faster, more focused answers while making your credits go further.
The Backpack of Your Conversation: Understanding the Context Window
Think of every AI chat as a conversation with a partner who has a very powerful, but limited, short-term memory. This "short-term memory" is what we call the context window.
Every time you send a new message, the AI doesn't just look at your latest question. To provide a relevant and coherent response, it re-reads the conversation history that fits within its context window — a limit measured in tokens.Here’s why this matters to you:
- Growing Costs: As a conversation grows, each new message requires processing more tokens, which in turn consumes more credits.
- Slower Responses: Asking the AI to carry a heavier "backpack" of conversation history can sometimes lead to a slower response.
- Potential for Drift: In long conversations, a massive context may cause the AI to lose focus on the most critical points.
Take Control: Introducing the Smart Context Control
We roll out the Smart Context Control feature—a simple, visual way to manage your conversation's context and optimize your credit usage. Here’s how it works:
Context Usage Meter
You will now see a clear, intuitive meter in your chat interface that shows how much of the context window your current conversation is using.
Quick Actions
Once the context size reaches a certain threshold where optimization would be beneficial, you'll be allowed to take two strategic options:
- "Compact Chat": This action intelligently summarizes the conversation up to that point, creating a condensed version that preserves the essential facts and core ideas. This significantly reduces the
tokencount for future messages. - "New Chat": This option gives you a clean slate, perfect for when you're switching to a completely new and unrelated topic.
Best Practices: When to Compact, When to Start Fresh
It gives you direct visibility and control over the conversation's memory, allowing you to balance detail with cost and performance. Knowing which to choose will make you an AI power user.
When should you compact your conversation?
This is your go-to option when you're deep in a single, continuous task and want to reduce token load without losing the plot.
- Example Scenario: building a single feature or component step-by-step. You've worked with the AI to define the logic, write some initial code, and are now in the process of refining and debugging it.
- What It Does: This action creates a concise summary of your entire conversation history. It’s not about abstracting a few key points; it's about summarization. The AI condenses the back-and-forth—the code snippets, the error messages, the revisions—into a dense block of context that preserves the full trajectory of the session.
- The Result: The AI retains the essential background of your task, but the token count for your next prompt is significantly lower. This helps you save credits and get slightly faster responses for the rest of your session. Note: This compaction is a "lossy" process. You are essentially trading some fine-grained details (which may be generalized or omitted) for lower cost and a longer conversation.
[!NOTE]
The Rule of Thumb: Use "Compact Chat" when you need to continue the same task or topic, but notice the context window is getting full.
When should you start a new chat?
This is the best choice when you are switching to a completely new topic or unrelated task.
- Example Scenario: You just finished a bug fix for a backend API, and now you need to build a new React component for a different requirement. The tasks are totally unrelated.
- What it does: It gives you a fresh chat with a clean context window.
- The Result: You avoid sending irrelevant context (like details from the API bug fix) when asking about the new React component, preventing token waste and potential model confusion.
[!NOTE]
The Rule of Thumb: Use "New Chat" when you are starting a completely new and unrelated task.
When should you avoid compacting?
You may notice that the "Compact Chat" option is sometimes unavailable. This is by design — we’ve intentionally disabled it in certain situations to balance efficiency with output quality. The button is grayed out in two key scenarios:
- Early in a Conversation: For short chats, token savings are minimal and not worth the risk of losing important details. The option automatically becomes available once your conversation uses over 40% of the context window, ensuring that compaction offers meaningful credit savings.
- While a Response is Being Generated: Attempting to compact the context mid-generation would compromise the ongoing response, resulting in a wasted query and ultimately costing you more credits.