My AI Assistant Started Losing Track of Time: The Session-Cache Optimization That Swallowed "Now"

I asked my always-on AI assistant what time it was, and it named a moment two and a half hours in the past as "now." The clock wasn't broken. A session-caching optimization I had added for speed had frozen the one thing that must be recomputed every time — the current timestamp — right along with everything else. This is a write-up of the trap that appears when you treat "the things that never change" and "the thing that changes every time" as one lump.

1. The Smell: The Longer the Chat, the Slower the Clock

This assistant keeps a conversation session alive for a long time. Instead of starting from scratch each time, it inherits prior context — faster responses, lower cost. But one early morning, mid-conversation, it said something like "it's already 3 a.m." The actual time was 5:47. It was calmly calling a value nearly three hours stale the "current" time.

Stranger still, the error grew with session age. Start a fresh session and the time was correct; the longer the conversation ran, the further into the past it drifted. The clock seemed stuck at the moment the session began.

2. Background: Why Keep the Session Alive

Re-sending an AI agent the full briefing (system prompt) on every question — "here is who you are, who the user is, what tools you have, and what time it is" — is expensive. It's long, identical every time, and eats tokens. So I added a "resume" optimization: keep the session alive and, from the second question on, send only what changed.

The implementation was simple: on resume calls, omit the entire system prompt (leave it empty). Identity and tool list are already alive inside the session, so there's no need to resend them. Cost and speed improved measurably.

3. Root Cause: Bundling the Unchanging with the Changing

The problem: that "one system-prompt lump" mixed two entirely different kinds of data.

• Things that never change per turn: the assistant's identity, user info, available tool list. These stay constant across the session — no need to resend.
• The thing that changes every turn: the current timestamp. This is time-varying data that must be recomputed on every question.

Both sat side by side in the same string. So the optimization "omit the entire system prompt" — meant to save on identity — threw away the current time along with it. On resume, no fresh timestamp reached the assistant from the second question on, so it kept referencing the stale time baked in when the session first started. The longer the session lived, the wider the gap between that time and reality.

And this was more than a verbal slip. The assistant had features whose decisions hinge on the time (is it working hours? is the market open?), and all of them were reasoning from a poisoned clock. A wrong time could bleed into a wrong decision.

4. The Fix: Give Time-Varying Data Its Own Channel

The solution wasn't to undo the optimization but to split the lump.

• Separate the time block. I extracted just the timestamp computation from the system prompt into its own function.
• Deliver it through a dedicated channel every turn. On resume calls, the full system prompt is still omitted — but in its place, only the time block is freshly computed and passed each turn. Identity and tool list remain omitted, so the cost savings stay intact. Only the timestamp flows fresh, every turn.

I tested just the isolated logic, then deployed. The time error vanished. The optimization's benefit was preserved; only the freshness of the time-varying data was recovered.

5. Lessons

• Before caching, ask "is this data that changes?" The essence of the optimization is "don't redo what doesn't change." If something that does change is mixed in, the optimization freezes that change too.
• Design immutable and time-varying data on separate channels from the start. The moment you fuse "identity + current time" into one string, every optimization that touches one risks corrupting the other. Different natures deserve different containers.
• Slowly growing errors are the last to be caught. Had the time been completely wrong from the start, I'd have noticed immediately. Because it was correct early and drifted with age, it was easy to shrug off as "occasionally weird." Bugs that diverge gradually are only caught when you pin the exact reproduction condition — here, an aged session.
• A seemingly trivial timestamp seeps into decisions. Get "what time is it" wrong and it's not just a slip — every time-dependent judgment goes wrong with it. The accuracy of time-varying data is the accuracy of every downstream piece of logic that consumes it.

FAQ

Q. Couldn't you just resend the full system prompt every time?

That avoids this bug but discards the whole speed-and-cost gain of keeping the session alive. Resending the long identity and tool list every turn was costly enough to justify the optimization in the first place; the right fix wasn't to abandon it but to carve out the time-varying data as an exception.

Q. How did you become certain of the root cause?

I opened the actual conversation logs and reproduced it. Cross-referencing "which session started when, at what real time, and what time it claimed" showed the error exactly matched the elapsed time since the session began. You have to reproduce the symptom with your eyes and match it with numbers to turn "probably this" into "this."

Q. How do I watch for the same trap elsewhere?

In every bundle you cache or delta-send, check whether values that change each time — time, location, exchange rate, balance — are mixed in. If so, pull just those out of the cache and flow them through a separate path. Separating static from dynamic data is the first button of any caching design.