Tag
#evaluation
Reports tagged evaluation — page 2 of 3.
Reports
22 reportsSep 2026
09-13 agent-harness When can an AI agent say the task is complete? Completion is a judgment about the requested result—not a synonym for stopping, passing a check, or sounding confident. 09-11 agent-memory How to make AI memory corrections and forgetting last A memory change should alter later answers and actions—not just the message that acknowledges it. 09-10 agent-memory When Should an AI Use Its Memory? Remembering is not permission to personalize. An AI should let the past shape an answer only when it helps the present task, still applies, and belongs in this interaction. 09-08 agent-memory What Should an AI System Remember? Persistent AI should preserve useful continuity without turning yesterday’s conversation into today’s rule. That requires deciding what to retain—and when not to use it. 09-07 model-routing Large Model, Small Model, or Both? The choice is never about how hard the task looks. It is an estimate of how well a candidate model will do, a threshold you own, and a bill you have to measure yourself. 09-07 agent-harness Can a Harness Make a Small Model Match a Large One? A harness is a bigger lever than most engineers assume, and the weaker model is the more exposed to it. Whether a small model can substitute is decided by the task, not the scaffolding. Aug 2026
08-28 finance From Language to Incremental Edge: Engineering LLM Semantics into Quantitative Models An LLM becomes useful to quantitative finance only when its semantic measurements improve a strong point-in-time numerical forecast, remain calibrated out of sample, and survive realistic trading costs. 08-26 finance The LLM Edge in Finance Is a Division of Labor Language models can widen a financial system's field of view. That does not make them universal forecasters, portfolio managers, or sources of alpha. 08-25 fine-tuning Can This Checkpoint Still Learn? Why retention and future learnability need separate tests in repeatedly trained neural networks. It frames a three-arm choice: continue, reset training state, or retrain from scratch. 08-20 long-context What Stops Working Before the Million-Token Window Runs Out The million-token window is a serving and pricing limit, not a measurement: literal lookup held in the one pilot to test a full million; nearly everything harder yet measured degrades far earlier.