A new benchmark asks whether enterprise agents can find facts hidden across records

A new research benchmark tests agents on company information that is distributed across records rather than stated in one obvious place. Its early results are a warning about synthesis under ambiguity, not a definitive scorecard for every workplace agent.

Enterprise software often stores related facts in separate places: a customer record, a project note, a contract, an email thread. An agent asked a seemingly simple question may need to locate the right records, connect them and distinguish similar entities. A September 24 paper introduces a benchmark designed to test that problem, which its authors call “hidden knowledge.”

The paper on arXiv, “Era by Eon: Benchmarking Enterprise Agents on Hidden Knowledge,” describes tasks built from synthetic organizational data. The challenge is not simply retrieving a matching phrase. It is answering questions whose evidence is implicit across multiple records, including cases where several people or projects look alike.

The paper reports a sharp drop on its hidden-knowledge tasks: the best agent answered 18 of 24 questions correctly on one difficult set. On the hardest similar-record questions, across 84 attempts, only one attempt got both questions right. These figures describe the benchmark’s tested systems and task setup; they are not measurements of every deployed company agent. Those figures describe the benchmark’s tested systems and task setup; they are not measurements of every deployed company agent.

That caveat matters. Synthetic environments allow researchers to control the records and know the correct answer, making systematic evaluation possible. They also simplify many features of real organizations: messy permissions, changing information, undocumented exceptions and the social context employees use to resolve ambiguity. The benchmark is best read as a focused test of whether agents can reconcile scattered facts under the conditions the authors created.

The practical lesson is still relevant for buyers. A fluent answer may hide a weak chain of retrieval and reasoning. Enterprise evaluations should include questions where the answer depends on several records, where near-duplicate entities compete, and where the system should abstain because the evidence does not settle the question. Audit trails should let a reviewer trace the answer back to each source record.

The paper does not establish that workplace agents cannot handle hidden knowledge. It identifies a failure mode and offers a way to measure progress on it. Independent replications, comparison across models and broader datasets will show whether the gap persists outside this benchmark.

For now, the benchmark helps move the discussion from “the agent can search our files” to a more demanding question: can it assemble the right facts, preserve their relationships and admit when the records do not justify an answer?