Many questions depend on a date. Who held a role in a given year? Which event came after another? In large companies, roles, contracts and rules all have start and end dates. A system that ignores time can give an answer that was once right and is now wrong.
The problem
A temporal knowledge graph stores facts together with the period in which they were true. Answering a question over one often takes several steps: find a fact, use its date, then find another fact.
Existing embedding-based methods struggle with these questions because they reason in a single pass. The paper also addresses two other gaps: telling which entity is the subject of a fact and which is the object, and reading both in the context of the question.
The method
SABET-QA works through a question in steps instead of one pass. At each step it scores candidate entities and dates in both directions, then updates a working memory that carries what it found into the next step. It also reads the subject, the object and the time against the wording of the question.
We chose four steps as the best compromise between accuracy and computing cost.
The results
We tested the method on four public benchmarks. The table shows Hits@1, the share of questions where the first answer is correct, from 0 to 1. “With time hints” means the system also receives the time boundaries of the question.
| Benchmark | SABET-QA | SABET-QA with time hints | Best method compared |
|---|---|---|---|
| CronQuestions | 0.843 | 0.954 | 0.914 |
| Complex-CronQuestions | 0.524 | 0.807 | 0.632 |
| MultiTQ | 0.373 | 0.403 | 0.337 |
| TimeQuestions | 0.502 | 0.504 | 0.419 |
With time hints, SABET-QA scores highest on all four benchmarks. The largest gap is on Complex-CronQuestions, 0.807 against 0.632. Without them, it leads on MultiTQ and TimeQuestions and falls behind on the two CronQuestions benchmarks.
The limits
- The method relies on pretrained components that stay fixed during training, which may limit how well it adapts to new domains.
- It loses accuracy when entities and dates are missing or unreliable, and extraction errors carry through to the answer.
- Working in steps adds complexity and may increase computing cost.
- This is a preprint and has not yet been peer reviewed. The tests use public benchmarks, not company data.
The authors are Brahim Touayouch, Mirette Moawad and Dmitry Akulov. The full method and all results are in the paper.
Read on arXiv↗