Intelligence analysts are typically required to process large volumes of data in a timely manner in order to extract useful information and detect potential security threats. This task relies on consistent judgements by the analyst in order to efficiently process the data and effectively identify useful information.

Research has shown that analysts’ judgements can be inconsistent due to the mass of data, the variation in types and nature of intelligence information and the time pressures the analyst is operating with. Consequently, intelligence analysts may take decisions that deviate significantly from those of their peers, from their own prior decisions, and from training rules that they themselves aim to follow. Such inconsistency is mainly due to two types of errors; noise and bias, which complicate the intelligence analysis process and can result in key pieces of data being misclassified or overlooked with potential security threat implications.

The project will develop, train and evaluate an innovative analytic approach to address these errors and enable analysts to achieve better judgements about the value of elicited information from intelligence reports.

Specifically, the project aims to address the following research questions:

  • How much does individual analyst bias affect the quality of the decisions?
  • Will incorporation of group decision support, as opposed to individual support, improve the quality of decisions?
  • Do additional facilities of feedback for consistency and sensitivity analysis provide support for better decision-making?

Project resources

Analysis of noise and bias errors in intelligence information systems

An intelligence information system (IIS) is a particular kind of information systems (IS) devoted to the analysis of intelligence relevant to national security. Professional and military intelligence analysts play a key role in this, but their judgments can be inconsistent, mainly due to noise and bias. The team-oriented aspects of the intelligence analysis process complicates the situation further. To enable analysts to achieve better judgments, the authors designed, implemented, and validated an innovative IIS for analyzing UK Military Signals Intelligence (SIGINT) data. The developed tool, the Team Information Decision Engine (TIDE), relies on an innovative preference learning method along with an aggregation procedure that permits combining scores by individual analysts into aggregated scores. This paper reports on a series of validation trials in which the performance of individual and team-oriented analysts was accessed with respect to their effectiveness and efficiency. Results show that the use of the developed tool enhanced the effectiveness and efficiency of intelligence analysis process at both individual and team levels.

(From the journal abstract)


Labib, A., Chakhar, S., Hope, L., Shimell, J., & Malinowski, M. (2022). Analysis of noise and bias errors in intelligence information systems. Journal of the Association for Information Science and Technology, 73(12), 1755–1775. https://doi.org/10.1002/asi.24707

Authors: Ashraf Labib, Salem Chakhar, Lorraine Hope
https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24707

Subscribe to the CREST newsletter.

Get the latest news, events and research into security threats delivered directly to your inbox.
Sign up now
Back to top