Newsletter #3 - SPIRIT PROTOTYPE 2.5

The SPIRIT consortium is indeed thrilled in announcing that its technology partners have managed to squeeze yet an additional evaluation cycle between Year 2 and Year 3 SPIRIT tools prototype environments. The community of end users have already received version Y2b (alias v2.5). This is an intermediate step before delivering the final system backbone. This decision came to resonate with a three months project extension that has been requested (final approval is imminent) to challenge whether we could have covid-19 free ‘showcasing events’.

Following the delivery of:

This M30 Year 2.5 Prototype extends Year 2 functionalities and features an improved end users experience. Moreover v2.5 strives and corrects performance efficiency issues, workflow technical hiccups and several end users’ driven operational requests.

In March 2021, SPIRIT technical partners delivered to the LEAs community (i.e. Hellenic Border Police GR, Thames Valley Police UK, West Midlands Police UK and STAD Antwerp Police BE) a working prototype that indeed aspires to quality TRL6 specifications.

Our next steps would be to upgrade the system with a final touch of Machine Learning Identity Resolution algorithms as part of a set of tools aimed to provide consultation to the LEA analysts..

Stay in touch as we tread towards the end of an exciting three years journey!!

Costas Davarakis,

SPIRIT Technical Coordinator (This email address is being protected from spambots. You need JavaScript enabled to view it.)

Refined Search

It is intended that the Refined Search is to be used when the investigator has already gained some knowledge about the target person. The actual search is undertaken in two stages:

1. Level 1; this search returns a list of web pages that you would expect to see from a Google search. The investigator should read the brief overview on the page and decide whether to include the page. If unsure, the page link can be clicked to open the web page in a new tab and view its content.

2. Level 2; this search uses all of the selected pages from the Level 1 search and uses any page links found to return another list of pages. The investigator goes through the same selection process as discussed above to select the required entries.

These two processes can take some time to complete but it is important that the investigator only selects the pages that are relevant to reduce the non-essential nodes that will be displayed in the graph. Once these two processes have been completed, all of the web pages will have the main text scraped and processed to label words and phrases (entities) that are relevant to an investigator. The entities are then linked based on a set of rules that have been devised in-house to produce a Relationship List that can be displayed through the UI:

The investigator can validate each entry be clicking on the document url and view the source web page. On looking and assessing the graphical output, should the investigator feel that there is a possibility that two or more of the persons be the same person, the Relationship List is used by the Identity Resolution processes to make that assessment.

The results of the refined search are displayed in a graph

The entities and their links can be manipulated as illustrated above. All manipulations are recorded in the database and added to the Relationship List for further analysis if needed. There is a “Development” mode within these processes which is linked to a set of fake web pages and can be used to demonstrate the functionality to a live audience.

Progress Monitoring

The investigator can validate each entry be clicking on the document url and view the source web page. On looking and assessing the graphical output, should the investigator feel that there is a possibility that two or more of the persons be the same person, the Relationship List is used by the Identity Resolution processes to make the SPIRIT generates results by orchestrating microservices according to proper chains of execution: SPIRIT services do not usually run in parallel since services may ‘consume’ information ‘produced’ by other services. For example, the Natural Language Processing service waits for contents extracted by the Scraper service from Web pages which, in turn, are downloaded by the Crawler service.

The Progress Monitoring Service tracks the status of a running investigation as well as of its corresponding execution chain. As soon as an investigation starts, each SPIRIT microservice communicates the status of its activities by notifying the Progress Monitoring Service about the number of consumed and produced items (such as Web pages, texts, images or videos). Messages are managed to monitor the number of consumed and produced items for each service as well as to infer the status of each service: the service status can be (i) running if the service is sending keep alive messages or (ii) completed if the service is no more communicating with the Progressing service. Consequently, a scheduled investigation is completed if all services in its execution chain are no more running.

The Progress Monitoring Service is a microservice written in Typescript by the Lutech Team. All SPIRIT service were integrated by adopting the architectural pattern adopted in SPIRIT where: (i) other Microservices communicate with the Progress Monitoring Service by sending messages over RabbitMQ and (ii) a REST API is used to allow the UI to collect information about investigations and microservices.

Dissemination events:

- Article publication of SPIRIT project in EU research magazine.