The SPIRIT consortium is indeed thrilled in announcing that its technology partners have managed to squeeze yet an additional evaluation cycle between Year 2 and Year 3 SPIRIT tools prototype environments. The community of end users have already received version Y2b (alias v2.5). This is an intermediate step before delivering the final system backbone. This decision came to resonate with a three months project extension that has been requested (final approval is imminent) to challenge whether we could have covid-19 free ‘showcasing events’.
Following the delivery of:
This M30 Year 2.5 Prototype extends Year 2 functionalities and features an improved end users experience. Moreover v2.5 strives and corrects performance efficiency issues, workflow technical hiccups and several end users’ driven operational requests.
In March 2021, SPIRIT technical partners delivered to the LEAs community (i.e. Hellenic Border Police GR, Thames Valley Police UK, West Midlands Police UK and STAD Antwerp Police BE) a working prototype that indeed aspires to quality TRL6 specifications.
Our next steps would be to upgrade the system with a final touch of Machine Learning Identity Resolution algorithms as part of a set of tools aimed to provide consultation to the LEA analysts..
Stay in touch as we tread towards the end of an exciting three years journey!!
It is intended that the Refined Search is to be used when the investigator has already gained some knowledge about the target person. The actual search is undertaken in two stages:
1. Level 1; this search returns a list of web pages that you would expect to see from a Google search. The investigator should read the brief overview on the page and decide whether to include the page. If unsure, the page link can be clicked to open the web page in a new tab and view its content.
2. Level 2; this search uses all of the selected pages from the Level 1 search and uses any page links found to return another list of pages. The investigator goes through the same selection process as discussed above to select the required entries.
These two processes can take some time to complete but it is important that the investigator only selects the pages that are relevant to reduce the non-essential nodes that will be displayed in the graph.
Once these two processes have been completed, all of the web pages will have the main text scraped and processed to label words and phrases (entities) that are relevant to an investigator. The entities are then linked based on a set of rules that have been devised in-house to produce a Relationship List that can be displayed through the UI:
The investigator can validate each entry be clicking on the document url and view the source web page. On looking and assessing the graphical output, should the investigator feel that there is a possibility that two or more of the persons be the same person, the Relationship List is used by the Identity Resolution processes to make that assessment.
The results of the refined search are displayed in a graph
The entities and their links can be manipulated as illustrated above.
All manipulations are recorded in the database and added to the Relationship List for further analysis if needed.
There is a “Development” mode within these processes which is linked to a set of fake web pages and can be used to demonstrate the functionality to a live audience.
The investigator can validate each entry be clicking on the document url and view the source web page. On looking and assessing the graphical output, should the investigator feel that there is a possibility that two or more of the persons be the same person, the Relationship List is used by the Identity Resolution processes to make the SPIRIT generates results by orchestrating microservices according to proper chains of execution: SPIRIT services do not usually run in parallel since services may ‘consume’ information ‘produced’ by other services. For example, the Natural Language Processing service waits for contents extracted by the Scraper service from Web pages which, in turn, are downloaded by the Crawler service.
The Progress Monitoring Service tracks the status of a running investigation as well as of its corresponding execution chain. As soon as an investigation starts, each SPIRIT microservice communicates the status of its activities by notifying the Progress Monitoring Service about the number of consumed and produced items (such as Web pages, texts, images or videos). Messages are managed to monitor the number of consumed and produced items for each service as well as to infer the status of each service: the service status can be (i) running if the service is sending keep alive messages or (ii) completed if the service is no more communicating with the Progressing service. Consequently, a scheduled investigation is completed if all services in its execution chain are no more running.
The Progress Monitoring Service is a microservice written in Typescript by the Lutech Team. All SPIRIT service were integrated by adopting the architectural pattern adopted in SPIRIT where: (i) other Microservices communicate with the Progress Monitoring Service by sending messages over RabbitMQ and (ii) a REST API is used to allow the UI to collect information about investigations and microservices.
- Article publication of SPIRIT project in EU research magazine.
See here the original Newsletter #3
This is another very exciting SPIRIT Newsletter indeed! We are treading our mid-summer steps towards shoring our three years project safely and effectively by the end of October 2021. We have chosen to pass the floor to our hands-on intelligence investigators who will harness SPIRIT’s outcomes beyond the life of this Horizon 2020 contract. To that end we spoke to Police Captain Theodoros Gylos from the Border Protection Division of Hellenic Police, about the challenges that Law Enforcement Agencies are facing in the fight against organised crime and terrorism and how the SPIRIT tools can help address these challenges.
Matching the above Interview, we have chosen to shed light to issues that relate to how SPIRIT may provide law enforcement agencies tools and methodology skills towards supporting the fight against organised crime.
Today, law enforcement is overcoming the deficiencies of traditional data mining by turning to entity/identity resolution technology. This technology could work through SPIRIT to:
- Integrate records from many sources
- Resolve conflicts between records
- Correct errors and complete missing field
With identity matching capabilities, law enforcement agencies can identify and match individuals, groups, events, and other critical data accurately and quickly across systems, regardless of language, structure, format, location, duplication, omissions, or errors. Identity matching enables law enforcement to gather, correlate, match, and share disparate data in ways not previously possible; this is what our partners London Metropolitan University with the assistance of AES Solutions UK and the overall SPIRIT ‘red thread’ technology team are describing in this Newsletter. Enjoy!
See you soon in our next Newsletter! Have a wonderful safe Summer and always keep an eye to the Greek Letters (especially Delta ? )
LAW ENFORCEMENT AGENCIES VIEW ON ICT TOOLS ΤΟ SUPPORT THE FIGHT AGAINST ORGANISED CRIME
Interview with Theodoros Gylos from the Hellenic Police
What are the main threats organised crime and terrorist groups pose to the International security?
In our days, transnational organized crime and terrorism undermine and international security, posing significant threats for public safety, public health, democratic institutions, and economic stability globally.
Specifically, organized criminal groups manage to penetrate the state apparatus, intensifying corruption, influencing political areas, undermining the rule of law, free press, subverting the principals of the judicial systems and transparency.
Furthermore, organized crime is a threat to democracy and social cohesion, raising concerns about state security, citizens’ security and conformity on government institutions.
Finally, organized crime represents a significant threat to economic growth and stability, through its subversion, exploitation, and distortion of legitimate markets and economic activity.
How SPIRIT may provide law enforcement agencies tools and methodology skills towards supporting the fight against organised crime?
Modernization of crime methods and techniques and invention of new modi operandi, used by the criminal organized groups, makes even more necessary the confrontation of the organized crime and the dismantling of organized criminal groups. Therefore, Law Enforcement Authorities must make use of technological innovation and develop new investigative measure to counter the threat of organized crime.
Research Innovation presented in SPIRIT tools are being guided and validated by a set of relevant use cases (scenarios) proposed and monitored by partner Law Enforcement Agencies; what may be highlighted as added value in the light of the deployed use cases?
Presenting an analysis of a case study drawn from a real life setting, provides the opportunity to check tools operability, propose improvements and help Law Enforcement Authorities to become more effective and better counter criminal threats emerging from serious and organizes crime.
Copping with big volume of data and diversity of information, makes data quality assessment essential. In real world data through scenarios inconsistencies and errors may be revealed and test data quality.
SPIRIT has pioneered a focused activity relating to ethics and privacy preservation. What would you consider as most important elements relating to ethics and personal data privacy (e.g. GDPR)? May we specify criteria for specific use cases, to enable intelligence investigation jobs whilst maintaining privacy? In SPIRIT by connecting entities to data spaces, how would you assess privacy preservation?
Intelligence investigation may reveal various links among entities, generating valuable data for the Law Enforcement Authorities, but also raising at the same time privacy concerns.
Firstly, law provisions (e.g. Data Privacy Principle) ensure data protection when accessing, using and collecting information. Moreover, on a technical level, safeguarding data from unauthorized or accidental access or loss, may ensure data privacy.
Randomization, k-anonymity and distributed privacy preserving, are proposed techniques, ensuring privacy-preserving data.
Towards a SPIRIT toolkit for Resolving Identities
By the SPIRIT ‘red thread’ technology team
An Identity Resolution toolkit, developed in SPIRIT, can help Law Enforcement Agencies (LEAs) to fill gaps and make ends meet in the sleuth process of addressing diverse organized crime threats. This service alleviates LEAs to identify and match individuals, groups, events, and other critical data accurately and quickly across systems, regardless of language, structure, format, location, duplication, omissions, or errors. Identity matching enables law enforcement to gather, correlate, match, and share disparate data in ways not previously possible.
This Identity Resolution service is derived from the SPIRIT prototype toolkit which provides the ability to recognize an entity, be it a person, place, or thing, along with associated relationships, consistently and accurately based on a Relationship List which is coming from a refined keyword search, where the LEA investigator exploits already existing overt and covert knowledge regarding candidate perpetrators.
In this job, Identity models and behavior patterns are being developed to enable the capability to resolve identities. This is accomplished through machine learning techniques that assess individuals, their roles, skills, communication channels among other elements. A SPIRIT designed identity resolution method enables clear connections and marking of possible single person duplicate identities, for the candidate perpetrators. This is exhibited in the SPIRIT prototype with links between people and entities while using relationship lists which are based on results coming out from another SPIRIT toolkit service, the Refined Search service.
After completion of the identity resolution service, a list of suspected similar identities is produced:
These entities are then linked, based on machine learning techniques, to produce a new relationship list linking suspected similar identities to be displayed through the SPIRIT presentation interface (UI):
The overall results of an Identity resolution service job are displayed in a graph, providing a holistic view of all identity data, across touchpoints, with the goal of referencing points accurately for up-to-date perpetrator identities to be determined (see nodes "Brooks" and "Mariett Snehh"):
See here the original Newsletter #4