This paper reports the activities done and the results obtained by the University of Bari within the COST Action DigForASP. The objectives of this project are to create a cooperation network for exploring the potential of logic-based Artificial Intelligence (AI) applications in the Digital Forensics field. Specifically, the challenges of our work were to develop a declarative AI approach, based on Answer Set Programming, to call pattern analysis, for extracting useful information or suspects “lifestyles” from anonymized phone recordings of real crimes, made available within the DigForASP project.
1https://digforasp.uca.es/ within the DigForASP project. Our contribution to the were thus able to identify the KR and AR techniques that project started with the MSc thesis in Cybersecurity (Uni- can be applied to EA and to suggest guidelines for the versity of Bari) of Gioacchino Sterlicchio under the super- creation and development of new techniques and methvision of Prof. Lisi. We developed special ASP tools that ods suitable for progress state of the art in both DF and automatically extract useful information from telephone AI, strengthening European research and innovation carecords instead of being found manually, reducing the pacity in these sectors. The long-term objective of the working time. In fact, evidence on possible crimes and “Network” is to increase the know-how and skills, so as to perpetrators of crimes collected from various electronic concretely design and implement projects and tools that devices (through specialized software, and according to must be applied by the Scientific Investigative Departspecific regulations) must be examined and aggregated in ments of the Police in the resolution of real cases in the order to reconstruct possible events, sequences of events countries COST members, COST Near Neighbor Counand scenarios related to a crime. tries (NNCs) and COST International Partner Countries
The paper is structured as follows. In Section 2 we (IPCs). This is also to promote coherent and eefctive briefly introduce the DigForASP project. In Section we cooperation with third countries. describe the case study considered in our contribution to DigForASP, which is then detailed in Section 4. Section 5 reports the collaborations with DigForASP members 3. The case study which helped us to carry out our research. Section 6 concludes with final remarks.
works that surround us run in parallel with the real world, turning people into traceable mobile identities. Informa2. The DigForASP project tion is automatically stored in phone records and in a device’s memory, an invaluable source of evidence for The project started on September 10, 2018 and ended on criminal court cases. Through telephone records it is not March 9, 2023. DigForASP constituted a timely challenge possible to trace the content of audio calls, messages and for both areas: DF and AI. From the point of view of AI, e-mails sent or received, the list of the web history of the research fostered the development of new theoretical the sites visited. It is a set of data relating to the external results, methods and techniques that will contribute in communications of the devices, that can be requested by the long term to the development of new software tools the Judicial Authority if it deems it useful to get hold of that will be based on a complex combination of concepts them in order to carry out investigations on the individand results from diferent areas of KR&AR such as diag- ual owner of the user. Telephone records contain all the nosis, causal explanation, temporal reasoning on actions, digital traces of communications relating to a specific epistemic reasoning, treatment of incomplete knowledge, user over a certain period of time like telephone calls, ethical and legal reasoning, inductive learning and analy- text messages/SMSs, and data trafic of the mobile phone. sis of formal concepts. At the same time, the application For example, such data are (a) caller telephone number, of (intelligent) automated tools to DF, capable of being re- (b) recipient telephone number, (c) communication cateliable and able to comprehensively explore evidence and gory (e.g. call, SMS, missed call, etc.), and (d) how long with a level of analysis that is beyond the reach of human the call lasted. As invaluable source of evidence, law observation and over time, it will be a breakthrough that enforcement can analyze these data for compare the geowill have a direct impact on the practical investigation graphical positions with respect to the declarations and of criminal scenarios. reconstruct the network of contacts for a single user to
To take up the challenge, the Action has built a “Net- trace which conversations he/she had and at what time. work” made up of researchers and engineers in the field of Analyzing phone data can be time consuming. So, the AI together with DF experts belonging to government in- (partial) automation of this activity limits the amount stitutions and NGOs alongside scholars from the world of of data to be processed during the EA phase. With our information and technologies. communication (ICT) law, research, we encouraged formal and verifiable AI methas well as social scientists, criminologists and philoso- ods and techniques to call pattern analysis, an intelliphers (the latter for ethical issues). The “Network” car- gence technique used to identify patterns in telephone ried out a series of activities and building resources to call trafic such as timing of events and actions, possible promote interaction, exchange and cooperation between causal correlations and contexts in which suspicious acthese diferent areas. It sought to understand the main tions have occurred. As said before, we have considered issues and open problems of DF, in particular the analysis the analysis of the real-world phone records, carefully of evidence, and is helping to promote the exploitation anonymized in order to preserve privacy and confidenof AI to address the key problems of this domain in an tiality, provided by Prof. David Billard (University of innovative, adaptive way. The partners of the “Network” Applied Sciences of Geneva) under non-disclosure agreement to DigForASP members for academic experimen- Table 1 tation. The dataset consists of four Excel files, one for An example of sequence database . each suspect: “Eudokia Makrembolitissa”, “Karen Cook McNally”, “Laila Lalami”, “Lucy Delaney”. Each file has ID the following features:
for the stream of research known as Declarative Pattern
Our goal was to address questions posed by Prof. Bil- Mining (DPM). DPM covers many pattern mining tasks lard and other DF experts within the DigForASP project such as sequence mining [8, 9] and frequent itemset minsuch as: ing [10, 11]. Very recently, the case of contrast pattern mining has been covered as well [12]. Besides ASP-based 1. From the Eudokia Makrembolitissa dataset, approaches like [9, 13], other declarative frameworks would it be possible to find her accomplices Karen have been considered such as Boolean Satisfiability (SAT) Cook McNally or/and Laila Lalami? [10], and Constraint Programming (CSP) [14, 11]. 2. From the Eudokia Makrembolitissa, Karen Cook Sequential Pattern Mining finds statistically relevant McNally and Laila Lalami dataset, would it be patterns within sequences of data examples [6]. It is possible to find Lucy Delaney? usually presumed that the values are discrete within a 3. Do same people gather physically often? time series. Given a sequences dataset (Table 1) the 4. When X calls Y, do always Y calls Z shortly after- cover of a sequence is the set of sequences of which wards? includes : (, ) = { ∈ | ⊆ } . The 5. At the time of the crime, who was at the same number of sequences that includes in is called suplocation, or called by Eudokia Makrembolitissa? port: (, ) = |(, )|. For an integer 6. The day before, who spoke with Eudokia Makrem- , frequent sequential pattern mining means discovering bolitissa? Or any other suspect? all sequences such that (, ) ≥ , where is called frequent sequential pattern and minimum sup
To this aim, we have proposed to analyze the dataset port threshold. As said before, sequential pattern mining by looking for sequential [6] and contrast [7] patterns can reason about sequences of events in a given time that could highlight habits of the suspects. The novelty frame and we encoded this in ASP by exploiting tempoof our work is the declarative approach followed, based ral information to create the suspects’ relationship neton ASP, as detailed in the next Section. work, to identify associations between individuals and to highlight the patterns or “lifestyles” of the suspects 4. Analysing Phone Calls with [15, 16]. To better understand the following example, we provide some clarifications regarding the syntax and Declarative Pattern Mining the semantics. Each answer set returned is a sequential pattern represented by means of the ℎ/2 predicate.
Data mining means the identification of information of Listing 1 shows one of the 15 generated run over 100
various kinds (not known a priori) through targeted ex- instances from the DigForASP dataset, with maximum
trapolation from large, single or multiple databases ap- pattern length equal to 3 and minimum support
threshplicable to the most varied fields: economic, scientific, old equal to 25%. It represents the sequential pattern
operational, etc. The techniques and strategies applied to which consists of two communication events. The first is
data mining operations are largely automated, consisting between Karen Cook McNally and Margaret Hasse (Line
of specific software and algorithms suited to the single 1), while the second is between Joan Aiken and Karen
purpose. To date, in particular, neural networks, decision Cook McNally (Line 2), finally between Lucie Julia and
trees, clustering and association analysis are used. Pat- Karen Cook McNally (Line 3). For example, considering
tern Mining is a data mining subtask in which rules that the day Sept. 8th, 2040, we know that Karen, the subject
describe specific patterns are identified within graphs, of the phone records, sent a text message to Margaret
sequences or itemset. A pattern is interesting if given at 1:1:34 (Line 5). Later, Karen received a incoming call
a threshold and set of data , occurs at least in ex- from Joan at 8:19:53 (Line 6) and once again Karen
reamples in and is called frequent pattern. ASP is used
Listing 1 Example of sequential patterns found
1: pat(1,(karen_cook_mcnally,margaret_hasse))
2: pat(2,(joan_aiken,karen_cook_mcnally))
3: pat(3,(lucie_julia,karen_cook_mcnally))
4: support((
Contrast Pattern Mining [7] involves the concept of contrast. It can therefore describe the significant diferences classes) of each suspect in the DigForASP dataset [12]. A between datasets under diferent contrast conditions. A single pattern is associated with each answer set and, in transaction dataset (Table 2) is a multi-set of transac- our approach, represented by means of the in_pattern/1 tions, a transaction is a non-empty set of items with and absolute_dif/1 predicates. The latter expresses the associated a transaction identifier TID and the TIDs are diference in support of the pattern between the class unique and can occur multiple times in . A dataset under consideration and the complementary class. Each may be associated with classes. In this case, some num- pattern conveys information that allows to characterize ber ≥ 2 of class labels 1, ..., are given, and is the considered class. In Listing 2, as an illustrative exampartitioned into disjoint subsets 1, ..., such that ple of the potential usefulness of contrast pattern mining is the dataset class. Contrast pattern mining could in the DF field, we report the results obtained on Karen’s exploit background knowledge to extract less but mean- phone records for the class “in call”. Here, we have set the ingful patterns. It is an interesting class of pattern mining minimum support threshold to 10% and the maximum problems halfway between discrimination and charac- pattern length to 3. Overall, the patterns found provide terization of a data set, thanks to the use of classes to rich information about Karen’s incoming calls in contrast guide the search for regularities. We have encoded a basic to other types of communication. Notably, they tell us contrast pattern mining problem with ASP and applied that incoming calls of Karen are mainly received in the the encodings to find the outgoing/incoming calls/SMS afternoon (Answer 3) and less in the morning (Answer characteristics and habits (from now on, referred to as 4).
Listing 2 Some contrast patterns for incoming calls 1: Answer: 1 in_pattern(callee(karen_cook_mcnally)) absolute_dif(216) 2: Answer: 2 in_pattern(callee(karen_cook_mcnally)) in_pattern(time(afternoon)) absolute_dif(106) 3: Answer: 3
in_pattern(time(afternoon)) absolute_dif(130) 4: Answer: 4
in_pattern(time(morning)) absolute_dif(43) 5: Answer: 5 in_pattern(callee(karen_cook_mcnally)) in_pattern(time(morning)) absolute_dif(72) results for the sequential pattern mining and the constrast pattern mining problems considered in our application to the DigForASP dataset can be found respectively in [15, 16] and [12].
funded by the DigForASP project, Gioacchino Sterlicchio (currently Ph.D. student in Aerospace Sciences and Engineering at the Polytechnic University of Bari under the supervision of Prof. Lisi) could visit Prof. Billard at the University of Applied Sciences of Geneva. During the mission it was possible to share the results obtained and establish future collaborations. This STSM was intended to exchange knowledge and obtain feedback for our work from DF experts. However, it turned out to be also a great opportunity to disseminate our work also in other application domains. Prof. Billard organized diferent interesting meetings for him, with experts from the DF field and from other fields interested in applying our approach to solve their problems to broaden the application horizon. We showed our work to Patrick Ghion, head of the forensic department of the Geneva Police. We talked about our work that attracted interest in the type of insights that can be extracted and we discussed how to apply a new privacy-preserving technique of anonymization to spatial data so that they can be used safely in the DigForASP dataset. After a meeting with Prof. Giovanna Di Marzo, head of the Computer Science Department at the University of Geneva, we have been invited to show our work at Digital Innovators3, a series of monthly seminars describing a digital innovation and its application in a use case. We also showed our work to Prof. David-Zacharie Issom, head of Algorithms and Programming Division Department of Information Systems at University of Applied Sciences of Geneva. He is 3https://cui.unige.ch/fr/pin/digital-innovators/ fascinated on how pattern mining can be applied to his problem in medical science, and extremely interested in using IA to translate knowledge into action, specifically to enable data analysis using IA methods and to visualize and reduce risks.
The challenge in DigForASP and in our work was to create a link between declarative AI and DF. In particular, we have contributed to the creation of innovative tools that automatically extract useful information about “lifestyles” of suspects from their telephone records. The application of our declarative AI techniques in investigative contexts is certainly an advantage, thus reducing the time to collect evidence and consequentially to capture the perpetrator(s) of a crime. The use of automated tools, such as these ones, capable of exhaustive research to explore evidence and go beyond human observation will surely become a breakthrough with an immediate impact on the practical investigation of crime scenes. Therefore, law enforcement, investigators, intelligence services, criminologists, prosecutors, lawyers and judges will have decision support systems in place, and can help make judicial proceedings clearer and faster essential properties in DF.
For the future we are interested in expanding the analysis by considering spatial data. Also, we intend to discover anomalous behavior by applying Rare Pattern Mining [18], and to enrich the sequential patterns with new knowledge about the sequences of actions done or not done by a suspect (Negative Sequential Pattern Mining [19]). Furthermore, we would like to improve the eficiency and scalability for the contrast pattern mining task. This implies diferent choices for: (a) the hardware, (b) the encoding, (c) the solver, and (d) the computing platform. For instance, experiments could be replicated with other ASP solvers, such as DLV2 [20]. Beyond the eficiency improvement, we would like to consider other variants of the contrast pattern mining problem. Last but not least, we would like to continue the collaboration with DF experts (from inside and/or outside the DigForASP network) in order to get their feedback as regards the validity and the usefulness of our work, and their suggestions for new interesting directions of applied research in this field.
This article is based upon work from COST Action 17124 “Digital forensics: evidence analysis via intelligent systems and practices (DigForASP)”, supported by COST (European Cooperation in Science and Technology). The work is also partially funded by the Università degli Studi di Bari “Aldo Moro” under the 2017-2018 grant “Metodi di Intelligenza Artificiale per l’Informatica Forense”.