This article describes ongoing research on the application of AI techniques such as Graph Neural Networks to assist investigators with the discovery of relations and patterns in digital forensic evidence. Digital forensic analysis of smartphones and computers reveals forensic artifacts that are extracted from structured databases maintained by the operating system and applications. Such forensic artifacts are part of a forensic ontology which can be used to build a relational graph of identifiers (e.g. users, documents) and a timeline of events. This information can assist with answering key investigation questions such as who, when, where etc. We propose to use a graph database and query language to assist in this analysis. Further, using key identifiers and aliases we want to augment digital forensic artifacts with entities, relations and events by extraction from the full-text of unstructured electronic contents such as emails and documents.
• Computing methodologies → Semantic networks; Neural networks; • Applied computing → Law; Investigation techniques; • Information systems → Users and interactive retrieval; • Human-centered computing → Visualization toolkits.
Digital evidence continues to grow exponentially in investigations
and prosecution of suspects in both criminal as well as civil cases.
Not only in advanced cybercrime investigations, as in, ransomware
investigations or as part of incident response, but also through
the use of digital forensics in homicide cases or internal
(corporate) investigations where the suspect’s smartphone and/or laptop
needs to be examined. Smartphones and other portable "wearable"
electronics leave digital traces that can be linked to persons and
locations. The exponential growth of digital traces, as well as the
expansion of cybercrime, and digitization of investigative methods
represent significant changes to society and lead to a broadening
horizon of digital investigation [
This article presents work in progress on research that focuses
on the use of Artificial Intelligence (AI) as an emerging technology
that can assist forensic examiners in the discovery of patterns and
relations in digital evidence. It builds further on the ideas presented
in earlier work on computer assisted extraction of identities in
digital forensics [
Our vision difers from existing applications of AI in E-Discovery
that typically rely on machine learning for classifying digital
content such as predictive coding and active learning [
Smartphones and Internet of Things (IoT) devices contain many other digital traces that are a treasure trove in a forensic investigation. Such traces can prove to be more personal than written communication because they do not only reveal our conscious but also our unconscious behavior. Also smartphones have become very personal because of their link with social media and biometric protection (e.g. fingerprint, iris). However, this type of information is machine generated and grows at an even faster pace than our personal communication. Forensic investigations are in need of more efective search strategies that can leverage the richness of detailed forensic in modern digital evidence (e.g. from smartphones, cloud, IoT devices etc.). We propose that investigators are assisted with discovery through using semantic nets that are obtained from digital evidence. We refer to this as technology assisted discovery as opposed to technology assisted review that is very common in E-Discovery investigations.
This paper is structured as follows. Section 2 describes digital forensic investigations and the key questions that are relevant when investigating a case. It also describes related work on a digital forensics ontology that can assist when taking a semantic AI approach and explains some use cases why this is helpful when investigating digital evidence. In section 3 we explain modern digital forensic investigations and illustrate how link analysis and time line visualisation are currently assisting forensic examiners in digital forensic investigations. Section 4 presents our vision on how AI techniques such as graph databases and entity extraction can help discovering patterns and relations in these semantic networks of digital artifacts. Finally, in section 5 we present conclusions and identify future research opportunities. 2
Digital forensic investigation typically has three phases: data collection, data examination and data analysis. Data collection involves the correct preservation and copying of digital data sources. Data examination relates to the investigation of copies of digital data sources to find files, extract fragments etc. without interpreting the resultant findings in the context of the case. Data analysis involves the analysis, reconstruction, interpretation and qualification of the evidence which is obtained from the digital data sources. The research proposed here focuses on the analysis of digital evidence. 2.1
In any investigation the investigators, regardless if they are senior legal counsel in legal E-Discovery or senior investigating oficers or detectives in a criminal investigation, try to answer the following ’golden’ investigation questions: 1 Who–was involved? 2 What–happened? 3 Where–did it happen? 4 How–was the crime committed? 5 When–did the crime take place? 6 With what–was the crime committed? 7 Why–was the crime committed?
The analysis of digital evidence in E-Discovery investigations
typically focuses on document review and analysis where reviewers
and senior investigators analyse textual content. They are assisted
by machine learning (also known as predictive coding and
continuous active learning [
Who-questions can often be answered by investigating which person is using an e-mail address, user account or phone number. Communication via text messages, chat and email may help to understand what has happened. Call details records, GPS-locations and WiFi-network tell something about the location of a smartphone and consequently of it’s user. Pictures and video can provide visual clues how a crime was committed and with what kind of weapon. Date and time of a file or trace, tell when data was last accessed, modified or created. Computers and smartphone maintain detailed records when apps and users were active and which ifles were involved. Besides messages that a user communicated via emails and chat messages, search history from a browser or specific apps can help understand motive and premeditation. 2.2
Document analysis in E-Discovery heavily relies on the review and
analysis of unstructured information that is contained in emails and
documents. The analysis of digital forensic artifacts described above
is more structured. In order to understand this structure and to be
able to analyse it, it is useful to have a digital forensics ontology.
The Cyber-investigation Analysis Standard Expression (CASE) [
Once a semantic network has been formed based on a digital forensics ontology, it can assist with identifying possible crime scenarios and with testing hypothesis which is becoming increasingly important in investigations. Sometimes it’s more important to know with who a victim, suspect or witness communicated, and where these persons were than actually knowing what has been communicated. AI can assist investigators with detecting correlations that can lead to the discovery of relationships that were not known. This is called link analysis. Analysing a social network from a collection of emails is not new but link analysis based on digital forensic artifacts relies on a much richer set of data. 2.3
Modern digital forensic tools have a feature that performs link analysis to assist forensic examiners with their investigations. Axiom is a commercial digital forensics processing tool created by Magnet Forensics that build a connections database from relationships between discovered artifacts (e.g. users, files etc.). Triples (subject, predicate, object) are extracted following the forensic ontology similar to the CASE ontology introduced in the previous section. These triples define a forensics ontology that is used by Axiom to automatically generate relationship graphs.
Subject Predicate Object ifle accessed on system ifle accessed on USB ifle accessed by user id ifle transferred with program name ifle transferred by user id ifle related cloud ifle emailed to email address ifle downloaded with program name ifle downloaded by user id contact name contacted with device contact name contacted by person picture hit similar to picture hit ifle/msg contains key words ifle/msg references ifle name call log call to contact name user id used program name user id searched for key words Table 1: Subset of triples of the forensic ontology that is used in Axiom
Link analysis has interesting use cases for forensic examiners: (1) Given a hit the examiner needs to see a visual representation of all related evidence. Where the ’related’ links are one of the concepts identified in the forensics ontology. (2) Given a link to related evidence the examiner should be able to follow the link and may want to pivot the data around the destination or choose a diferent visualisation. For example, the examiner identified a search query in browser history and then wants to review all events on the system before and event this query was executed.
To validate the idea and explore the potential of AI for assisting
with the discovery of patterns and relations in digital forensics data,
we have processed the M57-Jean scenario [
The M57-Jean scenario is a single disk image scenario involving the ex-filtration of corporate documents from the laptop of a senior executive. The scenario involves a small start-up company, M57.Biz. A few weeks into inception a confidential spreadsheet that contains the names and salaries of the company’s key employees was found posted to the "comments" section of one of the firm’s competitors.
The spreadsheet ’m57bis.xls’ only existed on one of M57’s oficersJean. Jean says that she has no idea how the data left her laptop and that she must have been hacked. The investigator has been given a disk image of Jean’s laptop and is asked to figure out how the data was stolen, or, if Jean isn’t as innocent as she claims. 3.1
Link analysis in itself is not a new concept in digital forensics as is
reflected by work published in 2015 [
Artifact relationship analysis goes beyond visualizing relationships between people and computers. It applies the link analysis concept to files and operating system artifacts, helping a forensic examiner to visualize relationships within artifacts and across evidence sources, such as, computers, mobile devices, and even cloud-based accounts.
Figure 1 above presents an example of link analysis in Axiom.
This example was discussed in a Magnet Forensics webinar [
In [
Figure 2 presents a screenshot of the new timeline visualisation and analysis feature in Axiom 3.0. The top section shows a timeline reflecting artifact counts for a period of 6 minutes starting from July 20, 2008 1:24:40am and ending 1:30:40am. The table below the timeline presents a detailed view of the artifacts presented in the graph. At the top is a "File download" record from email, followed by "File/folder opening", then the "File knowledge" reflecting the creation of a new file "m57biz.xls". Such a sequence of artifacts may help understand how a file came into existence on a computer and if it was opened on that computer.
Generation of timelines has also received much attention outside
the field of digital forensics. Many applications exist that allow for
creation of time lines in an investigation. For example, building case
chronologies with CaseFleet [
Both timeline analysis as well as link analysis are (separately from each other) considered powerful instruments in an investigation. However, we propose that in combination these features become even more powerful enabling an examiner to analyse links in the relation graph in a chronological order which provides more meaning and context then when simply filtering a timeline for selected entities or filtering the relation graph for a particular time frame. 4
Our research focuses on the combination of timeline and link analysis. In order to accomplish this we propose to use a graph database with a graph query language. The graph can initially be constructed from forensic artifacts. With modern graph databases and graph query languages it becomes easy to augment this graph with additional data. This could include data from non-digital sources but also by text mining the full-text of electronic documents and emails, new relations might be uncovered that previously would have required human inspection of the contents of such documents. 4.1
Visualisation of traces in a network, on a map or on a timeline can assist a forensic investigator to understand the story that is behind the data. By ingesting the information that is extracted by Axiom in the M57-Jean case in a graph database, it becomes possible to experiment further with visualisations and discovering relations.
Cypher is a graph query language that allows for expressive
and eficient querying of graph data [
Cypher describes nodes, relationships and properties as ASCII art directly in the language, making queries easy to read and recognize as part of your graph data. Figure 3 below presents an example of a simple Cypher query.
Cypher is supported by a variety of graph databases. We intend
to use Neo4j [
Once the digital artifacts from a case have been imported in the graph database, it becomes quite simple to add relations and objects in the same case that were discovered through other sources. These can either be other sources of digital evidence, e.g., other cases, call detail records, or from non-digital sources such as witness and victim statements, lawful interception, observation, open source intelligence or case time lines that were manually created assisted by software such as mentioned in paragraph 3.
By leveraging the scalability of modern graph databases a great
variety of additional information can be included in the automated
analysis [
More than 90% of the information around us is mostly unstructured, e.g., documents, emails and chat messages. Text mining can help investigators by turning this unstructured information into structured data. Entity extraction can extract entities (e.g. names of people, organisations, places etc) and events from full text. Unfortunately the extraction of entities is error prone and generates many false positives making the results useless. By using identifiers that have been discovered from digital forensic analysis the entity extraction can be targeted reducing the number of false positive identities.
Some interesting work in the field of entity-centric timeline
extraction has been reported in [
With digital forensics it is already possible to extract identities
from structured information through digital forensic analysis [
Analysing a graph using a visualisation tool seems simple enough. As graphs get bigger, traditional mathematics can help with the analysis of the graph but these methods also have their limitations. One of the problems is that there is no clear beginning or ending of a graph (assuming it’s cyclic) and that large scale matrix operations that are typically required for graph analysis do not compute due to memory and time restrictions.
Ontologies, graph database and graph query language are well established and are hardly considered AI techniques. Extracting relations and timelines from full text are well established AI techniques that we hope to leverage in our research but we have no intention of improving this. The core idea in our innovation is to use Graph Neural Networks (GNNs) as a new AI technique that can assist with the analysis of large time-based graphs of relations.
GNNs were first introduced in 2009 [
Our research hypothesis is that we can use GNNs to model interesting relation graphs which can assist investigators with the identification of relevant subgraphs from a highly complex case graph that is automatically constructed from digital forensic artifacts combined with other case data. 5
We propose to use a graph database and query language to assist in digital forensic investigations. We start with a relation graph that is based on connections from digital forensic artifacts. Further research and experiments are needed to study how forensic examiners can interact with this graph and how to extend the graph with other data sources. In particular we intend to study how events on a timeline can be added to the graph, how information from non-digital evidence can be added and how we can improve the performance of existing entity extraction techniques on unstructured data from emails and documents. Finally, we want to research if new machine learning techniques such as GNNs can be used to learn from investigators what link and event patterns are interesting from an investigator perspective.