This paper outlines the approach and technology employed to model and generate extreme volumes of synthetic ifnancial time-series data. We introduce the Graph-Massivizer project and its financial use case, focusing on green sustainable finance. One project objective is to create synthetic financial data in extreme volumes to facilitate advanced testing and simulations of investment and trading algorithms. Afterward, we provide an overview of the methodology, detailing the utilization of ontologies and knowledge graphs. Furthermore, we elaborate on modeling correlations between diferent markets' time-series and how we can benefit in combination with graph neural network models to generate financial data. We then present the current implementation status and conclude with a discussion of future work.
In the financial investment and trading domains, synthetic data—artificially generated datasets that
mimic real-world financial time-series characteristics—has become a robust solution for quantitative
analysis and back-testing. The demand for synthetic data has arisen due to increasingly complex
ifnancial models and algorithms driven by data-demanding machine learning (ML) models. These
models find real historical data time-series to have multiple limitations, such as reduced volumes,
high costs, incomplete data, or irrelevance as we go further back in time. The core characteristic of
synthetic data is its ability to capture the statistical properties of real-world markets while maintaining
a completely artificial nature. This allows for intensive testing before financial models and algorithms
are further validated on real-time financial data. The Graph-Massivizer project [
KGs by adding missing information, such as properties or classes that are not explicitly present but are implied based on existing relationships.
Green and sustainable finance This use case [
Graph-Massivizer toolkit The Graph-Massivizer toolkit is an integrated platform composed of five tools (Graph-Inceptor, Graph-Scrutinizer, Graph-Optimizer, Graph-Greenifier, and GraphChoreographer) that perform specific and unique functions for massive graph processing: • Graph-Inceptor: realizes a massive graph for the system to use. • Graph-Scrutinizer: provides analytic capabilities and probabilistic reasoning for insights. • Graph-Optimizer: ensures that large graph operations are completed eficiently. • Graph-Greenifier: evaluates the energy consumption of massive graph operation. • Graph-Choreographer: allows serverless deployment to use resources on-demand.
Further on, Figure 1 shows the overall Graph-Massivizer architecture and how the five tools are interconnected. Additionally, we can see in this diagram the external components, such as the metaphactory platform, the graph database, and the hardware and infrastructure used by the toolkit.
Modeling financial data presents several challenges due to financial markets’ dynamic nature and complexities. In addition to numerous variables, volatility clustering, fat tails, and noise, which are all specific to financial data, we focus particularly on five aspects to generate synthetic data. KG relations’ extraction and enrichment In large-scale financial data, extracting relationships among various data types can be complex and non-intuitive, often requiring inference methods to identify them. We aim to enhance the quality of KG relation extraction by utilizing ontologies and reasoning methods to identify and extract non-obvious relationships.
Heterogeneous time-series data Financial data consists of diferent types of time-series data with diferent semantics, domains, and dynamics. We can mitigate this diversity by using ontologies underlying their relationships and finding correlations among them.
Changing statistical properties Financial time-series often display changing statistical properties over time, such as means, variances, and covariances. These properties are used for measuring an asset’s performance and risk. These statistical patterns must be identified and replicated in the synthetic data to model the original data accurately.
Quality assessment of generated data The evaluation of synthetic data is ongoing research [
KGs and ontologies allow scientists and domain experts to model complex relations between data in a logically structured and machine-readable format. This capability allows ontologies to connect diverse sources of information, such as the use case presented here and similar related data.
In the Graph-Massivizer project, ontologies represent and integrate data from diverse use cases. The
metaphactory platform was chosen to manage ontologies and integrate data with a front-end interface.
Metaphactory has many applications for developing and managing ontologies, KGs, and other related
semantic artifacts [
By decoupling the data and the schema for the data, the ontology allows developers to model and prepare for handling massive amounts of data in an abstract way. For instance, a user or developer can write queries to inspect only the relevant data of interest inside the large data set. While queries like this are not themselves a direct algorithmic optimization, they do play a critical role in ensuring scalability is possible by identifying critical semantic information and metadata that can reduce a huge chunk of data into something more tractable.
In this section, we will describe the data represented by ontology and then show the ontology that schematizes it to integrate it with a KG.
This use case initially focuses on two types of financial products: stocks and commodities futures.
However, this paper will concentrate on one financial product: stocks. The financial ontology for stocks
consists of four main types of financial data:
Fundamental data Fundamental data [
ESG data Environmental, social, governance (ESG) [
Sentiment data Market sentiment data [
The financial ontology overview in Figure 2 shows the objects and data constituting the use case, namely historical and synthetic data and financial algorithms. They run inside the PeractonSecuritiesPlatform class that ingests the SyntheticStocksData and SyntheticCommoditiesData generated by the Graph-MassivizerPlatform class.
The synthetic financial data ontology in Figure 3 shows the created categories and mirrors the original historical financial data set structure. The SyntheticSymbol belongs to SyntheticCommoditiesData and SyntheticStocks classes, with the TechnicalData and FundamentalData classes as features. It also shows the SyntheticFinancialData class with SyntheticCommoditiesMultiverse and SyntheticStocksMultiverse subclasses, with further SyntheticCommoditiesData and SyntheticStocksData subclasses.
We briefly review the literature on applying KG in financial data analysis, then elaborate on other ifnancial data modeling and generation methods.
Several methods leverage ontology and KG in financial analysis, such as KG extraction and enrichment, querying and reasoning over KGs, extracting correlations, and modeling financial time-series.
[
Statistical models Various statistical linear models such as autoregressive models exist for stock prices, however, they can’t capture the complex non-linear structure of these data [18]. Event based models These methods formulate time-series data as event data and define it as remarkable changes in time-series in continuous time. The methods of [19] and [20] construct correlation (influence) graphs of time-series utilizing the Hawkes process. [ 20] defines events as long-lasting volatility values. It uses an attention layer to weigh and capture neighborhoods and an LSTM to predict the next price. The method in [21] uses an event graph and tackles dimensionality. It formulates the intensity function utilizing GNNs and the attention layer to dynamically embed node features and recurrent neural networks (RNNs) to embed event sequences. Graphical event models are another form of marked point processes event type utilizing graphical information for event graph construction [21]. Pattern recognition These methods perform pattern-matching techniques to predict trends in time-series, including perceptually important points, template matching, and dynamic tree wrapping algorithm. The survey [22] provides a detailed review of these methods.
ML models The review in [22] categorizes these models into supervised and unsupervised models. Supervised learning methods use various ML models such as support vector machines, random forest, Adaboost, kNN, and eXtreme gradient boosting methods for stock prediction. The unsupervised learning methods include clustering methods to help in finding correlations among markets [22].
Recently, several studies used deep learning models for modeling financial time-series data consisting of convolutional neural networks, RNNs, attention mechanisms, and generative adversarial networks (GANs) capable of capturing non-linear and complex data features. The survey [23] provides a comprehensive review of these models.
GANs are powerful data generation models appropriate for time-series and adjusted for event data
generation. The method in [24] proposes a marked event data generation method using separate
generators and discriminators for each event type and preserves type correlations using a central
discriminator. This method provides synthetic data for downstream tasks. The GAN model of [18]
constructs the correlation graph among stocks using diferent correlation analysis methods and uses
GCNs to encode interdependent time-series data. The method in [
The other category of methods uses natural language processing (NLP) to benefit from financial text data such as financial news and SEC filings. They consist of N-grams and word2vec embeddings as inputs for prediction models. The survey [22], and paper [16] introduce methods of this category.
Financial data can show various correlations across financial products, companies, and markets. These correlations may be based on deeper links between companies and industries or simply random, lacking any underlying economic or financial rationale. We aim to identify the relevant and meaningful correlations within historical financial time-series and use these insights to generate synthetic data.
Point process models are stochastic processes that successfully model event sequences [25]. They vary depending on the definition of the conditional intensity function, representing the expected number of events in a small time interval given the event history. We consider a multi-dimensional Hawkes (self-exciting) process [26] to model dependencies between various time-series of markets and obtain the influence graph. We convert financial time-series data to event sequences by defining events as relatively significant changes in time-series.
Self-exciting process This process represents the triggering efect of event history in the intensity and occurrence of future events, usually as an exponential exciting kernel (Eq. 1) [26]: where () is the intensity of event type at time t, is the base intensity for even type , is the excitation decay rate and , is the influence parameter between event types of and .
We infer model parameters ( and influence matrix = (, )) by maximizing the log-likelihood of events given in Eq. 2 using the expectation-maximization or stochastic gradient descent methods: (1) (2) where [0, ] is the time interval of events we consider for correlation analysis, and () is the sum of intensities of all event types.
() = + ∑︁ , · − · (− ),
;< ℒ = ∑︁ log () − :< ∫︁ 0 (),
processes for the time-series of stocks and obtain the correlation graph among them by analyzing event data and inferring the influence matrix of the multi-dimensional process [ 27]. This matrix reveals the weighted dependency (influence) graph among diferent market’ stocks. Due to the ever-changing dependencies among companies, we can update this graph over time and drive a dynamic dependency. As a remedy for high dimensional market data, which decreases the accuracy of these models, we can drive an initial dependency graph by processing various textual data using NLP techniques. Correlation analysis of internal stock data Despite the expressiveness of point process models, they are inaccurate in high-dimensional spaces. In correlation analysis of internal stock data (fundamental and technical), we can mitigate this problem by leveraging KGs and considering only relations appearing in the KG.
Formulating intensity function using neural networks In addition to high dimensionality, the assumption of solid intensity functions such as the formulation in Eq. 1 with predefined triggering efect (positive and additive efect of history), might not apply to every real scenario with intricate dependencies. Therefore, several methods benefit from neural networks such as LSTMs in [ 28] to formulate the intensity function of the point process models that capture variable and complex dependencies and adjust the model for the specific use case. Additionally, the attention mechanism [ 29], which can explicitly model the influence of event types, guides in finding the correlation graph.
Every correlation does not represent a causality relationship known as spurious correlation [30]. This correlation can be a random efect or caused by other hidden variables and, therefore, be spurious [ 30]. In particular, deep learning models usually result in spurious correlations without enough diverse data [31]. To mitigate this misinterpretation, we will apply methods to test the correlations. • Considering other factors in the stock market in correlation analysis, in addition to stocks, as much as possible. • Considering long-term correlations among time-series or comparing these correlations in the long term to test if they are not random. • Applying null hypothesis for finding significant p-values, such as the method of [ 32], which constructs a spurious relationship graph among time-series of stocks using Granger causal relation test to evaluate causality between time-series and T-test to estimate the p-value.
We aim to prune the initial dependency graph and reduce the dimensionality of point process models by detecting spurious correlations and driving a more meaningful model based on causality dependencies.
An important input in modeling synthetic time-series is using a correlation graph among stocks, as presented in Figure 4. The temporal and dependency features of stocks are embedded using this correlation graph. These features serve as inputs for ML models to generate time-series data. In the following sections, we propose various types of these models: Improving event-based models using GNNs and LSTM The correlation graph can be used directly for generating time-series. However, to enhance the initial point process models for modeling time-series (that consist of more details than event sequences) and to improve graph weights, we will add GNN with attention layers [33]. This model can simultaneously encode time-series and correlation graph structure and obtains graph weights [20] (Figure 4). Then, we will model the synthetic time-series data using an LSTM given the encoded data [20]. Correlation graph
GNN
ML-based time series generator (x11, x12, ....) (x21, x22, ....) (x31, x32, ....)
Multidimensional time-series
correlation graph and GANs to generate interrelated time-series data. We will apply this graph to embed time-series data, capturing their relationships and providing inputs for the GAN model.
Generating extreme volumes of synthetic financial time-series data has unique challenges and requirements from both software and hardware perspectives. Here, we outline the most relevant ones. Scalable data generation The five tools of the Graph-Massivizer platform are being implemented scalable to generate synthetic time-series data across a distributed system. One strategy employed is tasklevel parallelism, which divides the data generation process into small chunks processed simultaneously across diferent nodes in the HPC cluster. As a computing framework, Apache Spark provides libraries and functionalities for parallel processing and data management on large clusters. Data storage and streaming Managing and storing the produced synthetic data at the petabyte level requires various solutions, such as compression, third-party storage, and transferring data only when necessary. Ideally, the synthetic data should remain within the HPC cluster, with financial simulations and testing conducted in the same environment to minimize data movement.
Pre-exascale supercomputer [34] that will allow the parallel processing of synthetic data generation. A low-latency, high-bandwidth network is in place for communication between compute nodes within the HPC cluster to facilitate data exchange Energy consumption monitoring The Graph-Massivizer platform has a dedicated tool called ’Graph-Greenifier’ to monitor and analyze the energy consumption used for generating the synthetic data time-series.
Data security Even though the underlying historical financial time-series data does not contain personally identifiable information, its synthetic data can still be commercially sensitive. Therefore, security measures are necessary to ensure data confidentiality, data integrity, and secure coding practices, such as strict access control, synthetic data encryption at rest and in transit, and digital signatures of the synthetic data to ensure its authenticity and prevent tampering and data logging monitoring. Cost optimization Finally, cost optimization in generating synthetic data is critical to the entire process. It involves balancing hardware costs, licensing fees, and ongoing maintenance expenses with the desired performance and scalability. The goal is to ofer a more cost-efective solution than real historical financial data while maintaining competitiveness in pricing.
The work presented in this paper, as part of the Graph-Massivizer EU project, is currently at the halfway point and demonstrates the initial proof-of-concept developments for generating synthetic data in extreme volumes. The mechanisms and approaches identified here will be further implemented as a robust workflow within the five tools of the Graph-Massivizer platform. The focus will be on software implementation, integrating the five tools, and identifying relevant correlations in historical time-series to enhance the quality of the generated synthetic data. After generating synthetic data samples, they will be tested using Peracton’s back-testing engine with various investment and trading ifnancial algorithms. The behavior of these algorithms will be analyzed and compared with their performance on real historical data to assess diferences and similarities. Feedback will be provided to the Graph-Massivizer platform to fine-tune the quality of the synthetic data further. Acknowledgement The Graph-Massivizer project has received funding from the European Union’s Horizon Research and Innovation Actions under Grant Agreement Nº 101093202.1 1More information available at: https://graph-massivizer.eu/ [14] C. Erten, D. Kazakov, Ontology graph embeddings and ilp for financial forecasting, in: International
Conference on Inductive Logic Programming, Springer, 2021, pp. 111–124. [15] S. Ibrahim, W. Chen, Y. Zhu, P.-Y. Chen, Y. Zhang, R. Mazumder, Knowledge graph guided simultaneous forecasting and network learning for multivariate financial time series, in: Proceedings of the Third ACM International Conference on AI in Finance, 2022, pp. 480–488. [16] D. Matsunaga, T. Suzumura, T. Takahashi, Exploring graph neural networks for stock market predictions with rolling window analysis, arXiv preprint arXiv:1909.10660 (2019). [17] X. Fu, X. Ren, O. J. Mengshoel, X. Wu, Stochastic optimization for market return prediction using ifnancial knowledge graph, in: 2018 IEEE International Conference on Big Knowledge (ICBK), IEEE, 2018, pp. 25–32. [18] D. Ma, D. Yuan, M. Huang, L. Dong, Vgc-gan: A multi-graph convolution adversarial network for stock price prediction, Expert Systems with Applications 236 (2024) 121204. [19] J. Etesami, N. Kiyavash, K. Zhang, K. Singhal, Learning network of multivariate hawkes processes:
A time series approach, arXiv preprint arXiv:1603.04319 (2016). [20] T. Yin, C. Liu, F. Ding, Z. Feng, B. Yuan, N. Zhang, Graph-based stock correlation and prediction for high-frequency trading systems, Pattern Recognition 122 (2022) 108209. [21] K. Yoon, Y. Im, J. Choi, T. Jeong, J. Park, Learning multivariate hawkes process via graph recurrent neural network, in: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 5451–5462. [22] D. Shah, H. Isah, F. Zulkernine, Stock market analysis: A review and taxonomy of prediction techniques, International Journal of Financial Studies 7 (2019) 26. [23] W. Jiang, Applications of deep learning in stock market prediction: recent progress, Expert
Systems with Applications 184 (2021) 115537. [24] A. Seyfi, J.-F. Rajotte, R. Ng, Generating multivariate time series with common source coordinated gan (cosci-gan), Advances in neural information processing systems 35 (2022) 32777–32788. [25] D. J. Daley, D. Vere-Jones, et al., An introduction to the theory of point processes: volume I: elementary theory and methods, Springer, 2003. [26] A. G. Hawkes, Spectra of some self-exciting and mutually exciting point processes, Biometrika 58 (1971) 83–90. [27] E. Lewis, G. Mohler, A nonparametric em algorithm for multiscale hawkes processes, Journal of nonparametric statistics 1 (2011) 1–20. [28] H. Mei, J. M. Eisner, The neural hawkes process: A neurally self-modulating multivariate point process, Advances in neural information processing systems 30 (2017). [29] Y. Gu, Attentive neural point processes for event forecasting, in: Proceedings of the AAAI
Conference on Artificial Intelligence, volume 35, 2021, pp. 7592–7600. [30] H. A. Simon, Spurious correlation: A causal interpretation, Journal of the American statistical
Association 49 (1954) 467–479. [31] S. Wu, M. Yuksekgonul, L. Zhang, J. Zou, Discover and cure: Concept-aware mitigation of spurious correlation, in: International Conference on Machine Learning, PMLR, 2023, pp. 37765–37786. [32] G. Li, J. J. Jung, Dynamic relationship identification for abnormality detection on financial time series, Pattern Recognition Letters 145 (2021) 194–199. [33] P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio, et al., Graph attention networks, stat 1050 (2017) 10–48550. [34] CINECA, High performance computing, leonardo pre-exascale supercomputer, 2024. URL: https: //leonardo-supercomputer.cineca.eu/.