Permeability prediction of porous media system is very important in many engineering and science domains including earth materials, bio-, solid-materials, and energy applications. In this work we evaluated how machine learning can be used to predict the permeability of porous media with physical properties. An emerging challenge for machine learning/deep learning in engineering and scientific research is the ability to incorporate physics into machine learning process. We used convolutional neural networks (CNNs) to train a set of image data of bead packing and additional physical properties such as porosity and surface area of porous media are used as training data either by feeding them to the fully connected network directly or through the multilayer perception network. Our results clearly show that the optimal neural network architecture and implementation of physicsinformed constraints are important to properly improve the model prediction of permeability. A comprehensive analysis of hyperparameters with different CNN architectures and the data implementation scheme of the physical properties need to be performed to optimize our learning system for various porous media system.
Recent advances in multiscale imaging techniques for the analysis of complex pore structures and compositions have revolutionized our ability to characterize various porous media systems (Bultreys et al., 2016). Applications of imaging for porous media systems have been expanded for multi-interdisciplinary areas including fractured and porous natural media, biofilm, human bones/bodies, and various materials among many others. Flow and transport properties in porous media are very important to control and impact a variety of Earth science applications. Imaging methods have been tremendously advanced to produce 2D/3D structures and compositions of porous media over a range of scales, and numerical methods also have been advanced to fully understand Copyright © 2020, for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CCBY 4.0). Multiphysics behaviors in complex porous media (e.g., Yoon et al., 2013 and 2015). Although it is now largely possible to understand how pore topology, structure, and composition impact various processes affecting flow patterns, transport process, and evolution of porous media by combining a suite of imaging techniques and advanced numerical methods, integration of these techniques requires tremendous computational powers and expenses.
Recent advances in machine learning provide a great opportunity to enhance image-based property estimation and modeling capabilities (e.g., Raissi et al., 2018; Wu et al., 2018). In addition, combination of image data with other numeric and categorical data has improved the prediction of various quantities such as house prices (e.g., Rosebrock, 2019) and image classification as well (Aimone and Severa, 2017).
In this work, we explore how machine learning can be used to predict the permeability of porous media with physical properties. An emerging challenge for machine learning/deep learning in engineering and scientific research is the ability to incorporate physics into machine learning process. We used convolutional neural networks (CNNs) to train a set of image data of bead packing and additional physical properties such as porosity and surface area of porous media are used as training data either by feeding them to the fully connected network directly or through the multilayer perception network. We evaluated the effect of hyperparameters with different training dataset on permeability prediction.
Convolutional neural network (CNN) has been very successful for image classification and segmentation and has been adopted for various scientific and engineering problems including permeability estimation in network systems (Wu et al., 2018), physics-informed reduced order model combined with high fidelity turbulence simulations (Ling et al., 2016), and extraction of flow features (Ströfer et al., 2018). In particular, recent works (Ling et al., 2016, Raissi et al., 2018) demonstrated that deep neural network architectures have an ability to account for underlying physics behind the data.
First, a set of images as shown in Figure 1 was generated to represent a two-dimensional (2-D) porous media system with binary phase using an open source PoreSpy (Gostick et al., 2019) where a sphere packing module was used and porosity (fraction of void space as black in Figure 1) and surface area of porous media (i.e., beads as white in Figure 1). To represent a range of permeability which accounts for the capability of porous media system to allow fluid to flow through, different sizes of spheres were used to arrange the packing as shown in Figure 1. The set of images were used to compute the directional permeability of images using an open source OpenPNM (Gostick et al., 2016). The size of image is 192 x 192 and void and solid phases are shown in black and white, respectively. All physical data (permeability, porosity, and surface area) were normalized from 0 to 1. Since the logarithmic scale of the permeability is more correlated with porosity, we use a logarithmic permeability in this work. Figure 2 shows the relationship between permeability and porosity-surface area. As seen, the permeability has positive and negative correlations with porosity and surface area, respectively.
Additional physical information can provide physical constraints for training the model. The combination of image and numerical data allows us to build and train a hybrid physics-informed machine learning model. To handle processing of the porous media images, we have developed convolutional neural networks (CNNs) whose input consists of binary phase image. The first CNN used in our work (CNN1) include four convolutional layers with the number of kernels from 16, 32, 64, and 128, each followed by batch normalization, leaky Relu activation, and a max pooling. Each convolutional layer has a kernel of size 3 x 3 to extract the features from the corresponding input, and the max pooling with a kernel of size 2 x 2 were used. The two fullyconnected (FC) layers have 36 and 12 neurons and a dropout of 0.4 between two FC layers. The 12 neurons are combined with either the MLP output with the dense layers with 32 hidden nodes and 4 output or two numerical data (porosity and surface area) as shown in Figure 3. For the MLP the activation function was Relu.
The second CNN (CNN2) follows the CNN architecture from Wu et al. (2018) where 2 CNN layers with 10 channels, each of size 5x5 were followed by the three FC layers with 10,32, and 10 neurons. The porosity and surface area data are directly combined into the second FC layer. For the CNN2+Num model, two direction permeability values (in x and y directions) are trained. For the CNN1, a total number of 345 images are used with 80% and 20% of training and testing data. For the CNN2, a total number of 250 images (out of 345 images) are used with 70% and 30% of training and testing data. In particular, the CNN2 was trained with two directional permeability values compared to one horizontal permeability in the CNN1.
Key hyper parameters are the following: the (Leaky)ReLU activation function, the dropout of 0.4 for the CNN1, a batch size of 16 for the CNN1, Adam optimizer with a learning rate of 0.0005 and the decay of 0.0001. The number of epochs was 250 for most of cases with an observation of apparent no learning after 250 epochs based on cases with 750 and 5000 epochs. The loss function is the mean-squared error (MSE). A number of network sizes were evaluated, but in this work we focus on the impact of additional data and different CNN+Numeric data structure on permeability prediction. The mean squared error (MSE) values for training and validation data are reported in Table 1. Testing results with validation data sets for six different case are shown with a linear regression fitting. First, it is very clear that all four cases with the CNN1 and numeric data outperformed the CNN2 with numeric data in training and validation. Although we need to compare the results with the same training data, the CNN architecture significantly influences the learning process of the features of porous media image and numeric data. The CNN1 with image only performed similarly to the CNN2 with numeric data.
Second, all CNN1 models with additional physical numeric data performed better than two other models (Table 1). To compare the prediction with validation data, the predicted and validation data are plotted with the linear regression fitting and a R2 value in Figure 4. As a reference, the single perfect line is also shown. The slope shows the overall performance of each model with a better performance closer to one, while the R2 value shows the proximity of predicted data along the linear regression line. As expected, the CNN1 with both porosity and surface area performed better than the CNN1 with either porosity and surface area. As shown in Figure 2, the porosity and surface area are correlated with the permeability, so both information would provide additional physical constraints that are combined with features extracted from image data. Although there is need to study what features are extracted from image and how two input data can be used to learn the underlying feature to the permeability, Figure 4 shows that the CNN1 models with both numeric data tend to predict the lower and upper ranges of permeability better than the CNN1 models with single numeric data. This may imply that the physical constraints from the numeric data would influence the learning process of the features that impact either high and low permeability systems. For example, the high and low permeability (see an example in Figure 1) contains larger and smaller space (or cross-sectional distance) between spheres, respectively. The fact that the CNN1 with image data only tends to predict the permeability over a narrow range (between ~0.3 and ~0.7) may indicate that without physical property information the CNN tends to learn more common features rather than critical features for low and high permeability patterns. We evaluated how additional physical information can enhance the permeability prediction with the CNN models. As it is now well accepted in the community that a physics-informed machine learning model can overcome overfitting to the training data and improve the features underlying the physical processes, there is a strong need to improve how the physical constraints and/or additional information (e.g., equations and theory) can enhance the learning process in machine learning. Our results clearly show that the optimal neural network architecture and implementation of physicsinformed constraints are important to properly improve the model prediction of permeability. The analysis of the features learned through each layer and the output data from the MLP will reveal a better mechanistic understanding of the machine learning processes. A comprehensive analysis of hyperparameters with different CNN architectures and the data implementation scheme of the physical properties will be performed to optimize our learning system for various porous media system.
This work was supported by the Laboratory Directed Research and Development program at Sandia National Laboratories. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energys National Nuclear Security Administration under contract DE-NA-0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government. Aimone, J.B.; Severa, W.M. 2017. Context-modulation of hippocampal dynamics and deep convolutional networks: Using parallel pathways to limit network size. In Proceedings of 2017 NIPS Cognitive Influenced Artificial Intelligence Workshop. arXiv preprint arXiv:1711.09876.
Bultreys, T.; De Boever, W.; Cnudde, V. 2016. Imaging and image-based fluid transport modeling at the pore scale in geological materials: A practical introduction to the current state-of-the-art. Earth-Science Reviews 155: 93-128.