In this paper we propose a method of seismic facies labeling. Given the three-dimensional image cube of seismic sounding data, labeled by a geologist, we first train on the part of the cube, then we propagate labels to the rest of the cube. We use open-source fully annotated 3D geological model of the Netherlands F3 Block. We apply state-of-the-art deep network architecture, adding on top a 3D fully connected conditional random field (CRF) layer. This allows to get smoother labels on data cube cross-sections. Pseudo labeling technique is used to overcome training data scarcity and predict more reliable labels for geological units. Additional data augmentation allows also to enlarge training dataset. The results show superior network performance over existing baseline model.
Seismic sounding of the Earth gives some insight on the geological structure of the formation and allows predicting the presence of oil or gas traps inside. Usually, seismic sounding is carried out at the early stage of potential reservoir exploration, to mark prospective locations of exploration and production wells.
Seismic facies labeling task consists of assigning specific geological rock types to the seismic data. When done manually, this work is tedious and time-consuming. That is why automation of this procedure can save a lot of time and effort of geologists. A lot of work is already invested in the automation of seismic data labeling. The recent trend includes using modern approaches based on deep learning and semantic segmentation.
[
A lot of researcher efforts are now aimed at automating the process of seismic labeling task ([
A newly presented by [
2021 Copyright for this paper by its authors.
In case when no labels for facies are available, unsupervised learning methods can be applied. For example, [9] proposed to use deep convolutional autoencoders and clustering of deep-feature vectors. This approach allows performing fast analysis of geological patterns.
Recently self-supervised learning methods were applied on a wide range of different tasks. Starting from natural language processing, where a lot of data is already available in textual form, this technique made possible to train large scale neural nets. Similar technique was also applied to image data for learning representations. This helped to pre-train neural nets for the task where only a small amount of labeled data is available.
To overcome the problem of labeled data scarcity, semi-supervised learning techniques were proposed in the literature, such as transfer learning, where the model is first pre-trained on some data and then trained on available labeled data [10]; weak labels technique, which allows learning from non-accurate labels [11]; pseudo-labels – use a trained model to predict labels for unlabeled data, and possibly add these data to the training set ([12], [13]); meta-pseudo labels – state-of-the-art technique where two models are trained in parallel, teacher and student, first student learns from teacher, and after that teacher is learning on student outputs from class conditional probabilities of the unlabeled test set.
In this work we consider an open-source fully annotated 3D geological model of the Netherlands F3
Block [
Figure 1 shows full seismic image with labels, Figure 2 shows spatial position of training and testing cubes. inline slices test train t e s t s e T
Train set 300 100 300
inline 700
The train and test sets contain six groups of lithostratigraphic units. Table 1 below illustrates the percentage of present classes in the sets:
The test data is adjacent to the training cube, so, the slices that are closer to the boundary will be predicted better than the ones that are further away. 3.2.
We apply the well-known UNet architecture with Efficient net B1 as backbone ([14], [15], [16]) and five pooling levels, including scSE (Concurrent Spatial and Channel “Squeeze & Excitation”) attention layer [17]. The model has ~9M trainable parameters.
The model is two-dimensional, it takes patches of size 256 by 256 pixels as input and predicts the labels of the geological structures. Additional channel with depth gradient is introduced to the seismic image patches, so that the model does not mix up deep and shallow layers (Figure 3). a) b) c) d) simulate stretching and faults in the data (shown in Figure 4). Therefore, it was decided to use the approach of pseudo-labels.
Firstly, a model is trained. After that, a part of labels is predicted and added to the training set. Then the model is re-trained on the expanded dataset as shown in Figure 5.
Make inference of class probabilities by
3D step with CRF model is added to force label continuity [18], because very often inline label slices have a lot of artifacts near the class boundaries, where the model is not confident enough in prediction, and assigns probabilities close to 0.5 to two different classes. CRF layer helps to smooth these discontinuities and provide smoother labels as pseudo-labels for model re-training. We also apply an idea from ([19] and [20]), where authors propose to use LSTM block in the bottleneck of the UNet architecture but did not gain success.
Loss function is a combination of cross-entropy ( ), dice loss ( ) and total variation ( ) loss = + where C is number of classes, N is number of observations (e.g. pixels in the image), and , are output class probabilities and , is target distribution for observation i and class c. All loss functions use class weights , which are inversely proportional to the class sizes, with their sum equal to 1.
Dice loss usually performs better for imbalanced classes, which is often the case for the segmentation tasks, where background (non-object) area is larger than the area occupied by the object. Dice loss was (1) (2) proposed in the paper by [21], where it was stated, that it works better than logistic loss with class weights. We apply multi-class dice loss function as follows: 1 ∑
=1 ∑ =1(| 1( , )| + | 2( , )|), where ε is an additional term that solves the division-by-zero problem (it is set to be a small number).
TV loss is computed as predicted labels. 3.3.
where 1 and 2 are finite difference operations along the first and second spatial dimensions. This term enforces the predicted probabilities to be more ‘blocky’, which potentially leads to smoother The weights for loss terms were chosen empirically: = 0.25, = 0.65 and = 0.1.
The results show improvement after the application of the pseudo labeling technique. The test dataset was divided in two parts, after the first training round the labels of the first half of test cube were predicted, and a half of test set was added to the training set. Then the model was re-trained. The metrics (3) (4) are shown in the Table 2 below.
In this work we applied the pseudo labeling technique to the seismic facies segmentation task. We show that adding predictions of slices that are close to the training cube can improve network’s overall performance on the test set. We also applied the TV loss component that forced the predictions to be smooth, and the special type of non-linear warping of patches, to increase the diversity in the training data. The results show superior performance over reported techniques. 5. References [9] V. Puzyrev, C. Elders, Unsupervised seismic facies classification using deep convolutional autoencoder. arXiv preprint arXiv:2008.01995 (2020). URL:https://arxiv.org/abs/2008.01995 [10] D. Chevitarese, D. Szwarcman, R. M. D. Silva, E. V. Brazil, Transfer learning applied to seismic images classification. AAPG Annual and Exhibition (2018). URL:https://www.searchanddiscovery.com/documents/2018/42285chevitarese/ndx_chevitarese.p df [11] Y. Alaudah, S. Gao, G. AlRegib, Learning to label seismic structures with deconvolution networks and weak labels, in: SEG Technical Program Expanded Abstracts 2018, pp. 2121-2125. doi:10.1190/segam2018-2997865.1 [12] A. Saleem, J. Choi, D. Yoon, J. Byun, Facies classification using semi-supervised deep learning with pseudo-labeling strategy, in: SEG Technical Program Expanded Abstracts 2019, pp. 31713175. doi:10.1190/segam2019-3216086.1 [13] Y. Babakhin, A. Sanakoyeu, and H. Kitamura, “Semi-supervised Segmentation of Salt Bodies in Seismic Images Using an Ensemble of Convolutional Neural Networks,” Pattern Recognition 2019, pp. 218–231. doi:10.1007/978-3-030-33676-9_15 [14] M. Tan, Q. Le, Efficientnet: Rethinking model scaling for convolutional neural networks, in: Proceedings of International Conference on Machine Learning 2019, pp. 6105-6114 [15] B. Baheti, S. Innani, S. Gajre, S. Talbar, Eff-unet: A novel architecture for semantic segmentation in unstructured environment, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 2020, pp. 358-359. doi:10.1109/cvprw50498.2020.00187 [16] L. D. Huynh, N. Boutry, A U-Net++ With pre-trained EfficientNet backbone for segmentation of diseases and artifacts in endoscopy images and videos, in: EndoCV@ ISBI 2020, pp. 13-17.
URL:http://ceur-ws.org/Vol-2595/endoCV2020_paper_id_11.pdf [17] A. G. Roy, N. Navab, C. Wachinger, Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks, in: Proceedings of International conference on medical image computing and computer-assisted intervention 2018, pp. 421-429. doi:10.1007/978-3-030-00928-1_48 [18] L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L.Yuille, Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062 (2014).
URL:https://arxiv.org/abs/1412.7062 [19] A. A. Novikov, D. Major, M. Wimmer, D. Lenis, K. Bühler, Deep sequential segmentation of organs in volumetric medical scans. IEEE transactions on medical imaging, 38(5), (2018) 12071215. doi:10.1109/tmi.2018.2881678 [20] A. Pfeuffer, K. Schulz, K. Dietmayer, Semantic segmentation of video sequences with convolutional lstms, in: Proceedings of IV IEEE Intelligent Vehicles Symposium 2019, pp. 14411447. doi:10.1109/ivs.2019.8813852 [21] F. Milletari, N. Navab, S. A. Ahmadi, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in: Proceedings of 4th international conference on 3D vision 2016, pp. 565-571. doi:10.1109/3dv.2016.79 [22] M. Q. Nasim, T. Maiti, A. Shrivastava, T. Singh, J. Mei, Seismic facies analysis: a deep domain adaptation approach. arXiv preprint arXiv:2011.10510 (2020).
URL:https://arxiv.org/abs/2011.10510 [23] M. H. Vu, G. Grimbergen, T. Nyholm, T. Löfstedt, Evaluation of multislice inputs to convolutional neural networks for medical image segmentation. Medical Physics, 47(12), (2020) 6216–6231. doi:10.1002/mp.14391