From fat droplets to floating forests: cross-domain transfer learning using a PatchGAN-based segmentation model Kameswara Bharadwaj Mantha1,*,† , Ramanakumar Sankar1,*,† , Yuping Zheng1 , Lucy Fortson1 , Thomas Pengo2 , Douglas Mashek3 , Mark Sanders3 , Trace Christensen4 , Jeffrey Salisbury4 , Laura Trouille5 , Jarrett E. K. Byrnes6 , Isaac Rosenthal6 , Henry Houskeeper7 and Kyle Cavanaugh7 1 School of Physics & Astronomy, University of Minnesota, Twin Cities, 116 Church St SE, Minneapolis, MN, 55455 2 University of Minnesota Informatics Institute, 2231 6th St SE, Minneapolis, MN, 55455 3 Medical School, University of Minnesota, Twin Cities, 420 Delaware Street SE, Minneapolis, MN, 55455 4 Mayo Clinic, 200 First Street SW, Rochester, MN, 55905 5 Adler Planetarium, 1300 S DuSable Lake Shore Dr., Chicago, IL 60605 6 Department of Biology, University of Massachusetts Boston 100 Morrissey Blvd; Boston, MA, 02125 7 Department of Geography, University of California Los Angeles, Los Angeles, CA 90095 Abstract Many scientific domains gather sufficient labels to train machine algorithms through human-in-the-loop techniques provided by the Zooniverse.org citizen science platform. As the range of projects, task types and data rates increase, acceleration of model training is of paramount concern to focus volunteer effort where most needed. The application of Transfer Learning (TL) between Zooniverse projects holds promise as a solution. However, understanding the effectiveness of TL approaches that pretrain on large-scale generic image sets vs. images with similar characteristics possibly from similar tasks is an open challenge. We apply a generative segmentation model on two Zooniverse project-based data sets: (1) to identify fat droplets in liver cells (FatChecker; FC) and (2) the identification of kelp beds in satellite images (Floating Forests; FF) through transfer learning from the first project. We compare and contrast its performance with a TL model based on the COCO image set, and subsequently with baseline counterparts. We find that both the FC and COCO TL models perform better than the baseline cases when using > 75% of the original training sample size. The COCO-based TL model generally performs better than the FC-based one, likely due to its generalized features. Our investigations provide important insights into usage of TL approaches on multi-domain data hosted across different Zooniverse projects, enabling future projects to accelerate task completion. Keywords datasets, generative adversarial neural networks, UNET generator, patch-based discriminator, focal tversky loss, transfer learning 1. Introduction ing labels to train machine learning algorithms typically training models from scratch e.g., [3, 4, 5, 6, 7, 8, 9, 10, 11]. Citizen Science has established itself as a valuable method To accelerate labeling efficiencies across the platform, the for distributed data analysis enabling research teams from Zooniverse human-machine system should take advan- diverse domains to solve problems involving large quan- tage of transfer learning techniques, especially when tities of data with complexity levels requiring human volunteer engagement is at a premium. When applying pattern recognition capabilities [1, 2]. As the largest cit- transfer learning, a new project would require fewer la- izen science platform, Zooniverse.org has enabled over bels from volunteers to achieve the same performance as 2.5 million volunteers to provide over half a billion an- training a model from scratch. Volunteer labelers would notations on hundreds of projects across the sciences thus be able to focus on tasks more suited to humans and humanities. Many of these projects use the result- such as anomaly detection e.g., [12]. Transfer learning (TL) is an established approach, Human-in-the-loop data curation workshop at ACM CIKM 2022, Oct where the feature space from a pretrained model can 17–21, 2022, Atlanta, GA * Corresponding author. be transferred to another framework and fine tuned to KBM and RS contributed equally to the majority of this research. perform analogous or different tasks. Feature extraction † Additional authors contributed to specific aspects including initial is typically performed using Deep Convolutional Neu- models, data sets and project or platform development. ral Networks (CNNs) such as [13, 14]. Transfer learning " manth145@umn.edu (K. B. Mantha); rsankar@umn.edu generally uses models trained on data that is either “out- (R. Sankar) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License of-domain” (i.e., training data characteristics are different CEUR Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 from data at hand) or “in-domain” (data that are similar or closely relatable to the data at hand). Quantifying the 2.1. PatchGAN Framework gains provided by these different TL approaches is an The implemented PatchGAN framework is inherited from active area of research, where studies find several factors the Pix2Pix GAN architecture in [23], which is a condi- to be at play that govern its effectiveness: Accuracy and tional GAN for realizing paired image-to-image transla- architecture choice of the pretrained model [15], robust- tion. The PatchGAN architecture consists of a Generator ness of model to input adversarial noise [16], and type of (𝐺) and Discriminator (𝐷): task to which the TL is being applied [17]. Recent works The generator is composed of a U-Net [20], a U-shaped (e.g., [12, 8]) have demonstrated that transfer learning encoder-decoder neural network, with skip connections from a model pretrained on in-domain data performs bet- across the bottleneck layer (Figure 1). The encoder (de- ter than transfer learning from out-of-domain data. On coder) comprises of 6 downsampling (upsampling) blocks, the other hand, some studies find that TL models based each consisting of 4 × 4 convolution (transposed con- on out-of-domain data (e.g., ImageNet or COCO datasets) volution), Leaky ReLU activation, and a batch normal- perform on par with or better than the in-domain TL ization layer. All the blocks in the inner layers of the models [18, 19]. network also include a dropout layer which omits 50% In order to leverage the Zooniverse’s large library of of the extracted features during training. The outputs of image-label pairs across multiple domains, there is thus the transposed convolutions are also concatenated with a clear need to better understand the effectiveness of the corresponding skip connection feature map from the cross-domain transfer learning. In particular, we are in- encoder block. terested in the application of transfer learning specifically to projects that share task similarity across a wide range of domains. For example, image segmentation tasks vary across vastly different disciplines, from cell biology to satellite imagery. Frameworks such as the U-Net [20], Recurrent Convolutional Networks such as Mask-RCNNs [21], and Generative Adversarial Networks (GANs; e.g., [22, 23]) have been used to perform such object segmenta- tion across multiple domains and data sets. However, ro- bust learning of such segmentation models from scratch often requires large annotated training samples that may not be available (e.g., medical imaging), which can lead to poor generalizability of the learnt features to newer data, even in related domains. While Zooniverse can provide these large annotation sets per project, this comes at the cost of volunteer effort which we seek to optimize. In an effort to increase project completion rates, this study investigates potential machine performance gains Figure 1: U-Net Generator (top) and Discriminator (bottom) through transfer learning across domains by leveraging of our PatchGAN framework. the shared task similarity between Zooniverse projects. We use a PatchGAN-based [23] segmentation model1 The discriminator is a patch-wise binary classifier that to investigate the effectiveness of segmenting kelp beds takes a concatenation of the input image and its corre- from satellite images. Particularly, we test transfer learn- sponding ground truth or generated mask and outputs a ing from the COCO dataset (i.e., out-of-domain) and mi- 30 × 30 probability matrix. Each unit of this matrix rep- croscopy imaging of lipid droplets in liver cells (pseudo- resents a 70 × 70 patch of the input image, and provides in-domain) and compare them to their corresponding the probability that the patch is real. “trained from scratch” counterparts. 2.2. Data 2. Methods For this study, we use three sources for our image-mask In this section, we detail our PatchGAN architecture [23], pairs: the Floating Forests dataset, Etch-a-Cell dataset the training and testing data and its preparation, and the and the COCO-stuff. The former two are Zooniverse description of the five models analyzed in our work. projects focusing on image segmentation, while the lat- ter represents a generic image dataset that is used in computer vision, representing an out-of-domain dataset compared to the former two. These three data sources 1 https://github.com/ramanakumars/patchGAN/ represent a diverse feature set on which to perform our transfer learning experiment. Figure 2 shows an example of an image-mask pair from each dataset. 2.2.1. Floating Forests (𝐹 𝐹 ) Floating Forests is an ecology-based citizen science project hosted on Zooniverse.org2 to identify kelp beds in Landsat imagery. The project presents segments of Land- sat data to Zooniverse volunteers, who draw outlines around the kelp beds. These annotations are aggregated using a pixel-by-pixel consensus to create masks of the kelp beds in the corresponding Landsat segments. We use 4 channels from the Landsat data (Blue, Green, Red and near Infrared) to train the patchGAN on the image- mask pairs. This FF data comprises 6,967 (350 × 350 pix) image-mask pairs. We pre-process these data such that each pair is cropped into four 256 × 256 overlapping cutouts, and augment each crop 5 times (rotation and flipping). This resulted in 118, 440 training and 4180 testing images. Figure 2: Visualization of example input image, truth mask, and patchGAN predicted output mask. 2.2.2. Etch-a-Cell: Fat Checker (𝐹 𝐶) Etch-a-Cell: Fat Checker is a cell biology project hosted the ‘person’ class. This amounts to 63785 training and on Zooniverse.org3 to identify lipid droplets in electron 2673 testing image-mask pairs. microscopy data. The Zooniverse project presents 2D slices of the data to volunteers who annotate the outline 2.3. Experimental Design of the lipid droplet. The lipid mask is generated by ag- In this work, we investigate the potential of cross-domain gregating the annotations by multiple volunteers based transfer learning by training 5 models. The first 3 on consensus. The data set consists of 2341 image-mask models are trained from scratch – Λ𝐹 𝐹 , Λ𝐹 𝐶 , and pairs and each image is 1200 × 1200 pix in shape, with Λ𝐶𝑂𝐶𝑂 – using 100% of their corresponding data sets 3 channels. We split the sample into 2, 106 training and 𝐹 𝐹 , 𝐹 𝐶, and 𝐶𝑂𝐶𝑂, respectively. Next, we train 235 testing sets. We transform these images and masks the Λ𝐹 𝐶→𝐹 𝐹 and Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 by transferring the to work with our PatchGAN framework by resizing them weights from the trained Λ𝐹 𝐶 and Λ𝐶𝑂𝐶𝑂 models to to 512 × 512 pix and generating five crops (four corners the Λ𝐹 𝐹 . By comparing between the baseline Λ𝐹 𝐹 to and one center crop). We further augment them by apply- the transfer learnt models Λ𝐹 𝐶→𝐹 𝐹 and Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 , ing three rotations (90, 180, 270 deg) per image, yielding we quantify the impact of performing transfer learning augmented training and testing samples of 42120 and on the accelerated learning of the Λ𝐹 𝐹 model from 4700 images, respectively. two distinct feature initializations. During this transfer learning exercise, we also vary the amount of training 2.2.3. COCO-Stuff data used from 10%-100%. The Common Objects in COntext (COCO; [24]) is a large collection of several real-world images with objects set in 3. Training & Results various simple to complex scenes, which are annotated In this section, we outline the training strategy and pro- by outlines4 . [25] further processed the COCO data set vide details of the hyper parameters. We also present the to produce dense pixel-wise annotations for them (the results of our training and discuss the outcomes of our COCO-Stuff data set; hereafter COCO). These images transfer learning exercise. and annotated masks vary widely in their shapes, and therefore, we standardize these images by resizing them 3.1. Training Strategy to a 256 × 256 pix shape. For our PatchGAN training, Our Λ , Λ , and Λ 𝐹𝐹 𝐹𝐶 𝐶𝑂𝐶𝑂 models have been trained we limit the training and testing data to those that host for 50 epochs. For the generator, we use the Focal Tversky 2 https://www.zooniverse.org/projects/zooniverse/floating-forests/ Loss (FTL; [26]), which is a generalized version of the 3 https://www.zooniverse.org/projects/dwright04/ Tversky Loss (TL) defined in terms of the Tversky Index etch-a-cell-fat-checker 4 https://github.com/nightrome/cocostuff Figure 3: Comparison of generated mask from different model runs on the Floating Forests data, showing different perfor- mance gains from transfer learning. (TI) as: 𝑇𝑃 𝑇𝐼 = → 𝑇 𝐿 = (1 − 𝑇 𝐼) → 𝐹 𝑇 𝐿 = (𝑇 𝐿)𝛾 , 𝑇 𝑃 + 𝛼𝐹 𝑁 + 𝛽𝐹 𝑃 (1) For our training, we use 𝛼 = 0.7 and 𝛽 = 0.3. The 𝛾 parameter controls the non-linearity of the TL with re- spect to the 𝑇 𝐼, enabling the learning to focus on easier (𝛾 < 1) vs. harder (𝛾 > 1) examples. We use 𝛾 = 0.75 during our training. For the discriminator optimization, we use the Binary Cross-Entropy (BCE) loss. Specifically, our total discriminator loss is the average of two compo- Figure 4: Comparison of mean final loss on Floating Forests nents: the discriminator applied on the generated mask validation data across the different models. (i.e., against a fake label), and applied on the true mask (i.e., the real label). For both the generator and discrimi- model training runs. As expected, larger training sam- nator, we use the Adam optimizer with an initial learning ples provide much better performance, but we also find rate 5 × 10−4 and 1 × 10−4 respectively, decayed expo- that the model pretrained on the COCO dataset provides nentially by 𝜏 = 0.95, applied every 5 epochs. noticeably better performance on the Floating Forests 3.2. Transfer learning strategy data, compared to both Λ𝐹 𝐶→𝐹 𝐹 and also Λ𝐹 𝐹 . In fact, the Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 is able to match the performance of the For our transfer learning based model training of Λ𝐹 𝐹 model with between 50-75% of the training Floating Λ𝐹 𝐶→𝐹 𝐹 and Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 , we load the weights of the Forests dataset. Λ𝐹 𝐶 and Λ𝐶𝑂𝐶𝑂 models into the freshly initialized Λ𝐹 𝐹 In Figure 3, we show examples highlighting the dif- model architecture. To account for the 3 vs 4 channel ference between the generated masks from Λ𝐹 𝐹 and mismatch between the Λ𝐶𝑂𝐶𝑂 , Λ𝐹 𝐶 and Λ𝐹 𝐹 , we load corresponding masks from Λ𝐹 𝐶→𝐹 𝐹 and Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 . model layer parameters excluding the input layer. For The sharpness of the kelp beds is poorly reconstructed each model, we train 5 different versions, using random by the Λ𝐹 𝐹 model but is well captured by the transfer subsets of 10%, 25%, 50%, 75% and 100% of the full learnt models (particularly when training Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 Floating Forests data, to compare TL efficiency gains with more than 75% of the original data). The transfer from having a smaller dataset. For these experiments, learnt models are also better at capturing kelp beds not we also use only the first 6,967 un-augmented images identified in the original consensus data. For example, for re-training. We train the Λ𝐹 𝐶→𝐹 𝐹 and Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 both the ground truth and Λ𝐹 𝐹 fail to reveal the kelp models with the same hyper-parameter settings as the beds in the top left of the image, but these are picked up aforementioned “from scratch” models for 50 epochs. well by the transfer learnt models. This is likely due to the large diversity of the features 3.3. Results and discussion in the COCO dataset, making it a much more robust feature extraction network to transfer learn from. In- We find that our Λ𝐹 𝐹 , Λ𝐹 𝐶 and Λ𝐶𝑂𝐶𝑂 generally pre- deed, compared to Λ𝐹 𝐶→𝐹 𝐹 , the Λ𝐶𝑂𝐶𝑂→𝐹 𝐹 model- dict the annotation masks reasonably well (Figure 2), detected kelp beds are qualitatively better visually (e.g., qualitatively matching with the ground truths. Figures 3 Figure 3), especially at lower training data sizes. This is and 4 show our transfer learning results. In Figure 4, likely compounded with the lower feature diversity in we show our average validation loss for the different both the Floating Forest and Fat Checker data sets, given the fewer number of samples in the training data and low We also thank Lucy Collinson and the Electron Mi- variety in target classes. croscopy Science Technology Platform (The Francis Crick Institute, London UK) for their input into this project. 3.3.1. Transfer learning approaches for citizen This work was supported in part by the Francis Crick science datasets Institute which receives its core funding from Cancer Re- search UK (FC001999), the UK Medical Research Council For the Zooniverse platform, this study provides an av- (FC001999), and the Wellcome Trust (FC001999). This enue to build quick access for projects to use machine project has been made possible in part by grant number learning frameworks for simple tasks (e.g., image seg- 2020-225438 from the Chan Zuckerberg Initiative DAF, mentation), by transfer learning from existing models on an advised fund of Silicon Valley Community Founda- a small sample of volunteer annotated data sets. How- tion (H.S.). This publication uses data generated via the ever, despite the results presented here, there are still Zooniverse.org platform, development of which is funded several key questions which need to be answered: by generous support, including a Global Impact Award Domain dependency: It is unclear how much of the from Google, and by a grant from the Alfred P. Sloan performance gained from COCO was a ‘global truth’. Foundation. That is, whether COCO (or similarly diverse datasets) are immediately applicable to out-of-domain data, for all domains, or if there are domain-specific restrictions References which allow these performance gains to occur on data such as Floating Forests. This requires more experiments [1] L. Trouille, C. J. Lintott, L. F. Fortson, Citi- with increasingly different data sets on Zooniverse to zen science frontiers: Efficiency, engagement, and investigate the range of performance gains possible. serendipitous discovery with human–machine sys- Task dependency: Previous studies on transfer tems, Proceedings of the National Academy of learning across domains show significant variations in Sciences 116 (2019) 1902–1909. URL: https://www. performance across different task types. For example, im- pnas.org/content/116/6/1902. doi:10.1073/pnas. age classification tasks (e.g., [12, 17]) show lower gains 1807190116. than image segmentation based tasks (e.g., [18]). We need [2] L. Fortson, D. Wright, C. Lintott, L. Trouille, to further investigate the inherent difficulty associated Optimizing the human-machine partnership with different tasks on Zooniverse projects, and how ef- with Zooniverse, in: CI 2018: ACM Collective fectively they can be transferred between domains. [12], Intelligence, ACM, http://arxiv.org/abs/1809.09738, for example, show that significant boosts to performance 2018. URL: http://arxiv.org/abs/1809.09738. is only provided by using in-domain transfer learning. arXiv:1809.09738. Target data purity: For Zooniverse projects, data [3] C. N. Beaumont, A. A. Goodman, S. Kendrew, labels are generally provided by volunteers and are ag- J. P. Williams, R. Simpson, The Milky Way gregated based on volunteer consensus. In this study, we Project: Leveraging Citizen Science and Machine found that transfer learning can help mitigate data purity Learning to Detect Interstellar Bubbles, ApJS effects, since transfer learnt feature extraction models 214 (2014) 3. doi:10.1088/0067-0049/214/1/3. are generally robust to mislabeled data. The extent to arXiv:1406.2692. which transfer learning models are sensitive to data pu- [4] M. Zevin, S. Coughlin, S. Bahaadini, E. Besler, N. Ro- rity effects needs to be further investigated. hani, S. Allen, M. Cabero, K. Crowston, A. K. Kat- In conclusion, we find that transfer learning can pro- saggelos, S. L. Larson, et al., Gravity Spy: integrat- vide a significant boost to projects that contain similar ing advanced ligo detector characterization, ma- tasks on Zooniverse. However, the extent to which this chine learning, and citizen science, Classical and can be generalized across the full Zooniverse ecosystem Quantum Gravity 34 (2017) 064003. is a question of ongoing study. [5] M. Norouzzadeh, A. Nguyen, M. Kosmala, A. Swan- son, C. Packer, J. Clune, Automatically identifying wild animals in camera trap images with deep learn- Acknowledgements ing, arXiv preprint arXiv:1703.05830 (2017). [6] D. Wright, C. Lintott, S. Smartt, K. Smith, L. Fortson, The authors would like to thank the Zooniverse volun- L. Trouille, C. Allen, M. Beck, M. Bouslog, A. Boyer, teers without whom this work would not have been possi- K. Chambers, H. Flewelling, W. Granger, E. Mag- ble. RS, KM, LF, YZ, LT would like to acknowledge partial nier, A. McMaster, G. Miller, J. O’Donnell, B. Sim- support from the National Science Foundation under mons, H. Spiers, J. Tonry, M. Veldthuis, R. Wain- grant numbers IIS 2006894 and OAC 1835530. Partial scoat, C. Waters, M. Willman, Z. Wolfenbarger, support by RS, KM, LF, TP, MS, TC, JS is acknowledged D. Young, A transient search using combined hu- through Minnesota Partnership MNP IF#119.09. man and machine classifications, Monthly Notices of deep pre-trained models, International Journal of the Royal Astronomical Society 472 (2017) 1315– of Medical Informatics 159 (2022) 104669. URL: 1323. URL: http://dx.doi.org/10.1093/mnras/stx1812. https://www.sciencedirect.com/science/article/pii/ doi:10.1093/mnras/stx1812. S1386505621002951. doi:https://doi.org/10. [7] H. Domínguez Sánchez, M. Huertas-Company, 1016/j.ijmedinf.2021.104669. M. Bernardi, D. Tuccillo, J. L. Fischer, Improving [16] H. Salman, A. Ilyas, L. Engstrom, A. Kapoor, galaxy morphologies for SDSS with Deep Learning, A. Madry, Do adversarially robust imagenet models Monthly Notices of the Royal Astronomical Society transfer better?, in: H. Larochelle, M. Ranzato, 476 (2018) 3661–3676. URL: https://doi.org/10.1093/ R. Hadsell, M. Balcan, H. Lin (Eds.), Advances in mnras/sty338. doi:10.1093/mnras/sty338. Neural Information Processing Systems, volume 33, [8] M. Willi, R. T. Pitman, A. W. Cardoso, C. Locke, Curran Associates, Inc., 2020, pp. 3533–3545. URL: A. Swanson, A. Boyer, M. Veldthuis, L. Fortson, https://proceedings.neurips.cc/paper/2020/file/ Identifying animal species in camera trap images 24357dd085d2c4b1a88a7e0692e60294-Paper.pdf. using deep learning and citizen science, Methods [17] K. Thenmozhi, U. S. Reddy, Crop pest classification in Ecology and Evolution 10 (2019) 80–91. based on deep convolutional neural network and [9] M. Laraia, D. Wright, H. Dickinson, A. Simenstad, transfer learning, Computers and Electronics in K. Flanagan, S. Serjeant, L. Fortson, VERITAS Col- Agriculture 164 (2019) 104906. laboration, Muon Hunter 2.0: efficient crowdsourc- [18] M. Majurski, P. Manescu, S. Padi, N. Schaub, N. Ho- ing of labels for IACT image analysis, in: 36th taling, C. Simon Jr, P. Bajcsy, Cell image segmenta- International Cosmic Ray Conference (ICRC2019), tion using generative adversarial networks, transfer volume 36 of International Cosmic Ray Conference, learning, and augmentations, in: Proceedings of 2019, p. 678. the IEEE/CVF Conference on Computer Vision and [10] O. Ranadive, S. van der Lee, V. Tang, K. Chao, Ap- Pattern Recognition Workshops, 2019, pp. 0–0. plying Machine Learning to Crowd-sourced Data [19] J. Ma, L. Bao, Q. Lou, D. Kong, Transfer learning for from Earthquake Detective, arXiv e-prints (2020) automatic joint segmentation of thyroid and breast arXiv:2011.04740. arXiv:2011.04740. lesions from ultrasound images, International Jour- [11] H. Spiers, H. Songhurst, L. Nightingale, J. de Folter, nal of Computer Assisted Radiology and Surgery R. Hutchings, C. J. Peddie, A. Weston, A. Strange, 17 (2022) 363–372. S. Hindmarsh, C. Lintott, L. M. Collinson, M. L. [20] O. Ronneberger, P. Fischer, T. Brox, U-net: Con- Jones, Citizen science, cells and cnns – deep volutional networks for biomedical image segmen- learning for automatic segmentation of the nu- tation, in: International Conference on Medical clear envelope in electron microscopy data, trained image computing and computer-assisted interven- with volunteer segmentations, bioRxiv (2020). tion, Springer, 2015, pp. 234–241. URL: https://www.biorxiv.org/content/early/2020/ [21] K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask r- 07/29/2020.07.28.223024. doi:10.1101/2020.07. cnn, in: Proceedings of the IEEE international con- 28.223024. ference on computer vision, 2017, pp. 2961–2969. [12] M. Walmsley, A. M. M. Scaife, C. Lintott, [22] Y. Huo, Z. Xu, S. Bao, A. Assad, R. G. Abramson, M. Lochner, V. Etsebeth, T. Géron, H. Dickinson, B. A. Landman, Adversarial synthesis learning en- L. Fortson, S. Kruk, K. L. Masters, K. B. Mantha, B. D. ables segmentation without target modality ground Simmons, Practical galaxy morphology tools from truth, in: 2018 IEEE 15th international symposium deep supervised representation learning, Monthly on biomedical imaging (ISBI 2018), IEEE, 2018, pp. Notices of the Royal Astronomical Society 513 1217–1220. (2022) 1581–1599. doi:10.1093/mnras/stac525. [23] P. Isola, J.-Y. Zhu, T. Zhou, A. A. Efros, Image- arXiv:2110.12735. to-image translation with conditional adversarial [13] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learn- networks, in: Proceedings of the IEEE conference ing for image recognition, in: Proceedings of the on computer vision and pattern recognition, 2017, IEEE conference on computer vision and pattern pp. 1125–1134. recognition, 2016, pp. 770–778. [24] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, [14] K. Simonyan, A. Zisserman, Very deep convolu- D. Ramanan, P. Dollár, C. L. Zitnick, Microsoft tional networks for large-scale image recognition, coco: Common objects in context, in: European arXiv preprint arXiv:1409.1556 (2014). conference on computer vision, Springer, 2014, pp. [15] S. Hosseinzadeh Kassani, P. Hosseinzadeh Kassani, 740–755. M. J. Wesolowski, K. A. Schneider, R. Deters, Deep [25] H. Caesar, J. Uijlings, V. Ferrari, Coco-stuff: Thing transfer learning based model for colorectal cancer and stuff classes in context, in: Computer vision and histopathology segmentation: A comparative study pattern recognition (CVPR), 2018 IEEE conference on, IEEE, 2018. [26] N. Abraham, N. Mefraz Khan, A Novel Focal Tversky loss function with improved Attention U- Net for lesion segmentation, arXiv e-prints (2018) arXiv:1810.07842. arXiv:1810.07842.