A local region model with attentive temporal-spatial pathways is proposed for automatically learning various target structures. The attentive spatial pathway highlights the salient region to generate bounding boxes and ignores irrelevant regions in an input image. The proposed attention mechanism allows eficient object localization, and the overall predictive performance is increased because there are fewer false positives for the object detection task for medical images with manual annotations. The experimental results show that proposed models consistently increase the base architecture's predictive performance on the Medico dataset with satisfactory computational eficiency.
This study proposes a simple and efective solution that interfaces
an attention mechanism in a standard CNN model. The feature
maps are utilized more eficiently, and localization does not require
processing the entire image. The proposed attentive model, which
consists of tempo-spatial pathways, automatically learns to focus on
target structures without additional supervision. The spatial
pathway generates local region proposals on-the-fly using the salient
features for a specific task. The temporal attention model proposes
a sequence of locations for the local region search and not the
entire image, so the computational overhead is significantly reduced,
and many model parameters are omitted, similarly to multi-model
frameworks. CNN models that use the proposed attentive model
can be trained from scratch using standard methods or transfer
learning. Similar attention mechanisms have been proposed for
natural image classification and captioning [
This study uses a novel hybrid attention model (HAM) as an
interface between any feature extractors, such as a CNN, and a
decision-making module for end-to-end tasks, such as RL,
classification, regression. The proposed module determines spatial pinpoints
in feature space using a hard attestation pathway. The model also
synthesizes the context vector using a soft attention mechanism
and a GRU for decision-making downstream. Real images are used
to determine the eficacy of the proposed model and are used as
a pre-training data set for detection and classification for
colonoscopic images [
A hybrid attention approach allows an attention mechanism specific to local regions and the subsequent strategy or decisionmaking process. This improved model performs better than stateof-art methods that use global or local search schemes.
An attention interface is used for region proposals and sequential
search of glimpses on local regions simultaneously for medical
images. The proposed attention interface, which can be trained
from end to end, replaces the hard-attention approaches currently
used only for image classification. It eliminates the need for the
global generation of bounding boxes for a Faster R-CNN [
A masking scheme is applied to the distribution of attention scores to increase computational eficiency, instead of imposing directly on the feature map and influencing downstream operations. It ensures better classification performance than the baseline approach. It is shown that attention maps and an observation pinpoint allow fewer glimpses and fewer useful observations. A modification to the standard FPN is used for feature extraction, so the process is sensitive and specific.
The process for the proposed local search method for polyps
detection involves two stages [
Instead of an exhaustive search over the entire image, the proposed method uses a Faster RCNN for a sequential search directed by a hybrid attention module (HAM) to determine glimpse regions that are likely to contain an object. RoI’s are generated in a restricted area, where target objects are likely to be located. This local search reduces the amount of calculation for insignificant ROIs. The proposed model has four modules: a CNN-based feature extractor, the proposed HAM, a local RPN, and a detector for bounding box regression and object classification. Glimpse regions are pinpointed, and the length of the sequence of glimpses is determined sequentially. The local RPN generates bounding boxes of diferent sizes and aspect ratios within a glimpsed region. The detector regresses bounding boxes and classifies objects. The architecture of the HAM is shown in Figure 1.
The experiments were executed using the Ubuntu 18.04 operating
system, Python 3.7, Tensorflow. The data sets for the experiments
are provided in Medico Challenge [
The results for the colonoscopy dataset in Figure 2 show that the HAM-beta and HAM-beta-mask are similar to drl-RPN in terms of 50.There are fewer average glimpses and a smaller average glimpsed area than for the drl-RPN, and the AP density and glimpse contribution are better than peer methods.
The drl-RPN must search three times for important areas before terminating the glimpsing process, requiring more computation time. The HAM-beta and HAM-beta-mask accurately locate the correct in the first time search.
This study proposes an innovative attention module that uses soft and hard attention. This module can interface with any architecture that involves simultaneous spatial and temporal tasks, such as polys detection. A global search scans the entire image in an object detection task, but it requires much time and resources. The proposed approach obviates the need to use an extra model by learning to highlight salient local regions in images. The proposed temporalspatial attention module leverages the salient information in the state space for a policy learner, such as reinforcement learning, in addition to object detection in image tasks.
This work is supported by the grant of the Key Project of Yiwu Science and Technology plan, China. No.20-3-067.