MSIM: A change detection framework for damage assessment in natural disasters
Introduction
Before, during and after the natural disasters, information about the damages and the current situations are vital for people to make decisions for their next actions. For example, the Nepal earthquake in 2015 did a huge damage to everything, including buildings, roads, infrastructure, and people, which resulted in 8,019 people died and 17,866 people injured; in a Tokyo earthquake, people are still able to walk back home since the earthquake damage the infrastructure, but not the buildings and paths. A recent study (Hiroi, Sekiya, Nakajima, Waragai, & Hanahara, 2011) found that people would like to know earthquake size and epicentre. Moreover, knowing these information could prevent the secondary and tertiary disasters. It is also found that in Tokyo, only 67.8% of people managed to get back their house on the day of an earthquake, and the rest 32.2% had to become “homeless”, among them 2% failed to going back home only because they could not find the safe path. People need the information about earthquakes, but there is always the question “How could the people get the information of earthquakes?”.
Recently, many researchers had approached to detect the damages caused by natural disasters by using the change detection techniques with the aerial images. However, the aerial images consume longer time to retrieve and harder to get compare to the other images. On the other hand, social media is pervasive, and updates very quickly especially on large events, e.g., natural disasters, by millions of people all the time. Thus, we investigate the problem of change detection from social images, so as to let the public aware of the latest situations of natural disasters on the spot. One of the challenges here is the large-scale of social images, which makes current change detection techniques infeasible if not impossible. The limitation of using the large-scale images with current change detection techniques is time cost. As existing techniques detect changes based on pixel level comparison (İlsever & Unsalan, 2012) without index support or query optimization or both, the time cost for image comparison is high. When applying them to large scale social images, the efficiency issue becomes even unacceptable. The second challenge is the unavailability of some special features, like building shady, used in traditional change detection (Turker & Sumer, 2008). In shared communities, most social images do not have shady because of low shooting angles, thus the shady-based matching cannot be conducted. Forcing the existing techniques on the social images will cause low detection quality. Finally, traditional change detection for aerial images suppose the image pairs are known to the same location points, which is not true in media shared communities.
To address these challenges, we have proposed a basic solution (Kito et al., 2017) for effective and efficient change detection from social community. Each image is represented by its local interest descriptors called Principal Component Analysis-Scale Invariant Feature Transform (PCA-SIFT). Then each image pair from the same source is identified by one-to-one matching over their PCA-SIFT descriptors. After that, boundaries of each image are modeled as a number of Relative Position Annulus (RPA) representations, which shows the difference of neighbouring edge lengths and is robust to the view point rotation and other global transformations of the same objects in images. Finally, the changes of each image pair during disasters are detected by proposing an extended dynamic time wrapping metric over the RPA representations. In this paper, we extend our previous work with a novel semantic data model over locations and tags, and propose a hash-based clustering method to further improve the efficiency when identifying each image pair from the same source, so the whole process of change detection can be conducted in real time. Specifically, we first represent the metadata of each image as a set of weighted tags, and propose a Social Image Similarity function (SIS) over image metadata and locations. Then, we extend the recursive 2-means clustering algorithm (Shen, Ooi, & Zhou, 2005) from the vector space under to the key word set space, so the Jaccard-based SIS measure can be applied. A hash-based technique is applied to improve the efficiency of clustering process. By deploying this extended recursive 2-means clustering over the whole image dataset, a number of small clusters are generated. Images in the same cluster or neighbouring clusters will have the high probability of being the image pairs of the sources. Following that, we conduct PCA-SIFT based matching, which determines whether two images are really referring to the same source. We have conducted experimental evaluation over two large real datasets to demonstrate the effectiveness and efficiency of our framework. To summarize, the contributions of this paper are as follows:
- •
We propose a new social image model by exploiting tags and locations and a new similarity function SIS over it. The new model can effectively and efficiently capture the similarity between images in terms of semantics.
- •
We propose an SIS-based clustering method by extending the recursive 2-means algorithm, then deploy a PCA-SIFT-based matching over images for finding the images pairs of the same source. With clustering, the copies can be detected by only checking the images in neighbouring clusters.
- •
We use djb2 hashing techniques to reduce one by one tag comparisons, so the efficiency of whole process of SIS-based clustering is greatly improved.
- •
We conduct extensive experiments on two large real image datasets to verify the performance of our proposed solution in terms of effectiveness and efficiency.
The rest of the paper is structured as follows: Section 2 reviews the related work; Section 3 presents our change detection framework; Section 4 details the social image modelling; Section 5 presents the proposed change detection method; Section 6 includes the experiment evaluation; Section 7 concludes the paper.
Section snippets
Related work
In this section, we review the existing research closely related to this work, including the image copy detection and change detection.
Framework
In this section, we first define two terms, change and change detection, and then present the overview of the change detection framework for social images.
Definition 1 In image processing, change is defined as the difference between two pixels or the objects in different images. The difference varies in different situations. In this paper, the difference is limited to the damage to the roads, the buildings or the infrastructures, which are caused by natural disasters. For instance, when a bridge breaks
Data modelling
This section presents how to model images by tags and uploading location information. Intuitively, while tags describe the content information of images conceptually, the location information is the key feature to evaluate whether two different images come from the same place. When images are uploaded in social communities such as Flickr, users attach tags and locations with them. We collect tags and location to model social images. The missing tags can be supplemented by tag recommendation
Change detection over large data sets
This section presents our change detection method, including how to identify the image pairs from the same source and how to detect the changes in images.
Experiment
This section examines the effectiveness and efficiency of the proposed method.
Conclusion
In this paper, we study the problem of change detection from media sharing communities for damage assessment in natural disasters. First, we propose a new concept-level image data model over tags and locations, and extend recursive 2-means clustering to our model for fast detection of images from the same sources. Then, we propose a robust boundary representation model together with the matching over it for effective change detection for damage assessment. Finally, we have conducted extensive
References (29)
Fast hashing of variable-length text strings
Communications of the ACM
(1990)- et al.
Data mining: A preprocessing engine
Journal of Computer Science
(2006) - Appleby, A. (2008). Murmurhash...
- et al.
Surf: Speeded up robust features
Computer vision–ECCV
(2006) - et al.
Using dynamic time warping to find patterns in time series
International conference on knowledge discovery and data mining
(1994) - et al.
Finding the Jaccard median
Proceedings of the twenty-first annual ACM-SIAM symposium on discrete algorithms
(2010) - et al.
Earthquake-induced building damage detection with post-event sub-meter vhr terrasar-x staring spotlight imagery
Remote Sensing
(2016) - et al.
Questionnaire survey concerning stranded commuters in metropolitan area in the Great East Japan Earthquake
The Journal of Social Safety Science
(2011) - et al.
Two-dimensional change detection methods: Remote sensing applications
SpringerBriefs in computer science
(2012) Standardization and its effects on k-means clustering algorithm
Research Journal of Applied Sciences, Engineering and Technology
(2013)
A comparison of sift, pca-sift and surf
International Journal of Image Processing
Pca-sift: A more distinctive representation for local image descriptors
IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Change detection from media sharing community
Asia-Pacific web (APWeb) and web-age information management joint conference on web and big data
Scalability of local image descriptors: A comparative study
ACM international conference on multimedia
Cited by (18)
Superpixel-based multiobjective change detection based on self-adaptive neighborhood-based binary differential evolution
2023, Expert Systems with ApplicationsCitation Excerpt :In the field of remote sensing (RS) (Kurban, 2022; Zhang & Zhu, 2011), it obtains the RS images of the same area in different periods and analyzes the remote sensing image information so as to detect changes of the land surface of the geographical region. At present, it has attracted extensive interest due to a great quantity of applications in diverse fields such as disaster assessment (Qin, et al., 2018), land use monitoring (Gupta et al., 2019), urban expansion monitoring (Bouziani et al., 2010; Tison et al., 2004) and so on. As a common category of RS images, SAR images have strong penetration ability and accurate azimuth resolution as well as range resolution (Dong, et al., 2021).
Large-scale deep learning based binary and semantic change detection in ultra high resolution remote sensing imagery: From benchmark datasets to urban application
2022, ISPRS Journal of Photogrammetry and Remote SensingCitation Excerpt :The purpose of change detection is to discover land-cover changes and interpret the process of feature change, where the results determine which objects have changed, both when and where. This information can provide help for urban construction (Huang et al., 2017; Daudt et al., 2018b; Amini Amirkolaee and Arefi, 2019; Tian et al., 2021), disaster emergency response (Vetrivel et al., 2018; Qin et al., 2018; Liu et al., 2019a; Zhang et al., 2020b), environmental protection (Bindschadler et al., 2010; Tian et al., 2014; Zhu et al., 2019; Ochtyra et al., 2020), and other applications (Jin et al., 2017; Hu et al., 2018; Maiya and Babu, 2018; Liu et al., 2019b), thereby assisting with decision-making. As the process of urbanization has accelerated in recent years, how to better detect urban changes has become a research focus.
Assessment of land use land cover change detection in multitemporal satellite images using machine learning algorithms
2021, Cognitive Systems and Signal Processing in Image ProcessingAccelerated genetic algorithm based on search-space decomposition for change detection in remote sensing images
2019, Applied Soft Computing JournalCitation Excerpt :Change detection is one of the most important research topics in remote sensing image processing, and has been widely used in environmental monitoring [1], natural disaster damage assessment [2], forest resources monitoring [3], agricultural surveys [4], and so on.
C2F-SemiCD: A Coarse-To-Fine Semi-Supervised Change Detection Method Based on Consistency Regularization in High-Resolution Remote Sensing Images
2024, IEEE Transactions on Geoscience and Remote SensingChange detection - siamese based framework to detect changes over the earth’s surface (CD-CSNN)
2024, Multimedia Tools and Applications