Skip to main content

Advertisement

SpringerLink
Log in

A unified framework for interactive image segmentation via Fisher rules

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Interactive image segmentation has been an active research topic in image processing and computer graphics. One of its appealing advantage is the optimization of human feedback and interactions to generate user-desired results. Segmentation results of most previous methods usually depend on the quality and quantity of the user input. In this paper, we propose an algorithm to solve one important challenge arising from inputs with bad or limited quality. Our work is notably different from previous methods. First, the weakly interactive image segmentation is formulated and deduced in theory, then we propose to reconstruct enough samples via sparse reconstruction to enhance the robustness to weakly interactive labels. More importantly, we leverage interactive labels to extract a latent subspace which jointly optimizes multiclass classification and binary classification based on fisher rules. Numerous experiments are conducted on MSRC (Ning et al. in Interact Imaging Vis Pattern Recognit 43(2):445–456, 2010) and KIM (Kim et al. in: 2010 IEEE computer society conference on computer vision and pattern recognition, 2010) database. The results demonstrate effectiveness and efficiency of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bai, J., Wu, X.: Error-tolerant scribbles based interactive image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 392–399 (2014)

  2. Bai, X., Sapiro, G.: A geodesic framework for fast interactive image and video segmentation and matting. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)

  3. Bini, A.A., Bhat, M.S.: A nonlinear level set model for image deblurring and denoising. Vis. Comput. 30(3), 311–325 (2014)

    Article  Google Scholar 

  4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)

    MATH  Google Scholar 

  5. Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive Image Segmentation Using an Adaptive GMMRF Model, pp. 428–441. Springer, Berlin (2004)

    Google Scholar 

  6. Chen, Y.-N., Lin, H.-T.: Feature-aware label space dimension reduction for multi-label classification. In: Advances in Neural Information Processing Systems, pp. 1529–1537 (2012)

  7. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  8. Deeba, F., Bui, F.M., Wahid, K.A.: Automated growcut for segmentation of endoscopic images. In: 2016 International Joint Conference on Neural Networks, IJCNN 2016, Vancouver, BC, Canada, July 24–29, 2016, pp. 4650–4657 (2016)

  9. Dong, X., Shen, J., Shao, L., Van Gool, L.: Sub-markov random walk for image segmentation. IEEE Trans. Image Process. 25(2), 516–527 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  10. Donoser, M., Bischof, H.: Roi-seg: Unsupervised color segmentation by combining differently focused sub results. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  11. Duchenne, O., Audibert, J.Y., Keriven, R., Ponce, J., Segonne, F.: Segmentation by transduction. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  12. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)

    Article  Google Scholar 

  13. Friedland, G., Jantz, K., Rojas, R.: Siox: simple interactive object extraction in still images. In: Seventh IEEE International Symposium on Multimedia (ISM’05), 7 pp (2005)

  14. Ham, B., Min, D., Sohn, K.: A generalized random walk with restart and its application in depth up-sampling and interactive segmentation. IEEE Trans. Image Process. 22(7), 2574–2588 (2013)

    Article  Google Scholar 

  15. Hosni, A., Rhemann, C., Bleyer, M., Rother, C., Gelautz, M.: Fast cost-volume filtering for visual correspondence and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 504–511 (2013)

    Article  Google Scholar 

  16. Jian, M., Jung, C.: Interactive image segmentation using adaptive constraint propagation. IEEE Trans. Image Process. 25(3), 1301–1311 (2016)

    MathSciNet  MATH  Google Scholar 

  17. Kim, T. H., Lee, K. M., Lee, S. U.: Nonparametric higher-order learning for interactive segmentation. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3201–3208 (2010)

  18. Li, H., Wen, W., Enhua, W.: Robust interactive image segmentation via graph-based manifold ranking. Comput. Vis. Media 1(3), 183–195 (2015)

    Article  Google Scholar 

  19. Li, K., Tao, W.: Adaptive optimal shape prior for easy interactive object segmentation. IEEE Trans. Multimed. 17(7), 994–1005 (2015)

    Article  MathSciNet  Google Scholar 

  20. Li, Y., Sun, J., Tang, C.-K., Shum, H.-Y.: Lazy snapping. ACM Trans. Graph. 23(3), 303–308 (2004)

    Article  Google Scholar 

  21. Luo, L., Wang, X., Shiqiang, H., Xin, H., Chen, L.: Interactive image segmentation based on samples reconstruction and FLDA. J. Vis. Commun. Image Represent. 43, 138–151 (2017)

    Article  Google Scholar 

  22. Michailidis, G.-T., Pajarola, R.: Bayesian graph-cut optimization for wall surfaces reconstruction in indoor environments. Vis. Comput. 33(10), 1347–1355 (2017)

    Article  Google Scholar 

  23. Mille, J., Bougleux, S., Cohen, L.D.: Combination of piecewise-geodesic paths for interactive segmentation. Int. J. Comput. Vis. 112(1), 1–22 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Mumford, D., Shah, J.: Boundary detection by minimizing functionals. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 17, pp. 137–154. San Francisco (1985)

  25. Ning, J., Zhang, L., Zhang, D., Wu, C.: Interactive image segmentation by maximal similarity based region merging. Interact. Imaging Vis. Pattern Recognit. 43(2), 445–456 (2010)

    Article  MATH  Google Scholar 

  26. Pan, R., Taubin, G.: Automatic segmentation of point clouds from multi-view reconstruction using graph-cut. Vis. Comput. 32(5), 601–609 (2016)

    Article  Google Scholar 

  27. Rother, C., Kolmogorov, V., Blake, A.: “Grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)

    Article  Google Scholar 

  28. Rubinstein, R., Bruckstein, A.M., Elad, M.: Dictionaries for sparse representation modeling. Proc IEEE 98(6), 1045–1057 (2010)

    Article  Google Scholar 

  29. Sourati, J., Erdogmus, D., Dy, J.G., Brooks, D.H.: Accelerated learning-based interactive image segmentation using pairwise constraints. IEEE Trans. Image Process. 23(7), 3057–3070 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  30. Subr, K., Paris, S., Soler, C., Kautz, J.: Accurate binary image selection from inaccurate user input. In: Computer Graphics Forum, vol. 32, pp. 41–50. Wiley Online Library (2013)

  31. Tropp, J.A., Wright, S.J.: Computational methods for sparse solution of linear inverse problems. Proc IEEE 98(6), 948–958 (2010)

    Article  Google Scholar 

  32. Vapnik, V.N., Vapnik, V.: Statistical Learning Theory, vol. 1. Wiley, New York (1998)

    MATH  Google Scholar 

  33. Wang, L., Pan, C.: Robust level set image segmentation via a local correntropy-based k-means clustering. Pattern Recognit. 47(5), 1917–1925 (2014)

    Article  Google Scholar 

  34. Wang, T., Ji, Z., Sun, Q.-S., Chen, Q., Jing, X.-Y.: Interactive multilabel image segmentation via robust multilayer graph constraints. IEEE Trans. Multimed. 18(12), 2358–2371 (2016)

    Article  Google Scholar 

  35. Wang, T., Sun, Q.-S., Ji, Z., Chen, Q., Peng, F.: Multi-layer graph constraints for interactive image segmentation via game theory. Pattern Recognit. 55, 28–44 (2016)

    Article  MATH  Google Scholar 

  36. Wang, T., Wang, H., Fan, L.: A weakly supervised geodesic level set framework for interactive image segmentation. Neurocomputing 168, 55–64 (2015)

    Article  Google Scholar 

  37. Wang, W., Shen, J.: Higher-order image co-segmentation. IEEE Trans. Multimed. 18(6), 1011–1021 (2016)

    Google Scholar 

  38. Wang, X., Tang, Y., Masnou, S., Chen, L.: A global/local affinity graph for image segmentation. IEEE Trans. Image Process. 24(4), 1399–1411 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  39. Welling, M.: Fisher linear discriminant analysis. Department of Computer Science, University of Toronto, 3:1–4 (2005)

  40. Wu, J., Zhao, Y., Zhu, J.-Y., Luo, S., Tu, Z.: Milcut: A sweeping line multiple instance learning paradigm for interactive image segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

  41. Xiao, C., Gan, J., Xiangyun, H.: Fast level set image and video segmentation using new evolution indicator operators. Vis. Comput. 29(1), 27–39 (2013)

    Article  Google Scholar 

  42. Yang, W., Cai, J., Zheng, J., Luo, J.: User-friendly interactive image segmentation through unified combinatorial user inputs. IEEE Trans. Image Process. 19(9), 2470–2479 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  43. Yao, J., Huimin, Y., Roland, H.: A new sparse representation-based object segmentation framework. Vis. Comput. 33(2), 179–192 (2017)

    Article  Google Scholar 

  44. Yin, S., Zhao, X., Wang, W., Gong, M.: Efficient multilevel image segmentation through fuzzy entropy maximization and graph cut optimization. Pattern Recognit. 47(9), 2894–2907 (2014)

    Article  Google Scholar 

  45. Zemene, E., Pelillo, M.: Interactive image segmentation using constrained dominant sets. In: European Conference on Computer Vision, pp. 278–294. Springer (2016)

  46. Zhang, L., Gao, Y., Xia, Y., Lu, K., Shen, J., Ji, R.: Representative discovery of structure cues for weakly-supervised image segmentation. IEEE Trans. Multimed. 16(2), 470–479 (2014)

    Article  Google Scholar 

  47. Zhu, H., Meng, F., Cai, J., Shijian, L.: Beyond pixels: a comprehensive survey from bottom-up to semantic image segmentation and cosegmentation. J. Vis. Commun. Image Represent. 34, 12–27 (2016)

    Article  Google Scholar 

  48. Zhu, S.C., Yuille, A.: Region competition: unifying snakes, region growing, and bayes/mdl for multiband image segmentation. IEEE Trans. Pattern Anal. Mach Intell. 18(9), 884–900 (1996)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Aeronautics and Astronautics, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, China

    Lingkun Luo, Shiqiang Hu, Xing Hu & Huanlong Zhang

  2. LIRIS, UMR 5205 CNRS, Ecole Centrale Lyon, 36 Avenue Guy de Collongue, Ecully, 69130, France

    Lingkun Luo & Xiaofang Wang

  3. Department of Automation, University of Science and Technology of China, Hefei, 230026, China

    Yaohua Liu

  4. Institute for Intelligent Systems Research and Innovation, Deakin University, Melbourne, Australia

    James Zhang

Authors
  1. Lingkun Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaofang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiqiang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xing Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huanlong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yaohua Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. James Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiqiang Hu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, L., Wang, X., Hu, S. et al. A unified framework for interactive image segmentation via Fisher rules. Vis Comput 35, 1869–1882 (2019). https://doi.org/10.1007/s00371-018-1580-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1580-0

Keywords

Search

Navigation