Abstract
The detection and observation of mitotic event are the key to studying the behavior of the cell and used to examine various diseases. The existing cell detection methods are performed on two-dimensional images with time sequence. However, the complexity of mitotic and normal cells and the orientation of the mitosis generate high false positive when using 2D methods. On the other hand, 3D methods can perform higher performance than 2D methods but also face the problem of overfitting due to the limit of training data. With those problems, we propose a 2.5-dimensional convolutional neural network with convolutional long short-term memory to extract the information time sequence and combined with 3D anchors to gather the spatial information for final mitotic detection. Furthermore, we also propose the method with a parallel model on multi-GPUs to speed up the detection time. Compared with state-of-the-art methods, our method can reach high precision and also recall rate with detection time is speed up about 1.9 times by the use of the parallel model on 4GPUs.
Similar content being viewed by others
References
Ipponjima S, Hibi T, Nemoto T (2016) Three-Dimensional Analysis of Cell Division Orientation in Epidermal Basal Layer Using Intravital Two-Photon Microscopy, PLOS one
Hsu YC, Li L, Fuchs E (2014) Emerging interactions between skin stem cells and their niches. Nat Med 20(8):847–856
Jones P, Simons BD (2008) Epidermal homeostasis: do committed progenitors work while stem cells sleep? Nat Rev Mol Cell Biol 9(1):82–88
Watt FM (2014) Mammalian skin cell biology: at the interface between laboratory and clini. Science 346(6212):937–940
van Diest PJ, Baak JPA, Matze-Cok P, Wisse-Brekelmans ECM, van Galen CM, Kurver PHJ et al (1992) Reproducibility of mitosis counting in 2,469 breast cancer specimens: Results from the Multicenter Morphometric Mammary Carcinoma Project. Human Pathology 23(6):603–607
Konno D, Shioi G, Shitamukai A, Mori A, Kiyonari H, Miyata T et al (2008) Neuroepithelial progenitors undergo LGN-dependent planar divisions to maintain self-renewability during mammalian neurogenesis. Nat Cell Biol. 93–101
Wu M, Smith CL, Hall JA, Lee I, Luby-Phelps K, Tallquist MD (2010) Epicardial spindle orientation controls cell entry into the myocardium. Dev Cell. 114–125
Mao Y, Yin Z (2016) A hierarchical convolutional neural network for mitosis detection in phase-contrast microscopy images. Medical Image Computing and Computer-Assisted Intervention - MICCAI 685–692
Mao Y, Yin Z (2017) Two-Stream Bidirectional Long Short-Term Memory for Mitosis Event Detection and Stage Localization in Phase-Contrast Microscopy Images, Medical Image Computing and Computer-Assisted Intervention - MICCAI
Yang W, Li K, Mo Z, Li K (2015) Performance Optimization Using Partitioned SpMV on GPUs and Multicore CPUs. IEEE Transaction on Computers 64(9):2623–2636
Xiao G, Li K CASpMV: A Customized and Accelerative SpMV Framework for the Sunway TaihuLight, IEEE Transactions on Parallel and Distributed Systems, https://doi.org/10.1109/TPDS.2019.2907537
Chen Y, Li K (2019) Performance-Aware Model for Sparse Matrix-Matrix Multiplication on the Sunway TaihuLight Supercomputer. IEEE Transactions on Parallel and Distributed Systems 30(4):923–938
Chen J, Li K, Bilal K, Zhou X, Li K, Yu PS (2019) A Bi-layered Parallel Training Architecture for Large-scale Convolutional Neural Networks. IEEE Transactions on Parallel and Distributed Systems 30(5):965–976
Chen J, Li K, Deng Q, Li K, Yu PS (2019) Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/TII.2019.2909473
Kitrungrotsakul T, Han XH, Iwamoto Y, Takemoto S, Yokota H, Ipponjima S, Nemoto T, Wei X, Chen YW (2018) A 2.5D Cascaded Convolutional Neural Network with Temporal Information for Automatic Mitotic Cell Detection in 4D Microscopic Images, Proc. of ICNC-FSKD2018
Hegde V, Usmani S (2016) Parallel and Distributed Deep Learning
Ben-Nun T, Hoefler T (2018) Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis, arXiv:1802.09941
Xiao G, Li K, Zhou X, Li K (2017) Efficient monochromatic and bichromatic probabilistic reverse top-k query processing for uncertain big data. Journal of Computer and System Sciences 89:92–113
Xiao G, Li K, Li K (2017) Reporting l Most Influential Objects in Uncertain Databases Based on Probabilistic Reverse Top-k Queries. Information Sciences 405:207–226
Xiao G, Li K, Li K, Zhou X (2015) Efficient Top-(k, l) Range Query Processing for Uncertain Data Based on Multicore Architectures. Distributed and Parallel Databases 33(3):381–483
Xiao G, Li K, Zhou X, Li K (2016) Queueing Analysis of Continuous Queries for Uncertain Data Streams Over Sliding Windows. International Journal of Pattern Recognition and Artificial Intelligence 30(9):1–16
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, NIPS
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Alexander C (2016) SSD: Single Shot MultiBox Detector, ECCV
Huh S, Ker DF, Bise R, Chen M, Kanade T (2011) Automated mitosis detection of stem cell populations in phase contrast microscopy image. IEEE Trans. Med. Imag. 30(3):586–596
Suykens J, Vandewalle J (1999) least squares support vector machine classifiers. Neural Process. Nett 9(3):293–300
Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature Pyramid Networks for Object Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 936–944:
Simonyan K, Zisserman A (2014) Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv technical report
He K, Zhang X, Ren S, Sun J (2015) Deep Residual Learning for Image Recognition, arXiv preprint arXiv:1512.03385
Sauvola J, Pietikainen M (2000) Adaptive document image binarization. Pattern Recognition 33(2):225–236
Erik M (2012) Cell segmentation: 50 years down the road [life sciences]
Shkolyar A, Gefen A, Benayahu D, Greenspan H (2015) Automatic detection of cell divisions (mitosis) in live-imaging microscopy images using Convolutional Neural Networks. Engineering in Medicine and Biology Society (EMBC) 743–746
Xingjian S, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutational LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. NIPS 802–810
TC-IAIP AIA2017, http://www.tc-iaip.org/index-e.shtml
Chollet F et al (2015) Keras, https://github.com/fchollet/keras
Abadi M et al. (2015) TensorFlow: large-scale machine learning on heterogeneous systems. http://tensorflow.org
Sugano J (2017) Mitotic cell division event detection using classification of temporal feature histogram, ViEW 2017 visual inspection algorithm competition
Kitrungrotsakul T, Iwamoto Y, Han XH, Takemoto S, Yokota H, Ipponjima S, Nemoto T, Xiong W, Chen YW A (2019) Cascade of 2.5D CNN and Bidirectional CLSTM Network for Mitotic Cell Detection in 4D Microscopy Image, IEEE/ACM Trans. Computational Biology and Bioinformatics, in press
Kitrungrotsakul T, Iwamoto Y, Han XH, Takemoto S, Yokota H, Ipponjima S, Nemoto T, Xiong W, Chen YW (2019) A Cascade of CNN and LSTM Network with 3D Anchors for Mitotic Cell Detection in 4D Microscopy Image, Proc. of IEEE ICASSP, pp.1239-1243
Acknowledgements
This work is supported in part by Japan Society for Promotion of Science (JSPS) under Grant Nos. 16J09596 and KAKEN under the Grant Nos. 18H04747, 16H01436, 15H05954, 15H05953; in part by Zhejiang Lab Program under the Grant No. 2018DG0ZX01 and in part by A*STAR Research Attachment Programme.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kitrungrotsakul, T., Han, XH., Iwamoto, Y. et al. An end-to-end CNN and LSTM network with 3D anchors for mitotic cell detection in 4D microscopic images and its parallel implementation on multiple GPUs. Neural Comput & Applic 32, 5669–5679 (2020). https://doi.org/10.1007/s00521-019-04374-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-019-04374-8