Keywords

1 Introduction

Silkworm industry is an important economic source in some areas of southern China and silkworm disease has an important impact on silkworm breeding. The recognition of silkworm disease is the primary task to prevent and control silkworm disease, and it is mainly based on the subjective judgment of silkworm experts in the past, which the efficiency is low and the intensity is high. Therefore, the requirements of online experts system of silkworm diseases recognition based on mobile application come into being. At present, machine learning has been widely used in the fields of bio-metrics (such as face recognition) and license plate recognition. The use of image processing to solve specific problems has become more and more widespread in various industries. With the advancement of machine learning technology and the popularity of mobile Internet applications, many mobile applications based on intelligent recognition systems have emerged, such as the “flower companion” App for flower and grass recognition and the Taobao online commodity purchasing system. However, there is currently no nationwide coverage of silkworm disease automatic recognition mobile phone App that can directly serve for silkworm farmers.

At the same time, for the scattered breeding of silkworm farmers, mainly rely on the reading of silkworm disease prevention manual, using text-based silkworm disease retrieval diagnostic rules as the basis for reasoning. Because most of the symptoms of silkworm disease in manual are terminology, for some silkworm farmers who do not have professional knowledge background, it is difficult to grasp the way of silkworm disease by text description. In addition, in the past, most systems for the recognition of silkworm diseases were based on a single machine or a Web system, and the recognition process required computer terminal, which was not very convenient. The applications of artificial intelligence, system engineering and other multidisciplinary for the judgment and prevention of silkworm disease is of great significance to ensure the normal production of silkworm farmers. Flexible mobile devices have better portability and popularity, and can provide silkworm disease recognition and diagnosis services to silkworm farmers anytime and anywhere. However, this work is full of challenges. First, the structure, shape and details of the silkworm itself are complex. In the natural environment, it is highly susceptible to some background factors, such as the occlusion of mulberry leaves and changes in lighting conditions, which makes the recognition of silkworm diseases very different. Second, the existing silkworm disease data is scarce and messy, and the data set standards are not uniform. The third is the lack of application and system design for the recognition of silkworm disease.

In this paper, aiming at the technical requirements of silkworm farmers for accurate recognition and effective prevention of silkworm disease, we have developed a set of silkworm disease automatic detection and recognition system based mobile App, improving methods, achieving accurate recognition and prevention, filling the gaps in the industry. It is foreseeable that the App has broad prospects, great economic and social values.

2 Related Work

With the development and popularization of smart phones, the realization of silkworm disease recognition system based on mobile application is feasible and convenient. The design and development of silkworm recognition systems are relatively few at home and abroad.

However, there have been good progress and research on retrieval system in other fields. Dai et al. [1] developed an Android system for searching cotton pests and diseases, which realized the automatic diagnosis of cotton pests and diseases. Kumar et al. [2] developed the iOS software Leafsnap, which can identify trees. Zhao et al. [3] developed an Android application that can identify plant species. Qi et al. [4] developed an Android software for face detection. Wu et al. [5] developed an Android software for locating images. Li et al. [6] used different kernel function training support vector machines to realize the classification and recognition of cucumber, and the final result confirmed that SVM has good generalization ability in disease detection. Yang et al. [7] used color moments and lab operators as features to conduct SVM learning and training, and successfully identified various diseases of barley. Carmargo and Smith [8] used SVM to identify plant diseases in color images, extracted features such as gray scales on plant leaves, and used SVM to identify them, achieving a high recognition rate. Patil et al. [9] used SVM to classify cotton lesions, which had two processing: firstly, using pre-processing to extract image boundary and texture features, then using SVM to classify and recognize the features, the experiment results show that it may obtain a good precision. Sannakki et al. [10] used SVM to detect pomegranate disease. Chang [11] used SVM to study wheat pathological diseases.

Based on the research and reference of exiting image retrieval systems in different fields at home and abroad, by selecting and testing many classic classifiers under the factors of comprehensive retrieval efficiency and accuracy, we have developed a silkworm disease recognition system based on mobile App.

3 System Design

3.1 Overall System Design

To realize the recognition of silkworm disease based on the Android platform, the system is divided into a server module and a mobile client module. The system architecture is shown in Fig. 1.

Fig. 1.
figure 1

Overall system architecture.

Full size image

In the Fig. 1, the mobile clients and servers use wireless networks for data transmission. The mobile phone client is mainly responsible for real-time collection and automatic upload of local silkworm disease images, that is, taking photos by viewfinder frame or selecting photos from local album, and transferring the designated images to the server. The server processes the image according to the received image, extracts the image features, classifies them by the trained classifier, and returns the retrieval results to the mobile phone client. The process of silkworm disease recognition is shown in Fig. 2.

Fig. 2.
figure 2

Silkworm disease recognition process.

Full size image

3.2 Client Module Design

Using Java to develop mobile App on the Android platform, we have designed the mobile client interface. In order to better communicate with the server, the registered users log in according to the user name and password. The image example can be collected at mobile phone by taking photos or selecting photos, while the retrieval results can be pushed to the mobile phone by the server.

3.3 Server Module Design

The important function of the server is to search and classify the images example sent by the mobile client. The main parts of the server module are the image pre-processing, the image feature extraction and the generation of the classification model, which the basic process includes below.

Image Pre-processing.

In this paper, silkworm image samples are collected from the natural state, including 542 sick silkworm image samples and 610 normal silkworm image samples. Since the number of silkworm image samples is scarce, it is easy to produce an over-fitting in training, so image enhancement is performed. The basic enhanced method is to mirror, rotate and pan the existing silkworm image samples. After the silkworm image samples are expanded, 2,604 silkworm disease samples and 2,928 silkworm disease-free samples are obtained. Since the image is easily affected by lighting, environment and other conditions during the shooting process, there will be some noise and other interference in the image. Therefore, some simple pre-processing of the image is required, and then the image is derived from the characteristics of the silkworm disease itself. In this paper, the algorithm of marker and watershed is used to segment the image. The extracted marker is used as the minimum value of the gradient image, the gradient image is modified, and then the watershed algorithm is used to reconstruct the image to complete the image segmentation [12]. Finally, the image is subjected to adaptive threshold segmentation and morphological transformation to complete image pre-processing. The image pre-processing process is shown in Fig. 3.

Fig. 3.
figure 3

Image pre-processing process.

Full size image

Image Feature Extraction.

This paper uses color moments to describe image color features because useful color information is concentrated in the first, second and third moments. Compared to other traditional color features, the use of the color moment method does not require more processing of color features, which is a very big advantage. A color feature is formed by taking 3 low-order moments for each component on the 3 components of the image and a total of 9 components are required. In practical applications, color moments are often rarely used alone because it is global in color description. The first, second and third moments of the two systems RGB and HSV are evaluated to identify the image color information.

The formula for calculating the first moment of R, G and B, or H, S and V is:

$$ {\upmu } = \frac{1}{p}\sum\nolimits_{j = 1}^{p} {f_{j} p\left( {f_{i} } \right)} $$
(1)

The second-order moments of R, G and B, or H, S and V are calculated as:

$$ {\upsigma }^{2} = \frac{1}{P}\sum\nolimits_{j = 1}^{p} {\left( {f_{i} - \mu } \right)^{2} p\left( {f_{i} } \right)} $$
(2)

The third-order moments for R, G and B, or H, S and V are calculated as:

$$ {\varepsilon }^{3 } = \frac{1}{P}\sum\nolimits_{j = 1}^{P} {\left( {f_{j} - \mu } \right)^{3} p\left( {f_{i} } \right)} $$
(3)

Equations (1) to (3) constitutes the characteristics of the image color moment.

This paper uses the LBP operator to extract image texture features, while the traditional LBP operator is sensitive to subtle texture features. Because the images retrieved by the system come from the mobile phone client, they are easily affected by light, plant leaf occlusion and other factors, which the noise interference is a serious problem to be solved. So the backend system uses the LBP operator with parameters to extract the texture features of the captured image [13]. Add a parameter \( \alpha \) to the calculation process when the difference between the adjacent pixel and the central pixel is less than \( \alpha \). At the same time, the adjacent pixel is still marked as 0. The image processing process based on the LBP operator is shown in Fig. 4.

Fig. 4.
figure 4

Image processing process based on the LBP operator.

Full size image

After obtaining the LBP encoding of the pixel, the LBP encoded value of all the pixels of entire image is counted.

The formula for calculating the LBP encoded value after adding parameters is as follows:

$$ {\text{LBP}}_{{{\text{P}},{\text{R}}}} = \sum\nolimits_{0}^{{{\text{P}} - 1}} {{\text{s}}\left( {{\text{g}}_{\text{p}} - {\text{g}}_{\text{c}} } \right)2^{\text{p}} } $$
(4)
$$ {\text{s}}\left( {\text{x}} \right) = \left\{ {\begin{array}{*{20}c} {0, \;\;{\text{x}} < 0} \\ {1, \;\;{\text{x}} \ge 0} \\ \end{array} } \right. $$
(5)

Classification by Classifier.

The SVM classifier and the KNN classifier are used in the experiment, while the color moment and the LBP feature vector are as input to the classifier. Use the cross-validation method to select the optimal model. The positive sample of silkworm disease is 2,604 and the normal silkworm sample is 2,928, while 108 of them are taken out as test sets. Since the “s” takes a value of 10, the training set is divided into 10 parts, 9 parts as a training set and 1 part as verification set. Then calculate the accuracy of each verification set, which the results are shown in Table 1. The model is with the highest accuracy as the final test model. The test model is tested with the test set and the final test error is as the standard deviation for the model. The classifier work flow is shown in Fig. 5.

Table 1. Image feature extraction training results.
Full size table
Fig. 5.
figure 5

Classifier work flow.

Full size image

4 System Testing and Analysis of Experimental Results

4.1 System Testing

The system development environment is as follows:

  • *Development language: Java, Python

  • *Android platform: Android system 4.0 or above

  • *Operating system platform: CentOS 7.3

The mobile client user interface is shown in Fig. 6. There are two modes for user to collect the image example by mobile phone, taking photos by viewfinder frame or selecting photos from local album. When collecting the image example by taking photos, the user only needs to click the camera button, as shown in Fig. 6(a), or by selecting photos from the mobile phone album, as shown in Fig. 6(b). After the photo is collected and sent, the backend retrieval system returns the retrieval result at once, as shown in Fig. 6(c) and (d), which the Fig. 6(c) and (d) show the retrieval results of two different silkworm diseases. With the increasing of test data, the system can achieve higher recognition rate. Many of test results show that the system runs stably, and the retrieval accuracy and respond time are satisfactory.

Fig. 6.
figure 6

Mobile client image retrieval interface.

Full size image

4.2 Analysis of Experimental Results

Accuracy Test.

This paper mainly uses the KNN and SVM classifiers to train the extracted image features. As mentioned above, the accuracy results obtained by the s-fold cross-validation are shown in Table 1.

From the data analysis in Table 1, it can be shown that SVM has strong generalization ability and small computational cost compared to KNN. The KNN algorithm is more suitable for classification with larger sample sizes, but it is more likely to cause misclassification for smaller sample sizes, while the SVM can solve the problem in small sample cases. The test results show that SVM is better than KNN algorithm, which proves that SVM has strong generalization ability for classification of insufficient data.

Further experimental analysis shows that the mean of the cross-validation method under the KNN classifier is 0.681 and the standard deviation is 0.0246, while the mean obtained under the SVM classifier is 0.736 and the standard deviation is 0.0304. The best model is selected from different models and the average retrieval result is used to test the recognition effect, which the system response time is less than 0.5 s, the retrieval accuracy rate is 73.8% and the recall rate is 75.6%.

This system has certain characteristics compared with the domestic similar system, the “Flower Companion”. The image retrieval interface of the “Flower Companion” at the mobile client is shown in Fig. 7. After testing, the “Flower Companion” App has a recognition rate of less than 60% for most kinds of flowers and plants.

Fig. 7.
figure 7

“Flower Companion” App mobile client image retrieval interface.

Full size image

Real-Time Test.

The real-time test results for the silkworm disease recognition system based on mobile App are shown in Table 2.

Table 2. Real-time test results.
Full size table

According to the data analysis in Table 2, the different models of Huawei mobile phones are selected to test for silkworm disease recognition system. The results show that the retrieval time can be completed within 0.5 s, and the system has good fluency and user experience.

Application Scenario Test.

In order to test the recognition effect of the system in various scenarios, the experiment uses the control variable method to compare the recognition results under different conditions. We conducted experimental design based on the knowledge provided by the sericulture experts, and divided the test evaluation criteria into three levels: good recognition, qualified recognition, bad recognition. The experiment performs 240 tests on each group of scenes, while the result is judged as good recognition when the recognition ratio is greater than 80%, qualified recognition when it is between 50% and 80%, and bad recognition when it is smaller than 50%. The recognition results are shown in Table 3.

Table 3. Application scenario test results.
Full size table

From the data analysis in Table 3, we can see whether the silkworm body is occluded by the leaves, and whether the light intensity will affect the retrieval results. However, after a large number of experimental results analysis, the system has better recognition effect in the case of good illumination, while qualified recognition effect under the condition of blurred background, which can satisfy the basic application needs.

5 Conclusions

In this paper, a silkworm disease recognition system based on mobile App is designed for several typical silkworm diseases, which the client platform is developed based on the Android platform and the App can get the silkworm disease recognition results by taking photo in real-time or selecting photo from local album. Firstly, the captured image example of silkworm body is pre-processed. Then, the silkworm disease features are extracted by the color moment and the LBP features, and the optimal classification model is gotten by the SVM training. Finally, the best retrieval result is achieved by matching features between the image example at client and the image database at the server. The experimental results show that the system can effectually achieve the retrieval of several typical silkworm diseases with higher retrieval accuracy and shorter respond time, which can meet the basic application needs. Due to the captured samples of different silkworm diseases are not enough, some comparative tests between a few kinds of typical silkworm diseases and normal silkworm are complied in the paper, which can only make high accuracy of classification between the white muscardine silkworm and normal silkworm, but the extraction of more silkworm disease features can not achieve better results. Thus, it provides a large margin for further improving the recognition rate and more tests will be done in the future to reach higher application levels.