[go: nahoru, domu]

CN112633416A - Brain CT image classification method fusing multi-scale superpixels - Google Patents

Brain CT image classification method fusing multi-scale superpixels Download PDF

Info

Publication number
CN112633416A
CN112633416A CN202110058684.0A CN202110058684A CN112633416A CN 112633416 A CN112633416 A CN 112633416A CN 202110058684 A CN202110058684 A CN 202110058684A CN 112633416 A CN112633416 A CN 112633416A
Authority
CN
China
Prior art keywords
scale
brain
image
classification
superpixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110058684.0A
Other languages
Chinese (zh)
Inventor
冀俊忠
张梦隆
张晓丹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN202110058684.0A priority Critical patent/CN112633416A/en
Publication of CN112633416A publication Critical patent/CN112633416A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/245—Classification techniques relating to the decision surface
    • G06F18/2453—Classification techniques relating to the decision surface non-linear, e.g. polynomial classifier
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Nonlinear Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

A brain CT image classification method fusing multi-scale superpixels belongs to the field of medical image research. The method has the following characteristics: 1) by means of fusion of the multi-scale super-pixels and the brain CT image, redundant image information is removed, and gray level similarity of the focus and surrounding brain tissue pixels is reduced. 2) A multi-scale superpixel encoder based on regions and boundaries is designed, and focus low-level information contained in the multi-scale superpixel is effectively extracted. 3) A fusion model of multi-scale superpixel features is designed, and high-level features extracted by a residual neural network and low-level features of multi-scale superpixels are comprehensively utilized to realize classification of brain CT. 4) Compared with the traditional deep learning algorithm, the method can effectively utilize the focus information contained in the multi-scale superpixel, thereby more accurately classifying the diseases existing in the brain CT image, and the method is reasonable and reliable and can provide powerful help for the classification of the brain CT image.

Description

Brain CT image classification method fusing multi-scale superpixels
Technical Field
The invention belongs to the field of medical image research, and particularly relates to a brain CT image classification method fusing multi-scale superpixels.
Background
The diagnosis of brain injury in clinical emergency is extremely urgent, and even a short time delay may lead to deterioration of the patient's condition. Computed Tomography (CT) is one of the most commonly used diagnostic tools, and has the characteristics of fast imaging, low cost, wide application range, high lesion inspection rate, and the like. Although brain CT can detect key and time-sensitive abnormalities such as intracranial hemorrhage, raised intracranial pressure, and skull fracture, the conventional disease classification method usually requires a radiologist to visually observe information such as the size of the hemorrhage area and estimate the midline offset, which is a relatively time-consuming process. In recent years, with the progress and development of medical imaging technology, the number of brain CT images shows geometric growth, but the growth rate of the number of radiologists is relatively slow, and the cost and the period for cultivating a qualified radiologist are high, so that the work task of the radiologist in the work is increased day by day, and the social problem of difficult medical care is indirectly caused. Therefore, the brain CT automatic classification method can assist the work of the radiologist, improve the diagnosis efficiency, reduce the misdiagnosis and missed diagnosis rate, and has very important practical significance.
In recent years, great success of Deep Learning (DL) in the field of computer vision has also promoted rapid development of medical image analysis technology, and a Convolutional Neural Network (CNN) is a traditional Deep learning model, can capture local region information of an image, extract high-level semantic features, and is widely used for feature extraction and classification tasks of the image. CNN is widely used in medical image recognition and processing, and through continuous iterative optimization, many classifiers based on CNN are constructed.
However, the existing work does not consider the difference between the brain CT image and the natural optical image when using the conventional CNN to extract the brain CT image features: firstly, the brain CT image has low spatial resolution and low contrast; lack of natural visual features such as brightness, color, texture, etc. that are easily recognized; the boundary between the regions is not clear, and the texture difference is not large; images can vary significantly due to patient-to-patient variation and imaging position; the images also have many unstable factors such as displacement-caused artifacts, volume effect-caused errors, and equipment-caused noise. In addition, the medical images mostly belong to hospital private data, privacy protection rules can hinder sharing of the medical images, and the number of data sets can directly influence the deep learning effect. Researches show that the unsupervised generated superpixels are composed of a series of characteristic similar pixel points, the local details of the original image can be reserved, the local characteristics can be highlighted, the extraction and the expression of high-level characteristics of the image are facilitated, the number of the superpixels is far smaller than that of the pixels of the original image, the superpixels are used for replacing the pixels to serve as primitives of image processing, the calculation complexity of subsequent image processing can be greatly reduced, and the efficiency of an image processing algorithm is improved.
Disclosure of Invention
Aiming at the problem that the visual characteristics of the brain CT image are ignored in the existing method, the invention provides a brain CT image classification method fusing multi-scale superpixel (MSF). The method can extract the information of the focus area unsupervised by optimizing the image through multi-scale superpixels, thereby enhancing the expressiveness of the characteristics generated by the residual neural network and improving the accuracy of the classification task.
In order to achieve the purpose, the technical scheme adopted by the invention is a brain CT image classification method fusing multi-scale superpixels. The process of the present invention is shown in FIG. 1 and includes the following steps. 1) Firstly, constructing a data set and preprocessing the data set to obtain multi-scale superpixels; secondly, performing data enhancement through multi-scale super-pixel image fusion to obtain an optimized fusion image; then, a multi-scale superpixel feature coding algorithm based on region and boundary information is adopted to obtain multi-scale superpixel low-level features; finally, classifying the brain CT images by utilizing a multi-scale superpixel feature fusion classification model;
acquiring data and preprocessing:
step (1.1) data: acquisition of brain CT images to construct a data set, each patientData contains RGB matrix generated by brain CT image
Figure BDA0002901648850000031
With brain CT classification label vector Y ═ Y1，Y2，…YT]，YiE {0, 1}, where N represents the image pixel size and T represents the number of disease categories acquired.
Step (1.2) divides all patient data into a training set, a validation set and a test set. Wherein the training set is used for learning parameters of the neural network; the verification set is used for determining a network structure and a hyper-parameter; the test set is used for verifying the neural network classification effect.
Step (1.3) data preprocessing: based on Super Hierarchy segmentation algorithm (SH), for given brain CT image I and set segmentation scale { scale of Super pixel1，scale2，…scalesAnd S represents the number of the set superpixels, and a superpixel map under the S-th segmentation scale is generated
Figure BDA0002901648850000032
The calculation process of (2) is as follows:
Ps＝SH(I，scales)
where S ∈ {1, 2 … S }, scalesFor the s-th segmentation scale, each segmentation scale is calculated to obtain
Figure BDA0002901648850000033
Representing a multi-scale superpixel containing a map of S different-scale superpixels.
Step (2), multi-scale super-pixel image fusion model: for given brain CT image I and multi-scale superpixel thereof
Figure BDA0002901648850000034
The fused image I' is calculated as follows:
Figure BDA0002901648850000041
wherein, f (·) indicates a dot product, f (·) is a SoftMax function,
Figure BDA0002901648850000042
represents the weight of the training, PsWatch holder
Figure BDA0002901648850000043
And the middle s element, W, realizes the self-adaptive distribution of the proportion of each scale.
And (3) multi-scale superpixel feature coding:
step (3.1) for multi-scale superpixel of brain CT image I
Figure BDA0002901648850000044
Scale of middle divisionsSuper pixel map of
Figure BDA0002901648850000045
Generating a set of pixel values thereof
Figure BDA0002901648850000046
To pair
Figure BDA0002901648850000047
Generates a set of mapping matrices for each pixel in the image
Figure BDA0002901648850000048
Wherein the kth mapping matrix
Figure BDA0002901648850000049
Middle i, j element
Figure BDA00029016488500000410
The calculation method of (c) is as follows:
Figure BDA00029016488500000411
wherein k ∈ {1, 2, … scalesTherein of
Figure BDA00029016488500000412
Represents PsPixel value of the kth super pixel, Ms，kRepresenting a pixel value of
Figure BDA00029016488500000413
Is mapped to the super pixel area.
Step (3.2) set M of pairs based on area and boundary informationsEach mapping matrix is encoded to obtain a superpixel map PsCoding results
Figure BDA00029016488500000414
The calculation process is as follows:
Figure BDA00029016488500000415
wherein N is2Representing the number of pixels, s, contained in the superpixel mapkAnd the number of pixel points contained in the kth super pixel region is represented.
Step (3.3) pair
Figure BDA0002901648850000051
Repeating the step (3.1) and the step (3.2) for each superpixel graph, and sequentially generating an encoding result b1，b2，…bSSplicing them into a matrix
Figure BDA0002901648850000052
Obtaining multi-scale superpixels
Figure BDA0002901648850000053
And (4) encoding the multi-scale superpixel feature B.
Step (4), multi-scale superpixel feature fusion classification model:
step (4.1) constructing a Residual neural Network (ResNet) as a main Network, inputting the brain CT fusion image I' extracted in step (2), and selecting ResNetFeature activation output l of last residual structure (Basic Block) of four layers1、l2、l3、l4As a high-level feature.
Step (4.2) performs dimensionality reduction processing on the low-level feature B extracted in step (3.3), and generates features through a convolution layer formed by 256 convolution kernels with the size of 3 multiplied by 3
Figure BDA0002901648850000054
Will f is0From bottom to top in sequence1、l2、l3、l4Performing multi-layer fusion to generate fusion feature f1、f2、f3、f4. Wherein f isi(i ∈ {1, 2, 3, 4}) is calculated as follows: firstly, f is subjected to pooling operation(i-1)Conversion to andifeature matrices of the same size, then l convolutional layers consisting of 256 1 × 1 convolutional kernelsiThe channel number of the channel is converted into 256, and then the characteristic fusion is carried out through matrix addition to obtain the fusion characteristic f of the ith layeri。
Step (4.3) the obtained fusion characteristics f4Inputting a convolution layer, a pooling layer and a full-link layer which are composed of 512 convolution kernels, obtaining classification vectors x with the same length as the label quantity T, and generating a predicted value vector y through Sigmoid linear regression by using the classification vectors x [ y ═ y1，y2，…yT]Wherein y isi∈[0，1]The ith element in x generates the probability y that the corresponding label is a positive exampleiExpressed as: y isi＝Sigmoid(xi) Determining a classification result according to a set classification threshold value t when y isiWhen the value is larger than the set threshold value t, the model judges that the brain CT has the corresponding label disease, if yiAnd if the threshold value is smaller than the set threshold value t, the operation is normal. t is 0.5.
And (4.4) inputting the brain CT image I of the patient and a brain disease classification label Y into the brain CT image classification method fusing the multi-scale superpixels, and then obtaining the probability Y of the tested object belonging to each class. Given a data set of M patients D { (I)1，Y1)，(I2，Y2)，…，(IM，YM) For a given brain CT image }IiCorresponding label YiAnd label predictions y generated by the modeliWe calculate the classification error of each label in the sample by Binary cross entropy loss (BCE loss), and then get the sample error by averaging all label classification errors, and the loss function is calculated as follows:
Figure BDA0002901648850000061
wherein
Figure BDA0002901648850000062
The value of the jth label representing the sample,
Figure BDA0002901648850000063
the value of the jth label predicted by the model is represented, and T represents the number of labels in the sample.
Step (4.5) aiming at the training set in step (1.2), minimizing the loss function in step (4.4) by utilizing Adam adaptive optimization algorithm, observing the classification accuracy rate of the model on the verification set after the model is trained by the training set under different learning rates lambda, and setting the initial value of general lambda to be 10-3To 10-6The next learning rate is set to be 3 times of the last time, and the maximum value of λ is set to be 0.1 to 0.5, and then the learning rate in which the accuracy is highest is selected to train the model.
After all the steps are completed, the new brain CT data set can be input into the model, and the brain CT images are classified according to the prediction result output by the model.
Compared with the prior art, the method has the following obvious advantages and beneficial effects:
the invention provides a brain CT image classification method fusing multi-scale superpixels, which has the following characteristics compared with the traditional image classification network: 1) by means of fusion of the multi-scale super-pixels and the brain CT image, redundant image information is removed, and gray level similarity of the focus and surrounding brain tissue pixels is reduced. 2) A multi-scale superpixel encoder based on regions and boundaries is designed, and focus low-level information contained in the multi-scale superpixel is effectively extracted. 3) A fusion model of multi-scale superpixel features is designed, and high-level features extracted by a residual neural network and low-level features of multi-scale superpixels are comprehensively utilized to realize classification of brain CT. 4) Compared with the traditional deep learning algorithm, the method can effectively utilize the focus information contained in the multi-scale superpixel, thereby more accurately classifying the diseases existing in the brain CT image.
Drawings
FIG. 1: a brain CT image classification method flow chart fusing multi-scale superpixels.
FIG. 2: and fusing the classification model by the multi-scale superpixel features.
FIG. 3: and fusing models of different size features.
FIG. 4: and (3) fusing and visualizing the multi-scale superpixel brain CT image.
Detailed Description
In the present embodiment, the subject of the study is a cerebral hemorrhage patient, but the method is not limited thereto, and a cerebral CT image of a patient with a cerebral disease may be used as the subject of the study. The following takes a real cerebral hemorrhage CT data set as an example, and specifically describes the implementation steps of the method:
acquiring data and preprocessing:
step (1.1) data: the invention uses CQ500 data set (http:// headctstudy. qure. ai/data set) to collect brain CT image and construct data set, practically obtains 451 scan data, total 22773 brain CT images, each patient label information contains 14 diagnosis categories of brain diseases: intracranial hemorrhage, cerebral parenchymal hemorrhage, ventricular hemorrhage, subdural hemorrhage, epidural hemorrhage, subarachnoid hemorrhage, left-side cerebral hemorrhage, right-side cerebral hemorrhage, chronic hemorrhage, fracture, skull fracture, other fractures, midline shift, mass effects. First, for the label of the data set, since three radiologists may have different labels for the same label, we select the choice of most experts as the true label for the case of non-uniform labels. Then, according to the determined intracranial hemorrhage label information, the method willThe data set is divided into 204 cases of patients with confirmed cerebral hemorrhage and 247 cases of patients with undetermined cerebral hemorrhage, 742 images corresponding to the lesion positions are selected from the cases of patients with confirmed cerebral hemorrhage according to the diagnosis type labels, 1045 images at the same positions as the lesion positions in the cases of patients with cerebral hemorrhage are selected from the cases of patients with undetermined hemorrhage, and M is 1787 brain CT images in total to serve as the data set D { (I)1，Y1)，(I2，Y2)，…，(I1787，Y1787)}. Each data contains RGB matrix generated by CT image of brain
Figure BDA0002901648850000081
And T ═ 14 brain disease diagnosis label vector Y ═ Y1，Y2，…Y14]，YiE {0, 1}, where element Y of the tag i1 indicates the presence of a brain disease corresponding to the i-th label in the brain CT image, and YiWhen 0, it is normal.
Step (1.2) divide all patient data into training set, validation set and test set according to 8: 1. Wherein the training set is used for learning parameters of the neural network; the verification set is used for determining a network structure and a hyper-parameter; the test set is used for verifying the neural network classification effect.
Step (1.3) data preprocessing: based on Super Hierarchy segmentation algorithm (SH), a given brain CT image I and a set segmentation scale {5, 10, 15} of 3 superpixels are generated for a given brain CT image I and a set S
Figure BDA0002901648850000082
The calculation process of (2) is as follows:
P1＝SH(I，5)
P2＝SH(I，10)
P3＝SH(I，15)
wherein,
Figure BDA0002901648850000083
representing a multi-scale superpixel containing 3 different-scale superpixel maps.
Step (2) multi-scale superpixelAn image fusion model: for given brain CT image I and multi-scale superpixel thereof
Figure BDA0002901648850000084
Fusing images
Figure BDA0002901648850000085
The calculation process is as follows:
Figure BDA0002901648850000091
wherein, f (·) indicates a dot product, f (·) is a SoftMax function,
Figure BDA0002901648850000092
represents the weight of the training, PsWatch holder
Figure BDA0002901648850000093
And the middle s element, W, realizes the self-adaptive distribution of the proportion of each scale. .
And (3) multi-scale superpixel feature coding:
step (3.1) for multi-scale superpixel of brain CT image I
Figure BDA0002901648850000094
Super pixel map with a median segmentation scale of 5
Figure BDA0002901648850000095
Wherein scale1Generating its set of pixel values 5
Figure BDA0002901648850000096
To pair
Figure BDA0002901648850000097
Generates a set of mapping matrices M for each pixel1＝{M1，1，M1，2，…，M1，5Wherein the kth mapping matrix
Figure BDA0002901648850000098
Middle i, j element
Figure BDA0002901648850000099
The calculation method of (c) is as follows:
Figure BDA00029016488500000910
where k ∈ {1, 2, … 5}, where
Figure BDA00029016488500000911
Represents P1Pixel value of the kth super pixel, M1，kRepresenting a pixel value of
Figure BDA00029016488500000912
Super pixel area mapping of
Step (3.2) set M of pairs based on area and boundary information1Each mapping matrix is encoded to obtain a superpixel map P1Coding results
Figure BDA00029016488500000913
The calculation process is as follows:
Figure BDA00029016488500000914
wherein s iskAnd the number of pixel points contained in the kth super pixel region is represented.
Step (3.3) pair
Figure BDA0002901648850000101
Repeating the step (3.1) and the step (3.2) for each superpixel graph, and sequentially generating an encoding result b1，b2，b3Splicing them into a matrix
Figure BDA0002901648850000102
Obtaining multi-scale superpixels
Figure BDA0002901648850000103
And (4) encoding the multi-scale superpixel feature B.
Step (4), multi-scale superpixel feature fusion classification model:
step (4.1) constructing a 34-Layer residual neural network ResNet-34 as a main network, using the brain CT fusion image I' input extracted in step (2), and selecting the feature activation output of the last residual structure (Basic Block) of four layers in ResNet-34
Figure BDA0002901648850000104
Figure BDA0002901648850000105
As a high-level feature.
Step (4.2) performs dimensionality reduction processing on the low-level feature B extracted in step (3.3), and generates features through a convolution layer formed by 256 convolution kernels with the size of 3 multiplied by 3
Figure BDA0002901648850000106
Will f is0From bottom to top in sequence1、l2、l3、l4Performing multi-layer fusion to generate fusion feature f1、f2、f3、f4. Wherein f isi(i ∈ {1, 2, 3, 4}) is calculated as follows: firstly, f is subjected to pooling operation(i-1)Conversion to andifeature matrices of the same size, then l convolutional layers consisting of 256 1 × 1 convolutional kernelsiThe number of channels is converted into 256, and then characteristic fusion is carried out through matrix addition to obtain the fusion characteristic f of the f-th layeri。
Step (4.3) the obtained fusion characteristics f4Inputting a convolution layer, a pooling layer and a full-link layer which are composed of 512 convolution kernels, obtaining classification vectors x with the same length as the label quantity T, and generating a predicted value vector y through Sigmoid linear regression by using the classification vectors x [ y ═ y1，y2，…y14]Wherein y isi∈[0，1]The ith element in x generates a corresponding tag ofProbability of positive case yiExpressed as: y isi＝Sigmoid(xi) When the classification threshold is set to t 0.5, y isiWhen the value is greater than the set threshold value 0.5, the model judges that the brain CT has the corresponding label disease, if yiAnd if the value is less than the set threshold value of 0.5, the state is normal.
And (4.4) inputting the brain CT image I of the patient and a brain disease classification label Y into the brain CT image classification method fusing the multi-scale superpixels, and then obtaining the probability Y of the tested object belonging to each class. For an input brain CT image I corresponding to a label Y and a label prediction Y generated by a model, we calculate a classification error of each label in a sample by Binary cross entropy loss (BCE loss), and then obtain a sample error by averaging classification errors of all labels, where a loss function is calculated as follows:
Figure BDA0002901648850000111
step (4.5) aiming at the training set in step (1.2), minimizing the loss function in step (4.4) by utilizing Adam adaptive optimization algorithm, observing the classification accuracy rate of the model on the verification set after the model is trained by the training set under different learning rates lambda, and setting the lambda initial value to be 10-5The next learning rate is set to be 3 times of the last time, the maximum lambda value is set to be 0.1, and then the learning rate with the highest accuracy is selected to train the model.
After all the steps are completed, the new brain CT data set can be input into the model, and the brain CT images are classified according to the prediction result output by the model.
To illustrate the beneficial effects of the method of the present invention, in the implementation process, we compare with the traditional classification model ResNet-34 as the backbone network herein, and perform three parts of the brain CT classification model with multi-scale superpixel fusion: the super-pixel image fusion, the super-pixel feature coding algorithm and the different-level feature fusion model are subjected to an ablation experiment to verify the effectiveness of each part, the experiment adopts the currently widely used Accuracy (Accuracy, ACC), Sensitivity (Sensitivity, SEN) and F1 evaluation value (F-score, F) as evaluation indexes, and the experiment results are shown in table 1.
The multi-scale superpixel selection uses a combination of superpixel maps of two scales, 5, 10, 15 and 10, 15, 20. Wherein the experimental control group Baseline is classified by using brain CT images, MSF represents the method of the invention, and MSF001Representing the result of classification using only the feature fusion model using the original image of the brain CT as input; MSF011The method represents that a multi-scale super-pixel image fusion part in the MSF method is removed; MSF101Representing the super-pixel characteristic coding part in the MSF method; MSF100And the method represents the part for removing the multi-scale superpixel image fusion and the superpixel feature coding in the MSF method.
TABLE 1 brain CT classification model contrast experiment with fusion of multi-scale superpixels
Figure BDA0002901648850000121
By comparison of MSF011And the classification effect of MSF can be seen: the classification effect is slightly improved compared with Baseline under the condition of not using multi-scale superpixel brain CT image fusion, but is still not as good as the MSF method. By comparison of MSF101And the classification results of MSF can be seen: although the classification effect is obviously improved compared with Baseline under the condition of not using a superpixel feature coding algorithm, the multi-scale superpixel graph is directly used without reference area and boundary information, the expression is poor, the multi-scale superpixel graph cannot be directly used as a low-level feature, the fusion feature effect with a high-level feature is poor, and the MSF classification effect has a larger difference. By MSF001And Baseline, MSF100And MSF101Two groups of comparison experiments show that the classification effect is remarkably improved by using the feature fusion model compared with the method of only using ResNet to extract features no matter whether the input is an original image or an image subjected to multi-scale super-pixel fusion.
In addition, a weighted graph of the multi-scale superpixel and the weight and a multi-scale superpixel fusion image I' in the multi-scale superpixel fusion process are visually displayed, as shown in FIG. 4. Clearly, the weighted graph well removes redundant information outside the cranium, accurately marks a focus area and does not excessively cut; the fused image I' can obviously distinguish the focus area, reduce the gray level similarity of the focus and the surrounding brain tissue pixels and better express the focus area.
In conclusion, the effectiveness of the MSF method provided by the text in the brain CT image classification task is verified by comparing the Baseline method with the ablation experiment. The fusion image generated by the fusion based on the multi-scale superpixel brain CT image model effectively reduces the noise of the image and accurately divides the focus area; the multi-scale superpixel encoder extracts accurate low-level features, wherein the low-level features comprise region area and boundary information, so that small-area focuses can be better concerned; the feature fusion model generates more discriminative fusion features by fusing two different levels of features, and has more effective expression on focus regions with unfixed areas. Therefore, the method is reasonable and reliable, and can provide powerful help for the classification of the brain CT image.

Claims (1)

1. A brain CT image classification method fusing multi-scale superpixels is characterized by comprising the following steps: firstly, constructing a data set and preprocessing the data set to obtain multi-scale superpixels; secondly, performing data enhancement through multi-scale super-pixel image fusion to obtain an optimized fusion image; then processing the multi-scale superpixel by adopting a feature coding algorithm based on region and boundary information to obtain a multi-scale superpixel low-level feature; finally, classifying the brain CT images by utilizing a multi-scale superpixel feature fusion classification model;
acquiring data and preprocessing:
step (1.1) data: acquiring brain CT images to construct a data set, each patient data comprising its RGB matrix generated from the brain CT images
Figure FDA0002901648840000011
With brain CT classification label vector Y ═ Y1，Y2，…YT]，YiE {0, 1}, where
Figure FDA0002901648840000012
Representing a real number set, N representing the size of an image pixel, and T representing the number of acquired disease categories;
step (1.2) dividing all patient data into a training set, a validation set and a test set; wherein the training set is used for learning parameters of the neural network; the validation set is used for determining the hyper-parameters; the test set is used for verifying the neural network classification effect;
step (1.3) data preprocessing: based on Super Hierarchy segmentation algorithm (SH), for given brain CT image I and set segmentation scale { scale of Super pixel1，scale2，…scaleSAnd S represents the number of the set superpixels, and a superpixel map under the S-th segmentation scale is generated
Figure FDA0002901648840000013
The calculation process of (2) is as follows:
Ps＝SH(I，scales)
where S ∈ {1, 2 … S }, scalesFor the s-th segmentation scale, each segmentation scale is calculated to obtain
Figure FDA0002901648840000014
Representing a multi-scale superpixel comprising a map of S different-scale superpixels;
step (2), multi-scale super-pixel image fusion model: for given brain CT image I and multi-scale superpixel thereof
Figure FDA0002901648840000015
The fused image I' is calculated as follows:
Figure FDA0002901648840000021
wherein, f (·) indicates a dot product, f (·) is a SoftMax function,
Figure FDA0002901648840000022
represents the weight of the training, PsTo represent
Figure FDA0002901648840000023
The middle-s element, W realizes the self-adaptive distribution of the proportion of each scale;
and (3) multi-scale superpixel feature coding:
step (3.1) for multi-scale superpixel of brain CT image I
Figure FDA0002901648840000024
Scale of middle divisionsSuper pixel map of
Figure FDA0002901648840000025
Generating a set of pixel values thereof
Figure FDA0002901648840000026
To pair
Figure FDA0002901648840000027
Generates a set of mapping matrices for each pixel in the image
Figure FDA0002901648840000028
Wherein the kth mapping matrix
Figure FDA0002901648840000029
Middle i, j element
Figure FDA00029016488400000210
The calculation method of (c) is as follows:
Figure FDA00029016488400000211
wherein k ∈ {1, 2, … scalesTherein of
Figure FDA00029016488400000212
Represents PsPixel value of the kth super pixel, Ms，kRepresenting a pixel value of
Figure FDA00029016488400000213
The super pixel area mapping;
step (3.2) set M of pairs based on area and boundary informationsEach mapping matrix is encoded to obtain a superpixel map PsCoding results
Figure FDA00029016488400000214
The calculation process is as follows:
Figure FDA00029016488400000215
wherein N is2Representing the number of pixels, s, contained in the superpixel mapkThe number of pixels included in the kth super pixel region indicates a dot product;
step (3.3) pair
Figure FDA00029016488400000216
Repeating the step (3.1) and the step (3.2) for each superpixel graph, and sequentially generating an encoding result b1，b2，…bSSplicing them into a matrix
Figure FDA00029016488400000217
Obtaining multi-scale superpixels
Figure FDA00029016488400000218
B, multi-scale superpixel feature coding;
step (4), multi-scale superpixel feature fusion classification model:
step (4.1) constructing a residual neural network ResNet as a main network, using the brain CT fusion image I' input extracted in step (2), and selecting the feature activation output l of the last residual structure (Basic Block) of four layers in ResNet1、l2、l3、l4As a high-level feature;
step (4.2) performs dimensionality reduction processing on the low-level feature B extracted in step (3.3), and generates features through a convolution layer formed by 256 convolution kernels with the size of 3 multiplied by 3
Figure FDA0002901648840000031
Will f is0From bottom to top in sequence1、l2、l3、l4Performing multi-layer fusion to generate fusion feature f1、f2、f3、f4(ii) a Wherein f isi(i ∈ {1, 2, 3, 4}) is calculated as follows: firstly, f is subjected to pooling operation(i-1)Conversion to andifeature matrices of the same size, then l convolutional layers consisting of 256 1 × 1 convolutional kernelsiThe channel number of the channel is converted into 256, and then the characteristic fusion is carried out through matrix addition to obtain the fusion characteristic f of the ith layeri；
Step (4.3) the obtained fusion characteristics f4Inputting a convolution layer, a pooling layer and a full-link layer which are composed of 512 convolution kernels, obtaining classification vectors x with the same length as the label quantity T, and generating a predicted value vector y through Sigmoid linear regression by using the classification vectors x [ y ═ y1，y2，…yT]Wherein y isi∈[0，1]The ith element in x generates the probability y that the corresponding label is a positive exampleiExpressed as: y isi＝Sigmoid(xi) Determining a classification result according to a set classification threshold value t when y isiWhen the value is larger than the set threshold value t, the model judges that the brain CT has the corresponding label disease, if yiIf the threshold value is less than the set threshold value t, the operation is normal; t is 0.5;
inputting a patient brain CT image I and a brain disease classification label Y, and then obtaining the probability Y of the tested object belonging to each category; if it is given toData sets D { (I) of M patients were derived1，Y1)，(I2，Y2)，…，(IM，YM) For a given brain CT image IiCorresponding label YiAnd label predictions y generated by the modeliCalculating the classification error of each label in the sample through the two-classification cross entropy loss, and then obtaining the sample error by averaging all the label classification errors, wherein the loss function is calculated as follows:
Figure FDA0002901648840000032
wherein
Figure FDA0002901648840000041
The value of the jth label representing the sample,
Figure FDA0002901648840000042
representing the value of the jth label predicted by the model, and T representing the number of labels in the sample;
step (4.5) aiming at the training set in step (1.2), minimizing the loss function in step (4.4) by utilizing Adam adaptive optimization algorithm, observing the classification accuracy rate of the model on the verification set after the model is trained by the training set under different learning rates lambda, and setting the lambda initial value to be 10-5Setting the next learning rate to be 3 times of the last time, setting the maximum lambda value to be 0.1, and then selecting the learning rate with the highest accuracy to train the model;
and (5) after all the steps are finished, inputting a new brain CT data set into the model, and classifying the brain CT images according to a prediction result output by the model.
CN202110058684.0A 2021-01-16 2021-01-16 Brain CT image classification method fusing multi-scale superpixels Pending CN112633416A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110058684.0A CN112633416A (en) 2021-01-16 2021-01-16 Brain CT image classification method fusing multi-scale superpixels

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110058684.0A CN112633416A (en) 2021-01-16 2021-01-16 Brain CT image classification method fusing multi-scale superpixels

Publications (1)

Publication Number Publication Date
CN112633416A true CN112633416A (en) 2021-04-09

Family

ID=75294799

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110058684.0A Pending CN112633416A (en) 2021-01-16 2021-01-16 Brain CT image classification method fusing multi-scale superpixels

Country Status (1)

Country Link
CN (1) CN112633416A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113362347A (en) * 2021-07-15 2021-09-07 广东工业大学 Image defect region segmentation method and system based on multi-scale superpixel feature enhancement
CN113707278A (en) * 2021-08-30 2021-11-26 北京工业大学 Brain CT medical report generation method based on spatial coding
CN114141339A (en) * 2022-01-26 2022-03-04 杭州未名信科科技有限公司 Pathological image classification method, device, equipment and storage medium for membranous nephropathy

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100172567A1 (en) * 2007-04-17 2010-07-08 Prokoski Francine J System and method for using three dimensional infrared imaging to provide detailed anatomical structure maps
CN108268870A (en) * 2018-01-29 2018-07-10 重庆理工大学 Multi-scale feature fusion ultrasonoscopy semantic segmentation method based on confrontation study
CN110910404A (en) * 2019-11-18 2020-03-24 西南交通大学 Anti-noise data breast ultrasonic nodule segmentation method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100172567A1 (en) * 2007-04-17 2010-07-08 Prokoski Francine J System and method for using three dimensional infrared imaging to provide detailed anatomical structure maps
CN108268870A (en) * 2018-01-29 2018-07-10 重庆理工大学 Multi-scale feature fusion ultrasonoscopy semantic segmentation method based on confrontation study
CN110910404A (en) * 2019-11-18 2020-03-24 西南交通大学 Anti-noise data breast ultrasonic nodule segmentation method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
康文杰 等: "生成对抗网络及其在神经影像应用中的研究进展", 医疗卫生装备, no. 09, 15 September 2020 (2020-09-15) *
王玉丽 等: "基于深度学习的脑图像分割算法研究综述", 生物医学工程学杂志, no. 04, 31 December 2020 (2020-12-31) *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113362347A (en) * 2021-07-15 2021-09-07 广东工业大学 Image defect region segmentation method and system based on multi-scale superpixel feature enhancement
CN113707278A (en) * 2021-08-30 2021-11-26 北京工业大学 Brain CT medical report generation method based on spatial coding
CN113707278B (en) * 2021-08-30 2023-11-03 北京工业大学 Brain CT medical report generation method based on spatial coding
CN114141339A (en) * 2022-01-26 2022-03-04 杭州未名信科科技有限公司 Pathological image classification method, device, equipment and storage medium for membranous nephropathy
CN114141339B (en) * 2022-01-26 2022-08-05 杭州未名信科科技有限公司 Pathological image classification method, device, equipment and storage medium for membranous nephropathy

Similar Documents

Publication Publication Date Title
CN108268870B (en) Multi-scale feature fusion ultrasonic image semantic segmentation method based on counterstudy
CN112633416A (en) Brain CT image classification method fusing multi-scale superpixels
US12008757B2 (en) Method and system for automatic multiple lesion annotation of medical images
CN113744178B (en) Skin lesion segmentation method based on convolution attention model
Rajee et al. Gender classification on digital dental x-ray images using deep convolutional neural network
Angkoso et al. Multiplane Convolutional Neural Network (Mp-CNN) for Alzheimer’s Disease Classification.
CN113850753B (en) Medical image information computing method, device, edge computing equipment and storage medium
CN113012163A (en) Retina blood vessel segmentation method, equipment and storage medium based on multi-scale attention network
CN116664931A (en) Knee osteoarthritis grading method based on quantum-to-classical migration learning
CN115147600A (en) GBM multi-mode MR image segmentation method based on classifier weight converter
CN115661165A (en) Glioma fusion segmentation system and method based on attention enhancement coding and decoding network
CN117036288A (en) Tumor subtype diagnosis method for full-slice pathological image
CN114581459A (en) Improved 3D U-Net model-based segmentation method for image region of interest of preschool child lung
CN117635625A (en) Pancreatic tumor segmentation method based on automatic data enhancement strategy and multi-attention-assisted UNet
CN116563524B (en) Glance path prediction method based on multi-vision memory unit
CN117522891A (en) 3D medical image segmentation system and method
EP3862922A1 (en) Explaining the output of a trained model
Chen et al. Alzheimer's Disease Clinical Scores Prediction based on the Label Distribution Learning using Brain Structural MRI
Wijerathna et al. Brain Tumor Detection Using Image Processing
CN112967295A (en) Image processing method and system based on residual error network and attention mechanism
Vanmore et al. Liver Lesions Classification System using CNN with Improved Accuracy
CN118570202B (en) Ankylosing spondylitis rating method based on visual state space model
CN118570570B (en) Traditional Chinese medicine tongue picture identification system and method based on image identification technology
CN115131312B (en) Subarachnoid hemorrhage focus segmentation method based on UDT
Sauron et al. Learning Representations for MR Image Retrieval: Contrastive Models Versus Other Supervision Strategies

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination