Comparison of publicly available COVID-19 models
Predict COVID-19 from a Chest X-RayThe objective of our website is to offer an ongoing assessment of COVID-19’s. As the virus spreads globally, here you can know certain things about COVID-19 and whether the patient has COVID-19 or not and also the severity of the viruses, using our Deep Learning models.
Here you can know the classification of given input into either normal, Pneumonia or COVID and you can also read the related information from our blogs.
PyTorch Binary classification model
In this study, a new model for automatic COVID-19 detection using raw chest X-ray images is presented. The proposed model is developed to provide accurate diagnostics for binary classification (COVID vs. No-Findings). Our model produced a classification accuracy of 98.08% for binary classification tasks.
Diving into the Model:
Binary or binomial classification is the task of classifying the elements of a given set into two groups (predicting which group each one belongs to) on the basis of a classification rule. Here we have considered COVID and no-findings as the target classes.
Understanding the Data:
We have trained a classification model using Conv2d layers in the PyTorch framework. To train this model we have used X-ray images obtained from two different sources that were used for the diagnosis of COVID-19. A COVID-19 X-ray image database was developed by Cohen JP using images from various open access sources. This database is constantly updated with images shared by researchers from different regions. Also, the ChestX-ray8 database provided by Wang et al. was used for normal and COVID images.
In order to avoid the unbalanced data problem, we used 500 no-findings frontal chest X-ray images randomly from this database.
Training the Model:
The DarkNet model was used in our study as a classifier for the You Only Look Once (YOLO) real time object detection system. We implemented 17 convolutional layers and introduced different filtering on each layer.
We have trained the model using the PyTorch framework with learning rate=3e-3 and 100 epochs. We have plotted the train and validation loss plots while training the model.
We have considered the CrossEntropyLoss() as the loss function and from the plot we can observe the fluctuations in log values as the batch size changes. Training loss is always nearly equal to zero, but we got the max validation loss with batch_size=500.
Running Inference:
A confusion matrix is a table that is often used to describe the performance of a classification model (or “classifier”) on a set of test data for which the true values are known. It allows the visualization of the performance of an algorithm. We have plotted a confusion matrix for test data points. Among the data points that belong to COVID class, all the points are correctly classified as COVID and among all the data points of no-finding class, all the points are classified as no-finding. Here the misclassification rate is zero. Outputs the softmax probabilities of input datapoint belonging to COVID or no-finding class.
PyTorch Multi-class classification model
In this study, a new model for automatic COVID-19 detection using raw chest X-ray images is presented. The proposed model is developed to provide accurate diagnostics for Multi-class classification (COVID vs. No-Findings vs Pneumonia). Our model produced a classification accuracy of 87.02% for multi-class classification tasks.
Diving into the Model:
Multiclass or multinomial classification is the problem of classifying instances into one of three or more classes. COVID, Pneumonia and no-findings are the three target classes.
Understanding the Data:
We have trained a classification model using Conv2d layers in the PyTorch framework with X-ray image dataset. X-ray images obtained from two different sources were used for the diagnosis of COVID-19. A COVID-19 X-ray image database was developed by Cohen JP using images from various open access sources. This database is constantly updated with images shared by researchers from different regions. Also, the ChestX-ray8 database provided by Wang et al. was used for normal, COVID and pneumonia images.
In order to avoid the unbalanced data problem, we used 500 no-findings and 500 pneumonia class frontal chest X-ray images randomly from this database.
Training the Model:
The DarkNet model was used in our study as a classifier for the You Only Look Once (YOLO) real time object detection system. We implemented 17 convolutional layers and introduced different filtering on each layer.
We have trained the model using the PyTorch framework with learning rate=3e-3 and 100 epochs. We have plotted the train and validation loss plots while training the model.
Running Inference:
We have considered the CrossEntropyLoss() as the loss function and from the plot we can observe the fluctuations in log values as the batch size changes. Training loss is always nearly equal to zero, but we got the max validation loss with batch_size=500.
A Confusion matrix is a table that is often used to describe the performance of a classification model (or “classifier”) on a set of test data for which the true values are known. It allows the visualization of the performance of an algorithm. We have plotted a confusion matrix for test data points. Among the data points that belong to COVID class, 96 data points are correctly predicted as class COVID and only 4 data points are misclassified as Pneumonia. Among all the data points that belong to No-finding, 94 points are correctly classified as No-finding class and only 6 points are misclassified as Pneumonia. Outputs the softmax probabilities of input datapoint belonging to COVID, Pneumonia or no-finding class.
Binary classification model
In this study, a new model for automatic COVID-19 detection using raw chest X-ray images is presented. The proposed model is developed to provide accurate diagnostics for binary classification (COVID vs. Normal). Our model produced a classification accuracy of 90.01% for binary classification tasks.
Diving into the Model:
Keras provides keras.applications, which include multiple deep learning models, pre-trained on the ImageNet dataset and ready to use. Pre-training lets you leverage transfer learning – once the model has learned many objects, features, and textures on the huge ImageNet dataset, you can apply this learning to your own images and recognition problems.
Dense Networks are a relatively recent implementation of Convolutional Neural Networks, that expand the idea proposed for Residual Networks, which have become a standard implementation for feature extraction on image data. Similar to Residual Networks that add a connection from the preceding layer, Dense Nets add connections to all the preceding layers, to produce a Dense Block. So the idea of Densely Connected Networks, is that every layer is connected to all its previous layers and its succeeding ones, thus forming a Dense Block. Here we are performing transfer learning with Keras pretrained DenseNets.
Understanding the Data:
We have trained a classification model using the Keras framework with X-ray image dataset. It is an open dataset of chest X-ray and CT images of patients which are positive or suspected of COVID-19 or other viral and bacterial pneumonias (MERS, SARS, and ARDS.). Totally we have 2274 Chest X-rays for training, out of which 1200 X-rays belong to COVID patients and 1073 belong to normal and 568 Chest X-rays for testing, out of which 280 X-rays belong to COVID patients and 288 belong to normal.
Training the Model:
We have trained a DenseNet121 model on X-Ray images with learning rate=0.001 and 30 epochs. We have plotted the train and validation loss plots while training the model.
We have considered the CrossEntropyLoss() as the loss function and Adam Optimizer and from the plot we can observe the fluctuations in log values as the batch size changes. Training loss is almost nearly equal to zero, but we got the max validation loss with batch_size from 1000 to 1500.
Running Inference:
A confusion matrix is a table that is often used to describe the performance of a classification model (or “classifier”) on a set of test data for which the true values are known. It allows the visualization of the performance of an algorithm. We have plotted a confusion matrix for test data points. Among the data points that belong to COVID class, 256 data points are correctly predicted as class COVID and only 59 data points are misclassified as Normal. Among all the data points that belong to Normal, 242points are correctly classified as Normal class and only 6 points are misclassified as COVID. Outputs the softmax probabilities of input datapoint belonging to COVID, or normal class.
COVID-Net Geographic severity model
In this study, a new model for automatic COVID-19 lung disease Severity detection using a combined chest X-ray image dataset is presented.
Diving into the model:
Multiclass or multinomial classification is the problem of classifying instances into one of three or more classes. COVID, Pneumonia and no-findings are the three target classes.
Understanding the Data:
To train this model, we have combined three datasets to create COVIDx dataset and those three datasets are:
- COVID-chestxray-dataset(https://github.com/ieee8023/covid-chestxray-dataset)
- Figure1-COVID-chestxray-dataset(https://github.com/agchung/Figure1-COVID-chestxray-dataset)
- Kaggle RSNA Pneumonia Detection Challenge(https://www.kaggle.com/c/rsna-pneumonia-detection-challenge)
COVID chest X-ray dataset consists of 548 x rays, Figure1 COVID chest xray dataset consists of only 55 x rays and Kaggle RSNA dataset consists of 26,684 x rays for training and 1000 x rays for testing. By combining all these 3 datasets, we have created a new dataset and it consists of 13,857 training images and 1,510 testing images.
Pretrained Model
Here we are using the pretrained COVID-Net model for the Classification task, which is a deep convolutional neural network design tailored for the detection of COVID-19 cases from chest X-ray (CXR) images. It is trained using 13,857 training images with a learning rate of 0.002 and batch_size as 8 and AdamOptimizer is used to optimize the model along with CrossEntropy loss function.
This model takes as input a chest x-ray image of shape (N, 480, 480, 3), where N is the number of batches, and outputs the SARS-CoV-2 severity scores for geographic extent. COVIDNet-SEV-GEO predicts the geographic severity. Geographic severity is based on the geographic extent score for right and left lung. For each lung: 0 = no involvement; 1 = <25%; 2 = 25-50%; 3 = 50-75%; 4 = >75% involvement, resulting in scores from 0 to 8.
Running Inference:
The deep neural networks yielded R2 of 0.673 ± 0.004 between predicted scores and radiologist scores for geographic extent, in stratified Monte Carlo cross-validation experiments.
R Squared - The coefficient of multiple determination is a statistical indicator usually applied to multiple regression analysis. It compares the accuracy of the model to the accuracy of a trivial benchmark model wherein the prediction is just the mean of all of the samples. A perfect fit would result in an R squared value of 1, a very good fit near 1, and a very poor fit less than 0. If your neural model predictions are worse than you could predict by just using the mean of your sample case outputs, the R squared value will be less than 0. The best performing networks achieved R2 of 0.865 between predicted scores and radiologist scores for geographic extent.
Model repo: https://github.com/lindawangg/COVID-Net
COVID-Net Opacity Severity Model
In this study, a new model for automatic COVID-19 lung disease Severity detection using a combined chest X-ray image dataset is presented.
Diving into the model:
Multiclass or multinomial classification is the problem of classifying instances into one of three or more classes. COVID, Pneumonia and no-findings are the three target classes.
Understanding the Data:
To train this model, we have combined three datasets to create COVIDx dataset and those three datasets are:
- COVID-chestxray-dataset(https://github.com/ieee8023/covid-chestxray-dataset)
- Figure1-COVID-chestxray-dataset(https://github.com/agchung/Figure1-COVID-chestxray-dataset)
- Kaggle RSNA Pneumonia Detection Challenge(https://www.kaggle.com/c/rsna-pneumonia-detection-challenge)
COVID chest X-ray dataset consists of 548 x rays, Figure1 COVID chest xray dataset consists of only 55 x rays and Kaggle RSNA dataset consists of 26,684 x rays for training and 1000 x rays for testing. By combining all these 3 datasets, we have created a new dataset and it consists of 13,857 training images and 1,510 testing images.
Pretrained Model:
Here we are using the pretrained COVID-Net model for the Classification task, which is a deep convolutional neural network design tailored for the detection of COVID-19 cases from chest X-ray (CXR) images. It is trained using 13,857 training images with a learning rate of 0.002 and batch_size as 8 and AdamOptimizer is used to optimize the model along with CrossEntropy loss function.
This model takes as input a chest x-ray image of shape (N, 480, 480, 3), where N is the number of batches, and outputs the SARS-CoV-2 severity scores for opacity extent. COVIDNet-SEV-OPC predicts the opacity severity. Opacity severity is based on the opacity extent score for right and left lung. For each lung: 0 = no opacity; 1 = ground glass opacity; 2 =consolidation; 3 = white-out, resulting in scores from 0 to 6.
Running Inference:
The deep neural networks yielded R2 of 0.636 ± 0.002 between predicted scores and radiologist scores for opacity extent, respectively, in stratified Monte Carlo cross-validation experiments.
R Squared - The coefficient of multiple determination is a statistical indicator usually applied to multiple regression analysis. It compares the accuracy of the model to the accuracy of a trivial benchmark model wherein the prediction is just the mean of all of the samples. A perfect fit would result in an R squared value of 1, a very good fit near 1, and a very poor fit less than 0. If your neural model predictions are worse than you could predict by just using the mean of your sample case outputs, the R squared value will be less than 0. The best performing networks achieved R2 of and 0.746 between predicted scores and radiologist scores for opacity extent.
Model repo: https://github.com/lindawangg/COVID-Net