AI against COVID-19: Screening X-ray Images for COVID-19 Infections
The coronavirus disease 2019 (COVID-19) pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus, has generated an unprecedented global health crisis, with more than 2.7 million deaths worldwide. Do you want to contribute to the fight against this pandemic?
IEEE SIGHT (Special Interest Group on Humanitarian Technology) of the Montreal Section, Vision and Image Processing Research Group of the University of Waterloo, and DarwinAI Corp. invite data scientists, students and professionals working on Artificial Intelligence (AI) to participate in a virtual competition to help medical researchers diagnose COVID-19 with chest X-ray (CXR) images. The ultimate goal is to contribute to the development of highly accurate yet practical AI solutions for detecting COVID-19 cases and, hopefully, accelerating the treatment of those who need it the most. Moreover, this AI for Good initiative will also allow us to take action on at least one of the United Nations Sustainable Development Goals (SDGs), Good Health and Well-Being.
The competition is composed of 2 phases:
- In the First Phase, the challenge consists of designing robust machine learning algorithms to predict if the subjects of study are either COVID-19 positive or COVID-19 negative. The dataset for this competition is the dataset curated by COVID-Net, a global open-source initiative launched by DarwinAI Corp., Canada, and Vision and Image Processing Research Group, University of Waterloo, Canada, for accelerating advancements in machine learning to aid healthcare workers around the world in the fight against the COVID-19 pandemic. More about the COVID-Net initiative and available open-source resources are available here.
- In the Second Phase, the 5 top teams of the first phase will have the opportunity to refine their solution and submit a proposal for a follow-up project to positively impact society or the academic community.
This competition is organized in collaboration with the National Research Council Canada and co-hosted by the IEEE Young Professionals Affinity Groups of Montreal, Ottawa, Toronto and Vancouver Sections, Vancouver Circuit and Systems (CAS) Technical Chapter, the Student Branches of INRS (Institut National de la Recherche Scientifique) and Vancouver Simon Fraser University, and WIE (Women In Engineering) Ottawa. It is also largely sponsored by Microsoft, and partially by the IEEE Canada Humanitarian Initiatives Committee and the IEEE Montreal Section.
Note: This competition is intended for participants living in Canada, due to restrictions on funds transfer. Check the specific rules for eligibility of international participants.
First Phase
- You can participate individually or as a team. However, only submissions from a single account are allowed.
- Anyone involved in organizing the competition is not allowed to participate in this competition.
- You have 2 weeks to submit your solution.
- The winning teams are required to open source their code.
- The winning teams will have the opportunity to write a manuscript with the organizers on implementation details of their work.
- You can submit a maximum of 5+1 entries.
- You can select only one final submission for judging.
- By participating in the competition, you agree to the competition rules and code of conduct.
Second Phase
- Each applicant may submit only one proposal.
- Proposals must involve the deployment of technology, customization of technology, and/or development of technology.
- Projects developing a medical device must collaborate with medical professionals and/or organizations. The funds may not be used for a) tests of medical equipment on people/animals or b) mass production and/or deployment of devices that require but have not yet undergone the appropriate regulation, such as government approvals.
- Proposals must clearly articulate the impact of the project and how it will be measured, using the Project Assessment Matrix. Applicants can take the free online HAC/SIGHT course on IEEE Learning Network, “Project Assessment, Monitoring and Evaluation” for more information on how to complete the matrix and monitoring and evaluation best practices at all stages of a project.
- Proposals must contain the provided budget application form completed.
- Expenses should cover direct project costs, including necessary equipment, materials, supplies, software licenses, research publications, conference registration, etc.
- Given current circumstances, travel expenses should be avoided.
- Anyone contributing to the project cannot receive reimbursement of salary.
- No indirect costs will be considered for funding, including but not limited to overhead expenses, Facilities & Administrative (F&A) costs, tuition, etc.
- Funds must be expended within a maximum of four months from the announcement of the selected projects.
- Open Source: You hereby license and will license the source code used to generate your submission under an Open Source Initiative-approved license (see www.opensource.org). This in no event limits commercial use of such code or model containing or depending on such code. To the extent your submission makes use of generally commercially available software not owned by you, that you used to generate your submission, but that can be procured by the Organizing Committee of this Competition without undue expense, you do not grant the license in the preceding sentence to that software.
- Eligibility: If you are entering as a representative of a company, educational institution or other legal entity, or on behalf of your employer, these rules are binding on you, individually, and the entity you represent. If you are acting within the scope of your employment, as an employee, contractor, or agent of another party, you warrant that such party has full knowledge of your actions and has consented thereto, including your potential receipt of a Prize and Project Funding. You further warrant that your actions do not violate your employer’s or entity’s policies and procedures.
- Eligibility of international participants: If you live in a different country than Canada, you are allowed to participate only if (i) you give up the Prizes and Project Funding or (ii) if you are part of a team where there is at least one participant that resides in Canada. The Organizing Committee will only deal with participants from Canada and will not be responsible for any international transaction and fee charges associated with it in such a case (ii).
- Integrity: The Organizing Committee reserves the right to verify eligibility and to adjudicate on any dispute at any time. If you provide any false information relating to the Competition concerning your identity, residency, mailing address, telephone number, email address, ownership of right, or information required for entering the Competition, you may be immediately disqualified from the Competition.
- Competition Period: For the purposes of Prizes, the Competition will run from the Start Date and time to the Final Submission Deadline (such duration is the “Competition Period”). You are responsible for determining the corresponding time zone in your location.
- Submission: Submissions are void if they are in whole or part illegible, incomplete, damaged, altered, counterfeit, obtained through fraud, or late. The Organizing Committee reserves the right to disqualify any entrant who does not follow these rules, including making a submission that does not meet the requirements.
- Winner Notification: If a potential winner (i) does not respond to the notification attempt within one (1) week from the first notification attempt or (ii) notifies the Organizing Committee within one week after the Final Submission Deadline that the potential winner does not want to be nominated as a winner or does not want to receive a Prize, then, in each case (i) and (ii) such potential winner will not receive any Prize, and an alternate potential winner will be selected from among all eligible entries received based on the Competition’s judging criteria.
- Disqualification: The Organizing Committee reserves the right to disqualify any participant from the Competition if the Organizing Committee reasonably believes that the participant has attempted to undermine the legitimate operation of the Competition by not following the Code of Conduct, cheating, deception, or other unfair playing practices or abuses, threatens or harasses any other participants or the Organizing Committee.
As a participant of this competition, we ask that you follow the research integrity described more here: (https://uwaterloo.ca/research/office-research-ethics/research-integrity). In this regard, you might find several solutions for image classification on the Internet. However, even if you can get inspired by them, in the end, you should implement your own code. You might get some ideas from open source network designs like COVID-Net (https://arxiv.org/pdf/2003.09871.pdf). However, you cannot use their models and codes directly in this competition.
Also, you can only use the provided dataset for training your model. If you use additional datasets, you will be disqualified. Privately sharing code or data outside of teams is not permitted and is against the code of conduct.
NO PURCHASE NECESSARY TO ENTER OR WIN.
The competition is hosted on the Eval.ai online platform. To participate, you or your team will need to perform the following steps:
- Register individually at the link provided on the vTools page: https://events.vtools.ieee.org/m/269273.
- Register yourself or your team at the link on Eval.ai: https://eval.ai/web/challenges/challenge-page/925/participate. Follow the instructions here: https://evalai.readthedocs.io/en/latest/participate.html#.
- Download the dataset from https://www.kaggle.com/andyczhao/covidx-cxr2.
- Design an AI algorithm that gets CXR images as inputs and predicts the labels of the images in the output (COVID or non-COVID).
- Train your AI algorithm using the training dataset.
- Submit your predictions through Eval.ai for evaluation against the test dataset for the competition.
For the Second Phase, an application form needs to be completed. Clear instructions will be sent to the 5 top teams of the First Phase.
You can download the dataset from https://www.kaggle.com/andyczhao/covidx-cxr2. This dataset consists of more than 16000 (480×480) chest X-ray images gathered from 15000 patients for the training set and public test set, with a few hundred chest X-ray images held out for evaluation in this competition which has not been released. The dataset contains positive and negative classes to indicate COVID or non-COVID cases. An example of images can be seen below.
Example CXR images from the COVIDx dataset: (A) non-COVID19 infection, and (B, C, D) COVID-19 viral infection.
You will find the following directories and files in the dataset:
- train: Contains the training set images. Each image has a unique name.
- test: Contains the test set images. Each image has a unique name.
- train_labels.csv It is a CSV file with two columns. The first column indicates the names of the images in the training set and the second column indicates the labels of images. The labels are 1 or 0 (1 for positive COVID cases and 0 for non-COVID samples).
The competition consists of binary classification of covid-19 positive and covid-19 negative classes. The score given to the final test results of each team is based on the weighted sum of the Sensitivity, Positive Predictive Value (PPV), and the Runtime of the model. To be more specific, the final score is as follows:
Score = 6PN + 5PP + 3SN + 2SP – wt
Where SP and PP refer to the Sensitivity and PPV of the COVID-19 positive class. PN and SN refer to the PPV and the Sensitivity of the COVID-19 negative class. t is the runtime of the algorithm and w is a non-negative weight. The score can be subject to change before the opening of the competition.
For the Second Phase, the proposals will be reviewed based on the following criteria:
- The benefit to the affected local community
- Project scope well defined
- Strength of team and capacity to attain a goal
- The overall potential for success
Note: This competition is intended for participants living in Canada, due to restrictions on funds transfer. Check the specific rules for eligibility of international participants.
For the First Phase, the first five best solutions will be awarded monetary prizes and Azure credits:
- First place: 1,000 CAD + 500 CAD in Azure.
- Second place: 800 CAD + 300 CAD in Azure.
- Third place: 600 CAD + 300 CAD in Azure.
- Fourth place: 400 CAD + 300 CAD in Azure.
- Fifth place: 300 CAD + 300 CAD in Azure.
The top 5 teams on the leaderboard will also have the following opportunities:
- Participate in the 2nd phase to refine their solution and receive funding for a project.
- Write a scientific paper with the Vision and Image Processing Research Group, from the University of Waterloo, to explain their approach.
For the Second Phase, the best three projects can receive funds up to the following amounts:
- Project 1: 5,000 CAD.
- Project 2: 5,000 CAD.
- Project 3: 4,000 CAD.
Term of funding: Up to 4 months following the announcement of the selected teams. The deadline is December 31st, 2021.
First Phase:
- Opening ceremony: May 27th, 2021 (Video)
- The competition starts on: May 31st, 2021
- Workshop and Networking session: June 5th, 2021 (Video)
- The competition ends on:
June 14th, 2021(June 16th, 2021) - Closing ceremony: June 18th, 2021 (Video)
Second Phase (for the top 5 teams):
- Deadline for submission of project proposals: July 15th, 2021
- Announcement of the selected teams: August 22nd, 2021
* Mohammad Ghodratigohar, Microsoft
Organizers:
- IEEE SIGHT Montreal
- Vision and Image Processing Research Group (University of Waterloo)
- DarwinAI Corp.
Collaborator:
- National Research Council Canada
Co-hosts:
- IEEE Young Professionals of Montreal, Ottawa, Toronto and Vancouver Sections
- IEEE Student Branches of INRS and Vancouver Simon Fraser University
- IEEE Vancouver Circuit and Systems (CAS) Technical Chapter
- IEEE WIE (Women In Engineering) Ottawa
We wouldn’t be able to host this competition without the help of our incredible sponsor Microsoft Canada, as well as the IEEE Canada Humanitarian Initiatives Committee and the IEEE Montreal Section.