Current clinical data Dan-NICAD
The Dan-NICAD data set, still unpublished in full, is a composite of three studies totalling 2,271 patients, of whom 203 had CAD: 8.9%. Using the full data set, Acarix has cited a negative predictive value (NPV) of 97.1%, that is 97.1% of negative results will be correct. For more explanation and terminology, please see our initiation note published on 21 December 2016.
The three data sets are as follows.
■
The main study, Winther et al (2017), analysed a subset of 1,437 patients using the CADScor 3.0 algorithm (as used in the marketed device). Of these, 140 (9.7%) had coronary artery disease with obstruction of blood flow. The study is discussed in more detail in this report.
■
A negative control cohort of 606 individuals was selected from the 1,156-patient DanRisk study (Diederichsen 2012) and included in the overall data set.
■
228 patients from an earlier study, AC003, with 63 CAD cases, were included in the data set.
As CADScor V3 is a recent iteration of the algorithm, patients from the DanRisk and AC003 studies were retrospectively included in Dan-NICAD by the use of historic sound recordings if of adequate quality. CADScor V3 is more demanding in terms of acoustic quality and needs a clearly defined heartbeat profile separating the short heart contraction (systolic) phase from the heart filling (diastolic) phase. The acoustic measurements are made in the diastolic phase. In the Winther et al 2017 study, 14 patients tested with CADScor V2 could not be re-analysed with V3 and were excluded on these grounds. Clinically, this is not an issue so long as the device reliably recognises that a CAD-score cannot be calculated, it is meant to test sound quality.
Winther et al 2017 recruited 1,675 patients in Denmark who had been referred by primary care physicians for further specialist tests. The patients were split into test optimisation (593 patients) and validation (1,082 patients) cohorts. Of these, 325 had a detailed gold standard invasive imaging test and 145 were found to have partial blockages causing a functional reduction in heart blood supply (factional flow reserve reduction of over 80%). All patients had CACS and all but four had a CTA.
A new CADScor algorithm, V3, was developed based on the data. However, this algorithm cannot be applied to all patients due to the increased acoustic demands of CADScor V3. In particular, 14 patients were excluded because of a short systolic period.
The patients recruited are meant to be a typical referred patient group. On average, this Danish Caucasian population seems rather healthy: BMI was 26kg/m2, age 57, 15% active smokers, less than 6% diabetic, moderate waists (93cm/36 inches). Whether this profile will be the same in, for example, the US where obesity (BMI >30) is a major issue with 30%+ prevalence is an interesting question. Will CADScor get a good enough signal though layers of subcutaneous fat?
Interestingly, physicians seem to be very cautious about ruling out CAD. Only 28% of patients had typical angina, so nearly three-quarters were low risk. In fact, only 10% of the tested cohort had clinically relevant CAD after full evaluation. On clinical risk scoring systems like Diamond-Forrester (see Exhibit 2), only 2.4% were high risk (>85%) – the level recommended for immediate referral – with 15% as low risk (<15%), so debatable if even CADScor should be used.
This apparently high referral level of low and intermediate risk patients may reflect a medical concern that an incipient heart attack (acute myocardial infarction, AMI) might be missed. It shows that a rule-out test should be very valuable as a clinician tool so long as clinicians view it as reliable. Note also that these referrals were made after a specialist outpatient examination.
Exhibit 2: Procedures and terms to assess CAD in the Winther et al 2017 study
Term |
Definition |
Comment |
Pre-test probability (PTP) |
Chest pain can be caused by many different factors. To make an initial assessment, a primary care physician calculates the pre-test probability (PTP) of CAD using a questionnaire. If a patient has signs of CAD, for example typical angina and if the PTP level is high then the patient will be referred for hospital tests. This also applies to patients with major clinical risk factors like diabetes, hypertension and a history of smoking. |
The updated Diamond-Forrester (DF) score) is used in this study. DF was updated and recalibrated in 2012 and is combined with other clinical risk factors like diabetes under 2013 European Society of Cardiology (ESC) guidelines (see below). An online calculator gives the recalibrated DF score and allows it to be further modified with clinical risk factors like smoking, diabetes and hypertension. |
Coronary artery calcium score (CACS) |
The CACS test requires a 10-minute computerised tomography x-ray scanner (CT). If a high level of calcium is present in a coronary artery it will be due to calcified plaque, see Shah and Coutler (2012). CACS does not predict the extent of obstruction and non-calcified plaque can also be present. Score of 100, low; up to 400 some risk; over 4,000 is high risk. The test does have a very high negative predictive value of about 98%. |
CACS is still regarded as investigational in the US and it is not covered by Medicare. It is not recommended by the ESC. It is recommended as a negative screen in the UK health service guidelines. |
Coronary angiography (CTA) |
This procedure needs a CT scanner. A radiopaque dye is injected into the patient’s arteries and the scanner images the flow of blood through the coronary arteries. These arteries are very small and as the heart is beating the scanner needs to have both high spatial resolution and a fast time resolution. CTA can overestimate the level of stenosis if there is a high level of calcification (CACS score >400). |
This procedure is non-invasive as only a radiopaque dye is required. There is a radiation dose. The advantage of the procedure is that no invasive catheterisation is required. Most of the population data used for calculating the risk of CAD comes from patient groups that have all undergone CTA. |
Invasive coronary angiography (ICA) |
The only way to determine coronary artery disease status absolutely is through an invasive coronary angiogram (ICA) where a catheter is inserted through the groin and into the coronary artery. If a blockage is confirmed, it is usually followed immediately with angioplasty (opening of the artery) |
A diagnostic ICA will be done for patients who are either very high risk or have shown more than 50% occlusion on coronary angiography. |
Anatomically obstructive CAD |
This is an ICA procedure. Once the catheter is adjacent to the stenosis, a radiopaque dye is released, and the flow visualised on x-ray. This visually clearly shows any narrowing of the artery. |
This is an anatomical estimate. A reduction of 50% or more is classed as CAD. As it is not functional, since the blood flow is not measured, it can overestimate the number of CAD cases. |
Haematologically obstructive CAD |
This is also an ICA but here a pressure sensor on the catheter measures the blood pressure, as a measure of blood flow on either side of the stenosis. |
The technique measures the factional flow reserve (FFR). The clinical guidelines show that FFRs of <0.8 require angioplasty . This corresponds to a coronary artery narrowing by over 90%. |
Angioplasty (PTCA) |
Angioplasty is the most common treatment for CAD. If a blockage is found, a balloon catheter is inserted and inflated to crush the atherosclerotic plaque. |
This is a natural follow-on procedure to ICA. However, many coronary angiograms fail to find a blockage. |
Source: Edison Investment Research
The study started by using CADScor V2. However, CADScor V2 was less accurate, at 58.1% accuracy, compared with 68.9% accuracy using the updated DF scoring system. Accuracy is the number of correct test results, whether positive or negative.
Accordingly, the enhanced CADScor V3 algorithm was developed and the data re-analysed. The new CADScor V3 data set included 1,437 patients and 140 CAD cases. The training and validation set outcomes are combined in the paper as the authors state that these were statistically the same.
CADScor V3 analyses eight acoustic parameters, of which five are new. On acoustic analysis alone, CADScor V3 had similar accuracy to the DF score. Enhanced accuracy was achieved by incorporating three clinical parameters into the V3 algorithm: gender, age and hypertension. Age and gender are also key parameters used in DF scoring.
CADScor V3 was optimised against anatomically measured obstructive disease. Using this benchmark, the accuracy was 72.4% vs 65.9% for DF.
The results for both anatomic and haematological (FFR) outcome are in Exhibit 3. For clarity, we have also shown the data for the CADScor V3 validation set as shown in supplementary data supplied by Winther et al.
Exhibit 3: Winther et al 2017 CADScor results
Parameter |
Haematological obstruction |
Anatomical obstruction |
Full set |
Validation set |
Full |
Accuracy |
71.3% |
69.90% |
72.40% |
Sensitivity |
80.7% |
78.70% |
80.40% |
Specificity |
52.8% |
52.10% |
53.00% |
PPV* |
15.6% |
15.9% |
16.40% |
NPV* |
96.2% |
95.50% |
95.90% |
This was a complex study using a large patient cohort. There is some minor confusion within the paper on the exact patient numbers in different cohorts, but the overall results in Exhibit 3 are very consistent. However, this was a trial run to develop and validate a test and so might, using the algorithm V3, be over-fitted to the data (a possibility discussed by the authors) and have been affected by the specific Danish population recruited. For marketing, Acarix may need to run prospective studies in different populations with national opinion leaders and also investigate populations with higher levels of obesity and Type II diabetes. The new Dan-NICAD II study will add significantly more validated CAD patients to the Acarix CADScor algorithm database for further improvement of the scoring algorithm. The expected performance improvements should lead to further acceptance of this new risk scoring method among key opinion leaders.
The US is likely to require a large clinical prospective study if a PMA-based approval is sought. However, if the FDA regards the current data set as adequate, then a de novo 510(k) application may be possible. A de novo application, if accepted, is a relatively straightforward and much faster and cheaper process.