Feb. 17, 2023 – In January 2022, Anthony Arenz, a 51-year-old living in Mesa, AZ, breathed a small sigh of relief.
The blood test that screens for 50 types of cancer – known as the Galleri test – hadn’t detected any positive signs.
It would be welcome news to anyone, but especially to a firefighter, whose job brings a 9% greater risk of developing cancer and a 14% greater risk of dying from it than the average person. The Mesa unit had already lost two servicemen to cancer in the previous 3 years. Both were more than a decade younger than Arenz.
But when the city of Mesa offered more free screening – including a full-body MRI to firefighters over 50 – Arenz shrugged it off at first. With a negative Galleri test in hand, he didn’t want to spend more time dwelling on it.
Still, he began to feel a creeping guilt for skipping a test that many of his fallen colleagues hadn’t been offered. He tried to soothe his anxiety with research. A look through the company’s website didn’t set him at ease. According to Grail Bio, maker of the Galleri test, a result of “No Cancer Signal Detected” does not rule out cancer.
Arenz booked his free MRI.
The results left him heavy: stage I kidney cancer. The Galleri test had missed it.
Arenz received his free Galleri test through a cancer screening program funded by the city of Mesa at Vincere Cancer Center in Scottsdale. Under the leadership of radiation oncologist and Vincere co-owner Vershalee Shukla, MD, the program screens first responders in more than 10 Arizona cities at no cost to them.
Vincere began using Galleri shortly after the test launched for consumers in June 2021. Since then, the first responder program has become the largest commercial user of the test in North America.
But Galleri’s ability to identify cancer and, perhaps more importantly, the fallout from wrong results, have been under scrutiny since the test’s launch. Galleri, which has not yet been FDA-approved, is still so new that few know what incorrect results look like in practice and how often they might occur.
‘Good’ But ‘Not Ready Yet’
After running the test on about 2,000 first responders, Shukla can offer some insights about the test’s real-world value in a high-risk population.
“Cancer screening is a very complicated issue,” she says. And “being honest, the tests are good but are not ready yet [for wider use].”
Arenz was not the only firefighter who faced a surprise after taking a Galleri test.
In nearby Phoenix, 51-year-old Mike Curtis knew his risk for cancer was high, but he wasn’t that worried. Curtis had been running into fires since he was 17, and his dad, also a firefighter, had died of cancer at age 58.
He’d taken the Vincere Cancer Center up on every free screening service since the program began in late 2018 — well before Shukla started using Galleri in 2021. His most recent lung scan was clear. But he did the Galleri test just to stay vigilant.
His result was a shock. The test detected signs of cancer.
Curtis decided to tell no one, not even his wife. He’d bear the bad news alone until he was certain.
Shukla, however, doubted the blood test results right away. She rushed to do several follow-up tests. One week, and scans of Curtis’s belly and pelvis later, her hunch was confirmed. The Galleri test was wrong, Curtis did not have cancer.
The price of his peace of mind: an extensive work-up with a $4,000 price tag. Fortunately, the bill was covered by the screening program.
Overall, in just over 18 months of using the blood test, Shukla has only encountered one other false positive out of about 2,000 Galleri results.
She also discovered two positive signals for cancer using Galleri, which were confirmed with follow-up tests. One was a chordoma, a rare type of bone cancer, and the other was a squamous cell carcinoma of the head and neck. Galleri caught both remarkably early, in time for treatment.
But for Shukla, false negatives were particularly “horrible.” Arenz’s was just one of 28 cancers that the blood test missed. And because 500 negative tests are yet to be validated, the 28 false negatives may be an underestimate.
In her experience, the binary test result – a simple positive or negative cancer signal – is an oversimplification of risk, she says. It “gives a false perception that you have cancer, or you don’t,” when the test itself is not definitive.
Grail Senior Medical Director Whitney Jones, MD, agrees that the test is not meant to be a standalone when screening for cancer. The goal of Galleri is to “complement other screenings, not replace them,” he says.
According to an analysis of Galleri data and Shukla’s experience, the test’s specificity was over 99%. That means the test successfully minimizes false positives.
But the test’s sensitivity was much lower. Shukla’s data in first responders measured the sensitivity to be 6.7%. That means the test misses about 93 out of every 100 cancers. According to Grail’s latest data in more than 6,300 people over age 50, the test’s sensitivity was 29%.
Specificity and sensitivity are metrics used to credential a test and demonstrate confidence in its ability to detect the target disease. A test with high specificity can correctly identify patients without the condition in question, while a test with high sensitivity can correctly identify patients who do have the disease. But there are tradeoffs between sensitivity and specificity. One value is increased at the expense of the other.
It’s normal for a cancer screening test to prioritize specificity, says Aparna Parikh, MD, an oncologist at Massachusetts General Hospital Cancer Center in Boston. In a test like Galleri, which is meant to be used with other screening tests, “at least we are seeing a good specificity, which is important because we don’t want false positives, where the downstream impact on the patient can be high.”
Overall, Jones says, Grail Bio’s aim is to build a test that’s sensitive enough to catch the most dangerous cancers without flooding the health system with false positives. In addition, he says, the sensitivity varies by cancer type and tends to be lower for cancers that already have other available screening tests as well as for earlier-stage disease.
But the Galleri sensitivity values are “a little bit scary,” says Ji-Hyun Lee, a doctor of public health and director of the Division of Quantitative Sciences at the University of Florida Health Cancer Center. Lee, who is not affiliated with Grail, reviewed the company’s publicly available data as well as Shukla’s data for WebMD.
While there’s no definitive threshold for sensitivity, miss rates as high as 93% and 71% “provide little confidence in the [accuracy of the] test,” Lee says.
Positive and negative predictive values, however, are more clinically relevant measures of a screening test. These numbers inform how likely a patient’s results are true, and therefore how worried they should be about a positive result and how much they should trust a negative .
In first responders, Shukla found that only half of positive Galleri tests were confirmed cases of cancer. And an analysis of Grail’s data found only 38% of positive Galleri tests – 35 of 92 people – had a validated cancer diagnosis.
In the general population, if only 38% of positive Galleri results truly have cancer, the test is “not quite useful to make a decision for the patient or the providers,” Lee says.
Galleri may also be a costly prospect for patients, no matter the result, says Electra Paskett, PhD, an epidemiologist and cancer screening expert at Ohio State University. A positive Galleri test requires a cascade of follow-up diagnostic tests without the promise of payer coverage. For a negative result, Galleri recommends the patient be screened again in a year, an annual cost of $950 plus any follow-up testing when Galleri does pick something up.
“If a provider wants to offer the Galleri test, all those things need to be made abundantly clear, in my opinion,” Paskett says.
After a negative Galleri test, Arenz’s cancer didn’t slip through the cracks because he received other advanced imaging free of charge. But whether all doctors will go to such lengths to back up Galleri results, even in negative patients, is unknown.
A negative result can give patients “a huge false sense of security,” says Shukla. And if a test is positive, the workup isn’t simple, she says. Chasing cancer, especially one that’s not really there, can be nerve-racking and expensive.
The question, then, is: Why do the Galleri test at all if it requires so much validation?
Parikh says that a high-risk group like firefighters stands to be an ideal use case for Galleri and other liquid biopsy tests. But, she says, if it were used more widely in the general population, she would be “wary of the ability of the system to manage this test en masse.”
Shukla says it’s less about the results she’s getting today and more about making the test more effective for her patients in the future. First responders need a test like this that can quickly identify multiple cancers, she says. But for it to improve, Grail needs more data in this high-risk population. That’s what she’s after.
Curtis doesn’t regret taking the Galleri test. The emotional toll of thinking he had cancer for a few days wasn’t too high a price, in his opinion. It’s part of cancer screening.
But he says it would be a much more burdensome experience if he’d been financially responsible for the workup, or if he hadn’t had Shukla to manage his case from start to finish.
Because it was free, Arenz doesn’t regret Galleri either. But he tells his co-workers to check the site, do their research, and get more screening.
“Any medical center that’s just doing this one test, you just have to be careful,” Shukla says. “It’s not that easy.”
