Home DNA testing and genetics kit company 23andme collects customers’ DNA and matches it with information about genetic research to provide updates about ancestry, predisposition to illness and more. However, citing a lack of solid evidence that the kits work as marketed, the U.S. Food and Drug Administration has cracked down on the company, sending it a warning letter giving 15 days to submit proof of analytical or clinical validation of its product. Among the issues that 23andme may be facing is sloppy coding that could be giving customers an inaccurate view of the health risks they face.
According to the FDA, without proof of its effectiveness, 23andme presents a danger to its users, who may be using information gleaned from the genetic testing kit to dictate medical treatment in ways that range from inappropriate medicine dosing to dangerous surgical procedures. The 23andme website encourages the use of the device for such diagnostic behavior, but marketing it as such puts it under the purview of FDA approvals it has not earned, the agency noted. False positives or false negatives indicating genetic risk for certain conditions could lead people to take on expensive and invasive tests or, conversely, to ignore symptoms, among other dangers.
“Assessments for drug responses carry the risks that patients relying on such tests may begin to self-manage their treatments through dose changes or even abandon certain therapies depending on the outcome of the assessment,” the letter noted.
Despite significant work on the agency’s part to help the company comply with regulatory requirements, there is no definite assurance that the product works as it claims. As a result, the company will be forced to halt marketing until it can meet FDA compliance standards.
An algorithmic problem
One of the drivers of false negatives and false positives may be the code governing the product’s alerts, according to customer and Spacedeck co-founder Lukas Hartmann. In a column for io9, Hartmann explained that he was recently diagnosed by the 23andme algorithm as having two genetic mutations linked to an illness known as limb-girdle muscular dystrophy. He was informed by the service that people with two such mutations typically had the disease, and, upon further research, he discovered that those with LGMD typically lose the ability to walk and can ultimately die as a result of muscle degradation.
Concerned by the result, Hartmann downloaded his raw data and performed his own analysis. He discovered the two genomic variations, both of which had been linked to LGMD. Patients who had homozygous mutations in these genes – meaning that they had inherited the mutation from both parents – were highly at risk. However, in Hartmann’s case, the mutations were both heterozygous, meaning that only one of the two chromosomes displayed it.
“Yes, I really had two mutations,” he explained. “But they weren’t on the same gene, but on two different genes. By rare chance, both of these mutations are statistically linked to LGMD, but to two different versions of LGMD. So I didn’t have a homozygous mutation, but two unrelated heterozygous ones. The web programmer at 23andme had added those two mutations together into one homozygous one in their code. And so the algorithm switched to red alert.”
The company fixed the error when Hartmann submitted it. However, these kinds of algorithmic errors, which create false negatives or false positives, are exactly the types of things that have raised red flags at the FDA and serve as a warning to companies making medical devices. To meet FDA standards, vendors need to have rigorous source code analysis practices in place to prevent errors and catch algorithmic slip-ups such as the one Hartmann discovered. On a business level, such errors may mean legal action from regulators. On a human level, they could mean dangerous false diagnoses and improper treatment. To avoid that type of issue, vendors can use products such as static analysis software or carry out peer code reviews to catch potential errors.
Software news brought to you by Klocwork Inc., dedicated to helping software developers create better code with every keystroke.