By Alison Contreras, PhD
February 11, 2018
Part 1 of this article was published last week. Read on for other things to consider as Sweden’s Medical Products Agency investigates Natural Cycles.
A large, prospective study on an app-based FABM: it’s just what has been missing as the modern fertility awareness landscape expands with the available smartphone technology. Last month, Scherwitzl et al. published in Contraception the perfect and typical use Pearl Index of Natural Cycles based on input about a woman’s cycle length, basal-body temperature, and an optional luteinizing hormone (LH) measurement.
They found the typical-use Pearl Index to be 6.9 pregnancies per 100 woman-years, with a perfect-use Pearl Index of 1.0 pregnancy per 100 woman-years.[i] Furthermore, they estimate the number of pregnancies from cycles where the application flagged a fertile day erroneously as infertile was 0.5 days. This translates roughly to a 93% effectiveness rate to prevent pregnancy.
Calculating Pregnancy Rates of FABMs
What was particularly helpful in the article was the nuanced discussion of how effectiveness rates for FABMs can be inherently difficult to calculate, given the nature of how these methods work: by either avoiding intercourse during a set time, or using an additional method of contraception such as condoms. So, what happens when a user ignores the app’s instructions (to not have sex if they don’t want to get pregnant) or uses a barrier method, and still achieves pregnancy? Which method failed, and how can we tell? This doesn’t even begin to touch on the challenge to factor in pregnancies from women who simply stop recording data. Do you consider them as part of the study, or are they no longer “using Natural Cycles”?
The article reveals just how thoughtfully and clearly all these considerations were incorporated into the final calculated effectiveness rate. Here’s an example of challenges to determine pregnancy rates accurately in the study:
“We detected pregnancies primarily through the users logging a positive pregnancy test in the application. We attempted to follow up with women who discontinued the method through in-app messages and via e-mail. We then classified all users lost to follow-up in one of three categories, based on the BBT at discontinuation, and at what time in the menstrual cycle they discontinued.[ii]…
The real-life nature (of the app), free from any possible bias from contact with clinics and health care providers, allows us to investigate the typical use of the application in a more direct sense than many other studies of contraceptive effectiveness. On the other hand, the real-life nature means that it is less clear who is to be considered pregnant, and any such estimate must rely to some extent on credible assumptions about the pregnancy status. In this study, we have complemented the estimate of efficacy during typical use with an estimate based on the worst-case assumption that everyone who was lost to follow-up while possibly pregnant was in fact pregnant. These two estimates indicate the possible span of typical-use failure rates.”
Additionally, there is the inherent possibility, again, of women changing their mind about their family planning goals mid-cycle:
“Of the 1273 pregnant women, 259 actively changed the mode of operation to “plan a pregnancy” within the application on the same date as they reported the pregnancy. While this change is in part due to the inherent software design, it is likely that many of these women planned a pregnancy without changing their stated intent prior to their pregnancy. In order to be as conservative as possible, we considered all these pregnancies as failures during typical use. As a result, the typical-use Pearl Index is almost certainly biased upward (emphasis added). In the future, women who suddenly start logging a lot of unprotected intercourse around ovulation will be asked to either switch to the “plan a pregnancy” mode or actively state that they still do not plan a pregnancy.”
The investigation into those 37 pregnant women
Since 37 Natural Cycles users reported experiencing an unintended pregnancy, it seems appropriate to investigate the effectiveness of the Natural Cycles app further. This is especially true since it is a new method of preventing pregnancy and up to this point, almost every study of the effectiveness of FABMs is from peer-to-peer instruction vs. learning solely through an app. Primarily, it is important to determine what pregnancy rate the number 37 actually represents. Since Natural Cycles is a wildly popular app with hundreds of thousands of users, we need to know how many people were using the app to prevent or achieve pregnancy in the first place. If the 37 pregnancies reported are out of 100 women using the app to avoid pregnancy, 37% would be a high pregnancy rate compared to other methods to avoid pregnancy, and very different from what the app claims. Yet, if it’s based out of 1000 women using the app, that pregnancy rate (3.7%) is consistent with the recently reported effectiveness rates.
It will also be interesting to see whether an investigation reveals the habits of those users who achieved pregnancy. Were the users logging their information daily? Were they knowingly having intercourse on fertile days, or did they have intercourse on days identified as infertile and then became pregnant? Also, how many of those 37 were using condoms as well? Were they using them properly, and was it noted if the condom malfunctioned? A benefit of apps is their ability to track and store information to make it accessible for a retrospective investigation like this one.
It’s important for all FABM users to be appropriately informed about how to use their chosen method most effectively. While an app interface aims to simplify the process of learning and recording chart details, the technology as programmed may lack the ability to emphasize other instructions. Previous FABM effectiveness rates are based face-to-face, personal instruction rather than through an app, which allows an instructor to ensure the user understands before moving on. The developers have already acknowledged that in future versions of the app, they will prompt users to update their intention through a pop-up window to “achieving pregnancy” if they repeatedly document unprotected intercourse on fertile days. Perhaps there are other similar prompts that can be added to ensure the user understands other nuances to using Natural Cycles.
While it is difficult to determine all the factors contributing to the 37 pregnancies based on early reporting, some improvements can be made to the Natural Cycles app to help women use the method more effectively. Hopefully, the investigation can shed light as to how those women were using the method as app developers and teaching organizations make fertility awareness more accessible for women wanting alternatives to hormonal contraception.
Read Part 1 of “Don’t Discount that Natural Cycles App…” here.
[i] Not familiar with “woman-years”? Learn more about the Pearl Index. Due to documented biases inherent to the Pearl Index, it is not as good a measure of effectiveness rates as life table or survival analysis.
[ii] Users lost to follow-up were classified as likely to be pregnant, very unlikely to be pregnant, and possibly pregnant based on established study criteria.
- We considered users likely to be pregnant if they discontinued at a late day of their luteal phase and/or reported high BBT when discontinuing after ovulation. We treated these women as pregnant for all calculations unless they stated explicitly in follow-up that they were not.
- We considered users very unlikely to be pregnant if they discontinued at a point in their cycle when it was very improbable that they were pregnant (e.g., between menstruation and ovulation). We do not find it accurate to denote a possible later pregnancy of such a user as due to contraceptive failure. We treated these women as not pregnant for all calculations unless they stated explicitly in follow-up that they were.
- We considered users possibly pregnant if we could not easily place them in either of the above categories (e.g., a user who quit early in the luteal phase with BBT incompatible with their follicular phase). We treated such users as having unknown pregnancy status. We treated these users as not pregnant in the main efficacy analysis. However, we conducted a sensitivity analysis in which all possibly pregnant women were treated as pregnant, as a worst-case scenario.