How accurate are online doctor reviews?

A while ago my doctor needed to refer me to a specialist and asked me where I wanted to go.

I researched providers and practices at a few major research institutions online and was pretty surprised by what I found: They all had really bad reviews.

These were physicians at the top of their field and some of the most reputable health care organizations in the country. How accurate are these ratings? How much weight should I give these reviews in my decision?

As it turns out, I’m not the only one asking these questions. As online physician review sites become more popular, more researchers are looking at these sites. And according to a trio of recently released studies, these reviews need to be considered with a lot of caution.

A team from Mayo Clinic recently published a study that found a large disconnect between patient satisfaction surveys and online reviews. Doctors who received negative online reviews did not receive similar ratings in patient satisfaction surveys.

Likewise, researchers at the University of Utah compared reviews from Healthgrades and Vitals to Press Ganey surveys, which are patient satisfaction surveys sent out to patients after treatment. The study found a poor correlation between online review sites and Press Ganey surveys.

In addition to patient satisfaction, researchers have also found a poor correlation between online reviews and quality scores. A study published in the Journal of the American Medical Informatics Association compared online physician reviews to clinical performance, including 30-day readmissions, length of stay and peer review scores. The result? Online ratings didn’t predict clinical performance.

While more people are using online review sites to research physicians, it’s still a relatively small segment of the population.

A team from Mayo Clinic found that only 16 percent of their pre-surgical patients looked at online reviews prior to choosing a provider, which was consistent with studies by Pew Research and other groups. Further, the study found that only about a quarter of people said that a positive or negative review would influence their decision.

This also confirms what I’ve seen through market research: Among possible sources of information for providers, online reviews consistently ranked near the bottom – far below recommendations from friends, other providers or insurance group websites. When asked whether physician referrals or online reviews were more valuable in choosing a provider, the answer was overwhelmingly physician referrals.

Consumer decisions are incredibly complex and health care decisions can be even more complex. Like other products and services, cost and convenience can play a big role, but there are other factors like insurance coverage, physician referrals and ratings from federal regulators that can also influence decisions.

We are starting to understand how different sources of information influence our decisions. The Brookings Institution published some research earlier this year that indicated when people are choosing a family doctor, online reviews are just as important as government ratings.

But this is a little concerning as online reviews have a number of limitations; their methodologies are very different from quality or patient satisfaction ratings.

Let’s start with sample sizes. When you search for a provider on Google, the star ratings are featured very prominently in the search results. In some cases, these star ratings can be based on one single review and if you’re not careful you may just see the star ratings – not the fact there was only one review. Patient satisfaction scores, on the other hand, are based on much larger sample sizes.

There are also huge variations between rating sites. I found one provider who had at least 15 reviews on multiple rating sites, with the ratings ranging from 2 to 4 stars. As a consumer, this can very confusing.

And then there’s the validity of the data. While there are some mechanisms to remove fake reviews, online review sites can’t verify the reviews were from actual patients the way patient satisfaction surveys or quality data can.

Yet despite these limitations of online review sites, this doesn’t mean providers or patients should ignore them. One group of physicians wrote in JAMA that avoiding online reviews because of these issues would be “a self-fulfilling disaster for physicians and our patients.”

While ratings may be extremely subjective and may not directly correlate to quality or patient satisfaction, they do provide an important feedback mechanism for providers and a way to engage with the public.

For patients, the differences in methodology should make it clear that online reviews should not be considered in isolation. These issues can also expose ways patients can make online reviews better. If more people left reviews, the sample size would increase and this would increase credibility and impact of any fake reviews. You would have fewer outliers in the data and avoid star ratings determined by one or two votes.