Are physician reviews on hospital-owned websites biased?

Research examines difference between independent sites like Healthgrades and hospital-owned websites

Physician rating sites are everywhere. With a quick search of Google, Healthgrades, Vitals or many other sites, you can easily find reviews and star ratings for nearly any health care provider.

As the popularity of these sites continues to grow, health care systems have also started publishing physician reviews on their own websites, often sourced from patient satisfaction surveys like Press Ganey.

The idea is that it provides additional transparency and helps enable consumers to make an educated decision about their provider.

However, new research published last month raises some concerns about the accuracy and helpfulness of physician reviews on health system-owned websites.

The study, published in the peer-reviewed Decision Support Systems, suggested that ratings on hospital-owned websites were “generally skewed toward scores favorable to physicians, and less dispersed, than corresponding ratings reported on commercial websites.”

Owned vs. Independent

Dr. Nima Kordzadeh, assistant professor at Worcester Polytechnic Institute, collected review data from nearly 2,000 doctors at five hospitals, including data from hospital-owned websites and commercial rating websites like Healthgrades, RateMDs, Vitals, and Google.  

He found that ratings on hospital-owned sites were more concentrated around highly favorable scores compared to commercial rating sites. They were also less varied, making it harder to distinguish between high-quality, medium-quality and low-quality providers.

“In other words, these ratings may not be as useful as they could be to the patients who want to rely on the physician ratings and corresponding comments posted on hospital-owned websites to decide on which doctors to choose,” the study says.

Kordzadeh also points out that consistently positive comments and high ratings might degrade trust in the website because website visitors will become suspicious of the reviewers’ credibility.

This should raise a red flag for health care organizations because if patients don’t trust the information, it will not provide value to them in their decision process and they will seek information elsewhere.

Building trust

When it comes to building trust, hospital-owned websites have some advantages over commercial sites like Healthgrades and Google. Hospital-owned sites generally have a larger sample size of reviews. In one example in Kordzadeh’s study, a provider had 185 ratings on the hospital site, compared to just 13 on HealthGrades, 6 on RateMDs, and 23 on Vitals.

Hospitals also have the advantage of being able to verify that the reviews came from actual patients. Google and other sites simply can’t do that.

But at the same time, if you’re on a health system website and everyone has a rating between 4.75 and 5 stars, it seems a little suspicious.

For example, the doctor in the example mentioned earlier had a 4.8 rating on the hospital site, while her rating on commercial sites was between 3.5 and 4.

So, how do hospitals and health systems address this?

Kordzadeh provides a couple recommendations:

• Closely monitor the difference between commercial sites and owned website reviews and identify reasons for the differences
• Encourage patients to use a variety of sites to make an informed decision
• Emphasize that owned website ratings are not comprehensive of all patient opinions
• Use a combination of online reviews and objective quality indicators like readmission rates, patient safety indices, and timeliness of care.

Building credibility

For health systems that maintain reviews on their websites, its critical that consumers can trust the reviews.

In another study published earlier this year, Kordzadeh found that credibility in the reviewers was an important factor in the use of physician rating sites.

Oddly, the study found that perceived integrity of the website did not influence potential use of rating sites. Kordzadeh explained this by suggesting that people might believe that even if reviews are deleted or manipulated, this doesn’t happen often and doesn’t impact the usability of the site.

According to the study, a key takeaway is that website owners need to ensure the reviewers are credible and their reviews are reliable – and this is communicated to users.