A controversial scorecard evaluating the performance of orthopedic surgeons throughout the country is raising the ire of the American Board of Orthopaedic Surgery, while Oregon surgeons and their respective hospitals also appear on the list.
The scorecard was developed by ProPublica, an investigative news publication, which analyzed 2.3 million procedures including hip and knee replacements, three types of spinal fusion, gallbladder removals, prostate removals and prostate resections for in-patient hospitalizations between 2009-2013. The analysis is based on billing data submitted to the Centers for Medicare & Medicaid Services.
The scorecard reported complication rates for more than 16,000 surgeons who performed eight common operations on Medicare patients over five years. The ratings cover about 11,500 orthopedic surgeons on three types of spinal fusions and knee and hip replacements.
In Oregon, 47 hospitals performed knee replacement surgeries on Medicare patients while at least 11 (23 percent) had at least one surgeon where patients fared better overall, known in medical terminology as a low adjusted complication rate and 3 (6 percent) had at least one surgeon where patients suffered more harm than their peers, known as a high adjusted complication rate.
The risk of doing better following hip replacement surgeries had much better results at the 42 hospitals in Oregon. (6 surgeons had low complication rates, 8 scored at the high end.)
Here’s a look at the scorecard ProPublica developed for Oregon.
Meanwhile, the Oregon surgeons whose patients had worse outcomes included:
- Dr. Alex Fedorov (hip replacement) at McKenzie-Willamette Medical Center,
- Dr. Charles Skeeters (prostrate removal) who practices at Providence Newberg and Providence Willamette Falls Medical Centers,
- Dr. Edward Bynum (hip replacement) at Good Samaritan Regional Medical Center in Corvallis,
- Dr. Ronald Teed (hip replacement) at Tuality Hospital, Providence St. Vincent Medical Center and Tillamook County General Hospital,
- Dr. Eric Sandefur (hip replacement) at St. Alphonsus Medical Center in Baker City,
- Dr. Jonathan Sherman (lumbar spinal fusion, anterior column) with McKenzie-Willamette Medical Center
- Dr. Sajal Dutta (prostrate removal) from Providence Newberg, Providence Portland Legacy Good Samaritan and Legacy Emanuel,
- Dr. Scott Grewe (knee replacement) at Providence Portland and Legacy Meridian Park,
- Dr. Thomas McWeeney (knee replacement) with Providence Willamette Falls and Legacy Meridian Park,
- Dr. Todd Lewis (lumbar spinal fusion, anterior column) at Good Samaritan Regional Medical Center and
- Dr. Sajal Dutta (prostrate removal) from Providence Newberg, Providence Portland, Legacy Emanuel and Legacy Good Samaritan.
Strikingly, at the Oregon hospitals that performed the most knee replacement surgeries – no doubt because they had the greatest number of orthopedic surgeons – three hospitals stood out for having the highest risk level ranking -- Legacy Meridian Park, Providence Portland Medical Center and Providence St. Vincent, which had 13, 14 and 13 orthopedic surgeons respectively.
Among the other large hospitals, St Charles Medical Center in Bend came away with a lower ranking (for both knee and hip replacements) despite having 9 surgeons. Kaiser Sunnyside Medical Center, which reported only 1 surgeon, ranked in the middle of the pack.
Both Salem Hospital and Sacred Heart Medical Center, 8 and 7 surgeons respectively, ended up in the low to middle range.
Shifting gears to hip replacement surgeries, two small rural hospitals, each with only one surgeon, soared to the top – Tillamook County General Hospital and St. Alphonsus Medical Center while only one hospital ranked in the low to medium range. To review all the hospitals included in this analysis, click here.
The table below shows how Oregon’s hospitals faired in the surgeon scorecard ranking. From left to right the columns lists each of the hospitals, count of surgeons performing, the adjusted complication rate (ACR), and risk level (low to high or L-H). Hospitals with at least one surgeon ranking at the higher risk level (H) are highlighted. (It’s important to note that surgeons performing fewer than 20 procedures at each hospital are excluded from the surgeon scorecard ranking.)
Hospitals with surgeons performing procedures less common (spinal fusions, gallbladder removal, prostate removal or resection) are shown in the table below with adjusted complication rate and ranking (L-H). Hospitals with at least one surgeon ranking at the higher risk level (H) are highlighted in the table. Notably, a few hospitals ranked at the upper end of the adjusted complication rate (above 5 percent) for varying procedures: St Charles Medical Center, Good Samaritan Regional Medical Center, and McKenzie-Willamette Medical Center.
To provide a little more background information about the analysis, the table below lists the range of adjusted complication rates from low to high for each of the procedures included in the surgeon scorecard, along with the minimum, maximum, and average rankings.
The adjusted complication rate is an assessment of how a surgeon would perform at a hypothetical average hospital on a standardized group of patients. To identify risk rates for the surgeon scorecard, ProPublica’s analysts used two primary measures: in-house mortality, and readmission within 30 days of discharge (with the likely diagnosis resulting surgery complications). For more information about the ranking’s methodology, ProPublica has a short form here or a long form here.
After ProPublica published these scores, the board’s executive director, Dr. Shepard Hurwitz, said those ratings would be incorporated into the recertification process where surgeons are re-evaluated very 10 years.
In an article appearing in ProPublica, Horwitz acknowledged this technique was controversial but said, “But, the fact that we’re doing it is in the spirit of transparency and holding people accountable for what’s already in the public domain. We’re not revealing anything that’s given to us in confidence.”
Not long after that article, Hurwitz was forced to backtrack, saying his comments about recertifying surgeons had been based on his own willingness to use the scorecard but that decision had never been approved by his board.
For More Information
To learn more about the statistical analysis developed to determine the risk level of surgeons, click here
Diane can be reached at [email protected] and Kathryn can be reached at [email protected].
Comments
ProPublica scorecards
"The real danger of scorecards like ProPublica's lies not in their failure to objectively capture quality but in their pretense that they suceed."
Before reading this report, one should read: L. Rosenbaum. "Scoring No Goal--Furthur Adventures in Transparency.NEJM 2015;373:1385-1388. It is not "controversial." It is simply bad data. It is not that we would not benefit from data on these very issues, but we need to recognize when we do not have it. Sorting good from bad data is a major issue of our time. Bad data may trump no data, but it shoud not.Reading results when the methods are not valid has led us astray many times.
Jim Patterson
Lazy and Unprofessional Work
While the data may be from CMS, the "reporting" of it by ProPublica shows lazy, biased, and unprofessional work. In the opening statement of the white paper it states: "and hundreds of patients who have been harmed while receiving medical care" - why did they not include the millions of patients who suffered no harm? Apparently, they put this information out in the manner in which it is because no reputable publication would publish this "study". This is a prime example of someone creating bad information and putting it out on the web as fact. For example: Physicians who are on-staff at mutliple hospitals are listed with all of them - which skews the results for any hopsital, making it all meaningless. Hospital A is tarnished if a surgeon does one case there, while doing 99 cases at hospital B and has 5 complication there. If ProPublica actually used CMS data, it would have been very easy to separate out the surgeries by hospital, yet they failed to do so - why?
Additonally, there are many examples of surgeons who have zero complications, however the white paper states: "The Adjusted Complication Rate does not directly represent a surgeon’s past outcomes. It is an assessment of how s/he would perform at a hypothetical average hospital, on a standardized patient pool." (see Section 3.1, page 13) In other words... "we are not treating the physicians as individuals, we are treating them as an average physician at an average hospital." The why publish this data at all? By treating the data this way, it makes all of the physicians "average" rather than showing actual differences in complication rates: ProPublica has made this data have zero meaning. The ProPublica whitepaper is not even close to an accurate assessment of data or a reflection on these surgeons, and yet this is being presented to the public as gospel. Total Garbage!
Jake Thielen
I don't think orthopedists should be doing prostate removals.
Dr. Leigh Dolin