What appears to be a straightforward calculation isn’t. Here’s why.
Six years ago Paul Steinbach published an article in Athletic Business. The NCAA had just released its most recent set of graduation rates. “NCAA graduation rates hit an all-time high,” the headlines read. Steinbach, who’s senior editor at Athletic Business, was curious.
Courtesy: CSRI and YouTube
For perspective, Steinbach turned to an independent source, Richard Southall, director of The College Sport Research Institute at the University of South Carolina. Southhall’s data revealed a different portrait.
After consulting with Southall, this is what Steinbach concluded: “The overall gap between Division I men’s basketball players and the general full-time male student body is once again sizable (with the former group’s graduation rate lagging by 20.6 percentage points). Moreover, the gap for student-athletes in major conferences (a negative 32.4 percentage points) increased almost two points from the initial 2010 report’s benchmark (a negative 30.8).”
What accounts for the difference? It’s how you calculate graduation rate.
Southall’s group focuses on the Adjusted Graduation Gap (AGG), a figure that focuses on the gap (if any exists) between graduation rates of full-time student-athletes vis-a-vis full-time students in the general student body. The NCAA, on the other hand, uses the so-called Graduation Success Rate (GSR), which tracks the percentage of students who graduate 6 years after admission. The NCAA’s GSR differs from a parallel statistic–calculated by the U.S. Department of Education–in that the NCAA metric tracks student-athletes who have transferred from one school to another.
Well, that was the conclusion in 2011. I wondered what, if anything, had changed with the passage of time. So the first step was to scour recent headlines. Here’s what I found.
DI African-American student-athletes graduate at record rates, November 8, 2017, NCAA
College athletes continue to graduate at record rates, NCAA report says, November 8, 2017, LA Times
I also found multiple stories written by/about various schools, touting their respective graduation rates. Here’s an example: University of Oregon Graduation Rates Show Sustained Academic Success By Student-Athletes. November 8, 2017
Richard Southall (photo, Athletic Business)
How curious, I thought. It’s the very same storyline. So I took the second step, replicating the approach Steinbach took in 2011. I turned to a report published in April of this year by Richard Southall’s group. Here’s what I found.
“Players’ AGGs remain large and negative, and AGG trends continue unabated,” the report read. “The AGG is especially troubling for Black male basketball players in major conferences, at -37.1 percentage points. Compared to our initial report in 2011, the overall men’s AGG is 3.7 percentage points larger, while the women’s AGG is 3.3 points larger. These results contrast sharply with the narrative of improving athlete graduation rates in various NCAA reports. The overall D-I men’s AGG is very large: -23.7 percentage points, while the overall women’s AGG is sizable: -12.2 percentage points.”
What accounts for the discrepancy between Southall’s data and the NCAA’s data? Again, it’s how the data are calculated. As one might expect, different formulas yield different results. But there’s more to the story than that.
Southall et al. comment: “The (NCAA) Graduation Success Rate…excludes from its calculation all athletes—including transfers—who leave a school prior to graduating, but in good academic standing (Left Eligibles – LEs). The NCAA methodology also includes athletes who transfer into an institution in that program’s GSR. Essentially, the GSR removes athletes who leave and adds athletes who enter. The NCAA argues the GSR is more accurate than the Federal calculation, but the GSR is itself flawed, significantly exaggerating athlete graduation rates. (italics added) The NCAA contends ‘student-athletes who depart a school while in good academic standing, Left Eligibles (LEs) … are essentially passed from that school’s cohort to another school’s cohort.’ However, the NCAA does not acknowledge the number of transfers-in is significantly smaller than the number of LEs. Contrary to the NCAA’s claims, most LEs are not just passed to another school’s cohort.”
What’s the bottom line? “The number of missing LEs is large, causing the GSR to be significantly inflated (italics added),” Southall, et al. conclude.
Paul Steinbach (photo, Twitter)
Whom should we trust? I favor Southall’s metric and the analysis offered by Steinbach six years ago. For one thing, Southall offers a comparative (to non-athletes) metric. For another thing, I always question figures generated by an industry group/membership association (in this case the NCAA). Southall’s group is an independent operation.
Let’s face it, any organization–the NCAA, included–has a lot to gain and much to lose when “the numbers” look bad. That doesn’t necessarily mean a partisan organization will falsify the numbers, but it does mean that numbers and level of political capital are very much related.
It’s obvious that all formulas are not the same. And one way to look better, even good, is to use a formula that tilts in a rosy direction.
That’s the only way I can explain the stark contrast between Southall’s conclusion and what the NCAA tells us. I’m not at all surprised that the NCAA offers self-serving spin: everything-is-good-and-getting-better. Southall says things aren’t good and they haven’t improved.
But even if you prefer the NCAA’s data, the picture is still cloudy. Take the issue of connecting athletic performance and graduation rates.
This Saturday the Big Ten conference will hold its annual football championship game–the Best of the East (Ohio State) against the Best of the West (Wisconsin). But guess which teams have the lowest graduation rates among schools in their respective divisions? You guessed it: OSU (15% behind leader Penn State) and Wisconsin (25% behind leader Northwestern).
