Let Them Eat Data

A new Discovery series, The Curious Case of Natalia Grace, demonstrates once again that truth is stranger than fiction. In 2010, Michael and Kristine Barnett were raising three boys in a new home in Indiana when they had the idea to adopt a girl. They worked with an adoption center to identify a Ukrainian orphan and soon were united with Natalia. While the Barnetts were told Natalia was six, something was amiss from the start. Natalia towered over playmates and her bone structure and vocabulary were more advanced. Then they discovered Natalia appeared to have gone through puberty, raising suspicions that instead of a six-year-old orphan, they’d adopted an adult dwarf. Suspicions turned to nightmares as Natalia began reading the bible “to get rid of these evil thoughts” and then was caught trying to pour cleaning fluid into their coffee as well as standing by their bed, knife-in-hand. As the Barnetts discovered, the only thing worse than mistaking an adult dwarf for a six-year-old is adopting a sociopathic adult dwarf.

Adopting a child is a huge decision, not unlike enrolling in college – a commitment of years and tens (and sometimes hundreds) of thousands of dollars. By making bad enrollment and program decisions over the past two decades, millions of Americans have found themselves in predicaments perhaps not as perilous as Natalia Grace, but still worse off and curious to discover went wrong. While Democrats and Republicans differ on causes and solutions, they’ve found one plot of common ground: making more and better college data available to prospective students and their families.

Announced a decade ago and rolled out in 2015, College Scorecard was an Obama administration transparency revolution for the opaque and byzantine college selection process. On a new user-friendly site, each accredited college and university was listed along with data on graduation rates, net tuition, student loan debt, repayment rates, and earnings after graduation. In addition, for the first time College Scorecard made this information available by program.

Establishing a federal one-stop-shop for college data would inevitably result in better enrollment decisions, so the thinking went. Students would sidestep underperforming schools and newfound accountability would strike the fear of God into institutions heretofore uniquely unaccountable for outcomes. In the name of transparency, College Scorecard was a priority for the Trump administration and Republicans who care about college remain focused on data; The Lowering Education Costs and Debt Act recently introduced by Senate Republicans includes provisions on improving transparency to give students and families better information. Meanwhile, President Biden’s Department of Education has taken up the baton of improving College Scorecard, recently adding new debt and earnings data, and continues to promote it.

College Scorecard has attracted its share of critics arguing the data isn’t perfect e.g., data doesn’t include Parent Plus loans, private loans, or students not receiving federal financial aid, earnings data doesn’t account for cost of living, data not available for majors with a small number of students. But that’s the logic of letting the best be the enemy of the good. Unless, of course, there’s little good to begin with.

Early papers on the impact of College Scorecard on college decisions weren’t promising. Nick Huntington-Klein at CSU Fullerton found a negligible effect on Google search activity: no increased interest in top schools and programs and no reduced interest in schools and programs with awful outcomes. The College Board’s Michael Hurwitz and Jonathan Smith of Georgia State discerned no impact on SAT score sends from Scorecard graduation rate and affordability data and only a small increase from earnings data. While any improvement is welcome, Hurwitz and Smith’s problem was that the students whose behavior was altered by the prospect of making more money came from private schools; the effect on public and charter school students was not statistically significant. Worse, they only found a response to Scorecard earnings data for students with SAT scores of at least 1100 and for Asian and white students – not Black and Hispanic students.

Last month Lumina Foundation president and CEO Jamie Merisotis offered a stirring defense of higher education to the American Academy of Arts & Sciences:

We also know the severe toll of not having higher education. Journalist Jon Marcus… described it this way. People without education past high school earn significantly less than those with bachelor’s degrees and are more likely to be unemployed and live in poverty. They’re more prone to depression, live shorter lives, need more government assistance, pay less in taxes, divorce more frequently and vote and volunteer less often.

I’ve previously described higher education’s problem with information asymmetry: while colleges probably have a good sense of employment outcomes from the degrees and programs they’re selling, prospective students – particularly those in greatest need of assistance with skills and employment – have no earthly idea. And information asymmetry leads to bad decisions. So, to Jamie Merisotis’ lengthy and gloomy list, he might have added the following: “They’re also less likely to make a good decision about higher education.” College Scorecard is inherently limited by the fact that those who need college most are least equipped to access, interpret, and act on data about colleges.

A recent Chronicle of Higher Education article quotes a Department of Education official who wishes to remain anonymous acknowledging that College Scorecard usage numbers “are not necessarily where we’d want them to be.” Yale’s Zachary Bleemer agrees: “We have very little sense that students are using this information.” So while the bipartisan College Transparency Act reintroduced in the House of Representatives this spring aims to further improve College Scorecard and other available data, a data-only or even data-first cure for higher education’s maladies seems like wishful thinking – a “let them eat data” approach reminiscent of some doomed ancien régime. (You know who’s better situated than at-risk students to act on college outcome data? College leaders and trustees. But that’s even more of a stretch.)

From the outset, progressives recognized that aggregate data can’t predict individual outcomes and isn’t a replacement for good advice. As the former doyenne of the down-and-out, Sara Goldrick-Rab, once said, the Scorecard is “good information to have, but it doesn’t tell an individual student what to do.” “So Much Data, So Little Guidance” bemoaned a Chronicle of Higher Education headline. Rather than fussing with data, wouldn’t we be better off devoting resources to putting data we already have in front of the right students at the right time?

Researchers have demonstrated that targeted interventions can work. After the FDA required chain restaurants to put calorie counts right on menus, diners reduced consumption by as much as 100 calories per meal. Also a decade ago, Caroline Hoxby and Sarah Turner reported on their Expanding College Opportunities project which attempted to address the problem of low-income students under-matching to less selective institutions by mailing specific information at key points during the college decision process. The information included in Hoxby and Turner’s mailings – “the kind of advice that an expert college counselor would give a high-achieving student” – resulted in a dramatic increase in applications to better schools. More important, students were admitted to 31% more colleges and the most selective school to which students were admitted had a median SAT score that was 53 points higher.

While I wouldn’t want to accuse anyone in the Ohio House of Representatives of having read Hoxby and Turner, the idea of putting data we already have in front of the right students at the right time is apparent in a bill passed last month. Bill 27 requires state institutions to deliver the following to accepted students before the decision deadline:

· Total cost of attendance and net cost

· Expected student loan payments

· Graduate earnings for:

o Most recent class

o Class graduating five years ago

o Relevant earnings for declared major or school

Equally important, schools aren’t allowed to lose students in the fine print; the information “shall not exceed one double-sided page in length.” The bill passed 88-1 and is now up for consideration in the Ohio Senate.

An element of Senate Republicans’ new higher education salvo, The Informed Borrowing Act of 2023, takes a similar approach but gets the timing wrong. Senators Cassidy and Daines propose requiring cost, student loan, and earnings disclosures, but only as part of student loan entrance counseling i.e., after enrollment. New rules promulgated by the Department of Education would require students enrolling in programs that fail prescribed debt-to-earnings rates to sign a disclosure, but seem confused about whether acknowledgement would happen before or after the fateful decision (and given last week’s cacophony of critical comments from groups as important as the American Council on Education, will almost certainly be watered down). Hoxby and Turner had the timing right: during the admission process and ahead of matriculation (and, if necessary, again before a program is selected).

If we’re looking for earlier intervention, what about high school? An effective intervention would go further than Virginia did last year when it required its Department of Education “to collect and distribute to public schools and publicly post on its website information that… ensure[s]… students are aware of the costs and benefits of different educational and certificate programs.” The good news is that almost all states require high schools to report on a range of college & career readiness (CCR) metrics. As a result, districts adopt CCR platforms like Naviance which run students through box-checking exercises with much less emphasis on assessment and discovery than on the process of applying to colleges and requesting transcripts and letters of recommendation. Once students check every box on a CCR platform, the school’s work is done.

But the college application process is exactly when college data is needed. So it’s surprising no state has thought to task K-12 school districts with Hoxby-and-Turner-type targeted interventions as part of the CCR process. Building personalized “right data at the right time” interventions into CCR platforms could be win-win-win for states, high schools, and students (but probably lose for thousands of colleges). It’s also an ideal application for AI: ensuring students receive and understand College Scorecard data before they apply, matriculate, and choose a major. That’s an engine that can help accelerate socioeconomic mobility.

We haven’t heard the last of Natalia Grace. Following the success of the series, Discovery greenlighted a follow-up, The Curious Case of Natalia Grace: Natalia Speaks, in which Natalia presumably will claim she was actually six. But while Michael and Kristine Barnett would have benefited from relevant data before making their life-changing decision (e.g., Natalia’s actual age, emotional stability), we can’t say the same for most prospective students – particularly those in greatest need of postsecondary education.

The quickest path to better college decisions and outcomes is targeted interventions at the end of high school and before enrollment or major selection. The bottleneck isn’t more or better data, let alone federal action. We just need states to step up and ensure students are better informed than the Barnetts so they’ll avoid picking a Natalia Grace.