How many school shootings happen in the U.S. in a single school year? The answer is surprisingly hard to figure out.

In April, the U.S. Department of Education released a report on the 2015-2016 school year, stating that “nearly 240 schools (0.2 percent of all schools) reported at least 1 incident involving a school-related shooting.” However, the nonprofit Everytown for Gun Safety’s database lists only 29 school shootings for the same period.

When National Public Radio investigated the inconsistency, they found that 161 of the Department of Education’s 240 shootings either did not occur or could not be confirmed by the school districts involved. Similarly, the American Civil Liberties Union contacted each school that allegedly had a shooting and found that 138 of the reported shootings were errors.

So where does school shooting information come from? How could these counts be so far apart?

The laws on guns in schools

Students are legally prohibited from bringing guns to school. That means they can be in serious trouble, even if they never fire the gun.

The Gun-Free Schools Act of 1994 requires schools in any state receiving certain federal funding to implement a one-year expulsion rule for students who bring a firearm to school. Students found in possession of a firearm must also be referred to the criminal justice or juvenile justice system.

Each year, schools must report any firearm-related expulsions. Those can include shootings, but also firearm possession and other firearm crimes. Schools report firearm offenses like these to the Civil Rights Data Collection (CRDC), which is operated by of the Department of Education.

In a biennial survey, the CRDC asks schools and other public local educational agencies like charter schools: “For the regular (…) school year, not including intersession or summer, was there at least one incident at the school that involved a shooting (regardless of whether anyone was hurt)?”

The U.S. Department of Education used the results of this question to estimate the number of school shootings for 2015-2016.

Schools are expected to report any incidents that occur during school hours on school grounds. In addition to shootings, schools report the number of incidents that involved possession of a firearm, as well as the number of robberies, homicides, physical attacks and physical fights that involved a “firearm or explosive device.”

Room for error

A closer look at this survey shows why the U.S. Department of Education’s data was so inaccurate.

First, there are problems with the definition of the term “shooting.” Nowhere in the CRDC survey is that word clearly defined. Is it a shooting if a student has a gun that accidentally goes off? Could the term “shooting” refer to incidents involving toy guns?

Schools are asked to provide counts of various crimes involving firearms or explosive devices, so some counts of possession or attacks may be due to weapons other than firearms. The CRDC’s definitions of various terms may differ from those used by state agencies or even the school districts themselves. If schools don’t record disciplinary actions using the same terms, reporting becomes very difficult.

Second, there’s the issue of the burden on school administrators. Schools already report a great deal of information to state agencies. Locating the required information or re-reporting information to multiple agencies can be time-consuming, possibly leading to errors or confusion about what to report. The CRDC is making some changes to their 2017-18 data collection effort to reduce this burden.

It’s also easy to make mistakes. A school administrator might accidentally type a 9 rather a 0. Or they may accidentally enter the number of offenses for one crime category in the space allotted for another category. These errors may not be spotted until after the results of the survey are made public. There’s currently no procedure in place to verify that reported shootings are, in fact, shootings.

Why getting the count right is so important

Errors like these can have major consequences – especially if the number is inflated.

A Pew Research Center poll in April found that 57 percent of teens were somewhat or very worried that a school shooting could happen in their school. Parents of teens were even more likely to report worry over school shootings, especially those with low incomes. Some families are considering homeschooling their children out of fear of school shootings.

School shooting counts can also sway policy. In this year alone, 26 states considered bills or resolutions related to arming school personnel. Even more introduced bills or resolutions related to guns in K-12 schools more generally.

Some sources of school shooting data, like Everytown, rely on a different resource: the media. They look for news mentions of school shootings instead of reports from the schools themselves. However, these counts can also be inaccurate. Not all firearm-related offenses come to the attention of the media. Some media mentions must be verified with law enforcement to ensure the details are accurate.

Combining methods may be our best option. As a researcher who studies criminal justice, I believe verifying school shooting reports with a media search may reduce errors without placing undue burden on schools. Double-checking each reported shooting is time-consuming and costly. Unfortunately, so are the consequences of getting the count incorrect.

Lacey Wallace does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.