A Response To Consumers Digest's Waterslide Safety Report (Part 1): The Numbers Don't Lie

A story was published today in Consumers Digest entitled “Waterparks:  Is Public Safety Going Down the Tubes” that paints an extremely unflattering and misleading picture of the safety of the waterpark industry as a whole.  Authored by Sara Bongiorni, the piece makes a number of disturbing assertions to suggest that injuries at waterparks in the United States are increasing at an alarming rate and that the best solution to countering this trend is federal regulation of the fixed site amusement industry.  The piece takes great issue with the so-called “patchwork” of state regulations governing the amusement industry and even goes so far as to portray the industry as expending hundreds of thousands of dollars per year specifically to avoid federal regulation.  I’ve seen other pieces like this – usually in the wake of a tragic accident at a park or carnival.  Rarely, though, have I seen a piece that goes to the extent that this piece does in mis-characterizing the facts and ignoring gaping holes in the logic and reasoning underlying its conclusions.  I thus feel compelled to address some of the more troubling aspects of Ms. Bongiorni’s piece.  The article though is rather lengthy, so I thought the best way to address it was in two parts.  Today, I’ll discuss the problems with Ms. Bongiorni’s injury statistics, and how they were manipulated to paint a far direr picture than exists in reality.  In Part 2, I’ll address her contention that federal regulation is the answer to all the industry’s problems.

At the outset, though, and in the interest of full disclosure, I should mention what most of you already know.  I am an industry advocate.  I have spent over twenty years in and around the amusement park industry, both as a park operator and as an attorney.  I am a member of the International Association of Amusement Parks & Attractions and a member of its Government Relations Committee.  I am also a member of the New England Association of Amusement Parks & Attractions, and serve on its board of directors.  (To be perfectly clear, I speak for none of those organizations or committees – this piece represents only my personal views and opinions).  Does this make me biased toward the industry?  Perhaps.  But it also gives me insight into the industry that I am confident Ms. Bongiorni lacks.  I have written ride procedures, implemented new safety protocols, dealt with countless numbers of guests – both happy and sad, and managed thousands of ride operators.  I have stood on top of lift hills and unloaded guests from roller-coaster trains at two hundred feet.  I helped create a ride rating system later used, at least in part, by parks and facilities all over the country.  I helped design the safety protocols for ride operations at one of the most technologically state-of-art theme parks every constructed.  I have dealt with inspectors – both state and insurance and, unfortunately, have dealt with a few unfortunate ride incidents as well.  So yes, I might be biased about the industry, but I’m also well-versed and experienced in its operation.  I know what works for the industry and its guests, and what doesn’t.  And because of that, I know, without any reasonable doubt, that the industry is safe and that federal regulation of the waterpark and, more generally, the amusement industry will do little, if anything, to make it safer. 

 

Unfortunately, Ms. Bongiorni’s piece is available only to Consumers Digest subscribers, so I cannot link to its full text.  And, while I have a copy, I don’t have the authorization to reprint it.  I think I can make my point anyway, but please try to find a copy and take a look – I don’t want to be accused of hiding the ball here.  So …. Where to begin?  How about with this?

“Although waterpark operators insist that their facilities are safe and that injuries are extremely rare, our investigation found evidence of increasing injury rates and a lack of nationwide safety standards.  The number of estimated injuries that are the result of accidents at waterparks increased by nearly 38 percent since 2009, according to federal statistics, even though attendance at waterparks increased just 3.8 percent over the same period.” 

Well, that does not sound good, does it?  But let’s unpack that a bit, because I promise you that the reality is not nearly as dire as it appears from this statement.  

 Let’s focus on the statement that the “number of estimated injuries that are the result of accidents at waterparks increased by nearly 38 percent since 2009.”  This statement is the foundation upon which the rest of the article is built.  After all, absent such a dramatic increase in waterslide injuries, I doubt there would be much of a story to write. There are, however, at least two major problems with this statement.

First, it uses one of the oldest rhetorical tricks in the book.  Rather than just telling the reader how many injuries federal statistics claim to have occurred and putting those numbers in the context of total visitation to waterparks, Ms. Bongiorni begins by citing a dramatic percentage in isolation to give the appearance of a serious problem.  To illustrate, if I were to tell you that emergency room visits for first-graders at my child’s elementary school had increased from 3 to 4 last year (because my daughter got something in her eye and had to go to the ER), you would likely think little of it – particularly if I added the context that there are about 100 first graders there.  But, if I tell you that emergency room visits increased by 33% last year, you might well start to wonder what was going on at that school to cause such an extreme increase.  This is the same trick Ms. Bongiorni uses here. 

 She is not lying, of course, she’s just not telling the whole truth.  What she is leaving out is the context.  While she does state that “the estimated number of emergency-room visits that resulted from injuries that were suffered at ‘public waterslides’ increased to 5,200 in 2011 … from 3,779 in 2009,” Ms. Bongiorni does not give any immediate context to these numbers.  A couple of paragraphs later, she cites attendance statistics provided by the World Waterpark Association which state that, in 2009, North American waterparks entertained 80 million guests and in 2011, 83 million.  The author uses these figures to make the point that the 38% increase in injury rate between 2009 and 2011 cannot be attributed to increased attendance, which saw only a 3.8% increase.  But what she didn’t say was this.  

According to these same figures, even assuming the accuracy of the government statistics (more on that later) the injury rate at a waterpark in 2009 was approximately 0.0047%.  In 2011, it had “skyrocketed” to .0065%.  Put another way, in 2011, after a 38% increase in waterpark injuries according to federal statistics, 1 in every 15,385 guests visiting a waterpark incurred a reportable injury. 

When you put it that way, a visit to a waterpark does not seem nearly as scary as Consumers Digest wants it to, does it?

Now, let’s talk about those injury statistics themselves.  Ms. Bongiorni claims the statistics she cites, which are derived from the Consumer Products Safety Commission National Electronic Injury Surveillance System (NEISS) – a reporting system made up of approximately 100 hospitals distributed around the country, are indicative of “the number of estimated injuries that are the result of accidents at waterparks.”  Is this true?  Well … sort of, but not really.

The problem is in the use of the phrase “the result of.”  This implies causation.  When a reader sees that phrase, the clear inference to be drawn is that these are injuries that were caused by waterparks.  But that is not at all the case.  But don’t take my word for it.  Take the word of the CPSC.

The CPSC’s NEISS Coding Manual, the document given to hospitals to instruct them on how to do the reporting that is eventually compiled into the CPSC’s injury statistics, is very clear.  Hospitals that participate in the NEISS system are instructed to report 

all consumer product-related emergency visits … , including emergency department cases, hospital admissions, trauma center and burn center cases, and cases transferred to other hospitals…. ‘Product-related’ is defined to include … all injuries where a consumer product, sport, or recreational activity is associated with the reason for the visit or related to a condition treated. … It is not necessary to determine that a product was at fault to report an associated injury. … If in doubt, report it. 

Thus, the statistics on waterslide injuries cited in the Consumers Digest piece do not say anything about whether those injuries were actually caused by a waterslide, only that they were associated with or related to a waterslide.  According to the CPSC’s reporting manual, therefore, a person who slips and falls while climbing a waterslide staircase or on a sidewalk next to a waterslide would be a reportable event for a NEISS hospital.  In fact, here are some of the actual injuries that made up the 2011 statistics:

•  A 13 year old female bumped her elbow going down a waterslide and three weeks later went to a hospital complaining of pain;
• A 15 year old male sprained his foot after hitting it on a board at a water slide at camp;
  
  
• A 15 year old male sprained his foot sliding into a board on a water slide at camp (not a duplicate – two separate incidents apparently);
•  A 10 year old male fell while climbing the ladder of a water slide sustaining a contusion to his toe;
• A 9 year old girl fell while skating, injuring her hand, and had previously bumped her head on a waterslide.  She was treated for a hyperextended left hand and a headache;
•  An 11 year old male was diagnosed with second degree sunburn on his chest after visiting a waterpark

None of these have any obvious connection to the safe operation of a waterslide, but all of them were included in the government statistics cited in the Consumers Digest article.

And that is what makes it, quite frankly, amazing that the article also posits that “the estimates from NEISS might underestimate the number of injuries” occurring at waterparks.  Such a statement makes no sense in light of the broad reporting duty required of NEISS participating hospitals.  Given that a bad sunburn makes the cut as a reportable waterslide injury, it is difficult to imagine that the statistics are actually underestimating water park injuries.

Moreover, Ms. Bongiorni summarily rejects the position of IAAPA spokesperson Colleen Mangone, that “if the raw data NEISS collects come from hospitals in areas that have a disproportionate number of waterparks, NEISS estimates by be inflated.”  Indeed, scientific data backs up Ms. Mangone’s position.  In a study titled, “Limitations of child injury data from the CPSC’s National Electronic Injury Surveillance System:  the case of baby walker related data,” published in the journal Injury Prevention in 1996, researchers found that the NEISS reporting system resulted in over-reporting of injuries associated with baby walkers.  That study reported that only two hospitals

reported 14% of all the NEISS baby walker related injuries though they encompassed 2% of the patients among NEISS Samples.  In some years, … one very large children’s hospital in the sample contributed as much as 17% of all weighted walker related injury estimates. … For trend analysis of product related injuries at the level of occurrence studied for baby walkers, NEISS suffers from poor sensitivity due to relatively large sampling error. … For all children’s injuries, NEISS’s hospital type sampling scheme reflects a random geographic imbalance from 1991 to 1994 because one north eastern state contributes both of the reporting children’s hospitals.

If the NEISS reporting system has been scientifically proven to have resulted in over-reporting of baby-walker injuries due to uneven geographic dispersion of reporting hospitals, why could the same not also be true of waterslide injuries – particularly given that the product  - i.e. the waterslide – is not nearly as evenly geographically dispersed as a baby-walker.  When one considers that NEISS hospitals are instructed to broadly report any injuries “associated with” a water slide or waterpark regardless of causation, that these hospitals may well be unevenly distributed across the country in relation to waterparks, and that waterparks themselves are not evenly distributed either, it takes no stretch of the imagination to see how the injury data could be inflated.

But to recognize any of this would be to undermine the entire premise of Consumer Digest’s report:  that water parks are experiencing a dramatic rise in their injury rates and that the industry is working hard to keep its parks from becoming more safe by fighting federal regulation.  I think I’ve pretty well addressed the former point.  In part 2 of this piece, I’ll take on the latter.  See you then.