Most social scientists are interested in causality. Today's New York Times highlights a fascinating question. Did working to clean up ground zero cause significant bad health outcomes for those exposed? The conventional wisdom is "yes" but the Times reports today that this inference is based on a strangely collected sample and has been analyzed by a group of doctors and researchers who the New York Times claims has a political agenda tied to labor unions. The Times hints that this group have a bias towards findings that support worker claims that they are victims who merit compensation from the local government.
This case strikes me as a fascinating test to see if structural researchers can do a better job on a causal question than reduced form guys. Here are some of the issues;
Your goal would be to recover estimates of a health production function. We observe now whether a workers who was exposed to the aftermath of 9/11/2001 is in good health. Did the exposure causes "excess sickness"?
To answer this question, you need to impute the counter-factual of what this worker's health would have been had she not been there for those weeks after 9/11.
The challenge for the researcher is self selection and heterogeneity. Who chose to work at the site? How much were they exposed? Did they engage in any unobserved self protection? Did the researchers sample a representative sample of the 9/11 workers or collect a non-random sample where sick people are over-represented?
Do people have any incentives to over-state their sickness (whiplash!) to try to cash in and collect $ from the state?
Are the doctors competent enough to diagnose who is really sick or not? If people are sick today, how do you know that the "smoking gun" is 9/11 rather than other trends such as family history, smoking, other sources of stress?
Good luck separating out all of these factors but billions of $ are stake here in compensation and the lawyers will get rich!
September 7, 2007
Accuracy of 9/11 Health Reports Is Questioned
By ANTHONY DePALMA and SERGE F. KOVALESKI
Much of what is known about the health problems of ground zero workers comes from a small clinic in Manhattan that at the time of the trade center collapse had only six full-time doctors and a tiny budget.
Yet in the weeks after 9/11, its doctors stepped into the fray in the absence of any meaningful effort by the city, state or federal government to survey, interview or offer treatment to potentially sickened recovery and cleanup workers.
Since then, the clinic, the Irving J. Selikoff Center for Occupational and Environmental Medicine, based at Mount Sinai Medical Center, has examined more than 15,000 workers and volunteers and has overseen the examination of 5,000 more at clinics elsewhere.
Those programs have received more than $100 million from the federal government for tracking and treating those workers. The clinic’s doctors published the largest and most often quoted study of recovery workers’ ills. And they have testified about the health problems before city and federal committees.
But six years after the disaster, it is clear that while the center’s efforts have been well meaning, even heroic to some, its performance in a number of important areas has been flawed, some doctors say. For years after 9/11, the clinic did not have adequate resources or time to properly collect detailed medical data on workers exposed to ground zero dust.
The clinic’s doctors presented their findings in what other experts say were scientifically questionable ways, exaggerating the health effects with imprecise descriptions of workers’ symptoms and how long they might be sick.
Researchers in this field say that the clinic’s data collection was so badly planned that its usefulness may be limited. Others say that doctors at the clinic, which has strong historical ties to labor unions, have allowed their advocacy for workers to trump their science by making statements that go beyond what their studies have confirmed.
Dr. Albert Miller, a pulmonologist who spent more than three decades at Mount Sinai before moving to Mary Immaculate Hospital in Queens in 1994, worries that the actions of the center’s leaders have harmed the legitimate cause of workers who might be in need of help. “They are doing the workers a disservice,” he said, “because any time you veer from objective and confirmable statements, you’re destroying your own case.”
“They are people with a cause,” Dr. Miller said.
Even now, there is debate about how harmful the dust was, and whether it could cause cancer or debilitating chronic diseases, although there is emerging medical consensus that workers who arrived at ground zero early and stayed longest were at greatest risk of getting sick. Medical studies by the Fire Department, and most recently by the city health department, show that the dust has caused diseases like asthma and sarcoidosis (a lung-scarring disease) in a small percentage of rescue workers.
Although the Selikoff clinic’s research has found signs of ill health in more workers than other studies, it generally tracks the same trends. But that has not lessened the skepticism of critics.
The clinic’s leaders acknowledge that their efforts were troubled. But they challenge anyone facing the same hardships to have done better. The doctors point out that they took on ever-increasing responsibilities with federal financing that came in fits and starts. They had to continue their clinical care while collecting data, and clinical care had to come first. They tackled an unprecedented epidemiological challenge with too little money, too few records and too little time to plan properly.
“I’ll accept that we could have done some things better and there’s always room for improvement,” said Dr. Philip J. Landrigan, who has overseen the clinic’s efforts to help ground zero workers. “You have to have a thick skin in this business.”
While organized labor has steadfastly supported and praised the Selikoff Center’s efforts, other doctors say its missteps have heightened the anxiety of New Yorkers who expected the center to answer medical questions that have unsettled the city since 9/11.
There remains confusion about whether government officials should have done more to protect workers from toxic materials at ground zero. The city is still contesting thousands of lawsuits from workers who claim they were sickened while working at ground zero, even as it is providing millions of dollars to Bellevue Hospital Center to treat people sickened by the dust.
And experts agree that the clinic’s imperfect work — done alone and under difficult circumstances — might have long-lasting consequences if the poorly collected data eventually skew the results of future studies. Should the clinic come to conclusions different from other medical researchers, say experts, those contrary findings would confuse the overall health picture, delaying scientific consensus. The city would then have lost valuable time in developing a precise picture of diseases from this kind of disaster and the public health response needed.
Dr. Steven Markowitz, who runs a ground zero screening and monitoring program at Queens College, and who worked at the Selikoff Center in the 1980s, says there is no doubt that the clinic, for all it has accomplished, has also let people down.
“Frankly,” he said, “it was reasonable for the public to expect more.”
A Logical Choice
Forty-eight hours after the attack, Dr. Robin Herbert, Dr. Stephen Levin and other Mount Sinai doctors met at a Westchester County home to figure out how to respond to the disaster at ground zero. They agreed to volunteer extra hours to see sickened workers, and to gather medical information on them. And in the weeks and months that followed, the Selikoff Center was virtually the only place for workers to turn to.
While federal officials warned those on the pile to protect themselves from the dust, they also said that the chance of developing serious long-term illnesses was low. And city officials stressed that the risk of illness from exposure was minimal. They also faced enormous legal liability if workers on the smoldering pile got sick.
Thomas R. Frieden, commissioner of the New York City Department of Health and Mental Hygiene since 2002, said in a recent interview that the threat of lawsuits in no way shaped the city’s response. Rather, he said, the city did not step in more forcefully because clinical treatment is not one of the department’s responsibilities. But, he said, it was something the Selikoff Center did well.
Few people in New York’s medical community were surprised that the center had taken the lead. After all, the Selikoff Center, named after a pioneering asbestos researcher who died in 1992, was founded in the mid-1980s with political backing from New York labor leaders. It was well known for serving injured union workers, including those with lung diseases, a major concern of Dr. Selikoff’s.
But on 9/11, the center was focused mostly on repetitive strain injuries, the workplace hazard of the moment. Still, ground zero workers complaining of a persistent cough started showing up on Oct. 2. It was not until April 2002, six months later, that the Federal Emergency Management Agency provided the center with $12 million to support a program to give physical and mental health examinations to 9,000 workers.
But the clinic got no money to begin a comprehensive research program, or to make any long-range plans for tracking or caring for injured workers.
“We were told very unequivocally that we were not being funded to do research,” recalled Dr. Herbert, who has been a part of the of the screening program since its inception. “We were being funded to do screening.”
Without money or time to plan, they started collecting data anyway, knowing that it would be necessary to track the rise of symptoms related to dust exposure. But the medical history questionnaire they pulled together was an unwieldy 74 pages long, full of questions that were too vague to be useful. When combined with X-rays and breathing tests, the examination process took more than three hours and scared off many workers. Some of the data was collected on paper and stored in boxes.
“It took me three months just to figure out where the information was and how it had been kept,” said Dr. Jeanne Mager Stellman, a medical researcher who was hired as deputy director of the data center in April 2006. “I don’t think they knew what they were getting into.”
Dr. Stellman resigned last November for personal reasons but continued to work on several mental health studies of ground zero workers. “This is a program that’s done enormous good for 20,000 people,” she said, “but it’s a program that has not yet met expectations.”
The clinic’s doctors also faced significant problems because critical information was simply not available. There were no records of how many people worked at ground zero or for how long. No one knew exactly what was in the dust or how much contamination each person at the site breathed in. And since many workers had not seen a doctor regularly before Sept. 11, there was no reliable way to confirm when respiratory symptoms and ailments started.
By contrast, the New York Fire Department, which monitors its 15,000 firefighters, knew exactly how many firefighters had been exposed. And mandatory annual checkups provided precise medical histories.
It was not until 2004 that the Mount Sinai clinic started to receive federal financing for analysis — about $3 million a year for a data and coordination center. The money was part of $81 million in federal aid for medical tracking — half to cover firefighters, and the rest for ground zero workers.
By then, it was too late to undo some of the missteps made early on.
A Misleading Impression
The Selikoff Center has been criticized for blurring the line between scientific observation and alarmism in acting like an advocate for worker causes. But its doctors say that an aggressive approach is necessary in occupational health because employers tend to challenge complaints about workplace safety.
“I’ve spent my whole professional life walking that line,” said Dr. Landrigan, who founded the center in 1986 with Dr. Selikoff. “You can collect facts and be rock-solid certain about those facts, but you know quite well that those facts are only a piece of the puzzle. The intellectual question then is: ‘Do I have enough information to issue a call for action?’ ”
Last year, as the fifth anniversary of the attack approached, the center produced a major report that was published in Environmental Health Perspectives, a scientific journal of the National Institute of Environmental Health Sciences, a federal agency. The report said, and Dr. Landrigan declared at a major press conference, that 69 percent of 9,442 responders examined had reported “new or worsened respiratory symptoms.”
In fact, a chart accompanying the report showed that 46.5 percent reported the more serious lower respiratory symptoms, which lung specialists consider to be indications of significant health problems (17 percent reporting shortness of breath, 15 percent reporting wheezing, and 14 percent listing cough with phlegm), while 62.5 percent of the workers reported minor upper respiratory symptoms like runny noses and itchy eyes.
The decision to combine the two categories of symptoms was criticized by medical experts, but it made a powerful — and misleading — impression on the public and the press about the nature and scale of the health problems.
“There is not a scientific reason to lump those two together,” Dr. John R. Balmes, a professor of environmental health and medicine at the University of California, San Francisco, who reviewed a version of the report before it was published, said in a recent interview. “Science is better served separating them.”
Dr. Miller, who called the press conference a “public relations extravaganza,” said: “I’m not as worried about a runny nose as I am about shortness of breath.”
In fact, the 69 percent figure — though it deals with symptoms, rather than actual diseases — suggests a more alarming picture than other studies. For example, a report by the city health department released last week showed that about 4 percent of 26,000 ground zero workers reported developing asthma after working on the pile. And the Fire Department’s sarcoidosis study focused on 26 new cases of the disease since 9/11.
Dr. Landrigan, in an interview, defended the way he presented the findings, maintaining that symptoms like a persistent runny nose could have indicated more serious lower respiratory problems.
The clinic was also criticized for suggesting that the symptoms were longer lasting than their own evidence indicated at the time. No symptom, major or minor, had persisted for more than two and a half years when the study was done, and a condition is not generally considered chronic until it lasts at least five years, doctors say. Yet Dr. Herbert said at the press conference that many workers would “need ongoing care for the rest of their lives.”
Newspapers, including The New York Times, gave prominent play to Dr. Herbert’s statements about the lasting nature of the problems. For some experts, her words went too far.
“It’s very hard to predict the future,” said Dr. Markowitz. “I know people want answers, and I know people want to give answers, but we really have to stick to the scientific method if we want to understand the truth.”
One thing is certain. The press conference galvanized many more workers to seek medical exams. More than 1,000 additional workers signed up for monitoring and 500 new workers continue to enroll each month even now.
Dr. Landrigan said he and his colleagues did not exaggerate their findings to scare workers. But other experts said the doctors may have caused a panic.
“We have patients constantly saying after one of these pronouncements, ‘Am I going to die?’ ” said Dr. David Prezant, deputy chief medical officer of the New York Fire Department, who has overseen several epidemiological studies for the department.
Dr. Prezant said that the Selikoff clinic’s statistics sometimes so worried workers that they neglected proven treatments to seek unorthodox cures that have questionable results.
In what many critics regard as the clinic’s most disturbing recent miscue, Dr. Herbert said in a 10-minute audio interview posted in May on the Web site of The New England Journal of Medicine that she was seeing the beginning of a “third wave” of disease, referring to cancer. In her interview, which accompanied a separate article on ground zero health effects by doctors not affiliated with the Selikoff Center, she named specific types of cancer — leukemia, lymphoma, multiple myeloma — and expressed concern about “synergistic effects” caused by chemicals in the dust, a controversial contention among medical experts.
She was instantly criticized by doctors outside Mount Sinai, who felt her comments were irresponsibly speculative because there is no evidence yet to conclusively link exposure to the dust to cancer. But the city’s tabloid newspapers seized on Dr. Herbert’s comments, prompting another panic among some recovery workers.
In an interview last month, Dr. Herbert defended her comments, explaining that she was speaking as a clinician and sharing her observations about diseases she was seeing with other clinicians.
“I feel that it is our job to communicate as clearly as we can what we do know, what we worry about, what are possible red flags,” Dr. Herbert said. “We have to strike a balance between not exaggerating and not waiting to act until we have absolute proof.”
Praise From Unions
Today, union officials stand by the work the Selikoff Center has done.
“Sinai should be canonized for the services it is providing,” said Micki Siegel de Hernandez, the health and safety director for District 1 of the Communications Workers of America. “The doctors have really established relationships with responders who walk in. This is the place where workers know that the people care and have the expertise.”
Only late last year did the center and the other clinics begin getting federal money to treat ill workers — $17 million then and more on the way. About 10,000 are now receiving treatment, which generally consists of prescription medication or counseling.
Most days, dozens of ground zero workers make their way to the clinic on East 101st Street. Dr. Jacqueline Moline, who now directs the programs, said some workers show up to be examined for the first time. Others come back to be re-examined. All of them expect answers, but for most, uncertainty has become a constant part of their lives. The center continues to collect data from each of them, and Dr. Landrigan said he expected to publish as many as 10 new reports within the next 18 months.
Eventually, doctors and scientists analyzing the long-term effects of the dust will take into account not only Mount Sinai’s studies but those of the Fire Department, the city’s health department and other sources. Clinical studies will continue for decades.
The Selikoff doctors acknowledge their mistakes, but they do not apologize for speaking out aggressively about the potential health dangers.
“If our advocacy has brought in people and we’ve saved their lives because we’ve identified health problems, whether they’re World Trade Center-related or not, I’ll take that any day of the week,” said Dr. Moline. “And if that’s our epitaph — that we talked loudly and we brought people in for health care — so be it.”