Lots of good stuff here today from Ben Goldacre of Bad Science: In the Guardian newspaper (UK), he writes primarily about how Alasdair Philips of Powerwatch has a profit motive not disclosed in BBC's Panorama propaganda on Wi-Fi safety--he sells gear that "protects" people from electromagnetic radiation. Philips doesn't hide the fact on his lobbying site, but the program should have disclosed that fact, since it labeled other people with economic associations. (You can read the transcript of the show here.)

The BBC presenter Paul Kenyon says that Philips wasn't given a chance to interpret the results, but clearly in the show you can see Philips's reaction, and Philips's group's opinion essentially drove the tone and nature of the show, which lacked any skepticism about claims that are medically and fundamentally unproven. Further, we don't know what device or scale Philips used and how it was calibrated. We don't even know that he's an expert in conducting these measurements; he says he is. If so, another expert unassociated with his views or devices should have been able to reproduce the results. (Kenyon admits that in a follow-up interview at BBC; see below.)

Kenyon responded to directly to Goldacre in an interview posted on his site. Kenyon essentially says, we talked to a few people, and they all sort of vetted each other, so we're done. An actual documentary would have involved talking to dozens of people and then perhaps showing the opinions from a few.

Let me tell you how journalism works. When I wrote a 2,700-word item on municipal Wi-Fi for The Economist a year ago March (samizdat copy, paid version), I spoke to over two dozen people specifically for the article in addition to hundreds I'd interviewed or had talks with over the two years prior to that. I started out with a set of assumptions, but listened to what people said, and wound up writing an article somewhat different than I thought I was writing. (It's proved out to be accurate, too, so far.) That's journalism. Daily journalism often lacks that kind of time, so you read quicker, more facile stuff that gets facts or context wrong because it has to get out the door. An Economist feature or BBC documentary isn't typically produced on that scheduled if it's not a matter of breaking news, in which case journalists usually air on the side of what's best known at that time.

A number of people claim all journalists have preconceived notions and find subjects and quotes to fit those notions. Most of the journalists I know rather follow the story, and while the participants in the story may not like the piece we cut out that strikes our interest because we think that's the part that our readers will like--we're not tailoring the facts to fit our notions.

Had I been researching this documentary, I would have wanted to find out from several epidemiologists with no horse in this race how credible various claims were about risk based on studies. You can often find academics willing to read studies and read statements about them to give you a reality check. I would also have noted that there aren't two sides to this fence: there are people who promote the idea that EMF makes you sick; and mostly scientists and health officials on the other end evaluating research and finding no smoking gun but not stating definitively no shots have been fired. I would also have made sure that if I was finding something spectacularly different than what was widely accepted that I could explain the reason why effects weren't being seen--I wouldn't have speculated that the methodology for all measurement was wrong (as Panorama did, relying on a few scientists), but would have looked into how the methodology has worked in the past, and how it might be falling down now. That's a more involved set of research, but was what was needed to make the claims in the show.

The BBC itself grills Kenyon on its Newswatch program (see heading "Update BBC24 Newswatch transcript"). In it, Kenyon agrees it was a bad idea to use Philips alone to take measurements, and had no idea that the Swedish scientist he used a resource is considered controversial (read: a misinterpreter of results, which I know from reading his actual study, which includes a lot less correlation than he or anyone else reading it claims).

Kenyon twists himself into a pretzel, too. The program dismisses the fact that classroom levels measured by Philips were at 1/600th of the level deemed to be unsafe by the government, but then accepts Sir William Stewart's statements as fact, even though he's part of government. Go figure.

Kenyon makes this extraordinary claim: "But out position was that there is a mainstream accepted view on this and I think it’s a role that Panorama can play to challenge that mainstream sort-of culturally accepted norm." Perhaps the idea that Paris Hilton is a spoiled rich kid who avoids the consequences of her actions until now could be received wisdom and the idea that she's actually a modern rebel teaching a non-conformist view of the world would be the challenge. But for god's sake, this is about science with empirical, measurable results.

One of my favorite cartoons shows a scientist thinking, "What happens this time when I mix ammonia and bleach?" (or perhaps two similar agents that react together). The caption, as I roughly recall it: "Why all scientists are empiricists." I'm not representing it beautifully, but the cartoon shows what happens when you reject the idea that science is founded on reproducible experiments which, built up over time, produce an association of cause and effect founded on reason. That's science. Everything else is anecdote and conjecture, and, while useful, must be put in that context.

This newer cartoon rather well summarizes the situation (and is hilarious, to boot).

Goldacre also finds that a letter responding to criticisms was written by the BBC before the program had actually aired. Huh. Seems like someone had the idea that the problem might have weaknesses in sourcing. And Goldacre finds that Panorama tried to take measurements with students in a room, but the school balked at what they saw as bad science.

This would all just be a matter of one show not getting right except it's the Beeb, the venerable institution, and I've been reading about alarm spread worldwide by the program, and already seeing that some schools are turning off Wi-Fi.