Combinations of weak predictors can be very effective.

From Boosting Algorithms: Regularization, Prediction and Model Fitting:

Kearns and Valiant [52] proved that if individual classifiers perform at least slightly better than guessing at random, their predictions can be combined and averaged, yielding much better predictions.

The reference is to Cryptographic limitations on learning Boolean formulae and finite automata.

Scientific Criticism

Michael Eisen on a recent talk by Felisa Wolfe-Simon:

The acid test of a scientist is how they respond when their work is criticized. The best scientists listen and consider what is being said, defend the things they still believe and, most importantly, recognize where their work fell short and use criticism to make their work better. This is, of course, not always so simple. It’s easy to get defensive instead – to view criticism as an attack, see sinister motives in its sources, and ignore its substance.
But I think the worst response is to view criticism as a kind of virtue. And there were signs in Wolfe-Simon’s talk that she is beginning to relish the role of the iconoclast. She appears to see herself as someone who has unconventional ideas that the scientific community can’t deal with. And that criticism of her work is not an effort to get at the truth but a conspiracy to suppress it. At several points she made reference to other scientists whose ideas were not accepted when they were proposed, but which turned out in the long run to be correct. 

People laughed at Galileo. They also laughed at Groucho Marx, though in a different manner. Worst of all is being laughed at like so many laugh at Sarah Palin, as Galileo turned out to be right, and I always got the impression that Groucho was laughing too.

Link: Felisa Wolfe-Simon (of arsenic infamy) is no more convincing in person than in print

Where have all the bullies gone?

The Disappointing Taste of Revenge, a blog entry by Ta-Nehisi Coates:

It's sort of the same thing here. This kid--who shouldn't have put his hands on anyone--gets power-slammed on a concrete driveway, is stumbling out of the frame, and for all we know could be concussed, and you read the comments, and everyone's yelling "Damn right." This is a world filled with people who've been bullied--but no people who are, or ever were, actual bullies.

I'm certain that the real bullies are hanging out with Carl Sagan's fire-breathing dragon, or are orbiting the sun near Bertrand Russell's teapot.

Louise Glass on the cheapness of data and the importance of being focused

Louise Glass from Berkeley has just received a fellowship that will allow her to develop new projects relating to bioenergy. In a recent interview with Nature, she talks about the changes that she's experienced during her career in science:

What has been the biggest change in science during your career?

The pace. When I was a graduate student, a postdoc across the hall from me sequenced one kilobyte of DNA. We have just finished sequencing the 40-megabyte genome of 100 wild Neurospora isolates. In this day and age, it is so easy to get data. The advantage is being able to ask very elegant questions because you are not limited by data. But it is also easy to lose sight of the biological problem you are trying to address. That is the danger.

Difficulties in measuring the value of science

From "The mismeasurement of science," by Michael Nielsen:

In this essay I argue that heavy reliance on a small number of metrics is bad for science. Of course, many people have previously criticised metrics such as citation count or the h-index. Such criticisms tend to fall into one of two categories. In the first category are criticisms of the properties of particular metrics, for example, that they undervalue pioneer work, or that they unfairly disadvantage particular fields. In the second category are criticisms of the entire notion of quantitatively measuring science. My argument differs from both these types of arguments. I accept that metrics in some form are inevitable – after all, as I said above, every granting or hiring committee is effectively using a metric every time they make a decision. My argument instead is essentially an argument against homogeneity in the evaluation of science: it’s not the use of metrics I’m objecting to, per se, rather it’s the idea that a relatively small number of metrics may become broadly influential. I shall argue that it’s much better if the system is very diverse, with all sorts of different ways being used to evaluate science. Crucially, my argument is independent of the details of what metrics are being broadly adopted: no matter how well-designed a particular metric may be, we shall see that it would be better to use a more heterogeneous system.

Neilsen's argument is that measuring the value of science is very difficult, but we must make a judgment on the value of science (or the expected return on investment in a scientific project) whenever we allocate funds in science. His solution to this problem is to maintain multiple metrics by which we judge the value of science rather than using a single metric.

Script of the Week: sync.pl

I've started experimenting with gist for storing and sharing snippets of code online. Below is a summary of a script that I use to synchronize directories across different machines using rsync.

Forensic Bioinformatics and Clinical Trials

From a Nature News article titled Cancer trial errors revealed:

Now, in response to information obtained by Nature under the US Freedom of Information Act, Kornbluth and Cuffe have offered their account of the mistakes that led the trials to be restarted even after they learned of potential flaws in the underlying data. The affair will have an impact beyond Duke, as the Institute of Medicine, part of the US National Academies in Washington DC, begins to examine research on genome-based patient testing. Originally commissioned to investigate Duke's controversial trials, the institute's US$687,000 study is now expected to focus on providing broader recommendations for the design of clinical trials that similarly use genomic data from individual patients to tailor therapy.

This seems like a big mistake on the part of the panel.

Related:

Make Up Your Own Rules of Probability

Irreproducible Analysis

Resolving Debates in Statistics

Occasionally there are debates in science about what models are formally appropriate for thinking reasoning about scientific concepts. These include:

- The existence of race as a meaningful concept in light of current knowledge about human genetic diversity. See Richard Lewontin's article "The apportionment of human diversity" (1972) and Lewontin's Fallacy for more information.

- Quantum and classical physics.

- Bayesian and Frequentist methods for statistical inference.

Scientists have chosen sides in these debates based on education, philosophical preference, and sometimes practical concerns. Usually these debates can be reduced down to the appropriateness of certain calculations in solving scientific problems. For example the Bayesian-Frequentist debate largely centers on the incorporation of prior information into inferential calculations. In "Confidence Intervals vs Bayesian Intervals," by ET Jaynes suggests that all of these debates over methods can be resolved by benchmarking methods on a set of focused problems:

I suggest we apply the same criterion in statistics: the merits of any statistical method are determined by the results it gives when applied to specific problems. 

Sometimes Big Changes Are Better

In college, I was never formally taught the art of debugging. Often in my first few programming classes, I'd end up rewriting chunks of code that weren't giving proper results, as big changes can occasionally be easier than small changes. I'm also a fast typist, so the cost of typing a few hundred lines is relatively low for me.

I always felt strange about this, because it seemed like making big changes to the code in this way is a poor practice. Interestingly, that Peter Norvig occasionally does the same thing, and doesn't even need to understand the details of the bug in order to fix it.  Here's an excerpt from Peter Seibel's excellent book Coders at Work:

Seibel: On a different topic, what are your preferred debugging techniques and tools? Print statements? Formal proofs? Symbolic debuggers?

Norvig: I think it's a mix and it depends on where I am. Sometimes I'm using an IDE that has good tracing capability and sometimes I'm just using Emacs and don't have all that. Certainly tracing and printing. And thinking. Writing smaller test cases and watching them go, and breaking the functionality down to see where the test case failed. And I've got to admit, I often end up rewriting. Sometimes I do that without ever finding the bug. I get to the point where I can just fell that it's in this part here. I'm just not very comfortable about that part. It's a mess. It really shouldn't be that way. Rather than tweak it a little bit at a time, I'll just throw away a couple hundred lines of code, rewrite it from scratch, and often the bug is gone.

Sometimes I feel guilty about that. Is that a failure on my part? I didn't understand what the bug was. I didn't find the bug. I just dropped a bomb on the house and blew up all the bugs and built a new house. In some sense, the bug eluded me. But if it becomes the right solution, maybe it's OK. You've done it faster than you would have by finding it.

This is a really smart way to think about debugging. We don't have infinite time to understand the minute details of some mistake made in the past. What's important is to get things working as quickly as possible. As long as the code has the desired functionality and passes tests appropriately, sometimes rewriting big chunks of code can be an effective debugging method.

Using a Closure to Make Panel Labels in R

When making plots with several panels, I very often want to label the individual panels with letters of numbers. I like to have a simple function for labeling panels, like this:

# create a plot

plot( x, y )

# assign a unique label to the plot
panel_labeler()

Below the break is a definition for the panel_labeler() function using a closure in R. Shown are some examples of using panel_labeler() assign labels to plots. The plots show different methods for combining mixtures of Normal distributions, but this approach could be used for plotting anything.

Infrequently Asked Questions about Perl

I just read a few entries from Infrequently Asked Questions about Perl, a hilarious list of joke answers to serious questions by Mark Jason Dominus. Here are some of my favorite entries:

How do I get tomorrow's date?

Use this function:

sub tomorrow_date {
          sleep 86_400; 
          return localtime();
        }

How can I find out whether a number is odd?
sub odd {
            my $number = shift;
            return !even ($number);
        }
How can I find out whether a number is even?
sub even {
            my $number = abs shift;
            return 1 if $number == 0;
            return odd ($number - 1);
        }

Action Shots of Water Ballons

Just saw a great article on NPR's web site about Edward Horsford's photographs of water balloons as they burst. The photos are brilliant, they capture the rupturing plastic and swirling water as the balloon breaks. 

• Edward Horsford's Flickr Stream
• NPR Article: Water Balloons Without The Water

On his Flickr stream are several other fun shots, including a detailed close-up of a spider and another of a snowflake.  

Illegal Manipulation of Automated Trading Systems

From this CNBC Article:

Two Norwegian day traders have been handed suspended prison sentences for market manipulation after outwitting the automated trading system of a big US broker.

The two men worked out how the computerized system would react to certain trading patterns – allowing them to influence the price of low-volume stocks.

This could become a more serious problem as trading systems become more automated, and I suspect that there are cases similar to this one that we never hear about.

One argument made by the defense is interesting: the defense lawyer admits that his clients misled the trading algorithm in order to make a profit, but weren't responsible for the actions of the computer. They're appealing the case.

String Lengths in R

For some reason, in R this does not do what I expect:

string = "mystring"
print( length(string) )
[1] 1

What I actually want is this function:

print( nchar(string) )
[1] 8

An Idea Whose Time Has Come

If you enjoy literature, politics and satire, you'll love ShakesPalin.

After inventing the word "refudiate" and subsequently comparing herself to Shakespeare as a wordsmith, Twitter is abuzz with posts bizarrely fusing quotes from the Bard on Avon and the Thrilla From Wasilla. Here are some of the better ones:

Alas, poor Couric, I read them all.

But soft, what light from yonder window breaks? It is the East, and I can see Russia from my front porch.

Here's my personal favorite:

Be not afraid of mavericks: some are born mavericks, some achieve maverickry, and some have maverickness thrust upon 'em.

Scientists Drawn By Seventh-Graders

Children draw and describe their perception of scientists before and after visiting FermiLab. Quoting from the project description page:

Few young people have never met a scientist or engineer and most have an opportunity to see where they work. In 1985, middle school teachers asked us if they could bring their students to Fermilab. They helped us develop Beauty and Charm so that those students who came to a working research laboratory would come with a purpose.

Roger Ebert is a sarcastic genius

Some choice words from Roger Ebert's review of Sex and the City 2:

Some of these people make my skin crawl. The characters of "Sex and the City 2" are flyweight bubbleheads living in a world which rarely requires three sentences in a row. Their defining quality is consuming things. They gobble food, fashion, houses, husbands, children, vitamins and freebies. They must plan their wardrobes on the phone, so often do they appear in different basic colors, like the plugs you pound into a Playskool workbench.

It doesn't slow down from there. My favorite thing about this movie is the contempt and sarcasm that it inspires in Mr. Ebert.

R Graphics Parameters That I like

Reproduced here so that I don't lose them:

par( lwd = 1.5 )
par( mgp = c(3,1,0) )
par( mar = c(4,5,3,2) )

Rethinking Paper Money

Michael Tyznik has a great post in which he suggests new designs for American currency:

Graphic Design: Dollar Redesign by Michael Tyznik.

The best part is that rather than filling the reverse side of bills with a smorgasboard of random eyes and famous buildings, Tyznik's designs use tasteful pictures of important people in history and famous quotes from them.

Refusing to Stop

I just finished reading a story about Jure Robic, a Slovenian soldier who excels at ultra long-distance bike races. His races are characterized by mental instability, he frequently hallucinates and uses his hallucinations to motivate himself to work harder. It's very compelling to read the stories of how he and his racing team manage the intense demands that the racing exerts on both his body and his mind.

From Danny Coyle's article in the New York Times titled
That Which Does Not Kill Me Makes Me Stronger:

In a consideration of Robic, three facts are clear: he is nearly indefatigable, he is occasionally nuts, and the first two facts are somehow connected. The question is, How? Does he lose sanity because he pushes himself too far, or does he push himself too far because he loses sanity? Robic is the latest and perhaps most intriguing embodiment of the old questions: What happens when the human body is pushed to the limits of its endurance? Where does the breaking point lie? And what happens when you cross the line?

The brain is a machine made up of hundreds of billions of interconnected neurons, and when machines are pushed to their limits they can begin to malfunction. This story make me wonder how much potential exists in all of us to push harder in our jobs and our hobbies, and what the cost of such exertion would be.