Talk:Information theory

From EvoWiki

Just a minor question about formatting: isn't the LaTeC (<math></math>) mode supposed to wotk? It does on WikiPedia!

--FreezBee 14:20, 8 May 2006 (BST)

Re: Information and DNA

The article text concludes:

So the mutation has added 0.01041 bits of information to the genetic pool.

Except that creationists will look at it differently. They will start out assuming a uniform distribution of all 4 possible bases in these four loci. That gives an entropy of H = log₂(4⁴) = 8. The decrease in entropy (= uncertainty) before the mutation is 8 – 1 = 7, but after the mutation it is 8 – 1.01041 = 6.98959, a smaller decrease. Before the mutation you needed only ask one question to figure out, which “type” a person was, but after the mutation you may need to ask two questions. Look at it this way: before the mutation you could indicate the type of a person by “A” and “C”, two different letters, after the mutation you need three different letters, e.g. “A”, “C”, and “T” (using the last letter of the 4-base sequences as indicator), so you have more possibilities and therefore more uncertainty about which possibility is the actual case.

For further info, see the article The Spetner Anomaly.

--FreezBee 09:56, 8 May 2006 (BST)

Complexity and Randomness

It's a common creationist claim that mutations cannot increase genetic information. Mutations will introduce randomness and thereby decrease rather than increase the information. The complexity of genetic information cannot be increased by random mutations, therefore all genetic information now extant must always have been here. Something along these lines.

However, in information theory complexity and randomness are positively correlated. In Kolmogorov-Chaitin information theory a string is algorithmically complex or algorithmically random if it has a high entropy (relative to its length). The problem for the creationists is that they want "complex" to mean "non-random"; but information theory can't really help them here.

For a concrete example, read Nancy Pearcey's article DNA: The Message in the Molecule. Pearcey writes:

The information content of any structure is defined as the minimum number of instructions needed to specify it. For example, a random pattern of letters has a low information content because it requires very few instructions: 1) Select a letter of the English alphabet and write it down, and 2) Do it again. A highly ordered but repetitive pattern likewise has low information content. Wrapping paper with "Merry Christmas" printed all over in ornate gold letters is highly ordered, but it can be specified with very few instructions: 1) Write "M-e-r-r-y C-h-r-i-s-t-m-a-s," and 2) Do it again. By contrast, a structure with high information content requires a large number of instructions. If you want your computer to print out the poem "'Twas the Night Before Christmas," you must specify every letter, one by one. There are no shortcuts. This is the kind of order we find in DNA. It would be impossible to produce a simple set of instructions telling a chemist how to synthesize the DNA of even the simplest bacterium. You would have to specify every chemical "letter," one by one.

What Pearcey refers to in the first line is Kolmogorov-Chaitin entropy. However, "a random pattern of letters" will have a high, not a low, entropy. The entropy is measured by the instructions to produce this string and not some other string. Toss a coin 100 times and you will get some sequence of heads and tails. Toss a coin another 100 times, and you'll most likely get a very different sequence. In Pearcey's wording they would be the same sequence, because they can both be produced by tossing a coin 100 times; that is a simple repetition of a simple instruction. But how would Pearcey detect whether some sequence of heads and tails is "random" or designed? Pearcey's definition of randomness is statistical randomness, which isn't the same as algorithmic randomess - a string can be statistically random without being algorithmically random (the Champernowne sequence is an example).

If we have, as Pearcey writes, ``to specify every chemical "letter", one by one´´, then we have something that is algorithmically random, something with high entropy, but that's the opposite way of detecting design as for instance William Dembski's Explanatory Filter works.

--FreezBee 12:00, 10 May 2006 (BST)

This section seems more reasonably included in the article than here. Also note that sometimes your apostrophes are replaced by fancy directional ones that do not become wikiformating, as in the last paragraph which was apparently meant to have a section in italics. --Suttkus 13:37, 10 May 2006 (BST)

Yes, but since I'm not sure, what your plans are for the article, I wrote it here - you are welcome to copy it into the article, where you think it fits. As for the apostrophes - the problem was that Pearcey used quotation marks, so I couldn't use those, and double `'´ won't work either - that would have made the text be italic, which wasn't my intention. I simply didn't really know how to solve the problem :-)

--FreezBee 16:45, 10 May 2006 (BST)

Plays? I think you rate us too highly! We don't have plans! Besides, you're part of us now, mwa ha ha ha ha ha!

The gramatical convention is to alternate single and double quotes to contain text with quoted segments. So, the following would be gramatically correct, but stupid:

"My teacher said, 'Billy, you shouldn't tell people that Mary said, "I heard Donna say, 'Billy is a sleeze who makes overly referential quotations!'"'"

But I think we can patch this without looking that silly!  :-)

--Suttkus 17:42, 10 May 2006 (BST)

I have moved it to the article page - to where I thought it would fit the best. --FreezBee 10:26, 23 May 2006 (BST)