Today I Learned:
1) A great example for teaching Bayes' Theorem, courtesy of one Deniz Senyuz.
Here goes my attempt to explain Bayes' Theorem on Facebook.
Bayes' Theorem is really just a simple equation that tells you how to correctly change how much you believe something based on what you knew before and what you see. In other words, it's the math that explains how learning works (if done correctly).
A quick review of terms:
P(X) = the probability of x, which is a number between 0 and 1
P(X|Y) = the probability of X *given* that Y is true.
P(X,Y) = the probability of X and Y both being true.
Bayes' Theorem: P(X|Y) = P(Y|X) * P(X) / P(Y)
or, as it shows up in scientific contexts:
P(Hypothesis | Data) = P(Data | Hypothesis) * P(Hypothesis) / P(Data)
First, a quick derivation of Bayes' Theorem. I find that going through this proof and knowing how to reproduce it are helpful for understanding how the theorem works. It's a very simple theorem, but feel free to skip this bit to see the example, which is the novel thing I actually learned today.
<PROOF>
First, we note that P(X,Y) = P(Y,X). After all, the order you list them in really doesn't matter.
Now note that P(X,Y) = P(X|Y) * P(Y). For X and Y to be true, Y certainly has to be true, thus the P(Y) term. Once you know that Y is true, the probability of X is, by definition, P(X|Y).
Similarly, P(Y,X) = P(Y|X) * P(X).
Thus, P(X|Y) * P(Y) = P(X,Y) = P(Y,X) = P(Y|X) * P(X)
Divide both sides by P(Y), and you get P(X|Y) = P(Y|X) * P(X) / P(Y).
Next... there is no next. I just wrote Bayes' Theorem. That's the whole proof.
</PROOF>
<AN ASIDE ON THE PARTS OF BAYES'S THEOREM>
Before I actually get to Bayes' Theorem, let me mention how Bayes' Theorem describes the mathematically correct way of updating knowledge when you get some data. As I wrote before, a common use of Bayes' Theorem is the specific case of
P(Hypothesis | Data) = P(Data | Hypothesis) * P(Hypothesis) / P(Data)
The number on the left -- P(Hypothesis | Data) is the confidence you have, as a probability, that some hypothesis about the world is true, given that you just saw some evidence (the Data). This term is usually called the "posterior probability" or "posterior likelihood", that is, the probability (of the hypothesis) after seeing the data.
The first term on the right -- P(Data | Hypothesis) -- describes how probably the data were *IF* the hypothesis is correct. It's often called the "likelihood", short for "likelihood of the data". You can kind of intuit why the posterior likelihood might be proportional to the likelihood. After all, if your hypothesis strongly predicts an outcome, and you get that outcome... you're more likely to believe that hypothesis than if the hypothesis was wishy-washy on the outcome, or make a contradictory prediction.
The second term on the right -- P(Hypothesis) -- is the confiecne, as a probability, that you placed in the hypothesis *before* you saw the data. It's usually called the "prior", short for "prior probability", or the probability of the hypothesis *prior* to the data. (If it weirds you out that Bayes' Theorem relies on your prior beliefs... you're not alone. That's a bigger topic than I'm going to address here, but the single-sentence response is that any kind of learning or statistics is going to involve prior beliefs, and Bayes' Theorem makes them nice and explicit).
The last term is the probability of the data. Not under any particular hypothesis, mind you, just the probability of getting it under *any* possible hypothesis -- to calculate this, you have to sum up all of the probabilities of the data under the different hypotheses, weighted by the (prior) probability of those hypotheses. That's a hell of a pain, but for a lot of purposes you don't actually need to calculate this, for reasons I'll get to in the example. There's a name for this term, but it's terribly unhelpful, so I won't repeat it here. I also don't have a better word for it. For the purpose of this post, I'll call it the "overall likelihood of the data".
</AN ASIDE ON THE PARTS OF BAYES'S THEOREM>
Ok, now the example.
Say you just met me. You know I'm from the US, and you want to know what state I live in. I won't tell you, but I *will* tell you that my district representative is a Democrat. How likely is it that I come from Indiana?
The competing hypotheses here are statements like "Sam lives in Indiana" and "Sam lives in Virginia". The data here is (and yes I know it's "datum" in the singular but SCREW THAT PARTICULAR LINGUISTIC CONVENTION I'M AN ADULT I DO WHAT I WANT) that I live in a district with a Democratic Representative. Bayes Theorem will tell you how likely it is that I'm from a particular state.
What is the likelihood? That is, what's the likelihood that my representative is a Democrat, under the assumption that I live in Indiana? To save some math, let's assume that every representative has an equal number of constituents, so that if I'm from Indiana, I'm equally likely to have any of the Indiana representatives. The probability of my representative being a Democrat (given that I'm from Indiana), then, is the fraction of Indiana representatives that are Democrats. In this case, as of this writing, that number is 2/9. Not very likely.
What is the prior probability? That is, how likely did you think it was that I'm from Indiana *before* you learned that my representative is a Democrat? Well, that depends on what information you have coming in. You might say that since you don't have any idea what state I live in, you assign equal probabilities to my living in any state. You could also say that I'm a random person, so the probability that I'm from Indiana is the fraction of Americans who are from Indiana. Maybe you hear my accent and don't think it sounds very Indianaian, or you see my drivers' license and see that it's actually from Michigan, in which case the prior probability would be very low.
Let's make a pretty minimal assumption and use the assumption I made above, namely that every representative has the same number of constituents. Then the probability that I'm from Indiana is the same as the fraction of total representatives that are from Indiana, which is 9/435 ~ 2%.
What's the overall-likelihood-of-the-data? Well, this would be the probability of my having a Democratic representative if you have *no* idea what state I'm from. Currently, the Democrats hold 188 out of 435 seats, so the overall-likelihood-of-the-data is 188/435 ~ 43%.
Now we just plug those numbers into Bayes' Theorem and it tells you how likely I am to live in Indiana. In this case, it's (2/9) * .02 / 0.43 ~ 0.01, so it's about 1% probable that I'm from Indiana.
Incidentally, there's another question you might ask, which is "what state is Sam most likely to be from?". This kind of question gets asked a lot in science. If you want find the *most* likely state, you can calculate the posteriors for each of the 50 hypotheses involved and see which one's highest. If you want to save some calculation, you might note that the overall-likelihood-of-the-data, P(Democrat), is equally likely no matter which hypothesis you're considering. It doesn't care what state you're asking about. Since *every single* hypothesis you're considering is being divided by the same term, you could multiply all of them by that term and it wouldn't change which is most likely. That's why you often just don't bother calculating the overall-likelihood-of-the-data -- if you're comparing *relative* likelihoods of different hypotheses, it doesn't really matter and can be dropped.
What I love about this example is that there's a nice graphical, visual way to think about the terms of Bayes' Theorem in this example, which helps illustrate why it works. Draw out (or imagine drawing out, if you prefer (or go look up a picture of)) a map of the US divided up by representative districts, with the districts colored in by representative-party. In fact, here's a link to such a map: http://tinyurl.com/p6dojc6. Pretty, no?
Now you can start seeing the terms of Bayes' Theorem visually. The likelihood, for instance, is the probability that a given Indiana representative is a Democrat, so you can blot out all the states except Indiana, and the likelihood is the fraction of the districts that are left that are blue. What's the prior? It's the fraction of *all* of the districts that are in Indiana. What's the overall-likelihood-of-the-data? That's the fraction of Democratic districts that are in Indiana -- blot out all of the Republican districts, and the fraction of what's left is the overall-likelihood-of-the-data.
If you've never seen or used Bayes' Theorem, I hope this teaches you something and convinces you how awesome the theorem is! If you *have* seen Bayes' Theorem before, I hope this explanation helps!
2) Our library has a ninth-floor lounge with a fantastic view of the mountains. Those mountains are a lot more impressive when you're nine stories up and they look just as big.
Also, downtown Pasadena has a lot more trees than I thought.
3) Xbox One controllers still use the same screws as Xbox 360 controllers, the kind that have a pin in the middle so you need a special screwdriver to unscrew them. I also learned that it's supposedly possible to unscrew those with a 2mm flathead screwdriver, though I don't have one myself and thus wasn't able to test it myself.
Saturday, October 17, 2015
Beta Function, Baked Ziti, and British Intelligence (Tangentially)
Today I Learned:
1) The beta function is a really nice function to know when working with binomial distributions. For reference, the beta function is defined as the integral from x = 0 to 1 of [x^a * (1-x)^b] with respect to x. If you look at that thing, it's a heck of a lot like the core of a binomial function, minus the constant-with-respect-to-x combinatorial term.
Conveniently, the Beta function is *also* equal to x!y!/(x + y + 1)!. Taken together, you can turn an integral over the binomial distribution into a relatively straightforward product of factorials.
2) ...a decent vegan alternative to baked ziti is rigatoni stuffed with little blocks of frozen tofu, drowned in tomato sauce with whatever vegetables you like with sauce (for me, it's onions, greek olives, thai chili, and garlic), and bake the whole thing for a while. The tofu stuffing is surprisingly ricotta-like for what it is.
3) When Nazi Germany invaded the Soviet Union during the Second World War, British intelligence estimated that the Soviet Union would be completely taken over in about ten days. US intelligence gave the Soviets a month. This gross misestimation of the Soviet armed forces was based on several factors:
a) During the *last* world war, Russian forces had consistenly underperformed relative to their German counterparts. Early in the war, for instance, a tiny German force was deployed to the Eastern front to stall for time while the actual army in the West beat France (which was supposed to take days or weeks). It ended up crushing several Russian armies and taking serious chunks out of Allied territory in the East.
b) Germany, up to that point in the war, had beaten a number of major nations in ridiculously short amounts of time. They even occupied France -- France! the nation that had ground them to a standstill for *years* just a couple decades before -- in a matter of about a month.
c) Nobody at the time thought that the Soviet Union was a stable enough government to survive a major invasion. It was the product of a revolution from just a couple decades before, had suffered *another* revolution since then, and was not well-loved by many of its member states. A lot of experts thought that the Soviet government would implode in more or less the same way the Czarist regime had near the end of the First World War.
1) The beta function is a really nice function to know when working with binomial distributions. For reference, the beta function is defined as the integral from x = 0 to 1 of [x^a * (1-x)^b] with respect to x. If you look at that thing, it's a heck of a lot like the core of a binomial function, minus the constant-with-respect-to-x combinatorial term.
Conveniently, the Beta function is *also* equal to x!y!/(x + y + 1)!. Taken together, you can turn an integral over the binomial distribution into a relatively straightforward product of factorials.
2) ...a decent vegan alternative to baked ziti is rigatoni stuffed with little blocks of frozen tofu, drowned in tomato sauce with whatever vegetables you like with sauce (for me, it's onions, greek olives, thai chili, and garlic), and bake the whole thing for a while. The tofu stuffing is surprisingly ricotta-like for what it is.
3) When Nazi Germany invaded the Soviet Union during the Second World War, British intelligence estimated that the Soviet Union would be completely taken over in about ten days. US intelligence gave the Soviets a month. This gross misestimation of the Soviet armed forces was based on several factors:
a) During the *last* world war, Russian forces had consistenly underperformed relative to their German counterparts. Early in the war, for instance, a tiny German force was deployed to the Eastern front to stall for time while the actual army in the West beat France (which was supposed to take days or weeks). It ended up crushing several Russian armies and taking serious chunks out of Allied territory in the East.
b) Germany, up to that point in the war, had beaten a number of major nations in ridiculously short amounts of time. They even occupied France -- France! the nation that had ground them to a standstill for *years* just a couple decades before -- in a matter of about a month.
c) Nobody at the time thought that the Soviet Union was a stable enough government to survive a major invasion. It was the product of a revolution from just a couple decades before, had suffered *another* revolution since then, and was not well-loved by many of its member states. A lot of experts thought that the Soviet government would implode in more or less the same way the Czarist regime had near the end of the First World War.
Friday, October 16, 2015
Indigo Day!
Today I Learned:
Facts about dyeing jeans!
1) As of about ten years ago, about 3 BILLION jeans were sold annually, making jeans a rougly $66 billion/year industry. For scale, that's something like a fifth of the US military budget, or about nine times the budget of the NSF.
2) Indigo, the dye used to color jeans, has some very special properties. First off, it's gorgeous. But then, you already knew that*. What's really special is that it binds to cotton and other fibers without any covalent bonding. For reasons I don't really understand, this lets it dye the outside and *only* the outside of a thread, which is why it fades as the outside of the thread abrades away. The fading thing is very important.
Of course, there are plenty of dyes that don't covalently bind. What separates indigo from all the other non-covalent dyes is that it *is* persistent enough to stick to cotton even through heavy bleaching and extreme heat (i.e., the inside of a washing machine/dryer cycle). Most non-covalently binding dyes come right off in water, much less *bleach*.
* I mean, how could you not?
3) Ever seen an indigo plant? (actually there are several, but any one will do for these purposes.) The first thing you'll notice is that it isn't blue. At least, most of the time. See, the indigo plant actually stores a modified form of the molecular precursor to indigo, called indoxyl, in special vesicles in its cells. Indoxyl will spontaneously react with air to form indigo, but before it can, the cell caps its reactive group with a glycosyl group, which stabilizes it. When the indigo plant is stressed, for some reason it releases glycosylated indoxyl from its special vesicles, uncaps the glycosyl groups, and turns brilliant blue. You can induce this by spritzing an indigo plant with ethanol.
Facts about dyeing jeans!
1) As of about ten years ago, about 3 BILLION jeans were sold annually, making jeans a rougly $66 billion/year industry. For scale, that's something like a fifth of the US military budget, or about nine times the budget of the NSF.
2) Indigo, the dye used to color jeans, has some very special properties. First off, it's gorgeous. But then, you already knew that*. What's really special is that it binds to cotton and other fibers without any covalent bonding. For reasons I don't really understand, this lets it dye the outside and *only* the outside of a thread, which is why it fades as the outside of the thread abrades away. The fading thing is very important.
Of course, there are plenty of dyes that don't covalently bind. What separates indigo from all the other non-covalent dyes is that it *is* persistent enough to stick to cotton even through heavy bleaching and extreme heat (i.e., the inside of a washing machine/dryer cycle). Most non-covalently binding dyes come right off in water, much less *bleach*.
* I mean, how could you not?
3) Ever seen an indigo plant? (actually there are several, but any one will do for these purposes.) The first thing you'll notice is that it isn't blue. At least, most of the time. See, the indigo plant actually stores a modified form of the molecular precursor to indigo, called indoxyl, in special vesicles in its cells. Indoxyl will spontaneously react with air to form indigo, but before it can, the cell caps its reactive group with a glycosyl group, which stabilizes it. When the indigo plant is stressed, for some reason it releases glycosylated indoxyl from its special vesicles, uncaps the glycosyl groups, and turns brilliant blue. You can induce this by spritzing an indigo plant with ethanol.
Thursday, October 15, 2015
Point Gagues, Rheology, and The Hamilton STAR Liquid Handling Robot Programming Language
Today I Learned:
1) A point gauge is a device for measuring steady-state water height.
2) Rheology is the study of fluid flow deformation. A rheoid is a solid material that flows by shear forces at least a thousand times faster than it deforms by the same amount of force applied perpendicularly -- for example, granite, or salt under sedimentation.
3) How to program a Hamilton STAR liquid handling robot! It's really simple, conceptually, but whoever designed the programming interface... well, I have a feeling there were a bunch of decisions that were made "just for now to get things working" that ended up codified in the end. Frankly, it feels rather amateurish, especially for a company that sells several-hundred-thousand-dollar robots.
There's something C-like in the design... for instance, there's a way to loop over a sequence of positions on the robot, like a Python for-loop. The resemblances are more than skin-deep -- sequences in this language act much like generators, in that they're kind of functions that return positions in, well, a sequence. However, unlike Python generators, sequences have an explicit pointer that can be viewed and manipulated. Moreover, it *isn't set by the language* whether the pointer stays at the end of the sequence at the end of a loop or goes back to the beginning. You can do either. But you have to choose. Urgh.
1) A point gauge is a device for measuring steady-state water height.
2) Rheology is the study of fluid flow deformation. A rheoid is a solid material that flows by shear forces at least a thousand times faster than it deforms by the same amount of force applied perpendicularly -- for example, granite, or salt under sedimentation.
3) How to program a Hamilton STAR liquid handling robot! It's really simple, conceptually, but whoever designed the programming interface... well, I have a feeling there were a bunch of decisions that were made "just for now to get things working" that ended up codified in the end. Frankly, it feels rather amateurish, especially for a company that sells several-hundred-thousand-dollar robots.
There's something C-like in the design... for instance, there's a way to loop over a sequence of positions on the robot, like a Python for-loop. The resemblances are more than skin-deep -- sequences in this language act much like generators, in that they're kind of functions that return positions in, well, a sequence. However, unlike Python generators, sequences have an explicit pointer that can be viewed and manipulated. Moreover, it *isn't set by the language* whether the pointer stays at the end of the sequence at the end of a loop or goes back to the beginning. You can do either. But you have to choose. Urgh.
Tuesday, October 13, 2015
Simple Eigenvalues, 420, and Eusocial Takeovers
Today I Learned:
1) The eigenvalues of a triangular matrix are its diagonal entries! So freaking simple!
2) 420 = pot. Therefore, the appropriate time to smoke weed is 4:20. The MOST appropriate time to smoke is April 20, at 4:20.
3) Right now, eusocial insects (that is, the "truly social" insects like most ants, termites, and some bees and wasps) are arguably the dominant form of animal life on Earth by biomass (not sure if they outmass planaria or nematodes, TBH). Today I learned that was only true after about 50 million years ago, even though there have been eusocial insects since about 100 million years ago! In other words, eusocial insects (bees, specifically) evolved in the mid-Cretaceous, but were relatively rare until about 14 million years after the extinction of the dinosaurs, at which point they took off in a big way and haven't stopped since.
I always figured that insects had kind of always been the dominant animal life form. Apparently not.
(Bonus eusociality fact -- there are three species of eusocial shrimp! They form nests in coral, which they defend as a group while one female reproduces madly.)
1) The eigenvalues of a triangular matrix are its diagonal entries! So freaking simple!
2) 420 = pot. Therefore, the appropriate time to smoke weed is 4:20. The MOST appropriate time to smoke is April 20, at 4:20.
3) Right now, eusocial insects (that is, the "truly social" insects like most ants, termites, and some bees and wasps) are arguably the dominant form of animal life on Earth by biomass (not sure if they outmass planaria or nematodes, TBH). Today I learned that was only true after about 50 million years ago, even though there have been eusocial insects since about 100 million years ago! In other words, eusocial insects (bees, specifically) evolved in the mid-Cretaceous, but were relatively rare until about 14 million years after the extinction of the dinosaurs, at which point they took off in a big way and haven't stopped since.
I always figured that insects had kind of always been the dominant animal life form. Apparently not.
(Bonus eusociality fact -- there are three species of eusocial shrimp! They form nests in coral, which they defend as a group while one female reproduces madly.)
Monday, October 12, 2015
C. Elegans Reversal, Overwriting That-Which-Should-Not-Be-Overwritten, and Getting the Attention of Dark Gods
Today I Learned:
1) C. elegans fact: C. elegans, the model worm organism, has exactly four sensory neurons dedicated to detecting things-that-make-them-want-to-turn-around. Those four neurons relay their signals to two processing neurons, which integrate that information and, if they decide there's a good reason to turn around, they send a signal to the next layer of neurons, which triggers the worm to immediately reverse direction.
It is always the same four and the same two, in every worm.
2) Python fact: You can overwrite the map function in Python. It is not a good idea. You cannot, however, overwrite "for" nor "lambda".
3) Destiny fact: If you shoot Oryx in the face, he looks at you.
1) C. elegans fact: C. elegans, the model worm organism, has exactly four sensory neurons dedicated to detecting things-that-make-them-want-to-turn-around. Those four neurons relay their signals to two processing neurons, which integrate that information and, if they decide there's a good reason to turn around, they send a signal to the next layer of neurons, which triggers the worm to immediately reverse direction.
It is always the same four and the same two, in every worm.
2) Python fact: You can overwrite the map function in Python. It is not a good idea. You cannot, however, overwrite "for" nor "lambda".
3) Destiny fact: If you shoot Oryx in the face, he looks at you.
Sunday, October 11, 2015
Punic Update, Simon Stalenhag, and A Rice Recipe
Today I Learned:
1) Punic Wars update: Turns out the Fabian strategy didn't last particularly long, at least not initially. Relatively early in the war, Fabius Maximus voluntarily stepped down from Dictatorship (part of the honor of receiving the post was that it meant the government trusted you to hand back your power after some term), and pretty quickly after that one of the Consuls* in charge of the military decided to attack Hannibal with an army twice the size of Hannibal's. That battle was the Battle of Cannae, and it was a disaster. Something like 70,000 Roman soldiers were captured or killed (mostly killed). For a time after that battle, it looked very likely that Rome would fall, but Hannibal couldn't close out the war. After that, the Romans went back to the Fabian strategy, which over the course of the next decade would wear Hannibal down and lead to a Roman victory.
*The Romans had this interesting system where if two Consuls were present in the same army, they would alternate leadership day-by-day. So actually, one Consul elected to *not* attack Hannibal, and the next day the other one attacked. The first Consul died in the ensuing battle.
2) ...about the artist Simon Stalenhag. I absolutely love this art -- I would not mind living in that future. Check him out at http://tinyurl.com/ouglymn.
3) Dill weed, rice seasoning (nori komi furikake, to be exact), and rice wine vinegar is an exceptionally efficient way to make rice tasty.
1) Punic Wars update: Turns out the Fabian strategy didn't last particularly long, at least not initially. Relatively early in the war, Fabius Maximus voluntarily stepped down from Dictatorship (part of the honor of receiving the post was that it meant the government trusted you to hand back your power after some term), and pretty quickly after that one of the Consuls* in charge of the military decided to attack Hannibal with an army twice the size of Hannibal's. That battle was the Battle of Cannae, and it was a disaster. Something like 70,000 Roman soldiers were captured or killed (mostly killed). For a time after that battle, it looked very likely that Rome would fall, but Hannibal couldn't close out the war. After that, the Romans went back to the Fabian strategy, which over the course of the next decade would wear Hannibal down and lead to a Roman victory.
*The Romans had this interesting system where if two Consuls were present in the same army, they would alternate leadership day-by-day. So actually, one Consul elected to *not* attack Hannibal, and the next day the other one attacked. The first Consul died in the ensuing battle.
2) ...about the artist Simon Stalenhag. I absolutely love this art -- I would not mind living in that future. Check him out at http://tinyurl.com/ouglymn.
3) Dill weed, rice seasoning (nori komi furikake, to be exact), and rice wine vinegar is an exceptionally efficient way to make rice tasty.
Subscribe to:
Posts (Atom)