tashirosgt

2012-Sep-06, 03:15 AM

Much of the theory of statistics deals with random samples from continuous random variables. Is such sampling actually possible?

Sampling a continouous random variable is impossible in a trivial sense, in the same way that exactly bisecting an angle with a physical appartus is impossible. It's due to the limited precision of physical devices. Physical sources of random "noise" measured by physical devices produce approximate outputs.

There is a purely mathematical type of impossibility, for example, the impossibility of trisecting an angle "with a ruler and compass". The phrase "with a ruler and compass" restricts the operation to certain mathematical methods. The impossiblity of accomplishing the task is just as much a mathematical fact as the non-existence of a largest real number. It's interesting to consider whether sampling a continuous random variable is impossible in this mathematical sense.

To determine this, we must specify what methods are allowed. A common scenario in the theory of computability is to assume we use an idealization of a computer, such a Turning machine. For computations involving randomness, it's common to assume the algorithm can have external inputs of "real" randomness. These have limited precision. For example, an input might be a 0 or 1 representing the result of tossing a fair coin.

I'm not an expert in the the theory of computability, but it seems to me that, with those restrictions, an algorithm that terminates in a finite length of time cannot generate samples from a continuous random variable. The cardinality of the possible outputs appears to be a "countable" infinity and the cardinality of an interval of real numbers is an "uncountable" infinity.

Some may say "We can't trisect angles, but we can still define the trisectors of an angle as mathematical objects. We ave methods that approximate the trisectors to any desired degree of accuracy. We can't randomly sample continous random variables, but we can conceive of such samples and we can approximate sampling from a continuous random variable to any desired degree of accuracy".

You can define angle trisectors and approximate them, but how do you even define doing continuous random sampling "to any desired degree of accuracy"? Try to specify a rigorous epsilon-delta type definition. I can make one that "works" for the uniform distribution on [0,1], but I don't see how to do it for distributions like the Gaussian which have an unbounded range of values.

There are sources on the web that discuss "samplable" probability distributions from the point of view of the theory of computability but I haven't found one that deigns to explain what the various definitions of "samplable" say when applied to continouous real valued random variables. If anyone can give an interpretation, I'd like to hear it.

Sampling a continouous random variable is impossible in a trivial sense, in the same way that exactly bisecting an angle with a physical appartus is impossible. It's due to the limited precision of physical devices. Physical sources of random "noise" measured by physical devices produce approximate outputs.

There is a purely mathematical type of impossibility, for example, the impossibility of trisecting an angle "with a ruler and compass". The phrase "with a ruler and compass" restricts the operation to certain mathematical methods. The impossiblity of accomplishing the task is just as much a mathematical fact as the non-existence of a largest real number. It's interesting to consider whether sampling a continuous random variable is impossible in this mathematical sense.

To determine this, we must specify what methods are allowed. A common scenario in the theory of computability is to assume we use an idealization of a computer, such a Turning machine. For computations involving randomness, it's common to assume the algorithm can have external inputs of "real" randomness. These have limited precision. For example, an input might be a 0 or 1 representing the result of tossing a fair coin.

I'm not an expert in the the theory of computability, but it seems to me that, with those restrictions, an algorithm that terminates in a finite length of time cannot generate samples from a continuous random variable. The cardinality of the possible outputs appears to be a "countable" infinity and the cardinality of an interval of real numbers is an "uncountable" infinity.

Some may say "We can't trisect angles, but we can still define the trisectors of an angle as mathematical objects. We ave methods that approximate the trisectors to any desired degree of accuracy. We can't randomly sample continous random variables, but we can conceive of such samples and we can approximate sampling from a continuous random variable to any desired degree of accuracy".

You can define angle trisectors and approximate them, but how do you even define doing continuous random sampling "to any desired degree of accuracy"? Try to specify a rigorous epsilon-delta type definition. I can make one that "works" for the uniform distribution on [0,1], but I don't see how to do it for distributions like the Gaussian which have an unbounded range of values.

There are sources on the web that discuss "samplable" probability distributions from the point of view of the theory of computability but I haven't found one that deigns to explain what the various definitions of "samplable" say when applied to continouous real valued random variables. If anyone can give an interpretation, I'd like to hear it.