Underdetermination of infinitesimal probabilities

Abstract

A number of philosophers have attempted to solve the problem of null-probability possible events in Bayesian epistemology by proposing that there are infinitesimal probabilities. Hájek (Synthese 137:273–323, 2003) (more tentatively) and Easwaran (Philos Rev 123:1–41, 2014) (more definitively) have argued that because there is no way to specify a particular hyperreal extension of the real numbers, solutions to the regularity problem involving infinitesimals, or at least hyperreal infinitesimals, involve an unsatisfactory ineffability or arbitrariness. The arguments depend on the alleged impossibility of picking out a particular hyperreal extension of the real numbers and/or of a particular value within such an extension due to the use of the Axiom of Choice. However, it is false that the Axiom of Choice precludes a specification of a hyperreal extension—such an extension can indeed be specified. Moreover, for all we know, it is possible to explicitly specify particular infinitesimals within such an extension. Nonetheless, I prove that because any regular probability measure that has infinitesimal values can be replaced by one that has all the same intuitive features but other infinitesimal values, the heart of the arbitrariness objection remains.

Appendix

Let us start by reviewing some standard facts about extended reals and their standard and infinitesimal parts.

Suppose that ${\mathbb {R}}^*$ is an extension of the reals. Say that elements of ${\mathbb {R}}^*$ are infinitesimally close provided that their difference is zero or infinitesimal. Note that the only way two real numbers can be infinitesimally close is if they are equal. For the difference between two reals is real, and hence can’t be infinitesimal.

Now suppose $x\in {\mathbb {R}}^*$ is finite, i.e., $-y< x < y$ for some real y. There can be at most one real number infinitesimally close to x. For suppose there were two. Then they would also be infinitesimally close to each other (the sum of two infinitesimals is infinitesimal). But as we just saw, that’s impossible for two reals.

However, there is at least one real number infinitesimally close to x. For let $S = \{ z \in {\mathbb {R}}: z < x \}$. Then S is a non-empty set, as $-y \in S$ (we assumed $-y<x<y$ for a real y), and it’s bounded above because $z<y$ for every $z\in S$. Every non-empty set of reals that is bounded above has a supremum, i.e., a least upper bound (this is a defining property of the reals). Let $x_1$ be the supremum of S. I now claim that $x_2=x-x_1$ is infinitesimal or zero. If $x_1 = x$, this is trivial. So suppose $x_1 \ne x$. There are two possibilities. Either $x_1 < x$ or $x < x_1$. Suppose first that $x_1 < x$. If $x-x_1$ is not infinitesimal, there must be a real number $a>0$ such that $a\le x-x_1$. Then $x_1 + a \le x$, and so $x_1< x_1 + a/2 < x$. Hence, $x_1 + a/2 \in S$, and so $x_1$ is not an upper bound for S, contrary to the assumption that $x_1$ is the supremum of S. Hence, $x-x_1=x_2$ has to be infinitesimal. The other possibility is that $x < x_1$. If $x_1-x$ is not infinitesimal, there must be a real number $a>0$ such that $a \le x_1 - x$. Then $x \le x_1 -a$, so $x< x_1-a/2 < x_1$. But then $x_1-a/2$ is an upper bound for S and yet is less than $x_1$, so $x_1$ is not the least upper bound for S, contrary to $x_1$’s being the supremum of S. So, $x_1-x$ must be infinitesimal, and hence $x_2 = -(x_1-x)$ is infinitesimal.

Thus, $x_1$ is infinitesimally close to x, and so for every finite extended real x there is a unique real, which we will denote by ${\text {Std}}\,x$, infinitesimally close to it. This is called the standard part of x. Let ${\text {Inf}}\,x = x - {\text {Std}}\,x$ be the infinitesimal part of x, so $x = {\text {Std}}\,x + {\text {Inf}}\,x$.

Let $\phi _\alpha (x) = {\text {Std}}\,x + \alpha {\text {Inf}}\,x$. We can now prove the lemmas from Sect. 3.6, which will be useful for proving Theorem 1. First, Lemma 1 follows immediately from:

Lemma 3

${\text {Std}}\,$ and ${\text {Inf}}\,$ are ${\mathbb {R}}$-linear functions.

Proof of Lemma 3

If $x_1$ and x are infinitesimally close to each other and if $y_1$ and y are infinitesimally close to each other, then $ax_1+by_1$ is infinitesimally close to $ax+by$, if a and b are real, since $ax+by-(ax_1+by_1)=a\cdot (x-x_1)+b\cdot (y-y_1)$ and if $x-x_1$ and $y-y_1$ are infinitesimal, their product with a real number is still infinitesimal, and the sum of two infinitesimals is infinitesimal. Hence, if $x_1 = {\text {Std}}\,x$ and $y_1 = {\text {Std}}\,y$, it follows that ${\text {Std}}\,(ax+by)=a{\text {Std}}\,x+b{\text {Std}}\,y$. Hence, ${\text {Std}}\,$ is an ${\mathbb {R}}$-linear function. But ${\text {Inf}}\,x = x - {\text {Std}}\,x$, so if ${\text {Std}}\,$ is ${\mathbb {R}}$-linear, so is ${\text {Inf}}\,$. $\square$

Second, we prove the strictly increasing character of $\phi _\alpha$.

Proof of Lemma 2

Suppose $x<y$ are finite members of ${\mathbb {R}}^*$. By linearity $\phi _\alpha (y)-\phi _\alpha (x) = \phi _\alpha (y-x)$. Letting $z = y-x$, all we need to show is that if $z>0$ is finite, then $\phi _\alpha (z)>\phi _\alpha (0)=0$. To show this, suppose first that z is infinitesimal. Then ${\text {Std}}\,z=0$ and ${\text {Inf}}\,z=z$, so $\phi _\alpha (z)=\alpha z > 0$. Suppose now that z is not infinitesimal. Then ${\text {Std}}\,z$ is a positive real, but $\alpha {\text {Inf}}\,z$ is infinitesimal. Adding an infinitesimal to a positive real has to result in something positive, so $\phi _\alpha (z)={\text {Std}}\,z+\alpha {\text {Inf}}\,z>0$. $\square$

Note also that $\phi _\alpha$ fixes the reals, i.e., $\phi _\alpha (x) = x$ for any real x, as then ${\text {Inf}}\,x=0$. Finally, observe that $\phi _{1/\alpha }(\phi _\alpha (x))=x$, so $\phi$ is a bijection.

Lemma 4

Suppose P is an ${\mathbb {R}}^*$-valued probability function, and $\phi :{\mathbb {R}}^*\rightarrow {\mathbb {R}}^*$ is an ${\mathbb {R}}$-linear and strictly increasing bijection fixing the reals. Let $P_\phi (A)=\phi (P(A))$. Then:

• (i) $P_\phi$ is an ${\mathbb {R}}^*$-valued probability function,

• (ii) $P(A)<P(B)$ if and only if $P_\phi (A)<P_\phi (B)$,

• (iii) if $P(C)>0$, then $P(A\mid C)<P(B\mid C)$ if and only if $P_\phi (A\mid C)<P_\phi (B\mid C)$

• (iv) if $a_1,\dots ,a_n,b_1,\dots ,b_m$ are real, then for any events $A_1,\dots ,A_n,B_1,\dots ,B_m$ we have $\sum _i a_iP(A_i) < \sum _i b_i P(B_i)$ if and only if $\sum _i a_iP_\phi (A_i) < \sum _i b_i P_\phi (B_i)$

• (v) if $P(C)>0$, and $a_1,\dots ,a_n,b_1,\dots ,b_m$ are real, then for any events $A_1,\dots ,A_n,B_1,\dots ,B_m$ we have $\sum _i a_iP(A_i\mid C) < \sum _i b_i P(B_i\mid C)$ if and only if $\sum _i a_iP_\phi (A_i\mid C) < \sum _i b_i P_\phi (B_i\mid C)$.

Proof

We first verify the axioms of probability for $P_\phi$. Consider any P-measurable event A. If $P(A)=0$, then $P_\phi (A)=\phi (P(A))=0$ as $\phi$ preserves reals. If $P(A)>0$, then $\phi (P(A))>0$ by increasingness and real-preservation. Hence, we have $P_\phi (A)\ge 0$ since $P(A)\ge 0$ for all A.

Next, since 1 is real, $P_\phi (\Omega ) = \phi (P(\Omega )) = \phi (1) = 1$.

Finally, if A and B are disjoint, then by linearity and the finite additivity of P, we have $P_\phi (A\cup B)=\phi (P(A\cup B))=\phi (P(A)+P(B))=\phi (P(A))+\phi (P(B))=P_\phi (A)+P_\phi (B)$.

Thus $P_\phi$ is a finitely additive probability, as condition (i) requires. Conditions (ii)–(iv) are special cases of condition (v) (though (ii) also immediately follows from the strict increasingness of $\phi$).

So let’s prove (v). Suppose that $a_1,\dots ,a_n,b_1,\dots ,b_m$ are real and that events $A_1,\dots ,A_n,B_1,\dots ,B_m$ are such that

$$\begin{aligned} \sum _i a_iP(A_i\mid C) < \sum _i b_i P(B_i\mid C). \end{aligned}$$

(1)

We will now show that

$$\begin{aligned} \sum _i a_iP_\phi (A_i\mid C) < \sum _i b_i P_\phi (B_i\mid C). \end{aligned}$$

(2)

We don’t need to separately show the other direction of the biconditional in (iv), because we can write $P = (P_\phi )_{\phi ^{-1}}$ and apply the first direction with $P_\phi$ in place of P and $\phi ^{-1}$ in place of $\phi$, as $\phi$ is a bijection and its inverse satisfies the conditions of the lemma as well.

Let $D_i = A_i\cap C$ and $E_i = B_i\cap C$. Then if we multiply both sides of (1) by P(C) and both sides of (2) by $P_\phi (C)$, and apply the ratio definition of conditional probability, we see that the two inequalities are respectively equivalent to:

$$\begin{aligned} \sum _i a_iP(D_i) < \sum _i b_i P(E_i). \end{aligned}$$

(3)

and

$$\begin{aligned} \sum _i a_i P_\phi (D_i) < \sum _i b_i P_\phi (E_i). \end{aligned}$$

(4)

But since $P_\phi (U)=\phi (P_\phi (U))$ and by the ${\mathbb {R}}$-linearity of $\phi$, inequality (4) is equivalent to:

$$\begin{aligned} \phi \left( \sum _i a_iP(D_i)\right) < \phi \left( \sum _i b_i P_\phi (E_i)\right) . \end{aligned}$$

(5)

And by the strict increasingness of $\phi$, this is indeed equivalent to (3), thereby completing the proof of (v). $\square$

Proof of Theorem 1

By our lemmas, all that remains is to prove condition (vi). Suppose first that $P(C)\approx 0$. Then $P_\alpha (C) = \alpha P(C)$ by definition of $P_\alpha$. Moreover, $P(A\cap C)$ will also be infinitesimally close to zero, so $P_\alpha (A\cap C)=\alpha P(A\cap C)$. Thus

$$\begin{aligned} P_\alpha (A \mid C) = \frac{P_\alpha (A\cap C)}{P_\alpha (C)} = \frac{P(A\cap C)}{P(C)} = P(A\mid C), \end{aligned}$$

as desired.

Now suppose that $P(C)\not \approx 0$. Let $c_1 = {\text {Std}}\,P(C)$. This is a non-zero real number. Observe that if x is finite and $a \approx b$ with $a\not \approx 0$, then $x/a \approx x/b$. (For $\frac{x}{a}-\frac{x}{b}=\frac{(b-a)x}{ab}$, which is infinitesimally close to zero since $b-a\approx 0$ while $ab\not \approx 0$.) Now, $P(C) \approx P_\alpha (C) \approx c_1$. Thus,

$$\begin{aligned} P_\alpha (A\mid C) = \frac{P_\alpha (A\cap C)}{P(C)} \approx \frac{P_\alpha (A\cap C)}{c_1}. \end{aligned}$$

But if $y>0$ is real and $a\approx b$, then $a/y \approx b/y$, so using the fact that $P(A\cap C)\approx P_\alpha (A\cap C)$:

$$\begin{aligned} \frac{P_\alpha (A\cap C)}{c_1} \approx \frac{P(A\cap C)}{c_1}. \end{aligned}$$

Hence

$$\begin{aligned} P_\alpha (A\mid C) \approx \frac{P(A\cap C)}{c_1}. \end{aligned}$$

(6)

The right hand side of (6) does not depend on $\alpha$, so all the values of the left hand side as we vary $\alpha >0$ must be infinitesimally close. In particular, $P_1(A\mid C)\approx P_\alpha (A\mid C)$, and since $P_1=P$, we have (vi) as desired. $\square$

Proof of Proposition 1

As a convenient and usual abuse of notation, we will consider “z” to be a constant denoting z in our preferred interpretation in the model ${\mathbb {R}}^*$, and other similar abuses will be perpetrated. Start by noting that z cannot be defined over ${\mathbb {R}}$ in the first-order language of fields, i.e., there is no formula $F(a_1,\dots ,a_n,x)$ in the first order language of fields with $a_1,\dots ,a_n$ real-valued constants such that $F(a_1,\dots ,a_n,z)$ holds in ${\mathbb {R}}^*$ and $\forall x(F(a_1,\dots ,a_n,x)\rightarrow x=z)$. For if there were such a formula, then $\exists ! x F(a_1,\dots ,a_n,x)$ would hold in the model ${\mathbb {R}}^*$. But by the transfer principle for hyperreals (e.g., Keisler 2007, p. 9), likewise $\exists ! x F(a_1,\dots ,a_n,x)$ would hold in the model ${\mathbb {R}}$. Thus, there would be a real number u such that $F(a_1,\dots ,a_n,u)$, and this sentence would also hold in ${\mathbb {R}}^*$ by the transfer principle, which would imply that $u=z$, contradicting the fact that z is not real.

Now let $\Sigma _0$ be the set of all formulas F(x) in the first-order language of fields with constants from ${\mathbb {R}}$ and at most one free variable x such that F(z) is true in ${\mathbb {R}}^*$ and let $\Sigma _1$ be $\Sigma _0$ with the addition of the formula $z\ne x$. Observe that $\Vert \Sigma \Vert = \Vert {\mathbb {R}}\Vert$. Moreover, any finite subset of $\Sigma$ is satisfiable over ${\mathbb {R}}^*$, i.e., if $F_0(x),F_1(x),\dots ,F_n(x)$ are in $\Sigma$, then there is a u in ${\mathbb {R}}^*$ that satisfies all the formulas. To see this, without loss of generality, assume that $F_0(x)$ is $x\ne z$ and that $F_1(x),\dots ,F_n(x)$ are in $\Sigma _0$. Let F(x) be $F_1(x)\wedge \dots \wedge F_n(x)$. Then z satisfies F(x) in ${\mathbb {R}}^*$, and so by the observations of the previous paragraph it is false that $\exists ! x F(x)$ holds in ${\mathbb {R}}^*$. Thus in ${\mathbb {R}}^*$ there must be another element besides z that satisfies F(x), say $z'$. This other element then satisfies $F_0(x)$, and so it satisfies all of $F_0(x),F_1(x),\dots ,F_n(x)$. Hence every finite subset of $\Sigma$ is satisfiable over ${\mathbb {R}}^*$.

Since ${\mathbb {R}}^*$ is $\kappa$-saturated for $\kappa > \Vert {\mathbb {R}}\Vert = \Vert \Sigma \Vert$, it follows that $\Sigma$ is satisfiable over ${\mathbb {R}}^*$. Suppose it is satisfied by $z'$. Then $z' \ne z$, since $x\ne z$ is one of the formulas in $\Sigma$. Moreover, $z\ne u$ for every real u, so the formula $z\ne u$ is in $\Sigma _0$, so $z' \ne u$ for every real u, and hence $z' \notin {\mathbb {R}}$. Now define $\phi (x)=x$ for $x\in {\mathbb {R}}$ and $\phi (z)=z'$. Because $z'$ satisfies every formula with one free variable that z satisfies, it follows that $\phi$ is a partial elementary embedding. But a partial elementary embedding in a $\kappa$-saturated model of cardinality $\kappa$ extends to an automorphism of the model (Sacks 2010, pp. 72–73), which completes the proof. $\square$

Proof of Theorem 2

By Proposition 1 and Lemma 4, we just need to prove the analogue of (vi), namely that if $P(C)>0$, then $P(A\mid C)\approx P_\phi (A \mid C)$. To see this, observe that because $\phi$ is a field isomorphism:

$$ \begin{aligned} \begin{aligned} P_\phi (A\mid C)&= \frac{P_\phi (A\, \& \,C)}{P_\phi (C)} \\&= \frac{\phi (P(A\, \& \,C))}{\phi (P(C))} \\&= \phi \left( \frac{P(A\, \& \,C)}{P(C)}\right) = \phi (P(A\mid C)). \end{aligned} \end{aligned}$$

But $\phi (P(A\mid C))\approx P(A\mid C)$ which follows from the fact that $\phi (x)\approx x$ for all finite $x\in {\mathbb {R}}^*$ (and $P(A\mid C)$ is finite as it is between 0 and 1). For suppose x is finite so we can write $x=a+b$ for a real a and an infinitesimal b. Since $\phi$ fixes the reals, $\phi (x)=\phi (a)+\phi (b)=a+\phi (b)$. If we can show that $\phi (b)$ is infinitesimal, it will follow that $\phi (x)\approx a\approx x$. To see that $\phi (b)$ is infinitesimal, fix any real $c>0$. Then $-c< b < c$. Hence $\phi (-c)< \phi (b) < \phi (c)$ as $\phi$ is a field isomorphism of ${\mathbb {R}}^*$ to itself and hence preserves order (for $\alpha \le \beta$ in a real-line if and only if there is a $\gamma$ such that $\beta =\alpha +\gamma ^2$, and this algebraic characterization of $\le$ is preserved by field isomorphism). But $\phi (-c)=-c$ and $\phi (c)=c$ as $\phi$ fixes the reals. Thus $-c<\phi (b)<c$, and since this holds for all real $c>0$, $\phi (b)$ is infinitesimal. $\square$