Probability Theory is an Extension of Logic

Main ideas and concepts

• Core thesis: Probability theory can be viewed as an extension of logic (“common sense reduced to calculation,” attributed to Laplace).
• Goal of the talk: Derive the foundational rules of probability (notably the sum rule and product rule) from:
  • different “definitions”/interpretations of probability,
  • classical logic (truth tables, normal forms),
  • and lattice theory (posets, join/meet, distributive lattices).

Two major interpretations of probability today

• Frequentist: probability = limit of relative frequency of an event over repeated trials.
• Bayesian / plausibility approach: probability = degree of plausibility of a proposition given the current state of knowledge/ignorance.

Frequentist critique (as presented)

Frequentist reasoning is said to struggle with:

• assigning probabilities to single/unique events (e.g., “tomorrow it is sunny,” or a specific person winning),
• relying on concepts that are circular (e.g., the law of large numbers depends on definitions of probability),
• requiring a notion of randomness that is hard to define,
• typically reasoning with probability of data given hypothesis (likelihood / maximum likelihood), not probability of hypotheses given data,
• producing point estimates rather than a probability distribution over hypotheses/parameters,
• leading to awkward methodology such as reasoning in imaginary universes and interpreting p-values/confidence intervals (with an “inversion problem”).

Motivating shift to Bayesian reasoning

Bayesian reasoning emphasizes induction rather than deduction, illustrated via a crime/police thought experiment (below).

Logic → probability bridge

• Classical logic operations (AND/OR/NOT) relate to lattice operations (meet/join).
• Logical implication induces an ordering, and probability arises as a valuation of propositions in a lattice.

---

Methodology / derivation structure (instruction-like steps)

1) Start from definitions/interpretations

Treat probability as either:

• Frequentist (frequency limit), or
• Bayesian (plausibility under a context/knowledge state).

Use the coin-toss example:

• Frequentist: 50/50 because long-run frequencies approach 1/2.
• Bayesian: 50/50 because symmetry/ignorance implies equal plausibility.

2) Argue frequentist limitations

Frequentism is claimed to struggle with:

• probabilities for unique events,
• avoiding circularity,
• defining randomness,
• interpreting p-values/confidence intervals as probabilities of hypotheses.

3) Introduce induction vs deduction (thought experiment)

A police officer hears an alarm. A masked man exits a jewelry store with jewels. An arrest follows. The bag contains jewels.

Questions:

• Is it a crime?
• Was the conclusion reached by deduction or induction?

Presented conclusion: It’s induction, since alternative explanations are possible (even if implausible).

4) Establish probability’s “fundamental rules” conceptually

In Bayesian methodology:

• probability is a plausibility quantifier between 0 (impossible) and 1 (certain).

Identify foundational rules to derive:

• Sum rule (for combining probabilities of alternatives / complements)
• Product rule (for combining probabilities of joint statements)

5) Build foundations using classical logic and truth tables

• Represent logical propositions using truth tables with operators:
  • AND / OR / NOT
• Use reduction to Disjunctive Normal Form (DNF):
  • break a complex statement into smallest “atoms,”
  • then combine via OR of AND-components.
• Implication ordering yields a partially ordered set structure.

6) Translate logic into lattice theory

• Define a poset (partially ordered set) satisfying:
  • reflexivity, antisymmetry, transitivity.
• Define bounds:
  • upper bound / lower bound
  • join = least upper bound
  • meet = greatest lower bound
• Connect to logic:
  • proposition OR ↔ lattice join
  • proposition AND ↔ lattice meet
• Use distributive lattices as sufficient; Boolean lattices/complements are not required for the derivation.

7) Define probability as a generalized “inverse zeta function” / valuation

Start from a zeta function idea (indicator-like logic about order in the lattice), generalized to a function Z (the “probability valuation”) such that:

• Z(X above T) = 1 (maximally supported),
• Z(meets at bottom / no implication) = 0,
• intermediate values represent partial implication strength.

Probability depends on knowledge:

• include a context term C or T in conditional expressions.

8) Derive sum and product rules from “symmetries”

• Symmetry/order preservation: valuation respects implication ordering under joins.
• Associativity for combining disjoint elements:
  • leads to an operator corresponding to addition for disjoint events.
• Inclusion–exclusion via overlap:
  • subtract intersection contributions when sets/events overlap,
  • yielding the general sum rule.

9) Use product structure for independent systems

• Build a product lattice (cross-product structure) for combining systems.
• Treat “independence” as epistemological:
  • learning about one tells you nothing about the other (not necessarily “physically independent”).
• Derive product rule for joint probability of independent components.

10) Extend to conditional probability via chaining intervals

• Introduce b-valuation: probability of predicate X given context T.
• For propositions not directly comparable:
  • interpret relationships through chains of implication intervals.
• Implication chaining associativity implies a functional equation:
  • its solution yields multiplication corresponding to the product rule along chains.

11) Obtain Bayes’ rule from alternative decompositions

Use equivalences of joint/conditional forms:

• express P(A, B | C) in different ways,
• rearrange to obtain Bayes’ theorem.

12) Normalization from evaluating context with itself

Setting the context as the event itself implies normalization:

• probabilities sum to 1.

13) Apply Bayesian inference (parameter updating)

Standard form:

[ \text{Posterior} = \frac{\text{Prior} \times \text{Likelihood}}{\text{Evidence}} ]

Emphasis:

• prior, likelihood, and posterior all come from the same probability calculus.
• frequentists cannot reproduce this fully because they don’t treat parameters as probabilistic unknowns.

14) Handle nuisance parameters via marginalization

If there is a nuisance parameter A and a target ω:

• “average out” A using sum/product rules.

• Discrete case: sum over A values.

• Continuous case: integrate over A.

15) Model comparison and “Occam’s razor” effect

Compare models using:

• posterior odds / Bayes factors

Odds depend on:

• priors of models,
• integrated likelihoods (“global likelihoods”).

Argument:

• more complex models get penalized because their prior mass is diluted across a larger parameter space unless they improve likelihood enough.

16) Warnings / paradoxes and technical caution

• Deviating from the probability calculus can yield incorrect/paradoxical results.
• Special caution with infinite sets:
  • ill-posed limits can create paradoxes unrelated to probability properly defined.

Named examples:

• Bertrand’s paradox
• Borel–Cantelli paradox

Suggested remedy:

• treat infinite cases as well-behaved limits of finite sets.

---

Key lessons conveyed

• Bayesian probability is presented as logical consistency under partial information, derived from lattice/implication structure.
• Frequentist methods are critiqued for:
  • lacking direct probabilities on hypotheses/parameters,
  • interpretational issues around p-values and confidence intervals.
• Core Bayesian tools follow from the probability rules:
  • Bayes’ theorem (updating),
  • marginalization (handling nuisance parameters),
  • Bayes factors (Occam-razor penalization and model comparison).
• Proper handling of infinite structures is crucial to avoid misleading paradoxes.

---

Speakers / sources mentioned

• Pierre-Simon Laplace (1819 quote attributed)
• Sir Harold Jeffreys (criticized p-values/confidence interval interpretations)
• Edwin Thompson Jaynes (advocate of Bayesian logic/derivation perspective; author of Probability: The Logic of Science)
• Edwin Thompson James (mentioned as “legendary physicist and statistician” advocating the alternative—apparently referring to Edwin Thompson Jaynes)
• Kevin Knuth and John Skilling (authors of a referenced 2012 paper used in the derivation)
• Cox (mentioned for deriving probability rules under additional assumptions)
• de Finetti (mentioned in the context of subjective probability approach)
• Sherlock Holmes (used as an analogy for deduction vs inductive reasoning)
• Bertrand (via Bertrand’s paradox)
• Borel–Cantelli (via Borel–Cantelli related paradox)