Why agent foundations?
Prerequisites
Probability theory refresher
Formal logic refresher
Computability theory refresher
Information theory, causality, and statistical mechanics refresher
Coherence and consequentialism
Reading: Coherent decisions imply consistent utilities
Reading: Complete class theorem — consequentialist foundations
Exercise: The complete class theorem
Hint: Part 3(a)(i)
Hint: Part 3(a)(ii)
Embedded agency
Reading: Embedded agency
What is the dualistic assumption and why does it fail?
Logical uncertainty and self-reference
Reading: Logical induction
Reading: Vingean reflection
Reading: Lob’s theorem and the tiling agents problem
Exercise: Godel’s second incompleteness theorem
Hint: Part 1(a)
Exercise: Lob’s theorem
Hint: Part 2(a)
Decision theory
Reading: Functional decision theory
Reading: Updateless decision theory
Reading: Reflective consistency
Self-reference and multi-agent reasoning
Reading: Reflective oracles
Reading: Infra-Bayesianism
What is Knightian uncertainty?
Descriptive agent foundations
Reading: The ground of optimization
Reading: Optimization at a distance
Reading: Algorithmic thermodynamics and three types of optimization
Connection to Maxwell's demon and the second law
Reading: General purpose search
Reading: Selection theorems
Exercise: The do-divergence theorem
Hint
Exercise: Channel additivity
Hint: Part 5(a)
Further reading