Statistical Inference/Probability Theory - Revision history

Root at 22:25, 3 December 2018

2018-12-03T22:25:51Z

← Older revision		Revision as of 22:25, 3 December 2018
Line 17:		Line 17:
	We first need to define formally the following two relationships, which allow us to order and equate sets:		We first need to define formally the following two relationships, which allow us to order and equate sets:

	'''Containment:''' <math>A \subset B \Leftrightarrow x \in A \Rightarrow x \in B</math>		'''Containment:''' <math>A \subset B \Leftrightarrow x \in A \Rightarrow x \in B</math>

	'''Equality:''' <math>A = B \Leftrightarrow A \subset B \land B \subset A</math>		'''Equality:''' <math>A = B \Leftrightarrow A \subset B \land B \subset A</math>
Line 39:		Line 39:
	== Event operations ==		== Event operations ==

	The elementary set operations can be combined: for any three events, A, B, and C, defined on a sample space S, the following relationships hold [Theorem 1.1.4].		The elementary set operations can be combined: for any three events, A, B, and C, defined on a sample space S, the following relationships hold [Theorem 1.1.4].

	=== Commutativity ===		=== Commutativity ===
Line 75:		Line 75:
	== Axiomatic foundations ==		== Axiomatic foundations ==

	For each event A in the sample space S we want to associate with A a number between zero and one that will be called the probability of A, denoted by P(A).		For each event A in the sample space S we want to associate with A a number between zero and one that will be called the probability of A, denoted by P(A).

	=== Sigma Algebra ===		=== Sigma Algebra ===
Line 104:		Line 104:
	=== Defining Probability Functions ===		=== Defining Probability Functions ===

	Let <~~amsmath~~>S = \{s_1, \dots, s_n\}</~~amsmath~~> be a finite set. Let <~~amsmath~~>\cal{B}</~~amsmath~~> be any sigma algebra of subsets of S. Let <~~amsmath~~>p_1, \dots, p_n</~~amsmath~~> be nonnegative numbers that sum to 1. For any <~~amsmath~~>A \in \cal{B}</~~amsmath~~>, define <~~amsmath~~>P(A) = \Sigma_{\{i:s_i\in A\}}p_i</~~amsmath~~> (The sum over an empty set is defined to be 0.) Then P is a probability function on <~~amsmath~~>\cal{B}</~~amsmath~~>. This remains true if <~~amsmath~~>S = \{s_1, s_2, \dots\}</~~amsmath~~> is a countable set [Theorem 1.2.6].		Let <math>S = \{s_1, \dots, s_n\}</math> be a finite set. Let <math>\cal{B}</math> be any sigma algebra of subsets of S. Let <math>p_1, \dots, p_n</math> be nonnegative numbers that sum to 1. For any <math>A \in \cal{B}</math>, define <math>P(A) = \Sigma_{\{i:s_i\in A\}}p_i</math> (The sum over an empty set is defined to be 0.) Then P is a probability function on <math>\cal{B}</math>. This remains true if <math>S = \{s_1, s_2, \dots\}</math> is a countable set [Theorem 1.2.6].

	The physical reality of the experiment might dictate the probability assígnment.		The physical reality of the experiment might dictate the probability assígnment.
Line 112:		Line 112:
	=== Properties of the probability function applied to a single event ===		=== Properties of the probability function applied to a single event ===

	If P is a. probability function and A is any set in <~~amsmath~~>\cal{B}</~~amsmath~~>, then [Theorem 1.2.8]		If P is a. probability function and A is any set in <math>\cal{B}</math>, then [Theorem 1.2.8]
	* (a) <~~amsmath~~>P(\emptyset) = 0</~~amsmath~~>, where <~~amsmath~~>\emptyset</~~amsmath~~> is the empty set;		* (a) <math>P(\emptyset) = 0</math>, where <math>\emptyset</math> is the empty set;
	* (b) <~~amsmath~~>P(A) \le 1</~~amsmath~~>;		* (b) <math>P(A) \le 1</math>;
	* (c) <~~amsmath~~>P(\overline{A}) = 1 - P(A)</~~amsmath~~>.		* (c) <math>P(\overline{A}) = 1 - P(A)</math>.

	=== Properties of the probability function applied to any set pairs ===		=== Properties of the probability function applied to any set pairs ===

	If P is a probability function and A and B are any sets in <~~amsmath~~>\cal{B}</~~amsmath~~>, then [Theorem 1.2.9]		If P is a probability function and A and B are any sets in <math>\cal{B}</math>, then [Theorem 1.2.9]
	* (a) <~~amsmath~~>P(B \cap \overline{A}) = P(B) - P(A \cap B)</~~amsmath~~>;		* (a) <math>P(B \cap \overline{A}) = P(B) - P(A \cap B)</math>;
	* (b) <~~amsmath~~>P(A \cup B) = P(A) + P(B) - P(A \cap B)</~~amsmath~~>;		* (b) <math>P(A \cup B) = P(A) + P(B) - P(A \cap B)</math>;
	* (c) If <~~amsmath~~>A \subset B</~~amsmath~~>, then <~~amsmath~~>P(A) \le P(B)</~~amsmath~~>.		* (c) If <math>A \subset B</math>, then <math>P(A) \le P(B)</math>.

	=== Bonferroni's Inequality ===		=== Bonferroni's Inequality ===

	Formula (b) of Theorem 1.2.9 gives a useful inequality for the probability of an intersection. Since <~~amsmath~~>P(A \cup B) \le 1</~~amsmath~~>, we have, after some rearranging, <~~amsmath~~>P(A \cap B) \ge P(A) + P(B) - 1</~~amsmath~~>.		Formula (b) of Theorem 1.2.9 gives a useful inequality for the probability of an intersection. Since <math>P(A \cup B) \le 1</math>, we have, after some rearranging, <math>P(A \cap B) \ge P(A) + P(B) - 1</math>.

	This inequality is a special case of what is known as Bonferroni's Inequality. Bonferroni's Inequality allows us to bound the probability of a simultaneous event (the intersectíon) in terms of the probabilities of the individual events.		This inequality is a special case of what is known as Bonferroni's Inequality. Bonferroni's Inequality allows us to bound the probability of a simultaneous event (the intersectíon) in terms of the probabilities of the individual events.
Line 133:		Line 133:

	If P is a probability function, then [Theorem 1.2.11]		If P is a probability function, then [Theorem 1.2.11]
	* (a) <~~amsmath~~>P(A) = \Sigma^\infty_{i=1} P(A \cap C_i)</~~amsmath~~>, for any partition <~~amsmath~~>C_1, C_2, \dots</~~amsmath~~>;		* (a) <math>P(A) = \Sigma^\infty_{i=1} P(A \cap C_i)</math>, for any partition <math>C_1, C_2, \dots</math>;
	* (b) <~~amsmath~~>P(\cup^\infty_{i=1} A_i) \le \Sigma^\infty_{i=1} P(A_i)</~~amsmath~~>, for any sets <~~amsmath~~>A_1, A_2, \dots</~~amsmath~~> (Boole's Inequality)		* (b) <math>P(\cup^\infty_{i=1} A_i) \le \Sigma^\infty_{i=1} P(A_i)</math>, for any sets <math>A_1, A_2, \dots</math> (Boole's Inequality)

	There is a similarity between Boole's Inequality and Bonferroni's Inequality. In fact, they are essentially the same thing. We could have used Boole's Inequality to derive that <~~amsmath~~>P(A \cap B) \ge P(A) + P(B) - 1</~~amsmath~~> (the special case of Bonferroni's Inequality, presented above). If we apply Boole*s Inequality to <~~amsmath~~>\overline{A}</~~amsmath~~>, we have <~~amsmath~~>P(\cup^n_{i=1} \overline{A_i}) \le \Sigma^n_{i=1} P(\overline{A_i})</~~amsmath~~> and using the facts that <~~amsmath~~>\cup\overline{A_i} = \overline{\cap A_i}</~~amsmath~~> and <~~amsmath~~>P(\overline{A_i}) = 1 - P(A_i)</~~amsmath~~>, we obtain <~~amsmath~~>1 - P(\cap^n_{i=1} A_i) \le n - \Sigma^n_{i=1} P(A_i)</~~amsmath~~>		There is a similarity between Boole's Inequality and Bonferroni's Inequality. In fact, they are essentially the same thing. We could have used Boole's Inequality to derive that <math>P(A \cap B) \ge P(A) + P(B) - 1</math> (the special case of Bonferroni's Inequality, presented above). If we apply Boole*s Inequality to <math>\overline{A}</math>, we have <math>P(\cup^n_{i=1} \overline{A_i}) \le \Sigma^n_{i=1} P(\overline{A_i})</math> and using the facts that <math>\cup\overline{A_i} = \overline{\cap A_i}</math> and <math>P(\overline{A_i}) = 1 - P(A_i)</math>, we obtain <math>1 - P(\cap^n_{i=1} A_i) \le n - \Sigma^n_{i=1} P(A_i)</math>

	This becomes, on rearranging terms, <~~amsmath~~>P(\cap^n_{i=1} A_i) \ge \Sigma^n_{i=1} P(A_i) - (n -1)</~~amsmath~~>, which is a more general version of the Bonferroni Inequality presented before.		This becomes, on rearranging terms, <math>P(\cap^n_{i=1} A_i) \ge \Sigma^n_{i=1} P(A_i) - (n -1)</math>, which is a more general version of the Bonferroni Inequality presented before.

	== Counting ==		== Counting ==
Line 148:		Line 148:
	=== Fundamental Theorem of Counting ===		=== Fundamental Theorem of Counting ===

	If a job consists of <~~amsmath~~>k</~~amsmath~~> separate tasks, the <~~amsmath~~>i</~~amsmath~~>-th of which can be done in <~~amsmath~~>n_i</~~amsmath~~> ways, <~~amsmath~~>i = 1, \dots, k</~~amsmath~~>, then the entire job can be done in <~~amsmath~~>n_1 \times n_2 \times \dots \times n_k</~~amsmath~~> ways.		If a job consists of <math>k</math> separate tasks, the <math>i</math>-th of which can be done in <math>n_i</math> ways, <math>i = 1, \dots, k</math>, then the entire job can be done in <math>n_1 \times n_2 \times \dots \times n_k</math> ways.

	=== Possible Methods of Counting ===		=== Possible Methods of Counting ===
Line 156:		Line 156:
	=== Factorial ===		=== Factorial ===

	For a positive integer n, n! (read n factorial) is the product of all of the positive integers less than or equal to n [Definition 1.2.16]. That is, <~~amsmath~~>n! = n \times (n-1) \times (n-2) \times \dots \times 3 \times 2 \times 1</~~amsmath~~>. Furthermore, we define 0! = 1.		For a positive integer n, n! (read n factorial) is the product of all of the positive integers less than or equal to n [Definition 1.2.16]. That is, <math>n! = n \times (n-1) \times (n-2) \times \dots \times 3 \times 2 \times 1</math>. Furthermore, we define 0! = 1.

	== Enumerating Outcomes ==		== Enumerating Outcomes ==

Root: /* Probability Function */

2018-12-03T22:23:47Z

Probability Function

← Older revision		Revision as of 22:23, 3 December 2018
Line 92:		Line 92:
	=== Probability Function ===		=== Probability Function ===

	Given a sample space S and an associated sigma algebra <~~amsmath~~>\cal{B}</~~amsmath~~>, a probability function [Definition 1.2.4] is a function P with domain <~~amsmath~~>\cal{B}</~~amsmath~~> that satisfies		Given a sample space S and an associated sigma algebra <math>\cal{B}</math>, a probability function [Definition 1.2.4] is a function P with domain <math>\cal{B}</math> that satisfies

	* 1. <~~amsmath~~>P(A) \geq 0, \forall_{A \in \cal{B}}</~~amsmath~~>.		* 1. <math>P(A) \geq 0, \forall_{A \in \cal{B}}</math>.
	* 2. <~~amsmath~~>P(S) = 1</~~amsmath~~>.		* 2. <math>P(S) = 1</math>.
	* 3. If <~~amsmath~~>A_1, A_2, \dots \in \cal{B}</~~amsmath~~> are pairwise disjoint, then <~~amsmath~~>P(\cup^\infty_{i=1} A_i)=\Sigma^\infty_{i=1} P(A_i)</~~amsmath~~>. [Axiom of Countable Additivity]		* 3. If <math>A_1, A_2, \dots \in \cal{B}</math> are pairwise disjoint, then <math>P(\cup^\infty_{i=1} A_i)=\Sigma^\infty_{i=1} P(A_i)</math>. [Axiom of Countable Additivity]

	These three properties are usually referred to as the Axioms of Probability (or the Kolmogorov Axioms, after A. Kolmogorov, one of the fathers of probability theory). Any function P that satisfies the Axioms of Probability is called a probability function. The axiomatic definition makes no attempt to tell what particular function P to choose; it merely requires P to satisfy the axioms. For any sample space many different probability functions can be defined. Which one(s) reflects what is likely to be observed in a particular experiment is still to be discussed.		These three properties are usually referred to as the Axioms of Probability (or the Kolmogorov Axioms, after A. Kolmogorov, one of the fathers of probability theory). Any function P that satisfies the Axioms of Probability is called a probability function. The axiomatic definition makes no attempt to tell what particular function P to choose; it merely requires P to satisfy the axioms. For any sample space many different probability functions can be defined. Which one(s) reflects what is likely to be observed in a particular experiment is still to be discussed.

Root: /* Sigma Algebra */

2018-12-03T22:23:00Z

Sigma Algebra

← Older revision		Revision as of 22:23, 3 December 2018
Line 85:		Line 85:
	* (c) if <math>A_1, A_2, \dots \in \cal{B}</math>, then <math>\cup^\infty_{i=1} A_i \in \cal{B}</math> (<math>\cal{B}</math> is closed under countable unions).		* (c) if <math>A_1, A_2, \dots \in \cal{B}</math>, then <math>\cup^\infty_{i=1} A_i \in \cal{B}</math> (<math>\cal{B}</math> is closed under countable unions).

	The empty set <math>\emptyset</math> is a subset of any set. Thus, <math>\emptyset \subset S</math>. Property (a) states that this subset is always in a sigma algebra. Since <math>S = \overline{\emptyset}</math>, properties (a) and (b) imply that S is always in <math>\cal{B}</~~amsmath~~> also. In addition, from DeMorganªs Laws it follows that <math>\cal{B}</math> is closed under countable intersections. If <math>A_1, A_2, \dots \in \cal{B}</math>, then <math>\overline{A_1}, \overline{A_2}, \dots \in \cal{B}</math> by property (b), and therefore <math>\cup^\infty_{i=1} \overline{A_i} \in \cal{B}</math> . However, using DeMorgan's Law, we		The empty set <math>\emptyset</math> is a subset of any set. Thus, <math>\emptyset \subset S</math>. Property (a) states that this subset is always in a sigma algebra. Since <math>S = \overline{\emptyset}</math>, properties (a) and (b) imply that S is always in <math>\cal{B}</math> also. In addition, from DeMorganªs Laws it follows that <math>\cal{B}</math> is closed under countable intersections. If <math>A_1, A_2, \dots \in \cal{B}</math>, then <math>\overline{A_1}, \overline{A_2}, \dots \in \cal{B}</math> by property (b), and therefore <math>\cup^\infty_{i=1} \overline{A_i} \in \cal{B}</math> . However, using DeMorgan's Law, we
	have <math>\overline{\cup^\infty_{i=1} \overline{A_i}} = \cap^\infty_{i=1} A_i</math>. Thus, again by property (b), <math>\cap^\infty_{i=1} A_i \in \cal{B}</math>.		have <math>\overline{\cup^\infty_{i=1} \overline{A_i}} = \cap^\infty_{i=1} A_i</math>. Thus, again by property (b), <math>\cap^\infty_{i=1} A_i \in \cal{B}</math>.

Root: /* Elementary operations */

2018-12-03T22:22:17Z

Elementary operations

← Older revision		Revision as of 22:22, 3 December 2018
Line 27:		Line 27:
	'''Union:''' The union of A and B, written <math>A \cup B</math>, is the set of elements that belong to either A or B or both:		'''Union:''' The union of A and B, written <math>A \cup B</math>, is the set of elements that belong to either A or B or both:

	<math>A \cup B = \{ x : x \in A \lor x \in B \}</math>		<math>A \cup B = \{ x : x \in A \lor x \in B \}</math>

	'''Intersection:''' The intersection of A and B, written <math>A \cap B</math>, is the set of elements that belong to both A and B:		'''Intersection:''' The intersection of A and B, written <math>A \cap B</math>, is the set of elements that belong to both A and B:

	<math>A \cap B = \{ x : x \in A \land x \in B \}</math>		<math>A \cap B = \{ x : x \in A \land x \in B \}</math>

	'''Complementation:''' The complement of A, written <math>\overline{A}</math>, is the set of all elements that are not in A:		'''Complementation:''' The complement of A, written <math>\overline{A}</math>, is the set of all elements that are not in A:

	<math>\overline{A} = \{ x : x \notin A \}</math>		<math>\overline{A} = \{ x : x \notin A \}</math>

	== Event operations ==		== Event operations ==

Root at 22:21, 3 December 2018

2018-12-03T22:21:54Z

Show changes

Root: /* Factorial */

2018-08-07T15:40:28Z

Factorial

← Older revision		Revision as of 15:40, 7 August 2018
Line 160:		Line 160:
	=== Factorial ===		=== Factorial ===

	For a positive integer n, n! (read n factorial) is the product of all of the positive integers less than or equal to n [Definition 1.2.16]. That is, <amsmath>nl = n \times (n-1) \times (n-2) \times \dots \times 3 \times 2 \times 1</amsmath>. Furthermore, we define 0! = 1.		For a positive integer n, n! (read n factorial) is the product of all of the positive integers less than or equal to n [Definition 1.2.16]. That is, <amsmath>n! = n \times (n-1) \times (n-2) \times \dots \times 3 \times 2 \times 1</amsmath>. Furthermore, we define 0! = 1.

	== Enumerating Outcomes ==		== Enumerating Outcomes ==

Root: /* Counting */

2018-08-05T09:37:51Z

Counting

← Older revision		Revision as of 09:37, 5 August 2018
Line 146:		Line 146:
	== Counting ==		== Counting ==

	Most often, methods of counting are used in order to construct probability assignments on ~~fmite~~ sample spaces, although they can be used to answer other questions also.		Most often, methods of counting are used in order to construct probability assignments on finite sample spaces, although they can be used to answer other questions also.

	Counting problems, in general, sound complicated, and often we must do our counting subject to many restrictions. The way to solve such problems is to break them down into a series of simple tasks that are easy to count, and employ known rules of combining tasks. The following theorem is a first step in such a process and is sometimes known as the Fundamental Theorem of Counting.		Counting problems, in general, sound complicated, and often we must do our counting subject to many restrictions. The way to solve such problems is to break them down into a series of simple tasks that are easy to count, and employ known rules of combining tasks. The following theorem is a first step in such a process and is sometimes known as the Fundamental Theorem of Counting.

Root: /* Counting */

2018-08-04T11:46:17Z

Counting

← Older revision		Revision as of 11:46, 4 August 2018
Line 153:		Line 153:

	If a job consists of <amsmath>k</amsmath> separate tasks, the <amsmath>i</amsmath>-th of which can be done in <amsmath>n_i</amsmath> ways, <amsmath>i = 1, \dots, k</amsmath>, then the entire job can be done in <amsmath>n_1 \times n_2 \times \dots \times n_k</amsmath> ways.		If a job consists of <amsmath>k</amsmath> separate tasks, the <amsmath>i</amsmath>-th of which can be done in <amsmath>n_i</amsmath> ways, <amsmath>i = 1, \dots, k</amsmath>, then the entire job can be done in <amsmath>n_1 \times n_2 \times \dots \times n_k</amsmath> ways.

			=== Possible Methods of Counting ===

			Counting can be made with without replacement. Also, the order in which the task is performed may matter. Four types counting scenarios are thus possible: ordered or unordered, both of which may be done with or without replacement.

			=== Factorial ===

			For a positive integer n, n! (read n factorial) is the product of all of the positive integers less than or equal to n [Definition 1.2.16]. That is, <amsmath>nl = n \times (n-1) \times (n-2) \times \dots \times 3 \times 2 \times 1</amsmath>. Furthermore, we define 0! = 1.

	== Enumerating Outcomes ==		== Enumerating Outcomes ==

Root: /* Counting */

2018-08-04T11:01:43Z

Counting

← Older revision		Revision as of 11:01, 4 August 2018
Line 147:		Line 147:

	Most often, methods of counting are used in order to construct probability assignments on fmite sample spaces, although they can be used to answer other questions also.		Most often, methods of counting are used in order to construct probability assignments on fmite sample spaces, although they can be used to answer other questions also.

			Counting problems, in general, sound complicated, and often we must do our counting subject to many restrictions. The way to solve such problems is to break them down into a series of simple tasks that are easy to count, and employ known rules of combining tasks. The following theorem is a first step in such a process and is sometimes known as the Fundamental Theorem of Counting.

			=== Fundamental Theorem of Counting ===

			If a job consists of <amsmath>k</amsmath> separate tasks, the <amsmath>i</amsmath>-th of which can be done in <amsmath>n_i</amsmath> ways, <amsmath>i = 1, \dots, k</amsmath>, then the entire job can be done in <amsmath>n_1 \times n_2 \times \dots \times n_k</amsmath> ways.

	== Enumerating Outcomes ==		== Enumerating Outcomes ==

Root: /* Counting */

2018-08-04T09:31:36Z

Counting

← Older revision		Revision as of 09:31, 4 August 2018
Line 147:		Line 147:

	Most often, methods of counting are used in order to construct probability assignments on fmite sample spaces, although they can be used to answer other questions also.		Most often, methods of counting are used in order to construct probability assignments on fmite sample spaces, although they can be used to answer other questions also.

			== Enumerating Outcomes ==

	= Conditional Probability and Independence =		= Conditional Probability and Independence =