Reading Notes: The Master Algorithm – Metaphor (part 8)

If ML is a continent divided into the territories of 5 tribes, the MA is its capital city, standing on the unique spot where the 5 territories meet. The outer and by far widest circle is Optimization Town Each house is an algorithm, and they come in all shapes and sizes Some are under construction, … More Reading Notes: The Master Algorithm – Metaphor (part 8)

Reading Notes: The Master Algorithm – Analogizers (part 7)

How to learn from very little data? Analogy! Analogizers can learn from as little as one example because they never from a model. Analogical reasoning has a distinguished intellectual pedigree. Aristotle: law of similarity: if two things are similar, the thought of one will tend to trigger the tought of the other Locke and Hume … More Reading Notes: The Master Algorithm – Analogizers (part 7)

Reading Notes: The Master Algorithm – Bayesians (part 6)

At heart, Bayes’s theorem is just a simple rule for updating your degree of belief in a hypothesis when you receive new evidence: if the evidence is consistent with the hypothesis, the probability of the hypothesis goes up; if not, it goes down. Things get interesting when you have many pieces of evidence. To combine … More Reading Notes: The Master Algorithm – Bayesians (part 6)

Reading Notes: The Master Algorithm – Evolutionaries (part 5)

Ronald Fisher’s classic treatise The Genetical Theory of natural Selection formulated the first mathematical theory of evolution. The key input to a genetic algorithm is a fitness function. Evolution as a whole has no known purpose. Genetic algorithms are a lot like selective breeding. They breed programs instead of living creatures, and a generation is … More Reading Notes: The Master Algorithm – Evolutionaries (part 5)

Reading Notes: The Master Algorithm – Connectionlists (part 4)

Hebb’s rule is the cornerstone of connectionism. It says knowledge is stored in the connections between neurons. Donald Hebb stated in his 1949 book The Organization of Behavior: “When an axon of cell A is near enough cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes … More Reading Notes: The Master Algorithm – Connectionlists (part 4)

Reading Notes: The Master Algorithm – Symbolists (part 3)

Rationalists: Believe that the senses deceive Logical reasoning is the only sure path to knowledge Likes to plan everything in advance before making the first move Pundits, lawyers, mathematicians, theorists and knowledge engineers in CS Plato was an early rationalist, later Descartes, Spinoza, Leibniz Empiricists believe: All reasoning is fallible Knowledge must come from observation … More Reading Notes: The Master Algorithm – Symbolists (part 3)