The world is made of events and like any assortment of data points there can be patterns found. With these patterns we tend to find the story which underlies them, which makes sense of them. These patterns often help us predict the future, solve problems, and are sometimes just pleasing see. What can a deep dive on the concept of patterns reveal about their nature?
1. What is the form of a pattern?
When we use the word pattern what to which are we referring? To break this down let’s consider quite possibly the simplest pattern, number sequences.
(2, 4, 6, 8, …)
The pattern in the numbers above is obvious. For each consecutive number we add two. What makes it a pattern is way in which it repeats. We are following a simple mathematical computation to find the next number. This simple computation also is deterministic, in that it does not depend on any external randomness. The deterministic character of patterns allows us to predict with certainty any value in any position. Suppose I wanted to know what the 20th value of P1 would be. We find this question trivial. There are two ways of finding out the answer.
Continue The Pattern
Find and Use the General Function
Continuing the pattern is obvious, we proceed by adding 2 to the previous number until we get to the 20th value. Finding the general function is still easy but provides more theoretical richness. For each number we take the previous number and add 2. A general function which can explain the pattern is F1: y = 2x where x is the position of the number you’d want to find. Now with this general function we can find the 20th value without following the labor of counting each number prior. 2 * 20(th value) = 40. Aside from the mathematical power granted by this less computationally intensive process, we can get an insight into what likely governs the creation of these numbers.
Sidenote: Occam’s Razor and Parsimony
Suppose someone tried to come up with the general function for P1 and they came up with F2: y = 4x - 2x. This equation also maps the pattern in P1 which reveals that multiple functions could exist, however mathematics allows us to simple reduce equations such that we can see their really equivalent. This works well in the logic of mathematics, but reduction and other plausible explanations becomes much more complicated with other patterns.`
2. Patterns in Nature
While theoretical numeric patterns laid our groundwork for understanding what a pattern is, pure number patterns do little by means of utility for our world. A common pattern that all people tend to be interested in is known as a trend. In the world we often are not content with just observing trends we attempt to understand why they are occurring. We may try to find the general function of two values to draw conclusions.
From this graph, we could imagine someone drawing the connection that the more ice cream is sold the more forest fires will occur. This connection is not completely useless, this correlation is strong, and it certainly does make us look twice. This brings out the favorite catchphrase of statisticians.
Correlation ≠ Causation!
Even though there exists a strong correlation, to prove causation requires a theory. It demands a plausible explanation that directly can explain a dependent relationship. Clearly, we all know that the sales of ice cream over the year have no direct cause on forest fires. But yet what can account for this spurious correlation? In this case there exists a third factor which has causation effects on both metrics: Temperature. When the temperature is high, people buy more ice cream and dry plants are more likely to ignite.
3. Strong Patterns
If we find patterns which are extremely strong, occurring with utmost stability, then we often move to enshrine these patterns with a special status. Consider as an example the observed force of gravity. Experiences, whether in a lab or not, has demonstrated that things fall toward the ground. Not only do they always fall (when not supported), but they do so at a constant acceleration. This type of pattern seems different from those we have seen before. The increase in ice cream sales during summer months makes sense but is by no means as definite as gravity. The increase in ice cream sales is contingent on all types of factors. We could easily imagine this connection not being the case. However, gravity seems to be necessary for a world like ours to exist. Given this rigorous pattern, we tend to want an explanation for why it is so. We observe the correlation between objects falling and their speed increasing, but what is the cause? The search for a cause is where a pattern becomes a theory. We want a framework which allows us to understand the particular instances of falling objects. Don’t tell me why my book falls when I drop it. Or why when Alice walked off the cliff, she fell. Give me the theory which explains all of these instances. A theory which can make sense of our world. Aristotle wanted a theory for this phenomenon and did so by asserting that heavy bodies, by their nature, wanted to move downward toward their natural place and air, by its nature, upward. This explanation in a modern context seems woefully off, but we should recognize that this explanation does fit the pattern. Air and gases move up and books and stones move down when we allow them to follow their ‘nature’. So, because the theory matched all the known experiences at the time, it was accepted.
4. Bad Theories and Revisions
For a theory we find foolish now, the Ancient Greek theory lasted until the 16th century. Sir Isaac Newton is credited with “discovering” gravity and suppling humanity with a better theory. This theory asserted the existence of a gravitational “force”. This force was formulated using modern science techniques and mathematics. Now, instead of explaining brute sensory observations, we could use sophisticated technologies to find a formula of the force. This formula gave way to a theoretical explanation. Physical bodies, dependent on their masses, acted upon one another with a force that varied with their distance. Even this theory and deemed “law” of gravity was overturned when Einstein’s general relativity posited that the cause was the curvature of space-time fabric. Notice that even though the theory Newton had was wrong, the data and formulas developed on the pure experiments remained.
5. Proving Theories
We saw that Newton’s theory ended up being an incorrect interpretation of the data. It was overturned by another theory which could better explain the experimental phenomenon. But this leaves you with a theory which could still be replaced by another, better one. How can we ever gain confidence that the theory we have is the correct one? The answer is predictions. A theory may be based on historical data, but can in explain unseen and unaccounted for future data? What additional facts must be true for the theory to hold? Einstein’s theory of space-time fabric implied the existence of gravitational waves. These waves were not yet recorded due to technical limitations. When the technology progressed, researchers began looking and did find the waves exactly as predicted. This power for a theory to explain unforeseen phenomenon show that it’s interpretation of the data is extremely robust. Now it is possible there is an odd coincidence, like the ice cream and the forest fires, there may be another explanation which better accounts for the spurious correlation, however until we have the evidence for that theory, we accept the current as true.
6. Abductive Reasoning
Theorizing about the world, philosophical, scientific or otherwise all generally follow the form of Abductive Reasoning. We start from our confirmed observations and write them down as evidence. Then from assessing the evidence as present to us, we try to tell a story which people find to be the most plausible. Like a detective on a case, we gather this evidence to determine who the likely suspect is, even know we may never truly know. The best way to get better theories in this process is to get more evidence. We saw as in the case with Einstein, by formulating theories we can find better questions to ask that test our theories. Without the once “unconfirmed” general relativity, we would have never known to look for gravitational waves or that they would even pertain as evidence. A crucial part of this entire abductive process is creativity and intuition. To have the creativity to discover a brilliant idea like general relativity is not something you can reach by logic alone. The abductive approach, unlike deductive or even inductive reasoning, allows for much more open-ended process of theorization. Intuition is the faculty which seems to guide us on this process. When Aristotle posited his theory of falling objects it wasn’t accepted due to its mathematical rigor, but it was accepted on its intuitiveness. It just made simple common sense of the experience. It was this and this alone which allowed this theory to maintain its dominance for thousands of years. But as new evidence came available our intuition, our creativity, were given more data points to generate from.
7. Skepticism and Ever-growing Puzzles
The biggest downfall in this entire process is the pressure from skeptics. Skeptics would be right to point out our flawed history in theorizing. To be fair, nearly every theory has been disproven at one point or another. So why should we ever have confidence in a new theory, or engage in the process at all? I respond by pointing to the ever-expanding body of evidence. When a theory is overturned, it doesn’t always invalidate its explanatory power over the data. It still can be useful in that it was tracking something relevant. Aristotle was right, heavy bodies fall and light air floats, this is still true today. Like a puzzle, these incorrect theories can still retain coherence and a solution to one portion of a phenomenon. But as our knowledge increases, we demand a bigger picture and a bigger theory which can explain all the unsolved edges of that puzzle. It is no doubt that this puzzle will ever expand but notice how our solved sections (previous and current theories) tell us where to look. They tell us how to solve other unknown portions. They limit the field of possible theories, by taking up pieces. Our repeated attempts at theory give us some important knowledge about the world (our theory is looking stronger, or we will need another theory, with now a better evidence pool to abductively reason from). I view this history not one to hold our head in shame, but one where we can see the triumph of human creativity and intuition. A beauty in this whole process is not just in how new theories outperform the old ones, but how this process is not one left to pure mathematics or logical proof. Theorizing and explaining patterns depends on telling a story, a metaphysics of the reality that lies behind appearances and perceptions. You never see theory and probably never will, and yet we can’t help chasing this ghastly force, not waivered by our failures, but perhaps despite them.