For the implementation of TEC in IMEC, we exclude several of TEC’s principles. Specifically, we excluded Principles 2b (propositions that explain evidence together cohere), 3 (analogy), and 6 (competition). Thagard (1989) based his Principle 2b (part of Explanation) “For each Pi and Pj in and Pj cohere” on the Duhem-Quine thesis (Duhem, 1954) that explanation or prediction requires a bundle of mutually dependent cohypotheses and assumptions. However, Thagard (1989, p. 437) already notes problems with this principle. He states “any scientist who maintained at a conference that the theory of general relativity and today’s baseball scores together explain the motion of planets would be laughed off the podium. Principle 2 is intended to apply to explanations and hypotheses actually proposed by scientists.” There are many more cases where we can find problems with this principle. For example, if a depression is caused by one’s physiological predisposition as well as one’s life partner dying in a car crash, would this lead us to conclude that physiological predisposition for depression and life partners dying in car crashes cohere? It seems more that two unrelated causes together explain the depression. Therefore, we decided not to model Principle 2b.12

We also decided against modeling Principle 3, Analogy. While analogy often seems to be useful in generating new discoveries, it is not established as a criterion for the quality of a theory. In practice, it is also often hard to differentiate what kinds of explanations are actual analogies and what kinds are illustrative metaphors. Noteworthy for psychology, some theories that were based on analogies or metaphors turned out to be hardly replicable (e.g., Carter & McCullough, 2014). A case could be made that actual analogies, as differentiated from metaphors, increase the approximate truth of a theory. An example is the analogy between artificial selection of traits in the domestication of herd animals and the development of traits in animals through natural selection. However, without giving a detailed explication of what makes something sufficiently count as an analogy, scientists can easily “boost” their theory’s explanatory coherence by tagging on metaphors and selling them as analogies. Such demarcation is beyond the scope of the article; thus, we (currently) did not model analogy in IMEC.

Finally, we excluded Principle 6, the principle of competition “If P and Q both explain a proposition Pi, and if P and Q are not explanatorily connected, then P and Q incohere.” This principle seems to be a requirement of the connectionist architecture of ECHO. Namely, if there are too many excitatory links compared with inhibitory links in a network, then the activation of nodes increases with each iteration and the network does not settle in a stable state. Hence, Thagard (1992) seems to have added the principle of competition to create more inhibitory links. In contrast to ECHO, IMEC does perform well with excitatory links only. Therefore, we did not incorporate the principle of competition by default. However, researchers who want to incorporate the principle of competition can do so by adding inhibitory links between propositions of different theories that explain the same phenomenon. This absence of an inhibitory link requirement allows researchers to evaluate individual theories. In addition, some have advocated to use IMEC analogous to models of probabilistic reasoning such as Bayesian networks (Dahlman & Mackor, 2019). In this case the principle of competition can also be useful to model “explaining away.” If there are two independent explanations for a phenomenon and we learn that one of them is true, the other one will become less likely to be true. Therefore, in situations where increased plausibility of one explanation necessarily leads to decreased plausibility of another, we recommend adding a negative link between the relevant explanatory hypotheses. However, in practice, it is often difficult to know whether two causes are independent and hence whether the situation involves explaining away, which is why we did not model the principle of competition by default.

We are left with five out of seven of TEC’s principles. These principles are to be our meta explanatory hypotheses from which the qualities of good theories should follow as a matter of course. We summarize these principles as the following requirements of a model that incorporates TEC:

1. The model needs symmetric connections between hypotheses and phenomena to incorporate Principle 1 (symmetry).
2. It must be possible to establish positive as well as negative connections between phenomena and hypotheses to represent Principle 2 (explanation).
3. It needs some external force that gives the phenomena some degree of acceptability on their own to incorporate Principle 4 (data priority).
4. It must be possible to establish negative connection to represent Principle 5 (contradiction).
5. It requires a measure for the activation of phenomena and hypotheses to denote Principle 7 (acceptability).

One model that has these four properties is the Ising model (Ising, 1925). We developed a new computational algorithm to evaluate explanatory coherence based on the Ising model that we term the IMEC. To understand the proposed method, it is necessary to outline the basic properties of the Ising model.

The Ising model was originally developed in statistical mechanics to describe the polarization of ferromagnetic materials (Ising, 1925). However, its properties are useful to model a variety of systems and the model has been applied widely, for example, to model psychological constructs and opinion dynamics (Dalege et al., 2016; Epskamp et al., 2016; van der Maas et al., 2020). The Ising model describes a binary network in which each node can have states 1 and −1. This can be interpreted as the node being either “on” or “off.” For instance, a person either has a symptom or she does not. The probability of a node being in a certain state is influenced by three parameters (Epskamp et al., 2016).

1. The threshold parameter τ represents the tendency of a node to be in a given state. The higher (lower) τ, the more the node prefers to be in state 1 (−1). For example, in Figure 2,
   TABLES AND FIGURES

   

   Figure 2. A Simple Example Case to Illustrate The Ising Model

   View larger image > in this page > in new window|
   > Download slide
   Figure 2. A Simple Example Case to Illustrate The Ising Model
   if we add a threshold of one to node one, this would increase the probability of this node to be in state one. The probability would be 0.881 and, thus, higher than the 0.5 probability of being in state one that we would find without threshold.
2. The network parameter or edge weight ω denotes the pairwise interaction between two nodes. A positive ω between two nodes indicates that they prefer to be in the same state, while a negative ω between two nodes indicates that they prefer to be in different states. The larger the absolute value of ω the stronger this tendency. In Figure 2
   Figure 2. A Simple Example Case to Illustrate The Ising Model
   adding the threshold on node one does not only increase the probability of node one to be in state one but also of node two. The reason is that this node prefers to be in the same state as node one because of their positive connection. Thus, after adding the positive threshold to node one, the probability of node two to be in state one is also increased to 0.790.
3. The inverse temperature β determines the entropy of the model and is only of minor importance for our implementation. If β equals zero, the probability of each state is equal, whereas for very high values of β, the network can often be in only one state. Note that β is a scalar on ω and τ and can be modeled using these parameters. In other words, if we divide β by some value and multiply all ω and τ parameters with the same value, we obtain the same probability (Epskamp et al., 2016). For instance, a node with τ = 1 and β = 1 has the same probability of being in a given state as a node with τ = 2 and β = 0.5). Therefore, we do not need to model β directly.

We can think of the different nodes as propositions in terms of TEC. In other words, each node represents either a phenomenon or a hypothesis in a theory. The relationship between phenomena and hypothesis and their acceptability can then be determined using the principles of explanatory coherence. The pairwise interactions are by definition symmetric and, therefore, satisfy Principle 1 (symmetry). Principles 2 (explanation) and 5 (contradiction) are incorporated through the edge weights ω that allow us to establish positive and negative connections between nodes. In addition, to incorporate Principle 2 (c) that the degree of coherence is inversely proportional to the number of explanatory hypotheses, we divide the edge weights by the number of hypotheses needed for an explanation (e.g., if two hypotheses are needed to explain a phenomenon the edge weight is half compared with a situation were only one hypothesis is needed to explain a phenomenon). Lower edge weights imply that the hypotheses become less activated if the phenomenon is in an activated state (i.e., is represented as true). For instance, the hypotheses that organic beings have evolved and that organic beings undergo natural selection together explain that the embryos of different species are similar; thus, they would each receive half of the activation generated from the phenomenon.

Third, by incorporating a threshold τ parameter for the phenomena, we can model the impact of evidence, or data priority (Principle 4). Thus, the threshold parameter acts to represent an external field of empirical evidence that impacts nodes representing phenomena in a way that is independent of the theory: this leads empirical phenomena to have plausibility of their own. In the example above, the stronger the evidence for the similarity of embryos of different species, the stronger would be the threshold for this phenomenon. This would also imply that the hypotheses connected to this phenomenon (organic beings have evolved, organic beings are subject to natural selection) are also supported more strongly. In cases where there is no specific knowledge regarding the strength of evidence and strength of explanation, we use default settings inspired by Thagard (1989), which are thresholds of two on the phenomena, a weight of one for explanation, and a weight of minus four for contradiction.

Finally, we need a measure for the explanatory coherence, which is manifest in Principle 7 (acceptability). This explanatory coherence is the dependent measure of IMEC. Theories with higher explanatory coherence are preferable over theories with lower explanatory coherence. In the Ising model, we can consider the explanatory coherence of a state as the probability that a state occurs. States that are more likely to occur in the Ising model have higher explanatory coherence than states that are less likely to occur.

The probability of a state (i.e., a given configuration of all nodes) in the Ising model can be calculated with the following equation:

Here, Z denotes the sum of exp(–βH(x)) over all possible states. H(x) simply denotes the Hamiltonian function. This is a function of the form where τi denotes the thresholds on the node and ωij the connection to other nodes for a state x.

The explanatory coherence (defined by Principle 7 or acceptability) of a hypothesis can be represented by the probability of a given node to be in State 1. Therefore, we sum Pr(X = x) over all states X where a given node Xk is in state one. The explanatory coherence can be denoted as.

In other words, we calculate the probability of all given configurations in which a hypothesis is true.13 If we look at a set of hypotheses, rather than a single one, the explanatory coherence is simply denoted as the average over the coherence of the individual hypotheses. Although we use the probability of a given state in the Ising model as the foundation for our method, we want to emphasize here, that the resulting number cannot directly be interpreted as the probability of a theory being true. Different considerations, such as the role of contradictions, and the fact that we average over individual nodes, when calculating the coherence of a theory complicate this matter. Therefore, the explanatory coherence values should simply regarded as numbers between zero and one, and not as probabilities for the truth of a theory. In the next section we give some aid for the interpretation of these numbers.

The higher this explanatory coherence is, the better, but how high is enough to actually accept a theory? The question of how to interpret the values for the explanatory coherence of theories requires careful consideration. We do not want to establish a default benchmark. Plenty of problems have been ascribed to blindly following the p-value threshold of .05 (e.g., Lakens et al., 2018; McShane et al., 2019) without paying the necessary attention to context (e.g., sample size and effect size). In addition, what determines a good theory depends on many contextual factors, such as the typical quality of theories in a field of research. Therefore, we cannot offer default benchmarks to interpret explanatory coherence. At the moment, we can offer two approaches to construct rules of thumb, one for single theory evaluation and one for theory comparison. In the single theory case, we can derive reasonable values based on the hypotheses: phenomena ratio of typical theories in the field. Although these considerations are highly specific to the chosen research field and should, therefore, always be evaluated in context Figure 3 may serve as a simple example to illustrate a procedure of deriving explanatory coherence values. This indicates that an explanatory coherence of .867 can be considered low, an explanatory coherence .950 can be considered medium, and an explanatory coherence of .977 can be considered high. However, we want to reiterate that these benchmarks are only meaningful in the single theory case. In the theory comparison case, contradictions between theories complicate the picture and these rules of thumb should not be used. In the case of theory comparison, researchers should favor the theory with the highest explanatory coherence.

Explanatory breadth, the number of phenomena that a theory can explain, is a criterion scientists commonly use to compare theories. If a theory explains more, it is preferred over a theory that explains less. Therefore, given the same complexity, IMEC should prefer the proposition that explains more phenomena. This setting is illustrated in Figure 4, where H1 gives a broader explanation of observed phenomena and should, thus, be preferable. Let us consider how IMEC evaluates this case. Comparing the two theories with IMEC based on Equation 3 indicates an explanatory coherence of .867 for H1 and an explanatory coherence of .134 for H2. In other words, IMEC seems to correctly prefer theories of higher explanatory breadth.

Falsification—the abandoning of refuted theories—is sometimes considered the defining criterion of science (Popper, 1959). However, in practice scientists do not directly discard an otherwise successful theory after one failed prediction and it is probably also wise not to do so (Lakatos, 1978). The refutation of a theory might often depend more on whether better, alternative explanations become available. Figure 5A shows a setting where a proposition H1 explains two pieces of evidence E1 and E2 but also predicts a phenomenon E3 for which contradicting evidence exists. We use the same thresholds and edge weights as before but assign a threshold of −1 to E3. In this setting, IMEC assigns an explanatory coherence of .867 to this hypothesis. In other words, the hypothesis is not directly refuted because of one incorrect prediction. Figure 5B shows the same setting, however, now an alternative explanation H2 has become available which does not predict E3 and contradicts the old explanation. Now, the explanatory coherence of H1 drops to .135 and H2’s explanatory coherence is .867. This shows that the explanatory coherence of a proposition usually depends also on the availability of alternative explanations.

Because of Principle 2c, the connection between a phenomenon and its explaining proposition is inversely proportional to the number of explanations. Therefore, if more propositions are needed to explain a phenomenon, this should result in lower explanatory coherence, reflecting the principle of simplicity. Figure 6 shows two theories with either one proposition (yellow) or two propositions (green). Because the green theory needs two propositions to explain E1 each of them only has a connection of 1/2 = .5 to this phenomenon.14 All other weights and thresholds are one or minus one as before. With this setting, IMEC indicates that the simpler theory’s explanatory coherence is .722, while the explanatory coherence of the two proposition theory is only .500. In other words, IMEC captures simplicity as a criterion for theory appraisal.

Because of the principle of data priority, phenomena that are supported by evidence have some explanatory coherence on their own as modeled by assigning them positive thresholds. However, if a phenomenon contradicts a variety of well-supported propositions, researchers often disbelieve it or downplay this phenomenon. Let us consider the example in Figure 7. We can see that the phenomenon E1 is connected to the inferior H1 that contradicts a better supported hypothesis (H2). All edge weights are set to 1 and thresholds are set to 2 or −4 (red weights). In this setting, the explanatory coherence for phenomenon E1 is only .884 and smaller than for the other phenomena (.995, .995, and .995). In other words, the phenomenon was downplayed because it was only predicted by one very implausible proposition H1.15

The next paragraph shows the usefulness of IMEC as a tool to compare psychological theories by comparing the explanatory coherence of two theories of intelligence, the g-factor explanation (e.g., Spearman, 1904) and mutualism explanation (van der Maas et al., 2006). We also show that IMEC allows to evaluate the robustness of the theory to weak evidence or weak explanations and to identify critical experiments by thinking through counterfactuals.

Van der Maas et al. (2006) propose an alternative model of intelligence that explains the positive manifold (the positive correlation of different components of intelligence) not by commonly used latent variable explanations (g-factor) but by mutualism. Development by mutualism means that the positive correlation between different aspects of intelligence (e.g., verbal reasoning, logical-mathematical thinking) occurs solely due to positive interactions between several distinct cognitive processes during development. This idea is inspired by ecology, where for example, the correlation between different aspects of water quality in a lake (e.g., vegetation, water quality) is not explained by a “lake-factor” but by the positive interactions between different aspects of water quality (e.g., Scheffer, 1997; Scheffer et al., 1993).

Based on this model, van der Maas et al. (2006) explain a variety of phenomena in intelligence research, some of which latent variable models have struggled to explain for a long time. However, they also introduce some new assumptions (explanatory hypotheses that support the main hypothesis); therefore, it is interesting to investigate whether their new theory has more explanatory coherence than a latent variable explanation.

Table 1 shows different phenomena related to intelligence and the propositions of the mutualism and the g-factor explanations. The explanatory relations can be seen in Figure 8 with edge weights and thresholds based on the default explained above. Looking at the figure shows that it appears difficult to compare these two theories intuitively. The mutualism theory explains more phenomena than the g-factor theory. However, it is also more complex. Hence, an instrument like IMEC that can assess the trade-off between these different epistemic values seems required for effective theory comparison.

Implementing these explanatory relations in IMEC indicates that the mutualism explanation is superior with an explanatory coherence of .788, whereas the explanatory coherence of the latent variable explanation is only .504; in other words, the mutualism hypothesis seems preferable. This exemplifies how IMEC can smoothly compare theories in contexts where it is intuitively hard to decide between theories.

However, van der Maas et al. (2006) state regarding E7 (differentiation effects) that this phenomenon has not been replicated consistently. Differentiation effects imply that the g-factor is not uniform in the population. In particular it has been suggested, that the positive manifold declines with age (e.g., Tideman & Gustafsson, 2004) and that the positive manifold is stronger in lower IQ groups (e.g., Deary et al., 1996). However, both of these manifestations could sometimes not be replicated (e.g., Facon, 2004). In other words, due to the weak evidence for this phenomenon, we should consider stepping away from IMEC’s default settings and lowering the threshold on E7. In addition, van der Maas et al. (2006) state that the mutualism model allows for both differentiation and integration; in other words, the model makes a very general prediction with regards to differentiation effects that can easily be confirmed. Therefore, let us consider what happens if we reduce both the evidence for E7 and the weight between HM1 and E7 from 1 to .5. In other words, to account for the problematic state of this phenomenon and the vague prediction of the mutualism model, we give only half the evidence to the phenomenon E7 and we weaken the connection to show that the phenomenon is implied weaker by the theory than other phenomena. Computing IMEC with these settings results in an explanatory coherence of .779 for the mutualism model and an explanatory coherence of .516 for the latent variable model; a change of −.009 and .008, respectively. In other words, the superiority of the mutualism model seems to be robust to the problems with differentiation effects. This example shows how we can incorporate weak predictions as well as weak empirical support when specifying IMEC.

In addition, the latent variable model may be taken to imply the existence of a biological cause (e.g., Ackerman et al., 2005; Luciano et al., 2005; van der Maas et al., 2006) that constitutes the g-factor (E8). That this predicted phenomenon was never discovered reduces the explanatory coherence of the latent variable theory. Investigating how the discovery of such a biological basis for intelligence would change the explanatory relations of the two theories can tell us whether finding a biological correlate would constitute a critical experiment (e.g., Platt, 1964). In other words, an experiment that could discard the more coherent mutualism theory in favor of the latent variable theory. With IMEC we can test this by modeling a counterfactual world in which a common biological cause of g is discovered. To do so we assign a positive instead of a negative threshold to E8 and compare the theories again. Computing the explanatory coherence assuming a positive threshold on NE8 shows that finding a biological correlate to g would indeed constitute a critical test. After finding such a correlate the explanatory coherence of the mutualism theory would be .782, while the latent variable theory increases to .836, surpassing even the explanatory coherence of the mutualism theory in the configuration we started with. Thus, in this thought experiment, the latent variable explanation would be the preferable theory after discovering that it designates a biological feature of the human being.

The example illustrates how IMEC can compare psychological theories. We showed that the explanatory coherence of two theories can be compared with identify which of them constitutes a better explanation. In addition, by varying thresholds and weights we can account for higher or lower corroboration of phenomena. Finally, modeling counterfactual states of a theory (e.g., assuming a not yet discovered phenomenon to be discovered) can help us to identify critical experiments that would make a previously inferior explanation the better explanation. However, the example should also be considered a cautionary note as it shows how the conclusions derived from IMEC can depend on what are considered individual hypotheses. For example, one could make the case that when including the assumption of the mutualism model that growth can be described by a logistic model, it would be important to also include that the influence of intelligence on test scores is linear in the latent variable model. Determining inclusion and exclusion of explanatory hypotheses can indeed be considered a weakness of our model and we give more guidance about it in the discussion section.

One intuitive approach to theory appraisal is selecting the theory that best explains the relevant phenomena.16 The complement of this intuition is that one should not abandon a theory until there is another theory that is better at explaining or explains more of the relevant phenomena. Thus, a case could be made that the coherence of an explanation should depend on the quality of the available alternative explanations.

One historical case, where a theory lost explanatory coherence because a better alternative explanation became available is when creationism was surpassed by the theory of evolution. As Thagard (1989) points out, Darwin’s argument was explicitly comparative. In many places in the Origin of Species (Darwin, 1859) he points to facts that the theory of evolution explains while they were unexplainable by the previous idea that species have been created by God. It is also notable that creationism is a very simple theory—the only hypothesis it postulates is the existence of God. Therefore, creationism might seem attractive in light of parsimony. Only a model that considers different epistemic values will be able to refute creationism if a better alternative (evolution) becomes available. In the next section, we first reproduce an example from Thagard (1989) comparing creationism to the theory of evolution.17 Then we consider whether it was reasonable to believe creationism before the theory of evolution was proposed.

The explanatory relations can be found in Figure 9. Comparing the two theories with IMEC indicates that Darwin’s theory is the superior explanation (.820) compared with creationism (.445).

However, in the case of the theory of evolution, one could argue that not all hypotheses should be considered but only the two core hypotheses of the theory. These are DH2 (Organic beings undergo natural selection) and DH3 (Species of organic beings have evolved). Therefore, we can also calculate the coherence of Darwin’s theory as the mean explanatory coherence over only these two hypotheses, instead of all of Darwin’s hypotheses. If we do so explanatory coherence of evolution increases even further to .999. This shows how explanatory coherence changes depending on the set of hypotheses considered elemental for the theory.

However, creationism provided an explanation for the origin of species long before Darwin conceived the theory of evolution. It is interesting to consider whether creationism was a coherent belief system at that time or whether it was always foolish to believe that species have been created by God.

To do so, we evaluate the creationist propositions in isolation. In other words, we do not model Darwin’s theory and the resulting contradiction. Doing so shows that creationism would have had a much higher level of explanatory coherence before Darwin’s theory. Before Darwin’s theory emerged as a strong alternative the explanatory coherence of the creationist explanation was in fact .999. This high level of explanatory coherence may partly explain why people used to believe in creationism. It also illustrates that theories can lose coherence or be refuted once a better theory becomes available. Naturally, this example rests on the assumption that one considers the creationist explanation of the basic phenomena adequate; for those who did not accept this, the explanatory coherence of creationism was low even in the absence of alternative theories. Thus, the example also shows that the dependence of the IMEC system on prior coding of whether or not propositions explain phenomena introduces a subjective element into theory evaluation. One can either accept this (rendering theory choice a partly subjective matter) or develop an objective account of explanation that can be used to determine whether or not a proposition truly explains a phenomenon.

Finally, we can also consider contradictions between theory and phenomena. For example, E5 (species become extinct) and E7 (forms of life change almost simultaneously around the world) can be considered directly in conflict with creationism.18 We can use IMEC to check whether the high individual explanatory coherence of creationism when evaluated in isolation is robust to these contradictions by adding contradictions between creationism and E5 as well as E7. This results in an explanatory coherence value of .39. This low coherence indicates that after discovering evidence that species become extinct and change, the theory of creationism was in peril (or at least its versions that could not account for this), which could have motivated the search for more explanatory coherent theories.

As all models, our model has a variety of assumptions and limitations. First, the symmetry of explanatory connections. In practice, theories might be captured more precisely by directed networks as the Bayesian networks described above, even though these are more difficult to specify for researchers. Exploring a theory of explanatory coherence based on directed networks would be interesting future research. However, IMEC seems to trade off well between describing the complexity of theories and being simple enough for researchers to specify it in practice.

Second, while the principles of explanatory coherence seem suitable to compare a wide range of cases, they are only hypotheses themselves and, thus, open to revision. In other words, the principles that we based IMEC on may need to be changed again in the future. In addition, when assessing a theory researchers might sometimes want to incorporate criteria beyond explanatory coherence as incorporated in IMEC. For example—although explanatory breadth is already incorporated in IMEC—it seems conceivable that of two theories with the same explanatory coherence, researchers would prefer the one with more explanatory breadth. The reason for this could be that its wider set of predictions is more useful from an applied perspective. This shows how considerations beyond the principles of explanatory coherence as applied here can guide theory evaluation.

Third, a key weakness of using IMEC to compare different theories is that the parameter values need to be specified by the user. Although the default values seem to work well across a wide range of cases; it is sometimes necessary to specify the edges and thresholds manually to incorporate weaker (stronger) evidence or weaker (stronger) predictions. In addition, determining the correct number of assumptions and phenomena is a difficult task. This process introduces researchers degrees of freedom and might allow a form of “theory-hacking,” where scientists push the explanatory coherence of their own theory. Therefore, when using IMEC to settle theoretical debates in practice it is important to check the robustness of the conclusion by varying the parameter values. While much further research is needed, Thagard (1989) appealed to the following detailed maxim, which also applies to IMEC:

In analyzing the propositions and explanatory relations relevant to evaluating competing theories, go into as much detail as is needed to distinguish the explanatory claims of the theories from each other, and be careful to analyze all theories at the same level of detail. Following this maxim removes much of the apparent arbitrariness inherent in trying to adjudicate among theories.

Fourth, in some contexts theory and conceptualization scaffold each other, and it is difficult to construe them independently, or postulate that different theories manifest in identical phenomena (Danziger, 1985, 1993). For example, psychoanalysis and cognitive behavioral therapy might have different concepts of what counts as a valid measurement of depression (i.e., associations or projections vs. symptom checklist). However, while this might apply in specific cases, we believe that most phenomena, such as reaction times and error rates or the examples analyzed in this article, can likely be construed independently of the theories being evaluated. In addition, where this limitation applies theory comparison is also intractable with methods other than IMEC.

Fifth, IMEC is capable only of comparing specified theories. The process of generating explanatory hypotheses is not incorporated in IMEC at this point. In other words, finding a theory is still the responsibility of the human theorist. However, several computational algorithms for theory development have been proposed (Romdhane & Ayeb, 2011; Thagard & Nowak, 1988). These algorithms can infer hypotheses from data using certain inference rules. For example, an algorithm called PI (processes of induction) can infer explanatory hypotheses from simple statements (Thagard & Nowak, 1988). For instance, when instructed that sounds propagate and reflect PI will automatically create the concept of a sound-wave. Integrating such an algorithm with IMEC, for example, by evaluating how well the sound-wave theory predicts other observations about sound, could possibly facilitate smooth integration of theory development and theory appraisal.

These limitations indicate directions for future research. First, it would be important to establish ways to derive the hypotheses and edge weights more systematically, for example, using text mining approaches to directly extract the explanatory relations from a paper. In addition, researchers could rely on the aforementioned computational algorithms for abduction (e.g., Romdhane & Ayeb, 2011; Thagard, 1988). Finally, severe testing, Bayesian inference and information theory might also guide the choice of edge weights as described before. This line of research could result in general knowledge about the nature of theories that would apply beyond IMEC.

Second, the original ECHO model has a variety of implications beyond the evaluation of scientific theories (Thagard, 1989, 1992). Therefore, it will be interesting to generalize IMEC beyond the realm of theory appraisal in psychology. Thagard (1989) applied his original ECHO model to several cases of legal reasoning, which might also be a promising application for IMEC. In addition, IMEC could also be used to model human reasoning more generally. Ranney and Thagard (1988) show that ECHO could be used to model belief revision in naive physics. Coherence-based models have also been used to model cognition more broadly (Thagard, 2000; Van Rooij, 2008; Van Rooij et al., 2019). It should be possible to apply IMEC to model these kinds of problems with little adaption.

Third, the Ising model shows interesting properties such as phase transitions (Barabási, 2016) depending on the value for the entropy parameter β. This parameter describes the heat of the network or the entropy of the model; in other words, if β is zero, the probability of each state is equal, whereas for high values of β, the network tends to settle in a single state. An interesting property of the Ising model is, that it shows a phase transition for high values of β but not for low ones. A phase transition describes a sudden chance between the internal symmetry of the model (Solé, 2011); in the Ising model, this would be a chance from an unordered (i.e., some magnets north pole up, others north pole down) to an ordered state (i.e., all nodes north/south up) of the system. Phase transitions have not been used in the present article and it would be interesting to consider the progression of research lines in terms of phase transitions. One could for example argue, that Popper’s idea of falsification describes a phase transition (Popper, 1959), whereas Lakatos (1978) concept of progressive and degenerating research lines describes slower changes. Because the β parameter moderates the occurrence of phase transitions, the Ising model could theoretically incorporate both of these ideas, suggesting a possible unification of different theories in philosophy of science.

Theories are one of the most important tools of cumulative science (Lewin, 1943). Especially in psychology, there has been little progress in the area of formal theory development, despite the high attention that is paid to this topic (Borsboom et al., 2021). One reason for this might be that a systematic, computational approach to evaluating theories lacked in this field. By incorporating a theory of explanatory coherence in the Ising model we developed such an approach. We hope that this will help researchers to evaluate the quality of their theories and will aid in increasing the quality of theories in psychology.