The following is a transcript of the opening remarks given at the Science for Social Good satellite event at the Cognitive Computational Neuroscience conference organized by Anne Urai, Ili Ma and myself in Amsterdam on August 11, 2025. Check out Georgia Turner’s blog post about the event posted on Anne Urai’s website.

Prezi

I want to start us off this afternoon by providing an overview of what one could mean when they say ‘science for social good’. I’m going to use this tree as an analogy. Up top are the fruits I want to grow. This is my positive vision of a science for social good. Then in the trunk are the prerequisites—the parts that need to be in place before the fruits can grow. And in the soil are slightly more specific nutrients that we can add to our scientific system to help the tree to grow.

Starting with the fruits

A science for social good abandons the outdated notion of scientific institutions as isolated, ivory towers and instead recognizes that science is always embedded within our social, political, economic, and planetary systems. These systems, particularly systems of power, shape what science gets done and science in turn will reinforce or challenge particular worldviews and have material consequences on our planet’s life support systems.

So these interactions are always happening. A science for social good is trying to amplify the positive interactions while dampening the harmful ones. We want to acquire true beliefs while avoiding false ones, and we want the products of our science to benefit or be valued by citizens. But we recognize that this is a two way street. It needs to be done in conversation with policymakers, communities, and civil society groups who have much to contribute to science itself.

I want to live in a science-based society in which scientific resources are mobilized to collect evidence and develop new knowledge that helps humanity to make decisions about how to best pursue its goals. I want to contribute to science and education as a public good that serves the interests of citizens. This is one place where science meets politics. How do we decide what humanity’s goals and values are if not through politics? through democracy?

In order for that to become a reality, we’ll need these things in the trunk.

We need the inclusion of the public in the co-creation of research and setting priorities. Co-creation is a hot topic these days, particularly in certain sub-fields. You’ll probably be familiar with it if you work with any special subpopulations where you might have heard the saying ‘nothing for us, without us’. But there is room for all of us to reflect critically on how we orient towards our human participants, the labellers of our machine learning datasets, and the people affected by the technology with create, use, and study.

We need our science to be effective, reliable, and trustworthy. That means we need to nurture and protect the practices and systems that support scientific progress and objectivity.

Then, hopefully decision makers and the public alike can develop an informed trust in science and grant epistemic authority to science. By ‘informed trust’ I mean a critical acceptance of science which recognizes its limitations rather than a blind faith or scientism.

Now I’ll move on to this incomplete list of ingredients or nutrients. I’m sure that you can think of many more to add here.

To achieve an effective, reliable, and trustworthy science we need our science to be open and transparent. Allow me to highlight UNESCO’s Recommendations for Open Science which provides an international framework for open science policy and practice.

We also need critical discourse. We need to be encouraged to challenge each other and we need our scientific community to be sensitive to that criticism. We need diverse values and approaches so that we can best scrutinize each other’s assumptions. We need to reflect on whose interests are reflected in our science. Groups whose interests are not reflected in mainstream science will be justifiably distrustful of that science, and so we must take action to broaden participation in science. For more about the deep sociality of science, check out this easy to read article I wrote for Science for the People Magazine.

Science communication and science education will help to build that informed trust in science I mentioned earlier. A scientifically literate public will then be better able to participate in a science-based society. On this point, I want to emphasize that this is something we can easily get wrong. Not all science education and science communication is helpful on this front.

In preparing this talk, I was quite influenced by this article Sealing the gateways to post-truthism: Restablishing the epistemic authority of science which concludes that ‘educational measures should highlight the social and conversational nature of scientific knowledge production because these concepts lay the foundation for learners’ and citizens’ abilities to build an informed trust in science, and in turn, actively engage in a science-based society.’ See also the book Why Trust Science? by Naomi Oreskes.

Lastly, we need the political will, both to conduct this kind of science and to act on its results. This is another place where science meets politics. If we want a science-based society, we’ll need to advocate for it and engage in activism to bring it about. Personally, I pursue this through my trade union where we are currently building a campaign for education as a public good.

So I hope in this tree you’ll see that it’s not just what we research but how we research that makes a science for social good. There is a buffet of entry points and ways to contribute, regardless of your particular interests.

I deliberately made this a positive vision, but we can equally imagine a mirror image of this tree that summarizes the forces against a science for social good. Science is under attack right now in many parts of the world. Instead of campaigning for a return to the status quo, let us use this moment to remake our scientific systems and repair our relationship with the human and more than human world.

We are the gardeners of our scientific ecosystem. What will we grow?

Today I am on strike. My first time striking was in November when the University and College Union (UCU) national ballot came back in favour of both strike action and action short of a strike (side note: I find it telling that the ASOS involves only working to contract, meaning only doing the work that one is paid for and not taking on extra work or working overtime for free. In what world does ‘only’ doing your job count as industrial action?!). The national ballot means that basically all of higher education in the UK is on strike right now, along with almost every other worker it seems: teachers, nurses, paramedics, rail workers, fire fighters, postal workers… but walking around University of Oxford you would hardly know it. Most staff don’t join the union and definitely don’t join the picket line. And I can understand why. My experience striking has involved lots of unpleasantness. Not working when I am under so much pressure to work is really stressful (should I really cancel that meeting??). I’m anxious not knowing who supports the strike and who wants to tell me to ‘get back to work’. I feel depressed and abandoned without the support of others in my department. It feels like I’m only hurting myself by striking, especially when my work is primarily research. The first strike day I received work-related messages from students which felt absolutely terrible to ignore. I’ll lose quite a lot of pay as these strike days add up. I miss out on meetings/events that I want to attend. I get anxious about not striking ‘properly’ if I arrive at the picket late or if I’m too cold and tired to attend afternoon events or if I sneak in a bit of work. I’m nervous about my colleagues asking me for updates about the negotiations if I haven’t found time to read all the latest updates. What’s the point? I feel powerless. My contract is up in a few months anyway. Why bother? No one else seems to care. The impact of me striking is so much less than a nurse or rail worker striking. Plus, aren’t we lucky to have this job?

So why do I strike? Simply, because my union voted to strike. Independent of my own working conditions and whether I personally voted for strike action, I strike in solidarity with my colleagues who collectively said ‘enough is enough’. I’m not saying I don’t have complaints. These disputes concern issues that definitely affect me. But my decision to join the union and strike when they vote to strike is more basic than that. My idea of a better future involves a strong labour movement who fight for workers’ rights as part of broad, multifaceted efforts to reverse growing inequality. The proportion of unionized workers has been declining in places like the US, Canada, and the UK over the past several decades as service industries have replaced more traditionally unionized professions like manufacturing. Around the world, people are struggling to unionize and face union-busting and severe retaliation for taking action. I have a union and the protected right to strike. In the global struggle for workers’ rights, surely the least I can do is join my own union?!

UCU’s demands in these disputes are completely reasonable. The asks are mostly for the employers to give back what they’ve already taken. We want our old pension scheme back after they cut it by 25-30% for demonstrably no good reason. We want pay to rise with inflation. We want a plan to address the gender gap, race gap and disability gap. We want to stop replacing long term contracts with precarious short term contracts. And we want reasonable workloads that can be managed within the number of hours we’re paid for in a week. The universities can afford it. Last year, they ended the year with £2.4 bn more cash in the bank than they started with. Instead of investing in their staff, they are constructing new buildings and other vanity projects. They are doing this because they can. They can because we let them.

The increasing commodification of education and research is not surprising, of course. Neoliberalism has normalized the harsh reality of a world driven by profit and personal responsibility. But just because it’s normal doesn’t mean it’s right. Students are not customers. Education is not a commodity. The value of research is not determined by how easily it can be monetized. Attending my first UCU events made me realize that there are other people around me who are willing to resist, not just in a particular pension dispute that may not affect me much if I end up leaving the UK within the year, but in the larger fight to dethrone the destructive logics of neoliberalism.

So despite all the unpleasantness, striking and attending associated rallies and teach outs has been inspiring and motivating and full of solidarity. Yesterday on the picket we spoke with some fire fighters who asked how our negotiations are going and wished us luck. Last week we held a joint march with the teacher’s union. These events take me out of my ivory tower and help me to stay focused on the issues of working people in my community. Lots of students also participate and express their solidarity. They know that this affects them too. During the second week of strikes last autumn, I attended a teach out on radical pedagogy with staff and students. We read an excerpt from Pedagogy of the Oppressed by Paulo Freire. It was so inspiring to discuss how learning and mentorship relationships can be more equitable, non-hierarchical, respectful, inclusive, and therefore effective. I feel empowered and energized to build those kind of relationships in my daily work. Yesterday I attended a teach out on radical needlework: knitting and crochet. We plan to meet again next week to make some strike related paraphernalia in UCU colours. I vacillate between feeling amazed and inspired in these pockets of magic and utterly depressed when I remember how thin participation is.

I understand that not everyone has the capacity to strike and I respect that. Honestly, I don’t think I’ll have the capacity to keep it up. Some work feels too important, too tragic to abandon. We all have to navigate these decisions for ourselves. I try to prioritize being visible on the pickets and vocal about the strikes since other forms of withholding my labour feel less effective (and more personally harmful). But we wouldn’t have to keep it up so long if more people showed their support, however they can.

I think as a postdoc or anyone on a short-term contract, participating in the union can be especially unattractive. You haven’t necessarily been here long enough to understand the issues, you might not stay here long enough for any of the fallout of the industrial action to actually make a difference to you. And we’re so stressed about our work and figuring out what we’re going to do next, that we don’t have time or energy to think about much else. This is very convenient for the universities, who are increasingly hiring people on shorter and more precarious contracts. The way I see it, the struggles at my current institution are continuous with the struggles at my next institution. The specific issues might be different but the fight is the same. And it’s not a fight for me, it’s a fight for us. Even if I am not here long enough to benefit personally much from the reinstatement of our pensions, the next postdoc who comes in after me will, and the one after that. And I will benefit from the actions of those who are currently organizing at whereever I end up going next. I am trying to unlearn the individualism that limits my capacity for empathy and solidarity. It is so easy to feel isolated in academia. How can we recentre and rejoin the collective?

Hopefully at the next negotiation meeting we’ll receive a better offer and there will be no more strike days. When that happens, everyone will benefit, and it will be due in large part to the hard work of the overburdened volunteers who organized—thank you! I hope that more of my colleagues join the UCU so that we can better take care of one another. The longer the pickets the shorter the strikes. Let’s not take our rights for granted.

Onward in solidarity.

• Comparing the activity of artificial and biological networks at the 2020 Montreal Artificial Intelligence and Neuroscience Symposium (MAIN2020)
• Introduction to Scientific Explanation of Cognition at our Symposium on Explanation in Neuroscience and Artificial Intelligence

Figure 1. Models lie at the intersection of one or more constraints. The rectangle indicates the space of all possible models where each point in the space represents a different model of some phenomenon. Regions within the coloured ovals correspond to models that satisfy specific specific model constraints (where satisfaction could be defined as passing some threshold of a continuous value). If a constraint is well-justified, this implies that the true model is contained within the set of models that satisfy that constraint. Models that meet more constraints, then, are more likely to live within a smaller region of the hypothesis space and hence will be closer to the truth, indicated by the star in this diagram.

The following is a lightly edited excerpt from the Discussion chapter of my doctoral dissertation which was successfully defended on Nov 20, 2020.

On using deep neural networks as models of sensory systems

The use of DNNs as models of animal sensory systems is largely in the context of a model comparison approach to scientific discovery. Through evaluation, comparison, and iterative refinement, models hopefully get closer to some truth about the phenomena under study (or at least the models become more useful, if one prefers a more pragmatic, less realist account). Within this view, models that are more constrained are likely to be closer to the truth since they occupy smaller region of the search space known to include the true model (Fig 1). In practice, however, a priori we don’t usually know which constraints are necessary to answer a specific scientific question or how a particular set of constraints will affect our exploration of the hypothesis space.

The use of DNNs as models of sensory systems emphasizes a different subset of constraints than alternative modeling approaches. For example, Kell and McDermott (2019) discuss the importance of task constraints and of models that exhibit the phenomenon to be explained. For example, if one wants to study face recognition, a reasonable possible model constraint is that the model be capable of recognizing faces. Emphasizing task-performance and accounting for animal behaviour may come at the expense of other possible model constraints since it is usually impossible to satisfy all model constraints at once.

The various differences between DNNs and biological brains are often repeated to refute their usefulness as models. In particular, the biological (im)plausibility of DNN models and their limited ability to replicate high-level cognition are often cited. Marcus (2018) describes the limitations of current DNNs. They are not capable of relational reasoning, cannot accommodate non-stationarity, do not extrapolate well, and cannot separate correlation from causation, among other limitations that human brains have managed to overcome. Zador (2019) questions the relevance of models that required large datasets to learn when most animal behaviour is the result of eons of evolution and encoded in the genome rather than learned over the course of a lifetime. DNNs and animals fail in different ways. DNNs are susceptible to adversarial examples—examples that have been only slightly modified such that the differences are not noticeable by humans, but can severely affect the performance of a network (Goodfellow et al., 2015). Since DNNs are only loosely inspired by biological neural networks, there are enumerable physiological details that are missing. Biological plausibility has been presented as a requirement for models to be useful for studying biological neural computation (Gerven and Bohte, 2017) and the biological plausibility of learning in DNNs has been questioned. There are many differences between DNNs and biological brains, but what do they imply about how we ought to think about DNNs as models? Why are these differences meaningful?

Criticisms of the use of DNNs as models of sensory systems often amount to claims that a different subset of constraints should be privileged. The proposed requirement that models must be biologically plausible in order to have bearing on neuroscience prioritizes the purple region of Figure 1 which contains only models that are deemed biologically plausible. According to the view put forth by Love (2019), positions of this type reflect value judgements about which datasets are most important. On what basis are such value judgements made? Within the model comparison framework, constraints (or datasets) are selected to narrow the search space. Constraints could be privileged based on how much they narrow the search space. However, when comparing two constraints like biological plausibility and task performance, it is not obvious that one will narrow the search space more than the other. It is entirely possible that exploring the space of possible models that can perform some task of interest will lead to truth faster than exploring the set of models that are biologically plausible. These known unknowns can inform how we think about optimizing scientific progress in a model comparison framework.

We can try to reason about which constraints are more limiting and it may be more or less possible for different research questions. In Article 1, I emphasized the importance of specifying the phenomenon to be explained. Similarly, Love (2019) emphases a similar need to identify the datasets to be accounted for. Different researchers, even researchers who are concerned with the same natural phenomenon, may still choose to privilege different model constraints and this is a feature, not a bug. Due to our uncertainty about the nature of the hypothesis space to be explored, we need different researchers to come at the same problem from as many different angles as possible. This has been studied using simulations of scientific discovery in a model-centric framework to identify the relationship between several attributes of scientific communities and the success of their research program. Devezer et al. (2019) found that innovative research speeds up the discovery of scientific truth by facilitating the exploration of model space and that epistemic diversity, the use of several research strategies, optimizes scientific discovery by protecting against ineffective research strategies. The authors compare epistemic diversity to diversifying an investment portfolio to reduce risk while trying to optimize returns. If one knew how the market was going to change, one wouldn’t need a diverse investment portfolio. Similarly, uncertainty about scientific truth and how to search for it should lead us to embrace epistemic diversity.

The long list of differences between DNNs and brains has no general implication for the suitability of DNNs as models of biological intelligence and learning. Specific differences may be relevant to specific research questions. Nevertheless, researchers are currently working on addressing several of these differences to further narrow the model search space. Machine learning researchers are currently working on biologically plausible learning algorithms (Bengio et al., 2014; Lillicrap et al., 2014; Guerguiev et al., 2017), relational reasoning (Bahdanau et al., 2018; Santoro et al., 2017), and causal inference (Schölkopf, 2019; Goyal et al., 2019). Neuro-AI researchers have been exploring the effects of adding elements of biological realism to DNNs to see how they affect representational correspondence (Lindsay and Miller, 2018; Lindsay et al., 2019). Storrs and Kriegeskorte (2020) hypothesize that, as the field of deep learning continues to progress, neural network models will only become more relevant and useful for cognitive neuroscience. They discuss how the study of relational reasoning in artificial systems helps to identify the necessary and sufficient conditions for such abilities to develop and how artificial systems trained in simulated environments can be used as a tool for studying embodied cognition. The use of DNNs as models of biological neural system is one of several well-justified modeling approaches. DNN models focus on different regions of model space than alternative approaches, and thus constitute an innovative strategy that increases the epistemic diversity of computational neuroscience.

Unifying Neuroscience and AI: Disambiguating prediction, representation and explanation

Many terms are used in different ways at the intersection of neuroscience, AI and philosophy of science. An integration of neuroscience and AI will require a consistent language. Here, I try to map between related concepts in cognitive science, statistics and machine learning.

Representation and encoding

Much effort has been directed at representations and their role in cognition and explanation. Marr and Nishihara (1978) defines a representation as “a formal system for making explicit certain entities or types of information, together with a specification of how the system does this.” He denotes a specific instance of an entity in a given representational system as a description. For example, the Arabic numeral 37 is a description of the number 37 that makes explicit its decomposition into powers of ten. A binary representation of the same number would make explicit its decomposition into powers of two. A representation will often be a useful abstraction. For example, we can represent strands of DNA as sequences of nucleotides, represented by the letters A, T, C and G. Similarly, the information processing approach to cognitive neuroscience presumes that the brain is likely to use various representations of sensory information at different stages along some pathway to facilitate certain computations. The terms encoding and decoding refer to representational transformations from the sensory input (encoding) and to perception or behaviour (decoding). According to Diedrichsen and Kriegeskorte (2017), information-based analyses of neural measurements (encoding analysis, decoding analysis, representational similarity analysis, etc.) test representational models, which describe how patterns of activity relate to sensory stimuli, motor actions, or cognitive processes. Their definition of representation within this framework is that a represented variable can be linearly decoded from a down-stream area. This paradigm, sometimes referred to as neural coding, has led researchers to make statements about what is ’encoded’ in neural signals based on the results of encoding and decoding analyses.

This paradigm has received criticisms on several fronts. Brette (2019) points out that the language of the neural coding framework implies causal relationships for which the analysis typically does not provide evidence. That the activity of a population of neurons can be well predicted by a particular representational model does not in itself imply that the hypothesized representation is in fact used by the neural system to accomplish the task of interest. Many candidate representational models may predict the relevant neural activity equally well. Using predictive performance as the only arbiter of model fit does not establish the causal relevance of the hypothesized representation. This debate reflects tensions between functional and causal mechanical theories of explanation. The neural coding paradigm en- tails the functional analysis of a neural system: decomposition of the component operations of a phenomenon. According to the functional theory of explanation, the causal mechanical implementation of those component operations are not needed. Although not stated explicitly, in essence, Brette’s warning regarding the interpretation of results in the neural coding paradigm reflect a warning against a functionalist view of explanation in neuroscience.

The neural coding paradigm has also received criticism from the dynamical camp. The dynamical hypothesis, is that ‘cognitive agents are dynamical systems’ (Gelder, 1998). The antirepresentational stance adopted by some dynamicists and radical embodied cognitive scientists claims that cognition is not inherently representational (Chemero, 2009): “Unlike digital computers, dynamical systems are not inherently representational. A small but influential contingent of dynamicists have found the notion of representation to be dispensable or even a hindrance for their particular purposes. Dynamics forms a powerful framework for developing models of cognition that sidestep representation altogether” (Gelder, 1998, 622). A dynamical explanation may make no reference to representation and instead describe the details of a particular neural circuit, for example.

The definition of representation in cognitive science and neuroscience is distinct from the notion of representation in machine learning. The field of representation learning is concerned with procedures for automatically learning useful transformations of data. The input data, say a set of images, are originally represented by a set of three-dimensional (RBG) pixel values. This pixel space is one representational space. Learned representations will consist of one or more transformations of this original form. In this sense, machine learning representations are representations of some signal whereas in cognitive science literature, a representation is a representation of some variable. In a DNN classifer, the target could be seen as a variable of interest. From the data processing inequality, we know that the mutual information with the target will be maximal at the input layer. All the information related to the target class is present at the input. The subsequent representational transformations change the form of that information, gradually linearizing the decision boundaries, such that the target class can be linearly read out at the output layer. One can add linear classifier probes at each layer of a deep network to see how well the target class can be decoded from each layer. For a trained network, one should see that the performance of these linear probes will increase with depth, but the decoding performance could be above chance at all depths (Alain and Bengio, 2016). In this case, where would the cognitive neuroscientist say the target is represented? At every layer? Or maybe at the input since that is where the mutual information is greatest? Or at the final layer since the decoding accuracy is highest there? From a machine learning perspective, what can be linearly decoded from a layer’s activity only provides a snapshot of its representational form.

Prediction, explanation, and generalization

In machine learning, the output of a model is a prediction. In classification, the prediction takes the form of a categorical label which represents the model’s best guess of the category of the input example. Traditionally, the goal of supervised machine learning is to discover statistical regularities and invariances in the training data that enable accurate predictions for a given task. The data are typically assumed to be independently and identically distributed (i.i.d); all observations are sampled independently from the same data generating process. The goal is a model with good generalization performance, which means that the predictions are accurate for any other sample from that data generating process. A model that overfits to the training data will not generalize well. For some models, there are analytic bounds on the generalization gap. In practice, this is typically verified empirically by separating datasets into training and testing sets. The performance on the test set estimates how well the model would predict any random sample from the same data generating process; this is referred to as within-distribution generalization. Some efforts in machine learning are focused instead on out-of-distribution generalization, which refers to the setting where the training and test sets are not i.i.d.

One example of out-of-distribution generalization is systematic generalization in language, which refers to the ability to rationalize about logical rather than purely statistical relationships between tokens. For example, (Bahdanau et al., 2018) investigated the ability ‘to reason about all possible object combinations despite being trained on a very small subset of them’:

Clearly, given known objects X, Y and a known relation R, a human can easily verify whether or not the objects X and Y are in relation R. Some instances of such queries are common in daily life (is there a cup on the table), some are extremely rare (is there a violin under the car), and some are unlikely but have similar, more likely counter-parts (is there grass on the frisbee vs is there a frisbee on the grass). Still, a person can easily answer these questions by understanding them as just the composition of the three separate concepts. Such compositional reasoning skills are clearly required for language understanding models.

Systematic generalization is something that is relatively easy for humans but difficult for artificial natural language understanding systems. Out-of-distribtion generalization also shows up in other applications. For example, one may wish to train a robotic arm first in a simulated environment controlled by a physics engine and want it to generalize to the real-world. Out-of-distribution generalization is one of the frontiers of AI research at the moment and will be required for AI systems to mimic human cognitive abilities. In this way, not all predictions are equal. Different predictions will test different generalizations.

In statistical hypothesis testing, commonly employed in the analysis of neural data, the word predict is employed in a different way. One variable is said to predict another if a significant statistical relationship has been found between the two. This use of the term is more akin to what philosophers call accommodation: how well a scientific theory accommodate the data that was already known at the time the scientific theory was constructed. When regression is used for statistical hypothesis testing, one variable (or set of variables or interaction of variables) is said to predict another based on an assessment of the experimental data. When using regression in machine learning, the model as a whole is predicting the target. The model is evaluated by how well the model predicts held out data (data that was not used during the training of the model). However, neither of these uses seems to parallel the use of predict in philosophy of science where the emphasis is on the prediction being novel, i.e., something that hasn’t been observed yet.

Confusingly, the word explain is also used in the context of statistical hypothesis testing. The statistical measure R-squared (R2) is the proportion of variance in one variable that is explained by another in a linear regression. This use of the word explain in statistical hypothesis testing is distinct from scientific explanation, but the two are sometimes not clearly distinguished in scientific writing. For example, consider this motivating statement for the Algonauts project, whose 2019 edition is dedicated to “Explaining the Human Visual Brain”:

Currently, particular deep neural networks trained with the engineering goal to recognize objects in images do best in accounting for brain activity during visual object recognition (Schrimpf et al., 2018; Bashivan, Kar, & DiCarlo, 2019). However, a large portion of the signal measured in the brain remains unexplained. This is so because we do not have models that capture the mechanisms of the human brain well enough. Thus, what is needed are advances in computational modelling to better explain brain activity (Cichy et al., 2019).

When discussing unexplained signals, the authors allude to statistical explanation while talk of capturing neural mechanisms hints to scientific explanation. When in reality, this project is about evaluating models based on their ability to predict (in the machine learning sense) neural activity. When they lament that a “large portion of the signal measured in the brain remains unexplained”, they invoke the notion of explained variance. Rather than trying to develop a scientific explanation for a phenomenon of interest, they are concerned with statistically explaining, or in this case, being able to predict, the variance in the collected data—variance which may or may not be causally related to any number of different neural or cognitive phenomena.

Many of the issues described above can be subsumed under the notion of generalization. The philosopher’s term accommodation does not imply any generalization beyond the observations used during the construction of the theory (or training of the model). The typical notion of generalization in psychology is akin to within-distribution generalization in machine learning. One assumes (or tries to ensure) that their sample of subjects represents a random sample from a population. The goal of statistical inference is to make general statements about the population from the measurements made on a sample. The notion of novel prediction in philosophy of science could be seen as an example of out-of-distribution generalization in machine learning.

The goal to explain as much variance as possible or to predict as accurately as possible expresses a desire for completeness. Philosophers of scientific explanation warn against over-completeness.

It is important to note, in this connection, that particular [explananda] do not necessarily embody all of the features of the phenomena which are involved. For example, archaeologists are attempting to explain the presence of a particular worked bone at a site in Alaska. The relevant feature of the explanandum are the fact that the bone is thirty thousand years old, the fact that it was worked by a human artisan, and the fact that it has been deposited in an Alaskan site. Many other features are irrelevant to this sought-after explanation. The orientation of the bone with respect to the cardinal points of the compass at the time it was discovered, its precise size and shape (beyond the fact that it was worked), and the distance of the site from the nearest stream are all irrelevant. It is important to realize that we cannot aspire to explain particular phenomena in their full particularity. . . . In explanations of particular phenomena, the explanation-seeking why-question—suitably clarified and reformulated if necessary—should indicate those aspects of the phenomena for which an explanation is sought. (Salmon, 1984, pg.273-4)

The project of collecting large-scale neural datasets and building models that explain as much variance as possible in that data is one of mere description rather than explanation. Descriptive science is unambiguously crucially important to scientific progress. Recall the first aspect of Craver’s notion of mechanical explanations is characterization of the phenomenon to be explained. However, the distinction between explanation and mere description is still important. Specific why-questions may eventually be motivated by such descriptive characterizations, but only if we don’t mistake them for explanations prematurely.

General Conclusions

Comparing activations in biological and artificial neural networks is a promising approach to study the architectures and learning procedures that support brain-like representations and the nature of representations in intelligent systems. However, this scientific project is not just about chasing high accuracy. As much as one might like to, the scientific problems posed in neuroscience cannot be reformulated as engineering problems. The (long term) goal of science is to generate scientific explanations, which is not the same as statistically explaining the variance in our data. At this particular moment in computational neuroscience, there is a high degree of uncertainty about what such explanations might look like for many phenomena of interest. Therefore, the field may benefit from a closer relationship with philosophers of neuroscience concerned with scientific explanation. Philosophy of science offers conceptual scaffolds that can help refine a vision of an integrated science of intelligence.

Part of the value of deep learning from a neuroscience perspective could come from the fact that deep learning is theory-poor compared to other areas of machine learning. That there are a lot of open questions in deep learning theory may reflect generic challenges of studying learning in distributed networks. This positions deep learning science as a model science for neuroscience. The methods and concepts that prove useful for explaining phenomena in deep learning may inspire new methods and ways of thinking in neuroscience, due to the similar scientific problems posed in these two fields and the relatively ease with which artificial systems can be analyzed compared to their biological counterparts. In this way, the opportunities for transfer between deep learning and neuroscience span several scientific and meta-scientific levels.

The arguments presented here are not intended to advocate for a deep learning approach to neuroscience over other approaches. The purpose of the arguments presented is to justify and clarify the merits of a deep learning approach to neuroscience as one among many. The addition of a deep learning approach increasing the epistemic diversity of the set approaches employed. Innovative and diverse approaches in an epistemically humble research community will better lead us towards truth.

Bibliography

• Alain, G. and Bengio, Y. (2016). Understanding intermediate layers using linear classifier probes. arXiv, page 1610.01644v3.
• Bahdanau, D., Murty, S., Noukhovitch, M., Nguyen, T. H., de Vries, H., and Courville, A. (2018). Systematic Generalization: What Is Required and Can It Be Learned? pages 1–16.
• Bashivan, P., Kar, K., and DiCarlo, J. J. (2019). Neural population control via deep image synthesis. Science, 364(6439).
• Bengio, Y., Lee, D.-h., Bornschein, J., and Lin, Z. (2014). Towards Biologically Plausible Deep Learning.
• Brette, R. (2019). Neural coding : the bureaucratic model of the brain.
• Cadena, S. A., Sinz, F. H., Muhammad, T., Froudarakis, E., Cobos, E., Walke, E. Y., Reimer, J., Bethge, M., Tolias, A. S., and Ecker, A. S. (2019). How well do deep neural networks trained on object recognition characterize the mouse visual system? In Real Neurons & Hidden Units NeurIPS Workshop.
• Chemero, A. (2009). Radical Embodied Cognitive Science. The MIT Press, Cambridge, MA. Cichy, R. M., Roig, G., Andonian, A., Dwivedi, K., Lahner, B., Lascelles, A., Mohsenzadeh,
• Y., Ramakrishnan, K., and Oliva, A. (2019). The Algonauts Project: A Platform for Com- munication between the Science of Biological and Artificial Intelligence. In Computational Cognitive Neuroscience.
• Devezer, B., Nardin, L. G., Baumgaertner, B., and Buzbas, E. O. (2019). Scientific discovery in a model-centric framework: Reproducibility, innovation, and epistemic diversity. PLoS ONE, 14(5):1–23.
• Diedrichsen, J. and Kriegeskorte, N. (2017). Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis. PLoS Computational Biology, 13(4).
• Elsayed, G. F., Shankar, S., Cheung, B., Papernot, N., Kurakin, A., Goodfellow, I., and Sohl-Dickstein, J. (2018). Adversarial Examples that Fool both Human and Computer Vision.
• Gelder, T. v. (1998). The Dynamical Hypothesis in Cognitive Science. Behavioural and Brain Sciences, 21:615–665.
• Gerven, M. v. and Bohte, S. (2017). Editorial: Artificial Neural Networks as Models of Neural Information Processing. Front. Comput. Neurosci., 11(114).
• Goodfellow, I. J., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings.
• Goyal, A., Lamb, A., Hoffmann, J., Sodhani, S., Levine, S., Bengio, Y., and Schölkopf, B. (2019). Recurrent Independent Mechanisms.
• Guerguiev, J., Lillicrap, T. P., and Richards, B. A. (2017). Towards deep learning with segregated dendrites. eLife, 6(e22901).
• Jonas, E. and Kording, K. P. (2017). Could a Neuroscientist Understand a Microprocessor? PLoS Computational Biology, 13(1).
• Kell, A. J. and McDermott, J. H. (2019). Deep neural network models of sensory systems: windows onto the role of task constraints. Current Opinion in Neurobiology, 55:121–132.
• Lillicrap, T. P., Cownden, D., Tweed, D. B., and Akerman, C. J. (2014). Random feedback weights support learning in deep neural networks. page 14.
• Lillicrap, T. P. and Kording, K. P. (2019). What does it mean to understand a neural network? (July):1–9.
• Lindsay, G., Moskovitz, T., Yang, G. R., and Miller, K. (2019). Do Biologically-Realistic Recurrent Architectures Produce Biologically-Realistic Models? pages 779–782.
• Lindsay, G. W. and Miller, K. D. (2018). How biological attention mechanisms improve task performance in a large-scale visual system model. eLife, 7:1–29.
• Love, B. C. (2019). Levels of Biological Plausibility. PsyArXiv Preprints.
• Marcus, G. (2018). Deep Learning: A Critical Appraisal.
• Marder, E. (2015). Understanding Brains: Details, Intuition, and Big Data. PLoS Biology, 13(5):1–6.
• Marr, D. and Nishihara, H. K. (1978). Representation and recognition of the spatial or- ganization of three dimensional shapes. Proceedings of the Royal Society of London B: Biological Sciences, 200(1140):269–294.
• Mehrer, J., Spoerer, C. J., Kriegeskorte, N., and Kietzmann, T. C. (2020). Individual differences among deep neural network models.
• Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., and Mordvintsev, A. (2018). The Building Blocks of Interpretability.
• Ponce, C. R., Xiao, W., Schade, P. F., Hartmann, T. S., Kreiman, G., and Livingstone, M. S. (2019). Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences. Cell, 177(4):999–1009.
• Raghu, M., Gilmer, J., Yosinski, J., and Sohl-Dickstein, J. (2017). SVCCA: Singular Vector Canonical Correlation Analysis for Deep Understanding and Improvement. NeurIPS.
• Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., Gillon, C. J., Hafner, D., Kepecs, A., Kriegeskorte, N., Latham, P., Lindsay, G. W., Miller, K. D., Naud, R., Pack, C. C., Poirazi, P., Roelfsema, P., Sacramento, J., Saxe, A., Scellier, B., Schapiro, A. C., Senn, W., Wayne, G., Yamins, D., Zenke, F., Zylberberg, J., Therien, D., and Kording, K. P. (2019). A deep learning framework for neuroscience. Nature Neuroscience, 22(11):1761–1770.
• Salmon, W. C. (1984). Scientific explanation and the causal structure of the world. Princeton University Press.
• Santoro, A., Raposo, D., Barrett, D. G., Malinowski, M., Pascanu, R., Battaglia, P., and Lillicrap, T. (2017). A simple neural network module for relational reasoning. Advances in Neural Information Processing Systems, 2017-Decem(Nips):4968–4977.
• Schölkopf, B. (2019). Causality for Machine Learning. pages 1–20.
• Storrs, K. R. and Kriegeskorte, N. (2020). Deep Learning for Cognitive Neuroscience. In
• Poeppel, D., Mangun, G. R., and Gazzaniga, M. S., editors, The Cognitive Neurosciences. MIT Press, 6th edition.
• Xu, Y. (2020). Limited correspondence in visual representation between the human brain and convolutional neural networks.
• Zador, A. M. (2019). A critique of pure learning and what artificial neural networks can learn from animal brains. Nature Communications, 10(3770).

Michael Silberstein (2002) Reduction, Emergence and Explanation, in The Blackwell Guide to the Philosophy of Science

Two types of reduction:

• Ontological reduction: everything in the world can be reduced to or determined by some irreducible fundamental constituents. Has to do with real world items: entities, events, properties, etc. which can be linked via elimination, idenitity, mereological supervenience (part-whole relations), nomological supervenience/determination.
• Epistemological reduction: scientific theories and laws about the world at a macroscopic level can be reduced to or identified with more fundamental theories and laws. Has to do with representational items: theories, concepts, models, frameworks, schemas, regularities, etc. which can be reduced via replacement, theoretical-derivation, semantic/model-theoretic/structuralist analysis, or via pragmatic approaches. For example, the attempt to reduce thermodynamics to statistical mechanics or the attempt to reduce chemistry to quantum mechanics as failed or incomplete cases of intertheoretic reduction.

Reductionism: belief that both ontological and epistemological reduction is true. Phenomena at one level can be explained by the activities and relations among component parts at a lower level. The goal of science is to map all phenomena onto the ‘real’ fundamental ontology of the world, governed by the most fundamental theories. It is through this process that scientist unify natural phenomena by understanding how diverse sets of activities are governed by the same small number of fundamental theories.

Both types are related: “We would like to believe that the unity of the world will be described in our scientific theories and, in turn, the success of those theories will provide evidence for the ultimate unity and simplicity of the world; things are rarely so straightforward.” (Silberstein p. 81)

Emergentism: Rejection of both types of reductionism. There is no fundamental ontology. The whole is greater than the sum of its parts. Phenomenological theories cannot be reduced to or identified with more fundamental ones. The “best understanding of complex systems must be sought at the level of the structure, behavior and laws of the whole system and that science may require a plurality of theories (different theories for different domains) to acquire the greatest predictive/explanatory power and the deepest understanding.” (Silberstein p. 81)

Emergentism popular today: “emergent property”, “emergent phenomenon”, lots of examples of observations that are not well explained by reduction to properties and activities of component parts.

This perhaps over-simplified account assigns science’s goal to unify in the reductionist camp and the commitment to explanatory and theoretical plurality with the emergentist camp. But it seems to me that these two aspects need not be completely at odds with each other.

“Emergentism and reductionism might form a continuum and not a dichotomy” (p. 99) e.g. causal mechanical perspective rejects microreduction, says instead that explanatory mechanisms are multi-level, “emphasizing the gradual, partial and fragmentary nature of many real world cases.””

Context-specific strategies for reduction, rather than universal. Good reason to believe “unification of scientific theories will be local as best”— a nested hierarchy of theories rather than a pyramid.

Although some would still say that it is just our ignorance that prevents us from unification. (I’m not convinced)

Miłkowski, M., & Hohol, M. (2020). Explanations in cognitive science: Unification versus pluralism. Synthese.

Special issue in Synthese on Explanations in cognitive science: unification versus pluralism. “Does cognitive science need a grand unifying theory? Should explanatory pluralism be embraced instead? Or maybe local integrative efforts are needed? What are the advantages of explanatory unification as compared to the benefits of explanatory pluralism?”

Original motivation for the interdisciplinary field of cognitive science: multiple research perspectives would be required to explain cognitive phenomena. Founders of cogsci were largely pluralistic in their views on explanation, but advocated for a unified research discipline, i.e. a single object of study but pluralistic methodology.

Two, non-exhaustive types of unification:

• Reductive unity: that all sciences should be reduced to one grand unifying theory, gained some popularity in the 60s. (Carnap 1928, Oppenheim and Putnam 1958). (Can safely reject this at this point)
• Integrative coordination: Scientists from multiple fields should coordinate, gathering evidence from multiple sources but ultimately building integrated models (Neurath 1937, Potochnik 2011) (I’m still on board with this one)

Problems for the received view of reduction in cognitive science:

• Multiple realizability: “the kinds in the theory to be reduced cannot be neatly identified with (collections of) the kinds in the reducing theory, in which there are usually multiple possible kinds that could realize the kinds of the theory to be reduced”
• received view requires theories to take the form of propositional logic and law like statements (and typically assumes a variant of the deductive nomological theory of explanation which states that explanations are deductive arguments). Law like statements, generally defined as exceptionless regularities, are rarely found in cognitive science. Instead, theories in cogsci “are sometimes stated in terms of formal specifications in artificial languages, not all of which are declarative; some are stated in a mixture of diagrams and verbal descriptions, and some as software” (Cooper and Guest 2014) so we cannot derive one theory from another (as the received view of reductionism would have us do)

Integrative coordination

Coordinated or integrated fields, theories, or explanations are not fully autonomous, e.g. psychological theories are constrained by biological facts. Example: Cognitive neuroscience as the integration of cognitive psychology and neuroscience, searching for the neural mechanisms underlying cognitive phenomena.

Carl Craver’s ‘Mosaic Unity of Neuroscience’ view

Craver proposes that good explanations in neuroscience describe constitutive mechanisms, span multiple levels and integrate multiple fields. “mechanistic explanations force piecemeal integration” because

• describing constitutive mechanisms involves identification of the component parts, and how their organization and activities realize the phenomenon to be explained
• various types of evidence may be brought to bear on complex explanations of the phenomena that are realized by neural mechanisms “The mechanistic account of integration is compatible with non-extreme versions of explanatory pluralism.”

Explanatory pluralism is default in cog sci, and this can go hand in hand with integrative efforts or not (as in isolationist pluralism and eliminativist pluralism)

Unifying cognitive science

Alan Newell: “without unification, cognitive science will cease to be cumulative” In 1973, he wrote the article “You can’t play 20 questions with Nature and Win” in which he questioned where current trends in experimental psychology would lead. He finds a large chasm between the general concerns of high level theories and the day-to-day decisions of psychological experiments which occur at lower levels. He makes the observation that much of what the field of experimental psychology accomplishes is description of increasingly long lists of psychological capacities and phenomena—cataloguing psychological effects—without much attempt to unify these phenomena or to build theories that would encompass them. He sees this as a crisis, that, if left unchecked, would leave the field to stagnate. Later (1990), he published Unified Theories of Cognition, where grand unifying theories were supposed to be the cure for the crisis of non-progress he observed in psychology, but without an appeal to reduction. This is perhaps a third sense of unification that is more about understanding and explanation rather than reduction or integration. The alternative strategies proposed in cogsci might take the form of:

• building a theory of entities that are responsible for all observed phenomena (defended in mainstream cogsci, e.g. unified theories of cognition that are theories about mechanisms or representations underlying all of cognition),
  • Difficult to build and also to test.
• or study general principles that govern them, appeal to grand principles:
  • e.g. predictive coding, negative feedback, classical cognitivism (compuationalism)
  • Need not cover all cognitive phenomena
  • problems:
    • does not suffice to explain individual phenomena, does not offer local misunderstanding

Unification and pluralism in practice

“For defenders of grand principles, simplicity and the universal scope of a theory may be more important than its evidential support. Integration seems more related to evidential support and consistency instead. The defenders of the new mechanistic approach to explanation, for example, usually treat generality as an optional feature of scientific explanation (Craver 2008). Thus, a defender of integration may proceed in a different fashion than a defender of unificatory strategies.” (p. 9)

Unification, defined by dimensions like simplicity, generality and scope, and systematicity, still constitutes a notable virtue of research traditions, even if attempts for grand unifying theories fail to encompass all cognition.

One approach has been to build models that are able to predict neural activity while an animal is experiencing some task or stimuli. Traditionally, a model would be designed given what is already known about the system of interest and the researchers hypotheses about neural function. Modern DNNs, on the other hand, though originally inspired by biological neural networks, were designed to solve computer vision problems independent of any knowledge or specific hypotheses about neural function. That such networks trained to recognize objects in images learned representations that were similar to those found in the primate ventral stream caused much debate in the neuroscience community. Jim DiCarlo, one of the leading researchers in this area, has described his approach as turning the scientific problem of neuroscience into an engineering one where the primary goal is to optimize the accuracy of predictive models (Di-Carlo, 2018). Competitions such as the Algonauts project (Cichy et al., 2019) and BrainScore (Schrimpf et al., 2018) seek to identify the models that achieve the best score on standardized tasks, akin to engineering competitions such as Kaggle. The approach of ‘Predict, then Simplify’ prescribes first building a predictive model and then trying to explain why it successful (Kubilius, 2017). Critics of this approach have essentially claimed that these DNN similarity results don’t count as scientific progress, at least not in the same way that the results of traditional modeling studies do, because the models them selves are “uninterpretable” or biologically implausible (Kay, 2017). The approach has been also criticised as replacing one black box with another, i.e., modeling one thing we don’t understand with another that we also don’t understand (Middlebrooks, 2019). This criticism implies that no scientific progress has been made. How can the predictive performance of representations learned in DNNs tell us anything about the brain when they don’t encode specific hypotheses about neural function?

Many of the conversations alluded to above start by asking what would it mean to understand the brain?: e.g., the paper, “What does it mean to understand a neural network?” by Lillicrap and Kording (2019) or the Challenges and Controversies session, “What it would mean to succeed at understanding how cognition is implemented in the brain” at the 2018 conference on Computational Cognitive Neuroscience. Are these the right questions to be asking if we’re concerned with how our science progresses? In philosophy, understanding is viewed as a type of personal, cognitive achievement (Grimm and Hannon, 2014). When an individual comes to understand a language, another person, a proof or a scientific theory, it is a personal achievement. What role does understanding play in scientific progress? As scientists, are we not more interested in the products of scientific enterprises than in the cognitive achievements of individuals?

To begin, we assume that science eventually makes progress towards its goals. This doesn’t imply that science proceeds directly to its goals or that all its goals are achievable, but it does imply that the activity of doing science is more than just going in circles—that science moves towards something. Some applied sciences will have very clearly defined goals, such as improving patient outcomes for medical research. In the absence of clear applied research goals, many contemporary scientists and philosophers would agree that one the primary goals of fundamental science, of science for science’s sake, is to explain, i.e., to provide explanations of the phenomena that are the focus of scientific study.

So then, if we want to understand how fundamental science progresses, we need to know what constitutes a scientific explanation. This has been a central question for philosophers of science over the past century. The goal of this line of inquiry is to identify the norms of scientific explanation. In the analytic philosophy tradition, this amounts to identifying the necessary and sufficient conditions under which something may be deemed a successful scientific explanation.

Around the mid twentieth century, philosophers who tackled this questions ought to identify a universal and objective logic of scientific explanation us-ing what has been called an a priori approach, which reflects the belief that philosophy of science can pass judgement on and discern the rules of science from a non-scientific view point. Philosophers looked to physics as the model science and tried to develop general theories of explanation that would account for all scientific explanations. As will be discussed in more depth later, proposals included that explanations are deductive arguments based on laws of nature(deductive-nomological model) and that explanations are collections of statistical relevance relationships (statistical relevance model). These works viewed science as providing an objective window into truth and assumed that it must have a clear and universal set of rules. This enterprise is largely considered to have failed to achieve its goal as each theory has a number of counter examples for which it is unable to account.

In contrast, the field of science studies tackles the subjective aspects of science. Science is performed by bias-laden humans in a particular social and historical context which will affect what questions get asked, how science is per-formed, and how its results are presented and received. An extreme view on explanation within this tradition is that scientific explanation is simply what-ever scientists find to be explanatory at a given time and place—that there is no objective notion of explanation independent from its social and historical con-text. This view is unsettling to many scientists and philosophers who want to believe that there is something special about science compared to other ways of knowing. How can we theorize about science in general if everything is relative?Wesley Salmon writes,

First, we must surely require that there be some sort of objective relationship between the explanatory facts and the fact-to-be-explained. . . Second, not only is there the danger that people will feel satisfied with scientifically defective explanations; there is also the risk that they will be unsatisfied with legitimate scientific explanations . . . The psychological interpretation of scientific explanation is patently inadequate (Salmon, 1984, pg. 13).

Similarly, Carl Craver writes, “All scientists are motivated in part by the pleasure of understanding. Unfortunately, the pleasure of understanding is often indistinguishable from the pleasure of misunderstanding. The sense of understanding is at best an unreliable indicator of the quality and depth of an explanation” (Craver, 2007, pg. 21).

The philosopher of science assumes that explanations of natural phenomena exist, regardless of whether scientists are ever satisfied with them. In the later part of the twentieth century, after several failed attempts at a universal theory of scientific explanation, a naturalized philosophy of science emerged, where naturalized here refers to subsumption under the natural sciences. This naturalization is attributed in part to W.V.O. Quine who proposed that the project of epistemology is actually the “scientific inquiry into the processes by which human beings acquire knowledge” (Bechtel, 2008). Naturalized philosophy rejects the a priori approach and instead employs an interdisciplinary approach including methods from history, anthropology and psychology. This rejection of the a priori approach does not imply abandoning the normative goals of philosophy of science. This new philosophy of science has both descriptive and normative goals where a descriptive goal (how science is) maybe considered the first step towards a normative goal (how science ought to be) (Craver, 2007). Historical explanations can be evaluated on pragmatic terms; we can look at what an explanation facilitated—what new technology, medicine, control or future research was enabled by the explanation? A naturalized philosophy of explanation still assumes that there are good and bad explanations but uses different methods to go about elucidating the difference.

The naturalized approach acknowledges the context-specific elements of scientific progress and recognized that not all sciences look like physics. One consequence of adopting the naturalist perspective is the recognition that not all sciences are the same. Many of the traditional philosophical ideas about science, whether developed by philosophers pursuing the a priori approach or the naturalist approach, were most applicable to domains of classical physics. But, starting in the 1970s and 1980s, certain philosophers who turned their attention to the biological sciences found that these frameworks did not apply all that well to different biological domains. Recognizing this, the naturalist is committed to developing accounts that work for specific sciences, postponing the question of determining what is in common in the inquiries of all sciences. In this sense the naturalist is led to be a pluralist (Bechtel, 2008). Pluralist here refers to someone who accepts that there may be many forms of explanation that are equally valid in their own domains.

Neuroscience, being highly interdisciplinary, is not clearly separated from its sister sciences: biology, physics, psychology, artificial intelligence (AI). Thus, the search for a universal theory of explanation in neuroscience may be as ill-fated as the original quest of a theory of explanation for all science. We propose instead, that explanations are phenomenon-specific. Classes of phenomena, regardless of which domain of science they belong to, will share a particular form of explanation.

This view can be especially unifying at the intersection of neuroscience and AI where the behaviour of artificial systems are designed to mimic human and animal behaviour. Thus, to the extent that an artificial system and a biological system display the same phenomenon (e.g. recognizing faces), the form of the explanation of that phenomenon will be the same. To be clear, the previous statement does not imply that the explanations themselves will be the same, but that what constitutes an explanation in both cases will be the same (to the extent that they demonstrate the same phenomenon).

Within this framework, it becomes very important to clearly identify and delineate the phenomena to be explained. How a scientist conceptualizes the phenomenon to be explained may bias them towards one form of explanation or another. This is especially apparent in cognitive neuroscience which deals with the difficult task of “understanding how the functions of the physical brain can yield the thoughts and ideas of an intangible mind” (Gazzaniga et al.,2014). In the philosophy of mind, there are many different perspectives on the nature of mind and cognition. Is cognition computation? Are cognitive agents embodied dynamical systems? We cannot completely separate the ontological question (What is cognition?) from the epistemological question (How do we explain cognition?). Thus, a commitment to a particular theory of explanation in neuroscience may also suggest a related commitment to a theory of cognition.

What are the philosophical questions most relevant to contemporary debates at the intersection of AI and neuroscience? If we’re interested in how our science progresses, then we’re interested in explanation rather than understanding. We may colloquially use ‘understand’ to stand in for ‘explain’ or to refer generally to the acquisition of knowledge, but the difference between the terms is important in formal arguments. Thus, when Lillicrap and Kording (2019) use ‘understanding’ consistent with its accepted definition in philosophy to refer to a personal achievement state, their arguments do not imply anything about scientific explanation. They emphasize compactness and compressibility in their proposal for what it means to understand a neural network because humans are only able to argue about compact systems. According to their view, any meaningful understanding of a neural network must be compressible into an amount of information that a human can consume, e.g.a textbook. They and many others often phrase the question as being about how to understand or explain ‘the brain’ or ‘a neural network’, but science isn’t in the business of explaining objects. Science may produce descriptions or characterizations of objects, but explains phenomena. Since the form of a satisfactory explanation may depend on the phenomena to be explained and the fact that the brain participates in a plethora of phenomena at many different scales, we will likely never arrive at a unitary answer for what it means to explain ‘the brain’. If we are concerned with how a new science of intelligence might progress, let us not ask how to understand the brain. Let us ask what are the specific phenomena of interest at the intersection of artificial and biological intelligence and how ought such phenomena be explained? The inability of the human cognitive system to simultaneously conceptualize 100 trillion synapses may indeed inform our decisions as scientists, but it need not constrain our theories of the explanation of neural phenomena.

Great caution must be exercised when we say that scientific explanations have value in that they enable us to understand our world, for understanding is an extremely vague concept. Moreover—because of the strong connotations of human empathy the word “understanding” carries—this line can easily lead to anthropomorphism. (Salmon, 1989)

We assume that an explanation (or multiple explanations) exist, regardless of whether a human scientist can ever grasp it.

I recently presented a poster entitled ‘The effect of task and training on intermediate representations in convolutional neural networks revealed with modified RV similarity analysis’ at the Conference on Cognitive Computational Neuroscience (CCN) in Berlin, Germany.

The modified RV coefficient (RV2) is a particular case of centered kernel alignment (CKA) which was recently discussed at length in an ICML workshop paper by Simon Kornblith, who also gave a presentation at ICLR2019.

We used RV2 to compare activation patterns in networks of identical architecture trained in different ways on a transfer learning task. In particular, we investigated why freeze training achieves superior transfer performance, above and beyond similar transfer networks where only the final layers are trained on the target task.

My CCN paper is now on arXiv

Despite a desire to participate in developing open-source software and develop good open science practices, my PhD has involved a lot of proprietary and closed datasets and software due to who I was working with. This summer I have been returning to some fMRI data that I collected ages ago but haven’t gotten around to analyzing due to other projects. Here I want to summarize my experience trying to adopt some better open-science practices and using new tools and standards. I’m coming to this as a relatively seasoned python programmer and data analyzer but having never used these specific tools. I hope this could be useful for someone starting from scratch or someone interested in a newcomer’s experience with these tools.

I did this on my local desktop running Debian 9.9.

Main components:

• Brain Imaging Data Structure (BIDS)
• Datalad
• fmriprep
• nistats/nilearn

Also mentioned:

• Freesurfer
• dcm2niix
• jo
• jq
• PsychoPy
• mriqc
• nipype
• Docker

Step 1: Create your conda environment(s)

I ended up needing to setup two environments, one for BIDS/fmriprep and one for nilearn/nistats due to some conflicting version requirements.

Step 2: Rename Dicoms

Every fMRI analysis package has their own version of this tool which goes through the dicom files as retrieved from the scanner and organizes them into folders with one folder for each acquisition (e.g. T1w, run1). I used freesurfer’s dicom-rename.

Step 3: Create BIDS formatted Datalad dataset

I wanted to use Datalad to version control any changes that I make to the data and also because I ultimately want to share this dataset. I followed the tutorial here which links to a bash script to convert dicoms to nifti files in BIDS data structure format. In retrospect, I should have followed this one which would have made my life easier. My bash scripts here.

Step 3.1: Install Datalad

Step 3.2: Install Dependencies

Both of the tutorials listed above ultimately use dcm2niix to convert dicoms to Nifti files.

1. jo git clone git://github.com/jpmens/jo.git cd jo autoreconf -i ./configure make check make install
2. jq sudo apt-get install jq
3. dcm2niix conda install -c conda-forge dcm2niix

Step 3.3: Create Nifti files and associated metadata (.json) files

Like I said above, I did this with a [bash script](https://github.com/thompsonj/quilts/blob/master/fmri_data/toNifti.sh but wish I had done it in python with HeuDiConv.

Step 3.4: Create Event files and associated metadata (.json) files

I did this with a bash script that processed the logfiles saved during my experiment which I ran with PsychoPy.

Step 3.5: Verify BIDS validity and add additional info

BIDS specifies how to store information about your participants, tasks and scans in .tsv and .json files.

I used the online BIDS validator to verify everything was named and organized correctly. The validator found a number of errors that I had to correct which helped me to better understand the BIDS format.

Step 3.6: Put the BIDS data into Datalad datasets

If I had used HeuDiConv, I think this could have happened automatically. Instead, I created my Datalad dataset as I converted the dicoms. I honestly didn’t find the Datalad documentation very clear on what the conventions are for organizing a large Dataset, but gathered somewhat indirectly that the idea is to have several nested Datalad datasets. For example, the highest level folder containing your BIDS dataset would be the highest level Datalad dataset, within which each subject folder ‘sub-' would be it's own Datalad dataset. If data were collected over more than one sessions, each session directory 'ses-'. I also made my stimuli directory it's own Datalad dataset. This compartmentalization may make the dataset easier to work with and version control in the future.

Create nested Datalad datasets with datalad create

For example:

cd speechQuiltsfMRI
datalad create -d . sub-1
datalad create -d . sub-2
datalad create -d . sub-3
datalad create -d sub-1 ses-1
datalad create -d sub-1 ses-2
datalad create -d sub-2 ses-1
datalad create -d sub-2 ses-2
...

My dataset structure looks like this:

I used datalad add to add directories/files to a dataset. I just noticed that the Datalad documentation says: “Note: This is an obsolete interface. Use datalad.api.save or Dataset.save instead.”, but I don’t see how I would be able to do create these datasets without datalad add.

Save datalad dataset:

datalad save -r -m 'initialize BIDS compliant dataset' .

When all files and directory are added to and saved, running datalad diff -r -d <dataset> should print nothing.

Step 4: Quality Control

At this point I considered using mriqc to perform some quality assurance tests, but for now I skipped this step and went straight on to preprocessing.

Step 5: Preprocessing with fmriprep

fmriprep is built on nipype which “provides a uniform interface to existing neuroimaging software and facilitates interaction between these packages within a single workflow”. fmriprep munges a BIDS dataset, so as long as your data is organized according to BIDS, you can specify your entire preprocessing pipeline with a single call to fmriprep.

Step 5.1: Install Docker and fmriprep-docker

At first I resisted using the Docker container because I am oldschool and like to actually install the software that I’m using. I had many of the dependencies installed already so I thought it would be fine, but quickly ran into runtime issues that I couldn’t easily debug, so I succumbed to using the Docker container.

I followed these instructions for installing Docker including the post-installation steps for Linux which includes how to manage Docker as a non-root user.

Follow this tutorial for the fmriprep docker image.

Aside: I initially missed the line in that tutorial where it says “1) To do the susceptibility distortion correction, the IntendedFor metadata keys need to be defined for every field map that is to be applied.” and since the BIDS specification said that the IndendedFor field was optional, I initially excluded it, assuming that fmriprep would automatically determine that the files in sub-x/ses-y/fmap where intended for all niftis found in sub-x/ses-y/func. That was not the case and in order to get the PE Polar distortion correction to work, I needed to add the IntendedFor field to each .json file in my fmap folders.

Step 5.2: Run fmriprep-docker and save changes the Datalad datasets

I thought that I should be able to use datalad run to call the fmriprep-docker command, which ostensibly should automatically log any changes to the Dataset dataset and the command that led to those changes. So I tried running something like this:

datalad run -d /data1/quilts/speechQuiltsfMRIdata "fmriprep-docker /data1/quilts/speechQuiltsfMRIdata /data1/quilts/speechQuiltsfMRIdata/derivatives participant --participant_label 1 --nthreads 8 --omp-nthreads 16 --mem-mb 64000 -vv -w /data1/quilts/fmriprep_work --resource-monitor --fs-license /usr/local/freesurfer/license.txt"

fmriprep ran but the changes to the Datalad dataset could not be saved automatically because the outputs of fmriprep are saved with limited permissions. I’m not sure I fully understand the philosophy of the restricted permissions…I guess it to prevent users from making unintentional modifications since the idea is that the resulting derivatives should be fixed once created. It sort of backfires in this case where I’m unable to use datalad run. I found this comment on Neurostars which describes how to run the fmriprep docker container under one’s user name instead of root. In my case, I just ran frmiprep-docker directly followed by sudo chown -R <user> <dataset> and datalad save -r -d <dataset> instead.

I experienced some confusion related to how to organize the reference images that I collected for distortion correction. I originally thought that since they were collected with the same scanner protocol as the rest of my runs, that they should have the ‘bold’ suffix. I later realized that they should have the ‘epi’ suffix. Reading the BIDS specification section on fieldmap data I was also perplexed by the reference to Spin Echo EPI scans since, as far as I know, PEPOLAR distortion correction functions like TOPUP also work with Gradient Echo EPI scans. Why the specification of Spin Echo?

I inspected the reports generated by fmriprep to verify what was performed. I love these reports I just wish they were easier to share. I tried to send the html and corresponding images to my advisor but it seemed like the paths to the images were absolute rather than relative.

Step 6: First-level GLM model in nistats

Once the preprocessing was complete, my next goal was to do a simple sanity check to make sure that we see activation in auditory cortex for my auditory stimuli. I decided to use nistats which ostensibly should be able to automatically specify a GLM from preprocesed files in a BIDS dataset. Unfortunately, it didn’t work out of the box on the derivatives saved by fmriprep.

Issues

1. nistats uses different BIDS conventions than fmriprep

   Nistats was looking for files named according to older conventions than what was output by fmriprep. This problem was mentioned in an open issue on github.

   I created a fork of nistats and updated the relevant functions involved in creating a first level model from a BIDS dataset to use the latest conventions. I eventually submitted these changes as a pull request but it is still in progress. Eventually, it would make sense to do a more major overhaul of nistats to use pybids for this kind of thing.

   Note: The current official BIDS specification doesn’t cover derivatives other than to say that derivatives should be in a folder called ‘derivatives’ and should not share file names with their corresponding raw/unprocessed files. However, there is a BIDS Extension Proposal to cover derivatives which has been evolving gradually.

2. fmriprep Confounds fmriprep automatically prepares a number of nuisance variables and saves them in a ‘desc-confounds_regressors.tsv' file. Some of these confound variables are derivatives (calculus) and so are one element shorter than their counterparts, resulting in a leading NaN for these variables. This led to errors when nistats tried to automatically put these confounds into a design matrix. For now I just omitted these confound variables. One could easily omit only the confound variables including NaNs. Probably a better solution would be to replace the leading NaN with the mean of the variable.

3. Missing events in some runs In my experiment, there are 0–3 button presses in every run. Because some runs have no button presses, the design matrix constructed for those runs had one fewer column than the others. This made it impossible to calculate any contrasts. I tried adding a column of zeros to the design matrices for runs without button presses, but this also led to issues. I didn’t finish debugging this problem but it seemed related to this open issue on github. In my case I simply omitted the button presses from the events files since I didn’t care so much about them. This analysis is only a sanity check after all.

With these modifications, I was able to fit first level models and compute basic contrasts like the ones shown below. It looks like my subjects heard the sounds I played for them!

Summary of issues encountered

1. Couldn’t use datalad run to automatically track the effect of running fmriprep due to permissions issues
2. Confusion about how to organize my epis in opposite phase encoding directions to enable Phase Encoding POLARity (PEPOLAR) susceptibility distortion correction in fmriprep.
3. Difficulty sharing the reports generated by fmriprep
4. nistats uses different BIDS conventions than fmriprep
5. nistats doesn’t deal with NaNs in the confounds prepared by fmriprep
6. nistats doesn’t handle different design matrices for different runs

Brette claims that the neural coding metaphor “cannot constitute a valid basis for theories of brain function because it is disconnected from the causal structure of the brain and incompatible with the representational requirements of cognition” (p. 6). This criticism reflects long-standing debates about how to conceptualize and explain brain and behaviour. In this commentary, we will expand on the philosophical commitments of the neural coding paradigm and its alternatives with the goal of bringing improved conceptual clarity to the discussion. In particular, we will describe how theories of mind and theories of cognition are related to theories of explanation in neuroscience. How a scientist conceptualizes a phenomenon will partly determine what constitutes a satisfactory explanation of that phenomenon. The neural coding paradigm centers the brain as an information processing system. This makes it well-suited to functionalist theories of explanation, which explain complex phenomena by analyzing the function of sub-phenomena. Brette, on the other hand, is concerned with identifying causal mechanisms. According to causal mechanistic theories of explanation in neuroscience, neural phenomena are explained by identifying the physical mechanisms whose components and their causal relationships produce the phenomenon to be explained. Thus, Brette’s criticism can be reframed as a criticism of functionalist theories of explanation, on which much has been written. Brette’s closing section, “What else, if not coding?”, reflects the contemporary imprecision in neuroscience about what are the phenomena to be explained (what is mind? what is cognition? what is the brain? what is information?) and how such phenomena ought to be explained. Rather than prescribing a particular theory of explanation, we suggest that neuroscience will enjoy improved clarity of thought by understanding how different conceptual frameworks lend themselves to different theories of explanation. Additionally, it is possible that none of the existing theories of explanation reflect the types of explanations that many areas of neuroscience will ultimately seek to produce. In order to make progress, the field may need to first acknowledge this theoretical gap.

Abstract:

Characterization of the representations learned in intermediate layers of deep networks can provide valuable insight into the nature of a task and can guide the development of well-tailored learning strategies. Here we study convolutional neural network-based acoustic models in the context of automatic speech recognition. Adapting a method proposed by Yosinski et al. [2014], we measure the transferability of each layer between German and English to assess the their language-specifity. We observe three distinct regions of transferability: (1) the first two layers are entirely transferable between languages, (2) layers 2–8 are also highly transferable but we find evidence of some language specificity, (3) the subsequent fully connected layers are more language specific but can be successfully finetuned to the target language. To further probe the effect of weight freezing, we performed follow-up experiments using freeze-training [Raghu et al., 2017]. Our results are consistent with the observation that CCNs converge ‘bottom up’ during training and demonstrate the benefit of freeze training, especially for transfer learning.

TL;DR:

All but the first two layers of our CNNs based acoustic models demonstrated some degree of language-specificity but freeze training enabled successful transfer between languages.

Keywords:

CNNs, acoustic modeling, interpretability, transfer learning, language-specificity, freeze training