CHRISTIAN J. TARSNEY: (00:07) So, a lot of people in the EA community think that if we want to do the most good with a given unit of resources, we should focus primarily on improving the very long-run future, on positively shaping the trajectory of human civilization over timescales of thousands or millions or even billions of years. This view has come to be known as longtermism. The basic argument for longtermism isn't terribly subtle. In short the far future is potentially really big and most of the potential value or disvalue – happiness or suffering, beauty, oppression, and so forth that we might produce as a species or as a civilization – doesn't lie in the next hundred years but in the much further future. But the far future isn't just big, it's also unpredictable. To get a sense of how unpredictable the far future might be, just think about what predictions an intelligent well-educated man or woman, say in the sixteenth century, might have made about the year 2019 and then remember that we are trying to make predictions not on the scale of 500 years, but on the scale of thousands or millions of years. So a lot of people are skeptical of longtermism for this reason. They think that even if the far future is overwhelmingly important, our ability to predict the effects that our present choices will have on the far future is so limited that from our epistemic vantage point we should mainly focus on improving the near-run future, focus on near term considerations.

So the goal of this talk is to survey some considerations both for and against this kind of epistemic challenge to longtermism. More specifically, I'm going to survey three reasons for pessimism about our ability to predict and predictably influence the far future, then three reasons for optimism and I'm not going to try to reach a bottom-line conclusion or make the case either for or against longtermism on the basis of these epistemic challenges. Rather the main takeaway from the talk is going to be that these questions (02:00) are really really deeply underresearched and that there's a lot of room for high-value research contributions on all sorts of questions in this area. So what I'll try to do along the way is highlight some projects that I think would be particularly valuable for researchers who would like to make a contribution here.

Alright. Starting with reasons for pessimism then. The first and maybe the most obvious is a kind of pessimistic induction from our unimpressive historical track record of thinking about the far future. So to begin with, it's just very very hard to find instances of people making accurate non-trivial predictions about the future on the scale of, say more than a century let alone millennia or longer and it's relatively easy to find examples of notable predictive failures, for instance in forecasting the progress of science and technology, say Albert Michelson’s famous prediction in the 1890’s that future progress in physics would come in the “sixth place of decimals”. In other words just measuring the fundamental constants a little bit more precisely. Grand theories of history from thinkers like Karl Marx that just turned out to be wrong. The visions of Utopian social planners who thought that some new social system would take over and revolutionize human civilization.

Now, I'm not sure that we should read too much into these examples for all sorts of reasons. So to begin with, it's just a few cherry-picked data points. Each of them is individually open to dispute. So for instance, I'm not sure that what Michelson was saying is actually as silly as it might sound and we have tools available to us now for making predictions for forecasting that weren't available to people 100 or 200 years ago. But there's also a kind of more empirically robust basis for skepticism about our ability to predict the future from the recent literature on forecasting from researchers like Philip Tetlock, for instance. So Tetlock , in his 2004 book Expert Political Judgement, finds that even very well qualified experts (04:00) making predictions in their field of expertise, say about global politics often do little better than chance at predicting the future or as he puts it little better than dart throwing chimps.

Now here too, I think there are different lessons we could draw and there's definitely a more optimistic reading of this empirical literature on forecasting, but it's not unreasonable for someone to say, look we just have essentially no empirical evidence at all that people are capable of making predictions about, you know, interesting and accurate predictions about the future on timescales longer than a century and we have quite a bit of evidence that even on the scale of a few years, we're not great at predicting how the social world is going to unfold. So why should we believe that we have any meaningful ability at all to predict the future on the scale of say a million years or a billion years? So here in particular I think there is quite a bit of room for valuable research contributions. So for instance putting together a more systematic data set of historical predictions about the far future on a scale of, say more than a century or explicit attempts to influence the far future on timescales longer than a century. Luke Muehlhauser actually, just in the last couple weeks, has a really great write-up on the open philanthropy blog that tries to do this for more sort of medium-term predictions where his criterion for inclusion is predictions longer than 10 years. But there's a lot of room for improvement here and in particular for focusing on these very long-term predictions.

Another thing that I was sort of surprised to find doesn't really exist, is empirical studies of how prediction accuracy decays over time. So if we get people to make sort of qualitatively similar predictions on timescales of 5 years or 10 years or 20 years and we have some measure of accuracy, how does accuracy decay on that measure as we look further and further into the future? If we had more literature like that maybe we could, for instance, find functional forms that would allow us to (06:00) extrapolate outwards over much longer timescales and say something about how accurate we’re likely to be on the scale of thousands or millions of years.

Okay. A second reason for pessimism has to do with what are called “unknown unknowns”. One way to describe the problem is, I think we roughly often have a sort of pretty good idea of how our immediate environment works, but the further we get from our immediate environment in space or in time or in scale, the more likely it becomes that we're missing basic considerations that would radically change our understanding of how the world works. One particularly important kind of “unknown unknown” is new ideas that might come along and either fundamentally change our understanding of how the world already works or when they enable new technologies, might change how the social world in fact works. So to see what I mean, think about someone in, say the year 1800 trying to predict or influence the long-run future without the benefit of ideas like evolutionary theory or computing. Now imagine that we're trying to predict and influence the future on the scale of a million years or a billion years while missing similarly important crucial ideas that might shape either how the world just works over those long timescales or might create kind of exogenous or unpredictable trajectory changes in the social world by introducing new technologies or abilities or memes. So there's a lot of room for research here too. For instance modeling the emergence of these “unknown unknowns” over long timescales thinking about how they either change our understanding of the world or how they change the actual dynamics of the world. One potential way of modeling “unknown unknowns” is by the formal notion of awareness growth. My own suspicion is that awareness growth by itself isn't adequate to capture what's going on when these sort of fundamentally new ideas emerge, but it might be a worthwhile place (08:00) to start.

Okay. The third reason for pessimism is what I'll call “negative correlates of predictability”. So roughly the thought here is that the things that make something high priority from a longtermist perspective are also things that tend to make it unpredictable. In particular, effective altruists often say that what makes a cause area or an issue or a feature of the world high priority is that it's important, neglected and tractable. But things that are important, neglected and tractable also tend to be unpredictable, particularly in the long-term. Why? Well to begin with they're open to human influence. Agents like us can have some effect on them. Right? That's what tractability is. It's likely that other human agents will try to influence it – that's importance. Right? And it's not obvious how or to what extent or in what ways they'll try to influence it because it's neglected. So conversely when we look for features of the world that are predictable over very long timescales, we tend to find examples that conspicuously lack one or more of these features.

So for example, I can predict very confidently that in, say a million years the local supercluster will still contain more than a million galaxies. Why is that predictable? Well among other things even if agents like us wanted to change it, we just don't have the ability. That's not a feature of the world that appears to be open to influence to agents like us. Another example, here's an existential threat that effective altruist don't spend a lot of time thinking about. What if humanity went extinct because everybody just forgot to drink any water for a week? Could happen, but we can pretty confidently predict that it won't happen. Why? Because in this respect human behavior is quite predictable. The cause of people drinking enough water to survive is not particularly neglected. Right? So that gives us (10:00) like one little feature of the far future that we can predict with confidence. Finally suppose that I just go out into the middle of a field in the middle of nowhere and carve my initials into a rock. Possibly, I can predict that those initials will still be there in a very long time say in a thousand years if I carve them deeply enough maybe in a million years, but one thing that makes that potentially predictable is just that it's not very important. So there's no reason to think that any other agent would be motivated to go out and change them, to efface my initials from the rock.

So this is just a sort of thought that's been bouncing around in my head for a couple of months. To my knowledge it hasn't been rigorously explored in the literature, but it might be interesting to think about how general these trade-offs are, how strong they are and to what extent they can be usefully quantified or modeled.

Okay. So we talked about some reasons for pessimism. Now let's talk about reasons for optimism. The first and the most straightforward reason for optimism is that when we try to actually model these epistemic challenges to longtermism, even in a fairly conservative way, in a way that's unfavorable to longtermism, we seem to reach the conclusion that longtermism still does okay. In other words that we can still produce quite a bit of expected value by trying to improve the far future. So there have been a few efforts along these lines in recent years. For instance, Yew-Kwang Ng has a paper in 2016 that does a little bit of this. Tom Sittler has a work-in-progress doing one version of this and I have a paper that does another version. So in my paper, for instance, what I assume is that we're trying to put the world into some desirable target state and we hope that if we put it there it'll stay there for a long time. But there is some probability per unit time that some exogenous event will come along that knocks the world out of the desirable state into a less desirable state. And in my model I make the sort of conservative assumption that that probability is constant over time.

So thinking about existential risks, for instance, we do something that prevents an existential catastrophe (12:00) then we hope that civilization will remain in existence for the indefinite future, but there's always some probability of exogenous extinction events coming along and messing up our plans. Right. And the assumption that the rate of those exogenous events will be constant over time looks quite unfavorable to longtermism because we would generally expect that, for instance as we're settling the stars perhaps, the likelihood of those exogenous events will decrease. But even under that conservative assumption and even making conservative assumptions about the value of other model parameters, what I find is that the expected value of an intervention to reduce existential risk exceeds the expected value of a comparable benchmark short-termist intervention as long as the annual probability of one of these exogenous extinction events is less than about two in 10,000 per year.

So, that's the red number up there on the slide – which as a long-term annual probability of exogenous extinction events seems pretty high. So, this looks like a relatively favorable result for longtermism and other people who tried to do similar things find roughly similar results.

Now again, I don't think we should infer too much here. So, these models still rely on a lot of intuitive judgments. For instance, to find conservative or what we think are conservative lower bounds on parameter values – and they're not capturing everything. So for instance, they're not trying to model “unknown unknowns”. So again, here I think there's a lot of room for new research to improve and generalize these models and to apply them to longtermist interventions other than an existential risk mitigation, which is what all the existing work along these lines has been focused on.

Okay. A second reason for optimism about our long-term influence that's particularly salient in the EA community is the possibility of what we might call “lock-in” events. So a “lock-in” event is an event that can be influenced beforehand. It can be caused or prevented or changed but once it happens it puts the world on (14:00) a trajectory that's very persistent. In other words, very hard or at least very unlikely to be altered over very long timescales. So the standard example of a “lock-in” event in EA is the possible emergence of a superintelligent “singleton”. For instance, an artificial intelligence that is more intelligent and as a result more powerful than any agent in its environment or in human civilization as a whole and as a result it's just able to impose its utility function on the world into the indefinite future.

Now if we expect a “lock-in” event like this to happen in say the next hundred years, that gives us a mechanism for influencing the much further future because if we can cause or prevent or alter the details of that “lock-in” event say 50 years from now and that “lock-in” event determines what the world is going to be like a million years from now, then we have a pretty clear path to influencing the world a million years from now. Now it's not obvious exactly how persistent we should expect these kind of “lock-in” events to be, so for instance, you might think that even a superintelligent “singleton” once it's, say, settled a region of space the size of a galaxy or a galaxy cluster will be vulnerable to say mutations in its code, maybe evolutionary dynamics will take over, other sources of value drift, but it's been at least suggested that a sufficiently powerful agent will have devices for preventing this kind of drift, for instance error correcting codes that will allow it to impose its preferences on the world or more generally to sort of make the “lock-in” persistent over astronomical timescales of billions or even maybe trillions of years. So again, I think there's like some reason here to think that there is a plausible pathway to very long-term influence that only requires us to be able to really model and predict the world on kind of medium timescales.

The third and final (16:00) reason for optimism is the availability of what we might call “robustly positive strategies”. So these are strategies that say… maybe we can't predict the empirical details of the far future but nevertheless there are things that we can do that have a positive influence on the world across a whole range of scenarios including scenarios that we haven't even imagined yet and that therefore produce robustly positive expected value in the far future even from a standpoint of more or less total ignorance about how the future is going to unfold. So standard examples here are, for instance, saving money in a fund that will gain interest and eventually be passed on to future altruists who can use it to improve the world given their superior knowledge of what the future is like or improving institutional decision-making – say getting governments to adopt better voting methods so that whatever challenges we face a thousand years from now will respond to those challenges more rationally. Now these kind of strategies aren't a silver bullet and in particular they still rely on us making some fairly specific predictions about how the future is going to go. So if we invest in a fund that's supposed to gather interest over very long timescales, for instance, we have to predict that that fund isn't going to be expropriated and then just going to end up in the hands of the right agents, people who are going to use it in ways that we regard as valuable and positive. But nevertheless, I think that the availability of these strategies gives us some reason to think, even if we're deeply fundamentally ignorant about how the future is going to unfold, we may nevertheless be able to shape it for the better.

Okay. So we've surveyed some reasons for pessimism and some reasons for optimism but we've only scratched the surface of a very complex and difficult topic. A few of us at GPI over the last year have been trying to think about these epistemic aspects of longtermism and one of the things that's really struck, me at least, is that although there's quite a bit of existing academic research that's relevant in one way or another to these questions, there's really almost nothing that directly addresses the questions that we as potential longtermists are interested in, (18:00) namely the challenges of predicting and trying to influence the future over timescales, say longer than a century. So as I tried to highlight, I think there's a lot of room for novel and valuable research contributions and from quite a wide range of fields. So of course from the perspective and using the methods of the existing academic literature on forecasting, but also from areas like dynamic systems theory, computer science, history, political science, sociology, psychology and GPI’s home disciplines of economics and philosophy and probably a bunch of other disciplines that I forgot to mention.

So if you're interested in doing research in these areas we'd love to hear from you and we'd encourage you to get in touch with us at this expressions of interest form that I've linked to in the slides.

So just to leave you with a few final thoughts: First longtermism may very well be correct notwithstanding these epistemic challenges and if it is correct then it's important enough that we shouldn't just throw our hands up and take these challenges as an excuse to ignore the far future. In fact for whatever it's worth I'll say my own view is that longtermism is very probably correct even though I think these challenges are quite serious and often not taken seriously enough by people who incline towards longtermism.

Secondly, I've emphasized in this talk the kind of either/or question… Is longtermism true or false? But these epistemic considerations are equally relevant to what we might call intra-longtermist prioritization questions. So given that we're trying to influence the far future, what kinds of interventions should we adopt? What effects are most likely to be feasible or most likely to be persistent over very long timescales? Those questions too might be very sensitive to epistemic considerations.

Third and finally, I think with respect to both of these questions –  whether we should be trying to influence the far future and if so how? The most important takeaway from thinking about these epistemic challenges for the moment (20:00) is just the need for quite a bit of epistemic humility. So, if we want to shape the future over the scale of millions or billions of years, we should recognize how incredibly ambitious that is, how big and complicated and unpredictable the future is likely to be and how many ways there are for us to really get it wrong and we  should proceed accordingly. Thank you.

[applause]

QUESTION AND ANSWER

NATHAN LABENZ: (20:24) I wanted to go back to one of your charts.

CHRISTIAN J. TARSNEY: (20:26) Sure.

NATHAN LABENZ: (20:27) And see if we could develop this intuition a little bit more. So this one, you've covered twenty orders of magnitude there. But I didn't quite get it. So I'll confess to that. So help us understand what it's all about.

CHRISTIAN J. TARSNEY: (20:26) Yeah. Yeah. Yeah. So I didn't explain the details of what's going on in the chart. So the left-hand column is the annual rate or probability of these exogenous events which in the context of the model or the working example that I used, applied model to is the probability of exogenous extinction events. So there we're covering (what is it?) something like seven or eight orders of magnitude, right, and then the right-hand column is the expected value of a longtermist intervention that aims to reduce existential risk. So what you see there is that the expected value of longtermist interventions can be extremely sensitive to the annual probability of these kind of exogenous events.

NATHAN LABENZ: (21:22) Okay.

CHRISTIAN J. TARSNEY: (21:23) And I will say, I mean there's a lot of details, of course, that I [inaudible 21:25].

NATHAN LABENZ: (21:25) I'm still trying to work it out a little bit.

CHRISTIAN J. TARSNEY: (21:27) Yeah. Yeah. Yeah. Yeah. There's a lot of details I'm leaving out here but the paper is online if anybody's interested in following up.

NATHAN LABENZ: (21:32) Alright. Cool. So go there for that. I'll have to do that myself. So first question from the app.

CHRISTIAN J. TARSNEY: (21:39) Yeah.

NATHAN LABENZ: (21:39) I think you spoke to this a little bit when you talked about, kind of, you know, if we can effect something now that might have a “lock-in” effect there's kind of a path. But the question is… Is there an important distinction between forecasting the future broadly versus forecasting what we can do to impact the future or the impact of our actions on the future.

CHRISTIAN J. TARSNEY: (21:57) Yeah. Absolutely. So in particular… Well two distinctions I think that are important. One is, there may be features of the far future that are predictable precisely because they're intractables. I tried to highlight on one of the slides. So for instance, the number of galaxies in the universe, right, is quite predictable, but that doesn't give us any ability to predictably change the world, right, because that's not a thing that we can change. The other important distinction is that in order to predict the effects of our actions, we have to make not just a prediction but also a sort of counterfactual prediction about how the world would have gone if we had done something else and that raises a whole different set of challenges about, you know, how to think about counterfactuals, how to evaluate… You know if we can sort of empirically assess how the world in fact went in light of the things that people in fact did, but we're not able to directly empirically study how the world would have gone if people have done things other than what they did and when we're talking about sort of grand changes to the trajectory of history we of course can't do RCT’s, so yeah…

NATHAN LABENZ: (22:56) Yeah.

CHRISTIAN J. TARSNEY: (22:56) I think those are definitely distinct challenges.

NATHAN LABENZ: (22:57) A huge challenge. Philip Tetlock was actually at the EA Global in San Francisco earlier this year and talked a little bit about his work on counterfactual forecasting. They actually have to use the video game Civilization in order to even have some grasp on that problem because it's the only way they can kind of run the counterfactual history. So that is a deep problem.

CHRISTIAN J. TARSNEY: (3:20) And I think it's indicative of where our understanding of these things is right now that playing Civilization is like really genuinely at the cutting edge of research in this field.

NATHAN LABENZ: (23:30) So a couple more questions coming in from the app. What… I guess… Let's start with this one. How plausible do you think these “lock-in” scenarios really are over these super long timescales. That concept itself could be just totally wrong.

CHRISTIAN J. TARSNEY: (23:46) Yeah. Absolutely. So there are details here that very much are not within my domain of expertise like, you know, how error correcting codes work and so I think this is one area where we need people with a diversity of backgrounds to weigh in. I guess my own sort of prima facie view is that we should expect the future to be more complicated and contain more “unknown unknowns” than we intuitively think. So even if there's a model of the world that plausibly does generate these “lock-in” events that are persistent over say trillions of years, we should be very live to the possibility that that model of the world is missing things that are fundamentally important. Now one thing that people have said that I think is true and important is that we don't necessarily need to give very much credence to these kind of extreme “lock-in” models in order to generate a lot of expected value because, you know, if something is persistent over trillions of years then it has just lots of opportunity to gather value.

But that points to another worry that I didn't talk about in the talk but I think is important, namely that one place we could end up as a result of working through these epistemic challenges is at the conclusion that attempts to influence the far future produce an enormous amount of expected value but most of that expected value is coming from a very very small part of the state space. So we have this kind of Pascalian situation where we can produce astronomical expected value but we have only a very tiny probability of having any impact at all. And I think that's a separate challenge for philosophers and economists and decision theorists to wrestle with.

NATHAN LABENZ: (25:24) So to kind of make that a little bit more practical, it seems like there's a pretty well-known pretty short list of things that often come up at events like this. Like let's make sure an asteroid doesn't hit the earth and wipe us all out and, you know, a handful more. Right? Is that kind of where this ultimately gets focused for you pretty quickly or do you see possibility of people going beyond that kind of, you know, first five attempts at intervention and actually having some confidence that they're doing some good.

CHRISTIAN J. TARSNEY: (26:00) I think we've been thinking about this for such a short time that we shouldn't be particularly confident that we've identified the best interventions. My own sort of initial thought is that if we want to generate a lot of expected value over the long-run we need to care a lot about persistence and existential risk is an area where persistence seems particularly easy to find. So, if we go extinct it's relatively unlikely that another species will emerge to replace us in any given time period and conversely if we survive for a thousand years it seems like we might become relatively immune to further extinction events. So, I would say there is a kind of challenge for proponents of other kinds of interventions to really make the case that those interventions have the same potential for persistence.

NATHAN LABENZ: (26:51) So maybe one of those interventions… And this will probably have to be our last question, but do you have any thoughts on attempts to just move the world toward a more longtermist view? And the question there specifically notes governments and businesses tend to be pretty short-term in their orientation. If we could shift that, that might be helpful. Does that seem like a fruitful…?

CHRISTIAN J. TARSNEY: (27:12) Yeah. Absolutely. I mean I think… If we think that there's any longtermist intervention that's worth trying, then we want to get more people on board with longtermism and put more, you know, thinking and resources and whatever behind whatever longtermist interventions we think are successful, and of course how to spread a meme, how to convince people of a new idea, that's a big challenge that I haven't tried to address but a lot of people in the EA community have thought about that. But yeah, absolutely. I think we should be trying to move people in a more longtermist direction insofar as longtermism is true.

NATHAN LABENZ: (27:43) Awesome! Thank you very much. Christian Tarsney, office hours at noon. Another round of applause. Thank you. Fascinating talk.

[Applause]