CHRISTIAN J. TARSNEY: (00:07) Longtermism is roughly the view that in most of the choice situations or at least most of the most important choice situations that we face today, what we ought to do is mainly determined by the potential effects of our choices on the very far future. The case for longtermism starts from the observation that the future of humanity or human originating civilization is potentially really big, and big in two ways. First, in duration. We and our ancestors could be around for a long time and second, in spatial extent and in resource utilization. So we could potentially settle some significant fraction of the reachable universe and thereby have access to a very large pool of resources that we could convert into new value. You might think when you sort of do the math in a battery envelope way, in fact, that the scale of the future, the potential scale of humanity’s future is so big that almost any probability of positively impacting the future as a whole is just bound to produce more expected value than anything we can do in the near future. So for instance here is Nick Bostrom talking about the importance of reducing existential risks to human civilization. He gives a kind of estimate of how great the future could be. He says if we implement future, post human minds digitally and we convert the universe's available energy resources into these kind of minds in a reasonably efficient way, we can support up to 10⁵⁴ human-brain-emulation subjective life-years and then he says that even if you think that that estimate has only a 1% chance of being correct, it turns out that...

“the expected value of reducing existential risk by a mere one billionth of one billionth of one percentage point is worth a hundred billion times as much as a billion human lives.”

The point here I take it is (02:00) that 10⁵⁴ is just such a huge number that whatever the kind of decision relevant probabilities are, they're not going to be small enough to offset it. The expected value of trying to influence the far future or trying to reduce existential risks is still going to be astronomical.

But is that right? So on the one hand as we look further into the future the scale of the future, the potential scale of human civilization and the potential stakes of our actions grow. But on the other hand as we look further into the future, the future itself and the effects of our present actions on the future get progressively harder to predict. So there's a sort of positive force but there's an offsetting kind of negative force on the expected value of interventions that try to influence the further and further future. So we might ask very sort of imprecisely and informally, “Does the future grow?” “Does the future get bigger faster than our ability to predict the future shrinks?” Or, “Could our sort of limited ability to predict the future actually offset the kind of astronomical amounts of value that seem to be at stake such that really we ought to be more focused on the near-term future?”

Now there are some reasons kind of prima facie to at least worry about our ability to predict and predictably influence the very far future. So for instance there's this recent literature on expert political and economic forecasting starting with the work of Philip Tetlock and his colleagues that you could read in a sort of more optimistic or more pessimistic way but at least the the pessimistic reading is not unnatural. It seems like in many contexts even very highly trained, even very expert political and economic forecasters often do little better or no better than chance trying to predict social outcomes, even just a few years in advance. Then second, we know that in principle some complex systems, like perhaps human societies and civilizations, are chaotic in the sense that very small differences in initial (04:00) conditions or conditions at one time can lead to very very large differences in conditions at just a slightly later time. And that means that in fact if there's any limit on our ability to know the state of the system or measure the state of the system with arbitrary precision in the present, then we might be very limited in our ability to predict the state of the system or the impact of our present interventions more than a very short time in advance. And then finally if we look at the historical record it just doesn't look like there are that many examples of individuals or groups successfully predicting the future on the scale of say hundreds of years level over thousands of years or predictably influencing the future.

Now none of these are decisive and all of them could be to be argued with. For instance, you know, it's not obvious that human civilization or the aspects of it that we care about are sort of chaotic in relevant sense but all these are just sort of sources of worry that might get us thinking, “You know, maybe our limited predictive abilities really do offset the astronomical stakes and constitute a sort of defeater to the case for longtermism.”

So the goal of the talk is to try and evaluate this epistemic challenge to longtermism and the way that I'm going to do that is describe a simple model that incorporates the idea that as we look further in the future, the future itself and the effects of our present actions become less and less predictable. And then we're going to start to fill in some of the parameter values in that model and look at the implications. Now the question we're trying to assess is whether the case for longtermism is robust to at least a particular version of this epistemic challenge, this worry about the predictability of the far future. So on the one hand, I'm going to assume a normative framework that's relatively favorable to longtermisms. Specifically I'm going to assume that we're just total utilitarians, so the thing that we care about is the total amount of welfare in the world. That means we're setting aside (06:00) challenges to longtermism that come from the direction of ethics or population axiology. But on the other hand when we think about empirical questions, when we try to fill in parameter values in our model, I'm going to focus on estimates of those parameter values on empirical assumptions that are less favorable to longtermism because again our goal is to test the robustness of longtermism to this kind of challenge. So just to spoil the results a little bit it's going to turn out that longtermism is mostly robust, at least in the model as we spell it out, to the epistemic challenge, but we're going to be left with a couple of significant caveats: one is, it may turn out… It looks plausible within the model that the expected value of attempts to influence the far future is significantly driven by very very small probabilities of kind of astronomical payoffs and so we're left with this residual decision theoretic worry about how to respond to tiny probabilities of extreme events or extreme payoffs. Secondly, there's at least a weak suggestion coming out of the model that perhaps we should be longtermists but we should be longtermists only on the scale of say thousands or millions of years rather than on a scale of billions or trillions of years.

So let's get into the model. I'll start with a kind of intuitive loss [?] how the model works, what it's trying to do, then we’ll describe the model itself and then we'll go through and talk about each of the parameters. So the assumption of the model is that we're making a choice between two interventions, a longtermist intervention that we’ll call O_i  and a short-termist “benchmark” intervention that we’ll call  O_b  and it will help in assessing the model and filling in parameter values to have a kind of working example in mind. So in the working example we imagine that we're working for a large grant making organization and then we're deciding between two different ways of granting a million dollars. So on the one hand we could give a million dollars to some existential risk intervention, say funding biosecurity research. On the other hand we could spend the million (08:00) dollars on direct cash transfers – just give it to people, very poor people in the developing world. The expected value of O_b because our interest here is in O_i and the longtermist intervention. We're just going to assume that the expected value of O_b is specified independently and more specifically in the working example, we'll assume that it has an expected value of 3000 QALYs. That's partly a number of convenience, but it's also very closely in line with the latest GiveWell estimates on how much could we could actually do with a million dollars in direct cash transfers. And then for convenience we will normalize our value scale so that the expected value of the short-termist benchmark intervention is one (EV(O_b) = 1) and the expected value of just “doing nothing” or business as usual earning the million dollars or just spending it on something trivial is zero (EV = 0). So we have zero and we have a unit.

Okay. So then the longtermist intervention, O_i . The goal of that intervention we assume is to increase the probability that the world is in some desirable target state which we'll call S in the far future and in the working example where O_i  is trying to mitigate existential risks, the target state S can be read roughly as “The accessible universe contains an intelligent civilization.” Right? That's the state that we're trying to put the world in, or specifically what we assume about the sort of the mechanism in play is that O_i doesn't sort of continuously act on the future at every moment in future time, rather it acts on the future in the near-term or it acts on the world in the near future and then the hope is that we increase the probability that when we get to the far future we're already in that desirable target state and then that we stay there, right, that that state is stable. So the goal in the context of the working example is to increase the probability that we survive the near future, and in the working example we'll assume that means the next thousand years, so until (10:00) 3019 and then the hope is that if we're still around in a thousand years then we will persist. We will continue to hang around. And we can use this subscripted notation S_t just to denote the state of the world being in state S at the time t. Okay. So if we succeed in putting the world in the target state S then this produces a stream of value that represents how much better it is to be in state S than to be in state not S (¬S) and the stream of value continues until we encounter what I'll call an exogenous nullifying event (ENE).

So these come in two flavors… And this is the thing that’s sort of meant to represent the crucial phenomenon that our model is investigating, namely, the sort of increasing unpredictability of the further future. So on the one hand there are negative ENEs and these are events that just put the world into the less favorable state the less desirable state not S (¬S). In the context of the working example this is just any far future extinction event. So we survive the next thousand years say, but then 100 years after that we fight a self-destructive war and we go extinct. Then there are what I call positive ENEs, these are events that just do the opposite. They put the world into the target state S. So perhaps we go extinct in the next thousand years but then sometime after that, say a million years later another intelligent species evolves on earth or aliens show up from somewhere else in the universe and they start doing all the great things that we would have done. We'll assume, for the sake of simplicity, that ENEs occur with a constant probability per unit time in the far future. Right? So the assumption is more than a thousand years from now we just don't have any information that allows us to say that these kind of events are more probable or less probable in one time period as opposed to another. Now the sort of crucial thing about ENEs, the thing that makes them significant is, as soon as an ENE of either type occurs, (12:00) the state of the world no longer depends on its initial state or no longer depends on the state that it was in at the the sort of boundary between the near future and the far future. So if a negative ENE has occurred, then we're in state not S (¬S) regardless of what state we started off in. If a positive ENE has occurred, then we're in state S regardless of what state we started off in. So either kind of ENE brings an end to the stream of extra value that we got by starting off in state S rather than state not S (¬S).

Okay. So that's the sort of informal introduction to the model. Now let's actually see the model itself. It looks like this. So we have one big equation that's intended to estimate the expected value of the longtermist intervention O_i and then within it we have this function n. So let's take it part by part and discuss each of the parameters. So the first thing is this parameter p and this says how much can we increase the probability of starting off in the desirable state S. In other words how much can we increase the probability that we are in the state S at the near-term/long-term boundary, which again we're assuming is a thousand years from now. So we can express this formally just as the probability that S₀ obtains by being state S at time zero given the we perform action O_i  minus the probability that we're in that state given that we perform O_b. Then assuming that we do make the difference between being in state S and being in state not S (¬S), we want to know how much better is it to start off in state S rather than state not S (¬S) and to do that we take a time integral of expected value at a time. In other words how much better is it at a given time to be in state S rather than not S (¬S) and we assume that that integral is bounded because we assume that somewhere out there in the far enough future is (14:00) some event that just sort of brings everything to an end, which we'll call the “eschatological bound”. The sort of natural thing to think of here is like the heat death of the universe. Right? So once the heat death of the universe happens, maybe just nothing matters anymore, but certainly at least we are no longer able to have any sort of predictable effect on the value of the work. Right? So that's why we bound the integral. Then within the integral we want to know how much better is it at a time to be in state S rather than ¬S and that's what this big parenthetical expression is about.

And it turns out that it's helpful to make a sort of division in here between what's going on on earth and what's going on in the rest of the universe. In particular this lets us get our accurate results when we're thinking about the first few years of space travel. So v_e  just says how much better is it for a civilization to be in state S rather than state not S (¬S). And in the working example that just means how much better is it for a civilization to exist per year in terms of value realized on Earth. Then correspondingly we have this other parameter v_s which says how much better is it to have a civilization in state S rather than not S (¬S). In the working example existing rather than not existing. Now not thinking about Earth but thinking about per star system that we've settled outside of our own solar system. So then this tells us how much better it is, you know, to be an astronaut who [inaudible 15:27] per star system.

The next thing you want to know is how many star systems have we settled at the given time t and that's what this function n is for. So n says at time t  how many additional star systems on the solar system have we managed to settle and to figure that out we need to know first how fast are we settling the universe. So that's this parameter s that is just the long-term average speed of space settlement as a fraction of the speed of light c. And then we multiply that by how long have we been traveling (16:00) or how long have we been expanding and to do that we just take t minus this parameter t_l  which is the time at which we start settling the universe measured relative to the short-term/long-term boundary. So if we start settling the universe right at the boundary in 3019 then t_l  equals zero. And then finally there's this parameter r and there's this last term e^-rt. So this is the sort of crucial thing that we're really interested in.

The idea behind the model is that we know S is better than not S (¬S) but the probability that the state of the world at time t depends on its initial state at the short-term/long-term boundary decreases with time and this term e^-rt describes how it decreases. So r is not exactly but to a very good approximation just the annual probability of an ENE occurring. In other words the annual probability of either a positive or a negative ENE and    e^-rt tells us how likely is it that no ENE has occurred by time t. Okay. So then we have this function n which is again sort of embedded in this equation and which tells us how many star systems have we managed to settle at time t. So before we start expanding obviously the answer is zero. Once we start expanding then we want to know what's the density of stars per cubic light year in the region we've managed to settle. I use a relatively crude function here.  I just distinguish between the density of stars roughly in the Milky Way. What I really do is within one 130,000 light years of Earth which is a big sphere that encompasses the Milky Way and then the density of stars per cubic light year in the Virgo supercluster. The reason we talk about the Virgo Supercluster rather than the universe as a whole is it just turns out that the action in the model that really matters, (18:00) that determines whether the longtermist intervention has more expected value than the short-termist intervention all happens in the first few million years where we're still inside the Virgo Supercluster.

Okay. So in this function we have two parameters d_g and d_u. d_g is just the number of stars per cubic light year within 130,000 light years of Earth, roughly the Milky Way and  d_u is stars per cubic light year in the Virgo Supercluster. Okay. So the basic features of the model, the things to remember. Number one, growth is cubic and that's because we have this term here starting with v_s and n itself is a cubic function as you can see here and here. The assumption is that in the long run we learn to convert resources into value as efficiently as possible or at least we reach some kind of upper bound and after that point our ability to create more value or growth in value is driven by acquiring more resources which we do via space settlement. The second crucial feature is this term e^-rt which gives us a small exponential discount rate. So this might be familiar to people who know the claim in economics literature where it's sometimes proposed that one reason to discount the future is this ongoing probability of exogenous catastrophes. Right? The difference here is just a) we're thinking on a potentially much longer time scale and b) we're including these positive ENEs as well as negative ENEs.

Now if we have polynomial growth and an exponential discount rate, we know that in the long enough run the discount rate is eventually going to be like, it's eventually going to overwhelm the growth rate. But the question we're interested in is, “How long does that take and how much expected value are we able to realize in the meantime?” And to answer that question we need (20:00) to fill in the parameter values. So okay, starting with p. I use here a very conservative lower bound estimate of p equals 2 x 10^-14. The way I get this is I first ask, suppose that humanity as a whole did nothing else for the next thousand years besides trying to stay alive. In other words, all of our time and effort and resources just go into reducing the risk of existential catastrophe. If we did that how much could we reduce that risk? How much can we increase our chances of survival? And it seems like a sort of safe lower bound estimate on that question is 1% (.01). We should be able to increase our odds of surviving the next thousand years by at least 1% if we did nothing else.

Then I say, okay, let's assume (and here's the sort of conservative part), let's assume the marginal impact of resources invested in existential risk mitigation is constant. In other words, each unit of resource we invest has the same impact on the probability of existential catastrophe. That's conservative because in almost all domains, resources have diminishing marginal impact. Right? They'd be less as you spend more and more and more, less per unit [inaudible 21:12]. So then we can just ask, how much of humanity's resource endowment over the next thousand years can you buy for a million dollars? And the sort of back-to-the-envelope answer turns out to be 2 x 10^-12. We multiply that through by .01 and we get 2 x 10^-14. Okay. t_f , the eschatological bound… It turns out this shift doesn't really matter as long as it's more than 10^-6 it's not going to change our qualitative conclusions and it clearly should be more than that. I use 10 to the… Sorry, 10⁶ (t_f >∼10⁶). I use 10¹⁴ which is still quite a conservative value that's just roughly what the last stars burn out. Then v_e… How much value roughly does human civilization produce or add up on Earth per year? (22:00) Again being conservative, I use 10⁶ (v_{e ≈} 10⁶). Remember our units of value are 3000 QALYs, so this corresponds to three billion happy lives per year. Conservative because at the moment we're supporting, hopefully, more than three billion happy lives or at least healthy lives per year.

Now v_s… This parameter is tricky because you can get a very wide range of estimates depending on what kind of scenario you're envisioning. So in particular, if you're thinking about what I'll call a “Space opera” scenario where human space settlement just involves humans or broadly kind of human-like organisms living on planets doing human stuff, then you might get a value like 10⁵ (v_{s ≈} 10⁵). So this is meant to represent the fact that most star systems are not as hospitable to human life as ours is. They might just not have hospitable planets or those planets might be smaller or less hospitable. So this is a kind of order of magnitude rounding down of v_e. On the other hand, if you think about this in the way that say Bostrom does and you think that the future of humanity is likely to consist in one of these sort of high Kardashev level civilizations turning stars into “Dyson spheres” and converting all of their energy into artificial lines, then just using Bostrom's estimates you can get a value of e_s on the order of like 10²⁰, so 15 orders of magnitude larger. For the moment in the spirit of conservatism we’ll use that smaller value of 10⁵. Okay. So the function… And remember as these two parameters, and these are more or less known values, give or take a bit. It doesn't matter too much for our purposes. So d_g = 2.2 x 10^-5 and d_u = 2.9 x 10^-9.

Now this parameter s… What's the sort of long-term average speed of space settlement? Here I use .1 (s = .1). In other (24:00) words 10% of the speed of light. I'm cheating a little bit here because you can obviously be much more conservative than that about the speed at which we’ll settle the universe. The reason I use a bit of a more liberal value here is, that is a sort of easy way of accounting for uncertainty. So because the expected value O_i  grows cubically with s, if you specify all the other parameters and then have some probability distribution over s and calculate a kind of certainty equivalent of that probability distribution, as a certainty equivalent .1 is extremely conservative because even a very small probability of larger values like .5 or .8, which are values considered in the literature, gives you a certainty equivalent of more than .1. Okay. This parameter t_l… When do we start settling the universe? Obviously we have very little to go on here. I just assumed the t_l equals zero (t_{l =} 0). In other words that we start settling the universe at the near-term/long-term boundary over 2000 years from now. Luckily, it turns out this parameter doesn't matter all that much either, so within reason, if we set it to 500 or 1000 it wouldn't qualitatively change our results. Finally, there is this parameter r which is a kind of crucial parameter of interest for us. Unfortunately, it's the hardest parameter to estimate. Among other things because it requires us to ask questions like, how stable will a far future human civilization be? A million years from now what will be the annual probability that we destroy ourselves in say a war or a biological catastrophe or whatever?

And of course we just have very little to go on trying to fill in numbers there. So what I'll do instead of filling in a number is consider the output of the model for a range of r values and more or less let you make up your mind which of those values you think are most plausible. So here's what happens when we do that. In the left-hand column you (26:00) see a range of r values decreasing by orders of magnitude. In the middle column you see the expected value of a longtermist intervention O_i corresponding to those r values given the other parameters we filled in and then in the right-hand column we have this number that I call the “Horizon”. So two things to note here. Number one is this boldface row in the middle. So this is where the expected value of O_i [EV(O_i)] overtakes the expected value of O_b where r is about .000182. In other words, a little bit less than a 2 in 10,000 annual probability of an ENE occurring. Right? The other thing to notice is this horizon column. So this is intended to capture sort of the point at which the discount rate overwhelms the growth rate and where further contributions to the expected value of O_i  are no longer significant. I do this in a kind of crude and imperfect way and I'm very open to suggestions how to do this better, but it's just the point at which the product of the growth term t cube (t³) and the discount term e^-rt falls below one over t (1/t ) and you can see over here what this looks like. So here's the integrand of the expected value O_i , EV(O_i )  for r = .00001 (10^-5). So you see there's a sort of big spike where the growth rate is winning, then at some point the discount rate takes over and then the horizon is sort of way out here. So the horizon is a very sort of liberal estimate of when the expected value of  our intervention stops being significant. So it's kind of notable here that even when the longtermist intervention is winning, say in these columns, the horizon still comes maybe surprisingly soon in the order of hundreds of thousands or millions of years, not billions or trillions of years.

Okay. (28:00) So what should we make of this? Well you might think this is kind of a mixed result for longtermism because this kind of threshold roughly 2 in 10,000… Well, it seems pretty plausible that future civilizations could be that stable that the probability of big exogenous events that would for instance wipe out advanced civilization might be lower than 2 in 10,000 per year, but it doesn't seem totally obvious. And so this suggests that there is a kind of empirical way here to resist the piece for longtermism. If you think that value of r should be longer than this [inaudible 28:34]. But actually I think that's the wrong conclusion to take away because what we've been doing so far is just using point estimates for the parameters and plugging these point estimates into our favorite model.

What we really ought to do in the end is account for uncertainty both about the model itself and about the parameter values that go in. So there's a kind of open-ended list of sources of uncertainty we might care about, but here are some obvious ones. First of all on the model itself… So in the paper that this talk is based on, I consider in addition to the cubic growth model a “steady state” model where we assume that humanity just stays Earth bound forever. But then within the model we could also be uncertain about a number of the parameter values, for instance, we've already talked about v_s. Should we use this more conservative “space opera” value or the more ambitious “Dyson sphere” value? Of course we can be uncertain about r. What's the long-term rate of these exogenous events, positive or negative? And then I won't say much about this, but we could also… I guess I mentioned already be uncertainty about s, and that has important impacts on the model. We could be uncertain about p, about how much we can change the probability of starting off within the desirable target state. It turns out that the net effect of these uncertainties is very favorable to longtermism. Why? Because even a little bit of probability on more favorable parameter values can just vastly increase the (30:00) expected value of O_i.

So I'll just now sort of briefly give you a few examples of how that works. Of course if possible we'll give it sort of much more rigorously but I think these are sort of sufficient to illustrate the point. So if I have say, even with 1% credence in any values of r less than about 10^-5, in other words less than 1 in… What? Ten hundred thousand annual probability of ENEs. That alone is enough to guarantee that the expected value of O_i  is going to be greater than 100. In other words more than 100 times the expected value of the benchmark intervention. Now suppose I bring in uncertainty about the model and about the the sort of crucial parameter v_s. So suppose that I have 99% (.99) credence in this “steady state” model where we just never settle the universe at all. That looks bad for longtermism, but then suppose that also conditional on settling the universe, I have 1% (.01) credence in this more ambitious “Dyson sphere” scenario where we create 10²⁰ units of value per star per year and 99% (.99) credence in the more conservative value that we worked with so far. Then just because this “Dyson sphere” value is so large, it turns out that the expected value of the longtermist intervention exceeds the expected value of the short-termist intervention as long as r is less than about .1 ( r ≤ ∼ .1 ). In other words as long as the annual probability of an ENE is less than 10% which seems like a very very modest condition and then finally we can put these two things together. So suppose you have 1% (.01) credence in the cubic growth model and otherwise credence in the “steady state” model, you have 1% (.01) credence in this “Dyson sphere” scenario, 99% (.99) credence in the more conservative “space opera” scenario and you're uncertain about r. Well as long as you have even 1% (.01) credence in values of r less than about .032 ( r ≤ ∼ .032 ). In other (32:00) words at less than about 3% annual probability of ENEs it will still turn out that the expected value of O_i is greater than the expected value of O_b [EV(O_i) > EV(O_b)].

Okay. Final point here. It looks like once we come for uncertainty the sort of expectational case for longtermism is fairly robust. But we might worry that that expectational argument relies really heavily on a conjunction of several improbable assumptions. So to begin with, all of this sort of depends on the assumption that we have any impact at all, which is measured by this this parameter p. Now the lower bound estimate 2 x 10^-14 is probably too conservative by several orders of magnitude, but even if we rounded out generously say, 2 x 10^-10 well that's still quite a long shot. Right? And then even on top of that, even assuming that we have some impact, the majority of the expected value of our intervention still relies on a number of other assumptions e.g. the probability of space settlement. In other words of the cubic growth model conditional on surviving perhaps on the probability of one of these more ambitious like the “Dyson sphere” scenario, conditional on us settling in space in the first place. And then the the probability or possibility of a sort of more stable future where the value of r is low in the long-term. So quantifying this and figuring out what numbers and quantities we actually care about is a sort of difficult decision [inaudible 33:30], but at least there's a sort of indication that most of the expected value of the longtermist intervention, at least the one the we're considering here, might come from a very very small subset of the state space or a very low probability subset of the state space. So then that leaves us with this kind of crucial decision here about the question from longtermism, namely, whether we're still… still rationally required to go over or in some sense ought to maximize (34:00) expected value even in the face of these miniscule probabilities of astronomical payoffs. This is some of the decisions you are so worried about, although nobody so far seems to have a really good answer.

Okay. So some conclusions. First of all we've seen that if our goal is just to maximize expected value and we don't mind premising our choices on these tiny probabilities of astronomical payoffs, then the case for longtermism looks relatively robust. But it does seem like we’re left with this kind of residual decision-theoretic worry about “Pascalian” probabilities, these tiny probabilities of large payoffs, which it's I think one of the most important goals of future work to try to say something about. Third we've seen this suggestion. I would say this is only a suggestion but there's a hint here that perhaps we should be longtermists but only up to a point on the scale of thousands or millions of years, not billions or trillions. A thing that I haven't mentioned so far but that I think is consequential for people trying to work out the implications of longtermism, is we've seen that the value of this parameter r has a sort of enormous impact on the expected value of longtermist interventions. That means persistence is really really important. So if we're trying to influence the far future we should really prefer interventions whose effects we expect to persist robustly over long time frames and that has important implications for people deciding between different ways of influencing the far future. But finally and most importantly the cubic growth model and everything that I've said in this talk is only a first approximation and I hope what you take away more than anything else, is that if we care about the far future and if we're at least tempted by the sort of longtermist case for trying to positively influence it, then we just need to think a lot more about how and to what extent we can predict and predictably influence the far future. Thank you.