{"id":1242,"date":"2024-08-12T12:00:00","date_gmt":"2024-08-12T12:00:00","guid":{"rendered":"https:\/\/forecastingresearch.org\/?post_type=research&p=1242"},"modified":"2026-05-05T14:27:24","modified_gmt":"2026-05-05T14:27:24","slug":"ai-conditional-trees","status":"publish","type":"research","link":"https:\/\/forecastingresearch.org\/research\/ai-conditional-trees","title":{"rendered":"Conditional Trees: A Method for Generating Informative Questions about Complex Topics"},"content":{"rendered":"\n

Abstract<\/h2>\n\n\n\n

We test a new process for generating high-value forecasting questions: asking experts to produce \u201cconditional trees,\u201d simplified Bayesian networks of quantifiably informative forecasting questions. We test this technique in the context of the current debate about risks from AI. We conduct structured interviews with 21 AI domain experts and 3 highly skilled generalist forecasters (\u201csuperforecasters\u201d) to generate 75 forecasting questions that would cause participants to significantly update their views about AI risk. We elicit the \u201cValue of Information\u201d (VOI) each question provides for a far-future outcome\u2014whether AI will cause human extinction by 2100\u2014by collecting conditional forecasts from superforecasters (n=8).1<\/a><\/sup> In a comparison with the highest-engagement AI questions on two forecasting platforms, the average conditional trees-generated question resolving in 2030 was nine times more informative than the comparison AI-related platform questions (p = .025). This report provides initial evidence that structured interviews of experts focused on generating informative cruxes can produce higher-VOI questions than status quo methods.<\/p>\n\n\n\n

\n
View the full PDF Report \n \n<\/svg>\n\n \n<\/svg><\/a><\/div>\n<\/div>\n\n\n\n
\n
Acknowledgments<\/summary>\n

This research would not have been possible without the generous support of Open Philanthropy. We thank the research participants for their invaluable contributions. We greatly appreciate the assistance of Page Hedley, Kayla Gamin, Leonard Barrett, Coralie Consigny, Adam Kuzee, Arunim Agrawal, Bridget Williams, and Taylor Smith in compiling this report. Additionally, we thank Benjamin Tereick, Javier Prieto, Dan Schwarz, and Deger Turan for their insightful comments and research suggestions.<\/p>\n<\/details>\n<\/div><\/div>\n\n\n\n

Executive summary<\/h2>\n\n\n\n

Introduction<\/h3>\n\n\n\n

From May 2022 to October 2023, the Forecasting Research Institute (FRI) (a<\/u><\/a>)2<\/a><\/sup> experimented with a new method of question generation (\u201cconditional trees\u201d). While the questions elicited in this case study focus on potential risks from advanced AI, the processes we present can be used to generate valuable questions across fields where forecasting can help decision-makers navigate complex, long-term uncertainties.<\/p>\n\n\n\n

Methods<\/h3>\n\n\n\n

Researchers interviewed 24 participants, including 21 AI and existential risk experts and three highly skilled generalist forecasters (\u201csuperforecasters\u201d). We first asked participants to provide their personal forecast of the probability of AI-related extinction by 2100 (the \u201cultimate question\u201d for this exercise).3<\/a><\/sup> We then asked participants to identify plausible4<\/a><\/sup> indicator events that would significantly shift their estimates of the probability of the ultimate question.<\/p>\n\n\n\n

Following the interviews, we converted these indicators into 75\nobjectively resolvable forecasting questions. We asked superforecasters\n(n=8) to provide forecasts on each of these 75 questions (the \u201cAICT\u201d\nquestions), and forecasts on how their beliefs about AI risk would\nupdate if each of these questions resolved positively or negatively. We\nquantitatively ranked the resulting indicators by Value of Information\n(VOI), a measure of how much each indicator caused superforecasters to\nupdate their beliefs about long-run AI risk.<\/p>\n\n\n\n

To evaluate the informativeness of the conditional trees method relative to widely discussed indicators, we assess a subset of these questions using a standardized version of VOI, comparing them to popular AI questions on existing forecasting platforms (the \u201cstatus quo\u201d questions). The status quo questions were selected from two popular forecasting platforms by identifying the highest-engagement AI questions (by number of unique forecasters). We present the results of this comparison in order to provide a case study of a beginning-to-end process for producing quantitatively informative indicators about complex topics. (More on methods<\/a>)<\/p>\n\n\n\n

Results<\/h3>\n\n\n\n

The conditional trees method can generate forecasting questions that are more informative than existing questions on popular forecasting platforms5<\/a><\/sup><\/h4>\n\n\n\n

Our report presents initial evidence that structured interviews of\nexperts produce more informative questions about AI risk than the\nhighest-engagement questions (as measured by unique users) on existing\nforecasting platforms.<\/p>\n\n\n\n

Using predictions made by superforecasters (n=8), we compared the status quo questions to a subset of the AICT questions.6<\/a><\/sup> Most of the AICT questions (nine of 13) scored higher on VOI than all 10 status quo questions.7<\/a><\/sup><\/p>\n\n\n\n

VOI is based on each respondent\u2019s expected update<\/em> in their\nbelief about the ultimate question, not on how much a participant would\nupdate if an event happened. That is, it takes into account how likely\nthe forecaster believes an event is to occur. If an event would result\nin a large update to a participant\u2019s forecast, but is deemed vanishingly\nunlikely to occur, it would have a small VOI. If an event would result\nin a large update, and is also considered likely to occur, it would have\na high VOI.<\/p>\n\n\n\n

Table E.1 compares the top five AICT questions to the top five status quo questions, as measured by superforecasters\u2019 ratings of a standardized metric of informativeness, which we call \u201cPercentage of Maximum Value of Information\u201d (POM VOI).8<\/a><\/sup> In this table and throughout the report, we refer to questions by their reference numbers. For a full list of the AICT questions and status quo questions selected from forecasting platforms by reference number, with operationalizations and additional information, see Appendix 1<\/a>.<\/p>\n\n\n\n

Question<\/strong><\/td>Mean POM VOI<\/strong><\/td><\/tr>
AI causes large-scale deaths, ineffectual response (CX50<\/a>)<\/td>6.34%<\/td><\/tr>
Administrative disempowerment warning shot (CX30<\/a>)<\/td>3.55%<\/td><\/tr>
Deep learning revenue (VL30<\/a>)<\/td>1.68%<\/td><\/tr>
Power-seeking behavior warning shot (ZA50<\/a>)<\/td>1.59%<\/td><\/tr>
Extinction-level pathogens feasible (CQ30<\/a>)<\/td>1.37%<\/td><\/tr>
Superalignment success (STQ205 \/ STQ215<\/a>)*<\/td>0.28%<\/td><\/tr>
Kurzweil\/Kapor Turing Test longbet (STQ9<\/a>)*<\/td>0.27%<\/td><\/tr>
Brain emulation (STQ196<\/a>)*<\/td>0.23%<\/td><\/tr>
Human-machine intelligence parity (STQ247<\/a>)*<\/td>0.14%<\/td><\/tr>
Compute restrictions (STQ236<\/a>)*<\/td>0.13%<\/td><\/tr><\/tbody><\/table><\/div>
Table E.1:<\/strong> Ratings of how informative AICT questions are relative to status quo questions. The status quo questions are marked with an asterisk.<\/figcaption><\/figure>\n\n\n\n

Focusing on questions resolving in the near-term (by 2030), we found that questions generated with the conditional trees method were, on average, nine times more informative than popular questions from platforms (p = .025). While we did not find a statistically significant result for questions resolving in 2050-2070, in our sample AICT questions were still eleven times more informative on average. (More on VOI comparison<\/a>)<\/p>\n\n\n\n

Questions\ngenerated through the conditional trees method emphasized different\ntopics than those on forecasting platforms<\/h4>\n\n\n\n

We also analyzed the extent to which questions taken from existing\nforecasting platforms effectively captured the topics raised in our\nexpert interviews. We found that some topics (such as AI\nalignment-related questions and questions related to concrete AI harms)\nwere of substantial interest to experts but had not received\nproportional attention on existing forecasting platforms, and that\nquestions generated by the conditional trees method were meaningfully\ndifferent from those taken from existing forecasting platforms.<\/p>\n\n\n\n

The table below compares the topical distribution of the AICT questions to the status quo questions. (More on question uniqueness<\/a>)<\/p>\n\n\n\n

Category<\/strong><\/td>AICT question set<\/strong><\/td>Status quo question\nset<\/strong><\/td><\/tr>
Social \/ Political \/ Economic<\/td>24% (29)<\/td>33% (131)<\/td><\/tr>
Alignment<\/td>20% (25)<\/td>12% (47)<\/td><\/tr>
AI harms<\/td>20% (25)<\/td>7% (27)<\/td><\/tr>
Acceleration<\/td>36% (44)<\/td>48% (191)<\/td><\/tr><\/tbody><\/table><\/div>
Table E.2:<\/strong> Proportion of total questions that fell into each category; numbers in parentheses are total questions per category. While some questions fell into multiple categories (and thus proportions in each column should sum to more than 100%), proportions have been normalized for ease of comparison.<\/figcaption><\/figure>\n\n\n\n

We\nfound weak evidence that superforecasters and experts value different\ntypes of questions<\/h4>\n\n\n\n

Given the small sample sizes involved, we are reluctant to make\nconfident claims about the significance of the difference between the\nopinions of the superforecasters and the experts. However, we do see\nthese results as providing prima facie evidence about which questions\nare the most informative for each group when making updates on the\nprobability of AI-related extinction.<\/p>\n\n\n\n

Our most notable finding when comparing the views of the superforecasters to those of the experts was that the superforecasters tended to value questions that focused on concrete harms caused by AI, rather than the experts\u2019 preference for questions regarding advanced AI capabilities or whether AI had been successfully aligned. (More on AI risk takeaways<\/a>)<\/p>\n\n\n\n

Figure E.1 shows examples of how experts updated on the ultimate\nquestion conditional on three of the highest-VOI indicator\nquestions.<\/p>\n\n\n\n

\"\"\/
Figure E.1<\/strong>: A diagram showing how experts update on three relatively high-VOI questions for different resolution years that scored particularly well on our VOI metric. Since experts answered different sets of questions, we derived P(U|C) and P(U|~C) (the probabilities on the bottom level) by multiplying the whole expert group\u2019s average P(U) of 17% by the average relative risk factor for each crux.9<\/a><\/sup><\/figcaption><\/figure>\n\n\n\n

The table below provides more detail on each of the questions in the\nprevious figure.<\/p>\n\n\n\n

Question<\/strong><\/td>Summary<\/strong><\/td><\/tr>
Administrative disempowerment warning shot (CX30<\/a>)<\/td>This question resolves positively if any of the following occur by 2030: AI systems in a country freeze assets affecting \u22651% of the population or \u22650.5% of GDP for over a year; prevent \u22655% of voting-age citizens from participating in a national election; restrict >3% of residents’ movement for 3+ months; limit >10% of the population’s internet access for 3+ months; or legally restrict >5% of citizens’ access to certain housing or employment for 1+ year.<\/td><\/tr>
Power-seeking behavior warning shot (ZA50<\/a>)<\/td>This question resolves positively if, by 2050, AI developers must attempt to disable or destroy an AI (costing >$5M) after it exhibits power-seeking behaviors such as acquiring significant resources, controlling financial accounts, manipulating humans, infiltrating key systems, seizing control of weaponry, self-propagation, or engaging in hacking.<\/td><\/tr>
No aligned AGI (CX70<\/a>)<\/td>This question resolves positively if there is no AGI system which meets several “alignment” criteria by 2070. Those criteria include AGI-like capabilities (high question answering skill on hard questions in diverse subjects, high programming ability) and alignment (performing well on SOTA alignment tests, and expert surveys indicating that experts believe that AI is aligned).<\/td><\/tr><\/tbody><\/table><\/div>
Table E.3:<\/strong> Example summaries of questions that experts found to be particularly informative.<\/figcaption><\/figure>\n\n\n\n

The\nconditional trees method still has disadvantages<\/h4>\n\n\n\n

While this case study suggests that the conditional trees method can generate informative forecasting questions, a primary limitation of the method as implemented is its high labor cost. The process involved conducting more than 20 interviews with subject matter experts, writing 75 forecasting questions, and eliciting conditional forecasts. In future work, we expect it would typically be more efficient to elicit fewer indicators within a conditional tree and to operationalize only 1-2 forecasting questions per interview before eliciting forecasts. The intensive process described in this case study would be most appropriate for particularly high-value topics with large pools of resources for research. Additionally, it may be possible to use LLMs or incentivized crowdsourcing for the question generation or filtering stages, making the process cheaper and less labor intensive. (More on limitations of our research<\/a>)<\/p>\n\n\n\n

Key takeaways<\/h3>\n\n\n\n
    \n
  1. Preliminary evidence suggests that the conditional trees method\nof generating forecasting questions can result in questions that perform\nbetter on \u201cValue of Information\u201d metrics than popular questions on\nexisting forecasting platforms.<\/li>\n\n\n\n
  2. The conditional trees method produced questions with a markedly\ndifferent distribution of topic areas compared to those on existing\nforecasting platforms. Notably, the conditional trees approach led to a\ngreater proportion of questions focused on AI alignment and potential AI\nharms, reflecting that certain expert priorities may be underrepresented\nin existing forecasting efforts.<\/li>\n\n\n\n
  3. In our limited sample, experts tended to find questions related\nto alignment and concrete harms caused by AI to be the most informative.\nSuperforecasters also found questions relating to concrete AI harms to\nbe informative, but were less likely than experts to find questions\nrelating to alignment to be informative.<\/li>\n\n\n\n
  4. The conditional trees method as implemented in this case study is\nparticularly labor intensive. We expect the most broadly useful versions\nof this process would take the underlying principles and 1) apply them\nto shorter interviews with smaller numbers of forecasting questions to\noperationalize, 2) leverage LLMs for elicitation and synthesis, and\/or\n3) utilize crowdsourcing at the question generation and filtering\nsteps.<\/li>\n<\/ol>\n\n\n\n

    Key outputs<\/h3>\n\n\n\n

    In addition to the above takeaways, we highlight key outputs from the\nreport: the tangible resources developed during the course of the\nconditional trees process which we believe may be useful to others\ninterested in replicating parts of the process.<\/p>\n\n\n\n

      \n
    1. We created a guide and replicable process for using conditional tree interviews to generate informative forecasting questions (see Appendix 6<\/a>). This process can be implemented by organizations and individuals that need high-quality, informative questions.<\/li>\n\n\n\n
    2. We provide details of relevant metrics (e.g., \u201cValue of Information\u201d) that can be used to assess how informative each generated question is. See our public calculator for \u201cvalue of information\u201d and \u201cvalue of discrimination\u201d here<\/u><\/a>.<\/li>\n\n\n\n
    3. In total, the conditional trees process generated 75 new questions relating to AI risk. The full operationalizations and resolution criteria of these questions are available in Appendix 1<\/a> of this report. We have posted several of the highest-VOI questions to two forecasting platforms and encourage interested readers to submit their own predictions. (See Appendix 7<\/u><\/a> for links)<\/li>\n\n\n\n
    4. We used our question metrics to create aggregated conditional trees that visually summarize the most important AI risk pathways according to small samples of experts and generalist forecasters. These aggregated trees can be found here<\/u><\/a>.<\/li>\n<\/ol>\n\n\n\n

      Limitations of our research<\/h3>\n\n\n\n

      Limitations of our research include:<\/p>\n\n\n\n

        \n
      1. The total number of participants in this study was small (n=8\nforecasts on most questions, 24 interviewees to generate\nquestions).<\/li>\n\n\n\n
      2. The forecasting tasks in this study were unusually difficult,\ninvolving low probability judgments, long time horizons, conditional\nforecasts, and \u201cshort-fuse forecasts\u201d made very quickly.<\/li>\n\n\n\n
      3. Participants were all either experts who are highly concerned\nabout existential risks from AI or superforecasters who are relatively\nskeptical, so we are not able to separate differences caused by risk\nassessment from differences caused by forecasting aptitude, professional\ntraining, or other factors.<\/li>\n<\/ol>\n\n\n\n

        (More on limitations of our research<\/a>)<\/p>\n\n\n\n

        Next steps <\/h3>\n\n\n\n

        Further research related to this topic could include:<\/p>\n\n\n\n

          \n
        1. Studies on the same questions with larger numbers of forecasters,\nincluding by integrating the questions into existing forecasting\nplatforms.<\/li>\n\n\n\n
        2. Replicating the conditional trees process in domains other than\nAI risk.<\/li>\n\n\n\n
        3. Following up as questions begin to resolve in 2030 to assess\nwhether forecasters update their views in accordance with their\nexpectations.<\/li>\n<\/ol>\n\n\n\n

          (More on next steps<\/a>)<\/p>\n\n\n\n

          \n

          Glossary<\/h2>\n\n\n\n

          AI Conditional Trees (AICT) question set<\/strong><\/h4>\n\n\n\n

          The set of questions generated by the AI Conditional Trees process\ndescribed in this report.<\/p>\n\n\n\n

          Conditional tree<\/strong><\/h4>\n\n\n\n

          A simplified Bayesian network, in which each node is an event that\nmay or may not occur, and each connection between nodes has the factor\nby which the next node is more or less likely if that one happens. In\nthis report, the conditional trees ultimately ask how likely it is that\nAI causes human extinction by 2100, and each node is an event that\naffects the likelihood of that ultimate outcome.<\/p>\n\n\n\n

          Operationalization<\/strong><\/h4>\n\n\n\n

          The process of making a question about a future event into a\nresolvable forecasting question. For example, if a prompt said \u201cthere is\nmajor progress in interpretability by 2030\u201d the operationalized question\nwould contain a specific way to resolve that question so that there can\nbe no future dispute about whether the progress counts as \u201cmajor.\u201d<\/p>\n\n\n\n

          Percent of Max (POM)<\/strong><\/h4>\n\n\n\n

          When we present VOI for a question, we also present the percentage of\nthe maximum VOI (POM VOI) it captured in order to contextualize the\nmagnitude of the results. The POM VOI of a question can be interpreted\nas the fraction of the uncertainty about the ultimate question U the\nquestion resolves, in expectation.<\/p>\n\n\n\n

          Question prompts<\/strong><\/h4>\n\n\n\n

          General topics of questions that we then operationalized into\nforecasting questions. For example, \u201cmajor progress in interpretability\nby 2030\u201d could be a question prompt, although it is not a clearly\nresolvable forecasting question.<\/p>\n\n\n\n

          Short-fuse forecasts<\/strong><\/h4>\n\n\n\n

          Very quickly estimated forecasts, in which each participant spent no\nmore than one minute per question and gave a snap judgment.<\/p>\n\n\n\n

          Status quo questions<\/strong><\/h4>\n\n\n\n

          Questions on AI that we selected from existing forecasting platforms on the basis of their popularity (largest number of unique users) and other criteria. See 2.3 Selection of status quo questions<\/a>.<\/p>\n\n\n\n

          Ultimate question \/ Ultimate outcome (U)<\/strong><\/h4>\n\n\n\n

          The \u201cultimate question\u201d that all of the intermediate questions help\npredict. In this study: \u201cWill AI cause human extinction by 2100?\u201d<\/p>\n\n\n\n

          Value of information (VOI)<\/strong><\/h4>\n\n\n\n

          VOI is a measure of how much knowing the answer to a question would\nchange an individual’s belief, in expectation. This is useful for\nunderstanding why individuals believe what they believe and what would\nchange their minds.<\/p>\n<\/div><\/div>\n\n\n\n

          1. Introduction<\/h2>\n\n\n\n

          For policymakers to use forecasting in their work, they need accurate\nforecasts, but\u2014perhaps equally important\u2014the forecasts need to be about\ndecision-relevant questions. Knowing which questions will be the most\nvaluable to forecast on can be difficult. How can policymakers identify\nthe short-term events that are most relevant to important long-term\noutcomes?<\/p>\n\n\n\n

          Here we present a tool, the conditional tree method (figure 1.1.1),\nwhich can distill complex issues into a few key uncertainties. We apply\nit to a topic of increasing public concern: \u201cWill advanced artificial\nintelligence pose an existential threat to humanity in the 21st\ncentury?\u201d Using a specialized interview process, we learn what subject\nmatter experts believe are the best warning signs for this risk in the\ncoming decades. Then we use metrics based on conditional forecasting to\nquantitatively measure the relevance of these warning signs. This allows\nus to winnow down to a few highly relevant indicators of increased risk\nto humanity from AI.<\/p>\n\n\n\n

          \"\"\/
          Figure 1.1.1:<\/strong> The conditional trees process<\/figcaption><\/figure>\n\n\n\n

          The conditional trees approach10<\/a><\/sup> represents a new set of priorities in the field of forecasting. Most previous forecasting research focused almost exclusively on identifying accurate forecasters and improving forecasting accuracy. But comparatively little work was invested in choosing forecasting targets. In order to mature into a practically applicable body of knowledge, the field must look beyond optimizing forecasts and toward optimizing the questions we ask.<\/p>\n\n\n\n

          1.1 A\nmethod for generating and judging high-value questions<\/h3>\n\n\n\n

          Some forecasting tournaments and platforms have already begun to\nutilize domain experts to generate questions with real-world relevance.\nHowever, many of these efforts are relatively ad hoc<\/em>, producing\ninconsistent results and plausibly missing many high-value forecasting\ntargets.<\/p>\n\n\n\n

          For example, for the Existential Risk Persuasion Tournament<\/u><\/a> (XPT),11<\/a><\/sup> the question preparation phase enlisted domain experts to comment on the prospective question set in a relatively unstructured way. While this undoubtedly improved the question set, it did not identify the most informative questions within the set.<\/p>\n\n\n\n

          To leverage the expertise of domain experts more fully, we propose a\nmore in-depth, systematic approach: expert elicitation structured around\nconditional trees.<\/p>\n\n\n\n

          Why conditional trees?<\/h4>\n\n\n\n

          Conditional trees represent beliefs through a tree-like structure, using nodes to represent events that influence the probability of an ultimate outcome. In the tree in Figure 1.1.2, for example, if you know someone is vaccinated, they are half as likely to be infected than if you were unsure whether they were vaccinated. Then, if you know they have been exposed, they are 3.5x as likely to be infected.12<\/a><\/sup><\/p>\n\n\n\n

          \"\"\/
          Figure 1.1.2:<\/strong> Example conditional tree diagram<\/figcaption><\/figure>\n\n\n\n

          In this study, the ultimate outcome was the probability of extinction\ndue to AI by 2100, and the nodes are events that make that outcome more\nor less likely. The tree structure makes the conditional probabilities\nbeneath a forecast explicit and visible, and may help forecasters narrow\nin on specific, important factors.<\/p>\n\n\n\n

          Participants initially provided an estimate of the probability of\nAI-related extinction by 2100 (the \u201cultimate question\u201d), represented by\nO in Figure 1.1.2. Interviews then focused on identifying key indicators\non the pathway to AI-related extinction. Participants selected two to\nfive indicators for deeper analysis to understand how they might alter\nthe risk of AI-related extinction. These factors then became the\nantecedents in the tree: for each of the indicators selected to be\nincluded in the tree, participants gave forecasts for how much their\nforecast of the ultimate outcome would change if that event\nhappened.<\/p>\n\n\n\n

          The ultimate outcome (for our purposes, the probability of extinction\ndue to AI by 2100) is an important parameter: the rest of the network\u2019s\nrelevance cascades from the outcome. But provided we\u2019re able to identify\nan outcome with strong bearing on present policy decisions, we can ask\nexperts to decompose the intervening time into possible events which\nwould reflect a greater or lesser likelihood of reaching that outcome.\nThus, these intervening events must themselves possess policy-relevance,\nin proportion to the strength of their relationship with the outcome,\nand the likelihood of observing them.<\/p>\n\n\n\n

          Conditional trees are a type of Bayesian network (BN).13<\/a><\/sup> BNs explicitly represent probabilistic relationships between outcomes and their antecedents.14<\/a><\/sup> This structure encourages experts to generate maximally relevant antecedents, and also provides us with a framework for measuring question relevance. But unlike some other forms of BNs, conditional trees are a relatively easy tool to learn. In our study, interviewees were able to grasp the necessary basics in around 10 minutes. This means that conditional trees may be more practical for interviews with subject-matter experts, who may not be experts in statistics or other domains that more often use BNs.<\/p>\n\n\n\n

          How\ndoes the conditional trees method fit into the forecasting research\nprocess?<\/h4>\n\n\n\n
          \"\"\/
          Figure 1.1.3:<\/strong> Life cycle of an impactful forecasting project.<\/figcaption><\/figure>\n\n\n\n

          The AI Conditional Trees project is an in-depth investigation into\nhow to generate informative forecasting questions. Question generation\nis the first step in the life cycle of an impactful forecasting project,\nillustrated in Figure 1.1.3.<\/p>\n\n\n\n

          Many earlier forecasting research projects have focused on\nidentifying the most accurate forecasters and on improved methods for\naggregating their forecasts. But to be useful to decision makers,\nforecasting research must move beyond those questions and incorporate\nforecasting into a process that includes question generation,\nconsidering actions based on forecasts, communicating with policymakers,\nand generating new questions.<\/p>\n\n\n\n

          Before the cycle starts, we begin with \u201cscoping and gisting,\u201d in\nwhich we consider the questions we want to answer, the scope of the\npossible project, and the general arguments (\u201cgists\u201d) on each side. We\nthen begin the cycle by generating questions, through processes like the\nAI Conditional Trees method, aiming to find the forecasting questions\nthat would be most informative to decision makers. Next, we elicit\nforecasts on those questions, to assess risk and understand which\npotentially dangerous events are most likely and in what circumstances.\nWe then elicit \u201crisk mitigation forecasts,\u201d asking experts and skilled\nforecasters to predict which policies would most decrease risk and what\nthe costs might be for implementing them.<\/p>\n\n\n\n

          Once we have completed these stages, we communicate that information\nto policymakers, and ask them whether it is useful and what would make\nit more relevant to their work. Their feedback gives us more information\nwe can use for the next stage of question generation, and we begin the\ncycle again.<\/p>\n\n\n\n

          The cycle as depicted is somewhat stylized, and many forecasting\nprojects will not include all of these stages. But thinking of AI\nconditional trees in the context of the \u201cforecasting life cycle\u201d helps\nus contextualize this work and think about how to incorporate it into\nour future research.<\/p>\n\n\n\n

          Measuring question value<\/h4>\n\n\n\n

          In order to form the feedback loop necessary for a dramatic\nimprovement in the decision-relevance of forecasting questions, we need\na means of quantitatively measuring the value of a forecasting\nquestion.<\/p>\n\n\n\n

          Policymakers\u2019 actions are often guided by a few important questions\nin their domain, like \u201cWhat will be the effects of climate change over\nthe next century?\u201d or \u201cWill our economy remain competitive in the world\nin the long-term?\u201d Such questions are difficult to resolve because they\nrefer to the distant future, and they may also be relatively complex or\ndifficult to specify clearly. But often one can find nearer-term\nantecedent questions which are easier to resolve, and which would reduce\nsome uncertainty about the \u201cultimate\u201d question. For example, in a study\nforecasting the effects of climate change, with the ultimate question,\n\u201cWill more than 2 billion people die or be displaced due to climate\nchange by 2100?,\u201d the question \u201cWhat will the average global temperature\nbe in 2040?\u201d might be a good antecedent question. It would not give a\nforecaster the full answer to the main question, but knowing what the\nglobal surface temperature will be in 2040 would be at least somewhat\nhelpful for forecasting the effects of climate change by 2100.<\/p>\n\n\n\n

          Thus, one way of conceptualizing the value of a forecasting question\nis to ask, \u201cHow would the answer to this question affect our expectation\nabout an \u2018ultimate\u2019 question we care about?\u201d There are several distinct\nways of expressing this mathematically, which we collectively refer to\nas \u201cValue of Information (VOI).\u201d<\/p>\n\n\n\n

          Conceptually, VOI measures how important a potential crux question\n(\u201cC\u201d) is to a participant\u2019s forecast of the ultimate question we care\nabout (\u201cU\u201d, in this case: AI extinction risk by 2100), in expectation.\nThat is, how much would a participant update on AI extinction risk by\n2100 based on whether a crux happens, weighted by how likely that crux\nis to happen. A high VOI question for a given participant will therefore\nbe one that a) that participant thinks has a meaningful chance of\nhappening and b) meaningfully affects that participant\u2019s forecast on the\nultimate question.<\/p>\n\n\n\n

          VOI is a useful metric for understanding why individuals believe what they believe and what would change their minds. A technical explanation of VOI can be found in Appendix 4<\/a>. To build intuition for using the VOI metric, we provide this calculator<\/a> (a<\/a>) in which users can input their own values. We also provide a more comprehensive R software package<\/a> for calculating it.<\/p>\n\n\n\n

          2. Methods<\/h2>\n\n\n\n

          2.1 Question generation<\/h3>\n\n\n\n

          Sampling interviewees<\/h4>\n\n\n\n

          Our sample included 24 interviewees in total: 21 \u201cexpert\u201d\ninterviewees, and 3 \u201csuperforecaster\u201d interviewees. We aimed to include\nin our sample representatives of four quadrants of a strategically\nimportant belief space (see Figure 2.1.1):<\/p>\n\n\n\n

            \n
          1. short timeline for AI progress, high estimated risk from\nAI;<\/li>\n\n\n\n
          2. short timeline for AI progress, low estimated risk from\nAI;<\/li>\n\n\n\n
          3. long timeline for AI progress, low estimated risk from AI;\nand<\/li>\n\n\n\n
          4. long timeline for AI progress, high estimated risk from AI.15<\/a><\/sup><\/li>\n<\/ol>\n\n\n\n
            \"\"\/
            Figure 2.1.1:<\/strong> Target groups for sampling<\/figcaption><\/figure>\n\n\n\n

            We gathered our expert sample via snowball sampling, seeded from recommendations from our funders and our networks. We do not expect our interview sample was particularly representative of any given group, such as AI experts. The goal of this project was to develop the trees process and assess whether it led to higher value questions, which did not require a representative expert sample. Our superforecaster sample was taken from the set of superforecaster participants in the Existential Risk Persuasion Tournament (XPT)16<\/a><\/sup> who had shown particularly high engagement. Candidate interviewees were approached for interview with a monetary incentive for producing the \u201chighest value\u201d questions in our interview-derived question set.<\/p>\n\n\n\n

            \"\"\/
            Figure 2.1.2:<\/strong> Histogram of interviewees\u2019 original forecasts of probability of extinction via AI by 2100.17<\/a><\/sup><\/figcaption><\/figure>\n\n\n\n

            The majority of our expert sample had academic or professional\nexperience pertaining directly to AI risk, such as experience in\ntechnical AI safety or AI governance (13\/21 expert interviewees). Others\nwere included for having publicly expressed views on AI risk indicating\na high level of engagement with the topic and having expertise in a\ncomplementary field, such as machine learning (7\/21 expert\ninterviewees). Finally, a small number of our expert sample had\nexpertise in a complementary field, but had not expressed detailed views\non AI risk in public (2\/21 expert interviewees). Most of our expert\nsample held senior positions within their fields, as professors,\ndirectors of organizations, leaders of research teams, or similar (13\/21\nexpert interviewees).<\/p>\n\n\n\n

            Our expert sample skewed toward the top left quadrant in figure\n2.1.1, \u201chigh risk\/short timelines.\u201d Of 21 expert participants, 13\nestimated the risk of extinction from AI by 2100 to be >10%. Only one\nof our expert sample estimated the risk to be <1% by 2100, whereas\nthe median expert in the XPT predicted 3%. Although we did not solicit\nAI progress timelines directly from interviewees, interview content\ngenerally suggested a positive relationship between beliefs in increased\nrisk and shorter timelines in our sample.<\/p>\n\n\n\n

            Because of this skew in our expert sample, we chose to ensure some\nrepresentation of the bottom two quadrants in figure 2.1.1 (low risk\nfrom AI) by selecting three superforecaster interviewees who forecast\n<10% probability of extinction from AI by 2100.<\/p>\n\n\n\n

            Interview process<\/h4>\n\n\n\n

            Interviews were 1-on-1, ran for roughly 60 minutes and followed a semi-structured format. By default, interviews aimed to trace one plausible path of increasingly strong signals of heightened AI risk at three successive timepoints before 2100.18<\/a><\/sup> Interviewers19<\/a><\/sup> were allowed some latitude for individual approaches, but generally followed this basic structure:20<\/a><\/sup><\/p>\n\n\n\n

              \n
            1. Introduction, task instructions<\/li>\n\n\n\n
            2. Elicitation of P(AI-related extinction by 2100)<\/li>\n\n\n\n
            3. Node generation<\/li>\n\n\n\n
            4. Wrap-up questions<\/li>\n<\/ol>\n\n\n\n
              \"\"\/
              Figure 2.1.3:<\/strong> The conditional tree workflow (I)<\/figcaption><\/figure>\n\n\n\n

              Interviewees were first given a very brief summary of the aims of the project, a short explanation of conditional trees, and a statement of the goals of the interview. Interviewees were also told that they would be awarded $1,000 if a forecasting question derived from their interview was one of the \u201chighest value\u201d forecasting questions generated by the project.21<\/a><\/sup> This introductory section of the interview typically took 10 minutes or less.<\/p>\n\n\n\n

              Next, interviewees were asked to give their best guess probability for the project\u2019s \u201cultimate question,\u201d namely \u201cAI-related extinction by 2100,\u201d which was operationalized as in the 2022 XPT.22<\/a><\/sup> Following the probability elicitation, we sometimes asked participants warm-up questions, for instance asking them to name possible \u201cdriving forces\u201d influencing their views.<\/p>\n\n\n\n

              Interviewees would then begin the node generating phase of the interview, which comprised the majority of interview time. Although we began the project with a set of three predefined years to ask participants about (2030, 2050 and 2070),23<\/a><\/sup> it soon became clear that this was not the best choice of years for participants with short AI progress timelines. Therefore, we began in the node generating phase to ask participants to propose a suitable set of years for their own trees (see Appendix 3<\/a> for the distribution of years chosen).<\/p>\n\n\n\n

              For each node, we took interviewees through a process of\nbrainstorming, selection, and fleshing out. We would then elicit a\nprobability of AI extinction by 2100 conditional on the node. We will\nrefer to these pre-operationalization nodes as question\nprompts.<\/strong><\/p>\n\n\n\n

              Interviewers took detailed notes, and most interviews were recorded (with participants\u2019 permission). Further details on interview technique can be found in Appendix 6<\/a>.<\/p>\n\n\n\n

              Operationalizing\nquestion prompts as forecasting questions<\/h4>\n\n\n\n

              Question prompts were generally not fully resolvable forecasting questions, though some were operationalized in more detail than others. We considered it an inefficient use of interview time to focus on constructing forecasting questions with detailed resolution criteria, and also not the comparative advantage of expert interviewees generally. Instead, an internal question-writing team24<\/a><\/sup> turned question prompts into fully operationalized forecasting questions, with the help of notes from the interview and feedback from the interviewer.<\/p>\n\n\n\n

              The primary goals of question writing in this project were:<\/p>\n\n\n\n

                \n
              1. To capture as much of a question prompt\u2019s original intent as\npossible, while still making questions highly resolvable.<\/li>\n\n\n\n
              2. To optimize the value of information from the question by\nadjusting thresholds or removing elements which made the probability of\na positive or negative resolution too extreme.<\/li>\n<\/ol>\n\n\n\n

                We developed a template for the question-writing process, which\nencouraged question writers to first consider multiple distinct ways the\ninterview node could be operationalized. They then analyzed these\noptions with respect to several important criteria:<\/p>\n\n\n\n