{"id":961,"date":"2024-11-17T12:00:00","date_gmt":"2024-11-17T12:00:00","guid":{"rendered":"https:\/\/forecastingresearch.org\/?post_type=research&#038;p=961"},"modified":"2026-04-17T17:32:18","modified_gmt":"2026-04-17T17:32:18","slug":"identifying-good-forecasters-via-adaptive-cognitive-tests","status":"publish","type":"research","link":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests","title":{"rendered":"Identifying good forecasters via adaptive cognitive tests"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Abstract<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Assessing forecasting performance is a time-intensive activity, often requiring months or years before we know whether or not the reported forecasts were accurate. Cognitive tests can be quickly administered and are predictive of forecasting performance, but it is unclear which and how many tests are optimal. In this study, we develop adaptive cognitive tests that optimize the selection and efficiency of cognitive tests to assess forecasters of different skill levels. The tests are based on item response models and the adaptive testing procedures commonly used in educational testing. We show how the procedures can select highly informative cognitive tests from a larger battery of tests, thereby reducing the time taken to administer the tests. We use a second, independent dataset to show that the selected tests yield scores that are highly related to out-of-sample forecasting performance. The approach enables real-time, adaptive testing, providing immediate insights into forecasting talent in practical contexts.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"btn orange\" href=\"https:\/\/doi.org\/10.48550\/arXiv.2411.11126\" target=\"_blank\" rel=\"noreferrer noopener\">Read the full paper <svg width=\"7\" height=\"9\" viewBox=\"0 0 7 9\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\">\n  <path d=\"M0.000156283 8.60806L4.22416 4.33606V4.24006L0.000156283 6.10352e-05H1.80816L6.06416 4.28806L1.80816 8.60806H0.000156283Z\" fill=\"#102B23\"\/>\n<\/svg>\n<svg width=\"8\" height=\"10\" viewBox=\"0 0 8 10\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\">\n  <path d=\"M0.601719 8.85794L4.82572 4.58594V4.48994L0.601719 0.249939H2.40972L6.66572 4.53794L2.40972 8.85794H0.601719Z\" fill=\"#102B23\"\/>\n<\/svg><\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"In this study, we develop adaptive cognitive tests that optimize the selection and efficiency of cognitive tests to assess forecasters of different skill levels.","protected":false},"featured_media":864,"template":"","meta":{"footnotes":""},"research_type":[5],"class_list":["post-961","research","type-research","status-publish","has-post-thumbnail","hentry","research_type-academic-article"],"acf":[],"yoast_head":"<title>Identifying good forecasters via adaptive cognitive tests &#8211; Forecasting Research Institute<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Identifying good forecasters via adaptive cognitive tests &#8211; Forecasting Research Institute\" \/>\n<meta property=\"og:description\" content=\"In this study, we develop adaptive cognitive tests that optimize the selection and efficiency of cognitive tests to assess forecasters of different skill levels.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests\" \/>\n<meta property=\"og:site_name\" content=\"Forecasting Research Institute\" \/>\n<meta property=\"article:modified_time\" content=\"2026-04-17T17:32:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/forecastingresearch.org\/wp-content\/uploads\/2025\/09\/FRI-illustration-library-5.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1232\" \/>\n\t<meta property=\"og:image:height\" content=\"928\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests\",\"url\":\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests\",\"name\":\"Identifying good forecasters via adaptive cognitive tests &#8211; Forecasting Research Institute\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/forecastingresearch.org\\\/wp-content\\\/uploads\\\/2025\\\/09\\\/FRI-illustration-library-5.jpg\",\"datePublished\":\"2024-11-17T12:00:00+00:00\",\"dateModified\":\"2026-04-17T17:32:18+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests#primaryimage\",\"url\":\"https:\\\/\\\/forecastingresearch.org\\\/wp-content\\\/uploads\\\/2025\\\/09\\\/FRI-illustration-library-5.jpg\",\"contentUrl\":\"https:\\\/\\\/forecastingresearch.org\\\/wp-content\\\/uploads\\\/2025\\\/09\\\/FRI-illustration-library-5.jpg\",\"width\":1232,\"height\":928},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/research\\\/identifying-good-forecasters-via-adaptive-cognitive-tests#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/forecastingresearch.org\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Identifying good forecasters via adaptive cognitive tests\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/forecastingresearch.org\\\/#website\",\"url\":\"https:\\\/\\\/forecastingresearch.org\\\/\",\"name\":\"Forecasting Research Institute\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/forecastingresearch.org\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>","yoast_head_json":{"title":"Identifying good forecasters via adaptive cognitive tests &#8211; Forecasting Research Institute","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests","og_locale":"en_US","og_type":"article","og_title":"Identifying good forecasters via adaptive cognitive tests &#8211; Forecasting Research Institute","og_description":"In this study, we develop adaptive cognitive tests that optimize the selection and efficiency of cognitive tests to assess forecasters of different skill levels.","og_url":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests","og_site_name":"Forecasting Research Institute","article_modified_time":"2026-04-17T17:32:18+00:00","og_image":[{"width":1232,"height":928,"url":"https:\/\/forecastingresearch.org\/wp-content\/uploads\/2025\/09\/FRI-illustration-library-5.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests","url":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests","name":"Identifying good forecasters via adaptive cognitive tests &#8211; Forecasting Research Institute","isPartOf":{"@id":"https:\/\/forecastingresearch.org\/#website"},"primaryImageOfPage":{"@id":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests#primaryimage"},"image":{"@id":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests#primaryimage"},"thumbnailUrl":"https:\/\/forecastingresearch.org\/wp-content\/uploads\/2025\/09\/FRI-illustration-library-5.jpg","datePublished":"2024-11-17T12:00:00+00:00","dateModified":"2026-04-17T17:32:18+00:00","breadcrumb":{"@id":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests#primaryimage","url":"https:\/\/forecastingresearch.org\/wp-content\/uploads\/2025\/09\/FRI-illustration-library-5.jpg","contentUrl":"https:\/\/forecastingresearch.org\/wp-content\/uploads\/2025\/09\/FRI-illustration-library-5.jpg","width":1232,"height":928},{"@type":"BreadcrumbList","@id":"https:\/\/forecastingresearch.org\/research\/identifying-good-forecasters-via-adaptive-cognitive-tests#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/forecastingresearch.org\/"},{"@type":"ListItem","position":2,"name":"Identifying good forecasters via adaptive cognitive tests"}]},{"@type":"WebSite","@id":"https:\/\/forecastingresearch.org\/#website","url":"https:\/\/forecastingresearch.org\/","name":"Forecasting Research Institute","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/forecastingresearch.org\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/research\/961","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/research"}],"about":[{"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/types\/research"}],"version-history":[{"count":7,"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/research\/961\/revisions"}],"predecessor-version":[{"id":1790,"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/research\/961\/revisions\/1790"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/media\/864"}],"wp:attachment":[{"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/media?parent=961"}],"wp:term":[{"taxonomy":"research_type","embeddable":true,"href":"https:\/\/forecastingresearch.org\/api\/wp\/v2\/research_type?post=961"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}