Skip to main content
Digital Fluency

Oecd 2024 — Survey Of Adult Skills Readers Companion

Research · grey-literature

Citation: OECD (2024). Survey of Adult Skills 2023 — Reader's Companion. OECD Skills Studies. Paris: OECD Publishing. https://doi.org/10.1787/3639d1e2-en

PDF: oecd-2024-survey-of-adult-skills-readers-companion.pdf · 134 pages · December 2024

Why this is in our research: This is the OECD's official methodological companion to the 2023 PIAAC results. It is the primary source for (a) what the new Adaptive Problem Solving (APS) domain actually measures, (b) why PSTRE was retired, (c) the Level 1–4 descriptors we cite in the pitch, and (d) the framework dimensions that determine whether APS is a valid digital-fluency proxy. We rely on this document for the thesis footnote [^aps] in pitch-and-overview.md.

What it is

The Reader's Companion is the OECD's plain-language methodological volume for the 2023 PIAAC cycle. It accompanies the headline results report (Do Adults Have the Skills They Need to Thrive in a Changing World?) and is intended to help readers interpret what the survey measured and how. Six chapters: the human-capital framing, the assessment domains, results reporting, the background questionnaire, methodology, and the relationship to PIAAC Cycle 1 and other adult-skills surveys.

The relevant sections for our project:

Why PSTRE was retired (verbatim)

The OECD's stated rationale, Reader's Companion p. 37:

"Adaptive problem solving (APS) is a new domain whose conceptual framework was specifically developed for the 2023 Survey of Adult Skills. APS was introduced to replace the assessment of problem solving in technology-rich environments (PSTRE) that was administered in the first cycle of the survey. PSTRE was exclusively focused on problems in digital environments and, as a consequence, it conflated problem solving and information and communication technologies (ICT) skills, as only test-takers with some (basic) ICT skills could participate and display proficiency in this domain. There was a sizeable non-response due to lack of familiarity with ICT devices or poor computer skills (between 8% and 57%, depending on the country). APS was therefore conceptualised as 'general' problem solving, relevant to a range of information environments and contexts and not limited to digitally embedded problems."

Three explicit reasons in this passage:

  1. Construct conflation. PSTRE could not separate "can the respondent problem-solve" from "can the respondent operate a computer." The OECD calls this a confound.
  2. Coverage failure. 8–57% per country (a fatal range) were excluded from PSTRE because they could not pass the ICT screener — which means the PSTRE statistic was systematically biased toward more digitally capable adults.
  3. Conceptual broadening. APS was reframed to cover physical, social, and digital information environments — measuring adaptive cognition rather than tool-use.

What APS measures

Definition (Greiff et al. 2021, quoted in Reader's Companion p. 38):

"[APS is] the capacity to achieve one's goals in a dynamic situation, in which a method for solution is not immediately available. It requires engaging in cognitive and metacognitive processes to define the problem, search for information, and apply a solution in a variety of information environments and contexts."

Two features that distinguish APS from PSTRE (p. 38):

"One critical feature of adaptive problem solving tasks is that these problems involve dynamic situations, where resources and information needed to solve a problem are not readily available, or some aspect of the problem changes while the solution is being developed. Additionally, it emphasises the importance of metacognition. Metacognitive skills are called upon in order to monitor the problem-solving process and adapt solution strategies as a problem changes."

Item-pool composition (65 items total, Table 2.4, p. 40):

Information environment Items %
Digital 26 40%
Physical 24 37%
Social 15 23%

APS is not a digital-skills measure. 60% of items are non-digital. The OECD repeatedly stresses that the construct is general adaptive cognition; the digital interface is a delivery mechanism for some items but not the construct.

Sample item — "Best Route" (p. 41–42): Respondents use an interactive map to drop a child at school by 8:30, buy groceries (20 min), and return home before a 9:30 meeting. Question 1 is static. Question 2 introduces dynamism: "It is now 8:30. You have dropped your child at school. You receive a news alert that your chosen shop has closed due to a water main break and flooding. Adjust your route to accomplish the rest of your tasks." This is the prototypical APS task — the problem changes mid-solution, and the respondent has to re-plan against multiple constraints.

APS proficiency level descriptors (Table 3.6, p. 54)

Verbatim from the Reader's Companion:

Below Level 1 — score < 176

"Adults performing below Level 1 understand very simple static problems situated within a clearly structured environment. Problems contain no invisible elements, no irrelevant information that might distract from the core of the problem, and typically only require a single step to solve the problem. Adults at this proficiency level are able to engage in the basic cognitive processes required to solve problems if explicit support is given and if they are prompted to do so."

Level 1 — score 176 to <226

"Adults at Level 1 are able to understand simple problems, and develop and implement solutions to solve them. Problems contain a limited number of elements and little to no irrelevant information. Solutions at this level are simple and consist of a limited number of steps. Problems are embedded in a context that includes one or two sources of information and presents a single, explicitly defined goal."

Cognitive processes:

  • develop mental models of simple and clearly structured problems
  • understand connections between tasks and stimuli that are explicit and embedded in a well-structured environment
  • solve problems that do not change and thereby do not require adaptivity

Level 2 — score 226 to <276

"Adults at Level 2 can identify and apply solutions that consist of several steps to problems that require one target variable to be considered in order to judge whether the problem has been solved. In dynamic problems that exhibit change, adults at this level can identify relevant information if they are prompted about specific aspects of the change or if changes are transparent, occur only one at a time, relate to a single problem feature and are easily accessible. Problems at this level are presented in well-structured environments and contain only a few information elements with direct relevance to the problem. Minor impasses may be introduced but these can be resolved easily by adjusting the initial problem-solving procedure."

Level 3 — score 276 to <326

"Adults at Level 3 understand problems that are either static but more complex, or have an average to high level of dynamics. They can solve problems with multiple constraints or problems that require the attainment of several goals in parallel. In problems that change and require them to adapt, they can deal with frequent and, to some extent, continuous changes. They discriminate between changes that are relevant and those that are less relevant or unrelated to the problem."

Level 4 — score ≥326

"Adults at Level 4 are able to define the nature of problems in ill-structured and information-rich contexts. They can integrate multiple sources of information and their interactions, identify and disregard irrelevant information, and formulate relevant cues. They are able to distinguish between relevant and irrelevant changes, predict future developments of the problem situation, and consider multiple criteria simultaneously to judge whether the solution process is likely to lead to success."

Why this matters for the thesis

The Level descriptors are why we can cite APS data as evidence about AI workflow readiness even though APS is not a digital-skills measure per se. The cognitive shape of "useful AI work" — hold a goal, decompose, recover from errors, evaluate output as conditions change — is exactly the APS construct.

A reviewer who pushes back with "but APS isn't a digital skills measure" can be answered with: that's the point. The cognitive operation below Level 2 is "cannot handle a problem that changes mid-task, that has multiple relevant criteria, that has more than one source of information." That operation is what AI workflows demand, regardless of the interface. The 32% Level-1-or-below figure is therefore a defensible floor for AI-augmentation readiness.

The Level 1 descriptor includes the line "solve problems that do not change and thereby do not require adaptivity." That is a remarkable plain-English match for "cannot use an AI tool inside a real workflow" — AI workflows are nothing but problems that change as you iterate. We should consider quoting this phrase directly in the pitch.

How to cite from this document

Open questions / things to verify later