The Psychological Record

https://doi.org/10.1007/s40732-021-00457-y

ORIGINAL ARTICLE

Assessing the Development of Relational Framing in Young Children

Elle B. Kirsten 1 & Ian Stewart 1

Accepted: 1 February 2021

# Association for Behavior Analysis International 2021

Abstract
Relational frame theory (RFT) sees operant acquisition of various patterns of relational framing (frames) as key to linguistic and
cognitive development, and it has explored the emergence of a range of psychological phenomena (e.g., analogy, perspective-
taking) in these terms. One potentially important advance for RFT research is to obtain more detailed information on the
normative development of relational framing in childhood. This was one of the aims of the present study, which sought to
measure relational responding of various types and at various levels of complexity in young children across a range of ages. A
second aim of the study was to focus in particular on analogy, or the relating of relations, as one particularly important pattern of
relational responding. The present study examined a range of frames including coordination, comparison, opposition, temporal-
ity, and hierarchy at a number of different levels of complexity (nonarbitrary relating, nonarbitrary relating of relations, arbitrarily
applicable relating, and arbitrarily applicable relating of relations) in young children ranging in age from 3 to 7. Performance
overall as well as under various subheadings was correlated with both age and intellectual ability. Outcomes and their implica-
tions are discussed.

Keywords Relational frame theory . Relational assessment . Normative development . Analogical relations . Children . Derived
relations

Relational frame theory (RFT) is a contextual behavioral ac- on physical properties and, from an RFT perspective, is the
count of human language and cognition (Hayes et al., 2001a; result of exposure to multiple exemplar training (MET) pro-
Stewart, 2016; Stewart & Roche, 2013) that views arbitrarily vided by the socioverbal community in which this particular
applicable derived relational responding (AADRR), or rela- pattern of comparison framing has been reinforced across a
tional framing, as the key operant underlying these repertoires. variety of different stimuli in the presence of these cues. In this
Many species can be trained to engage in nonarbitrarily appli- and many other frames, such training initially involves nonar-
cable relational responding (NAARR), which involves relat- bitrary relations (e.g., learning to pick a relatively smaller or
ing stimuli based on their physical properties (e.g., selecting a larger quantity of items on being asked to select “less” or
stimulus physically similar, different, smaller, or larger than “more,” respectively) but eventually the child can apply the
another). However, humans alone can learn AADRR, which response pattern in purely arbitrary situations in which nonar-
involves relating stimuli based on contextual cues that specify bitrary relations (e.g., physical size) no longer function as a
the relation rather than on the formal or physical properties of guide, such as in the coin example.
those stimuli. According to RFT, all forms of framing are characterized
For example, consider AADRR of comparison. If I show a by three properties, and it is these properties that underlie the
sufficiently verbally advanced child two novel, similarly sized substantial generativity of language (Stewart et al., 2013).
coins, A and B, and tell them that A is worth less than B, then Mutual entailment is the property of bidirectionality of stim-
they will readily derive that B is worth more than A. This ulus relations, whereby if A is related to B, then B will be
derivation is based on the cues “more” and “less” rather than related to A (e.g., if A is less than B, then B is more than A).
Combinatorial entailment involves the combination of previ-
ously acquired relations to allow derivation of novel relations
* Elle B. Kirsten (e.g., if A is less than B, and B is less than C, then A is less
e.kirsten2@nuigalway.ie
than C, and C is more than A). Transformation of functions is
the property whereby the psychological functions of a
1
National University of Ireland, Galway, Ireland
 Psychol Rec

stimulus in a derived relation can change depending on the relations. However, as yet there is little or no empirical work
functions of other stimuli in the relation as well as the nature of on the normative acquisition of frames. Previous research on
the relation itself. For example, if a child has already learned relational framing and age has found that particular relational
that A has monetary value and then derives that a novel coin C repertoires (deictic and hierarchical, respectively) increase as a
is worth more than A, they will choose C over A despite the function of age in typically developing children (McHugh
fact that C is novel. In an alternative scenario, if an arbitrary et al., 2004; Mulhern et al., 2017). There is tentative evidence
stimulus A has acquired an aversive function (e.g., by being suggesting that equivalence emerges at a young age (Lipkens
paired with shock), then despite not being directly paired with et al., 1993), but on a number of other basic frames there is
shock itself, C might come to be perceived as even more little relevant research. Barnes-Holmes, Barnes-Holmes, and
aversive than A based on the derived “larger than” relation. Smeets (2004) trained comparison in 4- to 6-year-old children,
Indeed, such an outcome has been empirically demonstrated and Barnes-Holmes, Barnes-Holmes, and Smeets (2004)
(see, e.g., Dougher et al., 2007). trained opposition in 4- to 6-year-old children, but no evi-
RFT research has by now provided evidence of a variety of dence was provided to suggest when these frames tend to
relational frames. Coordinate (sameness) framing or equiva- emerge under natural conditions. In the case of other frames
lence, as initially demonstrated by Sidman (1971), is the ear- such as distinction and temporality, for example, there is as yet
liest and most researched example of AADRR, but subsequent no relevant research. A key aim of the current cross-sectional
research has provided evidence of various other patterns of study therefore was to provide more comprehensive data on
relational framing, including difference (e.g., Steele & the emergence of basic patterns of framing across ages 3 to 7
Hayes, 1991), opposition (Barnes-Holmes et al., 2004), com- years. In this study, we investigated the normative develop-
parison (Dymond & Barnes, 1995), hierarchy (Gil et al., ment of a number of specific relational frames (coordination,
2014), analogy (Barnes et al., 1997), temporality (O’Hora comparison, opposition, temporality, and hierarchy) in young
et al., 2005), and deixis (McHugh et al., 2004). children and measured relational framing against standardized
Furthermore, in accordance with the thesis that relational tests of cognitive abilities. This is one of the first attempts to
framing is an operant, studies have not simply demonstrated provide a cross-sectional investigation of the acquisition of a
relational framing but have successfully assessed and trained variety of frames in a number of different age groups of
various patterns of framing in young children. For example, children.
Barnes-Holmes, Barnes-Holmes, and Smeets (2004) success- Apart from this, another focus of this relational assessment
fully trained comparative AADRR as generalized operant be- and of the study in general was on the relating of relations, or
havior using multiple exemplar training (MET) in 4- to 6-year analogical responding, an important form of relational fram-
old children and this outcome has since been successfully ing, and one that is relevant for intellectual development
replicated and extended (Berens & Hayes, 2007; Gorham (Gentner & Christie, 2010; Goswami & Brown, 1989;
et al., 2009). In several of these studies, some participants Sternberg, 1977). Analogy is pervasive in human language
who did not readily learn to derive the arbitrary relations were and critically important in various key domains of human
aided in doing so via additional training in nonarbitrary rela- activity, including science, technology, and education.
tions, which emphasized the importance of NAARR as a pre- Furthermore, it is frequently used as a metric of intelligent
requisite repertoire for AADRR. Apart from demonstrating behavior (e.g., the Miller Analogies Test, the Law School
that frames not already in place can be established, RFT re- Admissions Test; Gentner, 1983; Morsanyi & Holyoak,
search has also shown that already acquired framing reper- 2010; Sternberg, 1977; Stewart et al., 2004, 2020). Given
toires can be substantially strengthened and that doing so the apparent importance of analogy for intellectual develop-
can affect general cognitive ability (Cassidy et al., 2011; ment, cognitive developmental psychologists have examined
Hayes & Stewart, 2016; Mulhern et al., 2018). the emergence of this skill in young children. Early re-
Hence, RFT has provided substantive evidence that searchers believed that analogical reasoning developed at the
AADRR underlies human language and cognition (Hayes et age of 12 or later, and that children younger than this relied on
al., 2001a), including work showing that training this reper- simple associative strategies (Levinson & Carpenter, 1974;
toire can boost intellectual skill. However, despite the evi- Lunzer, 1965; Piaget et al., 1977/Piaget et al., 2001;
dence that relational framing is a core skill underlying cogni- Sternberg & Nigro, 1980). However, it has been argued that
tive ability, research on the sequence of frame acquisition and children as young as four can show analogical reasoning
the normative development of relational responding remains (Goswami & Brown, 1990) with prior knowledge playing a
limited. Luciano et al. (2009) proposed a training sequence for critical role. For example, Goswami and Brown (1989) exam-
early frames of coordination, opposition, distinction, compar- ined the effect of children’s previous experiences and found
ison, and hierarchy and provided suggestions for intervention, that children as young as 3 could complete analogical tasks
including multiple exemplar training, bidirectional relational when they had relevant knowledge about the domains in-
training, and transitioning from nonarbitrary to arbitrary volved (see also Alexander et al., 1989; Goswami, 1989).
Psychol Rec

Although research on analogy has been mostly the province of and testing with compound–compound matching tasks with
cognitive psychologists, during the last 2 decades behavior trained relations (e.g., A1B1–A3B3 and A1B2–A1B3) before
analysts have also begun to research analogy. The impetus they could successfully pass the derived compound relations
for this has primarily come from researchers who take an (BC–BC) test. Carpentier et al. (2003) extended this work by
RFT perspective. assessing whether this additional training could also facilitate
Working within an RFT framework, Barnes et al. (1997) the 5-year-old children’s ability to pass equivalence–
provided the first functional analytic definition of analogy as equivalence tests before receiving the prior equivalence tests.
the derivation of a sameness or equivalence relation between This was something that Barnes et al. had shown that adults
equivalence relations, called “equivalence–equivalence” and 9-year-old children could do and this was replicated by
responding. For example, consider the analogy apple is to Carpentier et al. (2003). However, despite providing consid-
orange as dog is to sheep. In this case, apple and orange erable additional training, only 2 of 18 of the 5-year-old par-
participate in an equivalence relation in the context of fruit; ticipants were successful in this task. The Carpentier et al.
and dog and sheep participate in an equivalence relation in the (2002, 2003) studies thus provided additional insight into
context of animal, and thus, because these are both equiva- the development of equivalence–equivalence responding as
lence relations, we can derive a relation of equivalence be- a functional analytic model of analogy. By providing a pre-
tween the relations themselves. In order to empirically model cise, functional-analytic model of this behavior it could be
this phenomenon, Barnes et al. first trained and tested four 3- argued that this work has shed additional light on this phe-
member equivalence relations in adults and 9-year-old chil- nomenon beyond that provided by mainstream, cognitive psy-
dren. They used a matching-to-sample (MTS) procedure to chological work by not only confirming a developmental di-
train conditional discriminations amongst three-letter non- vide in the analogical ability at a particular age but also sug-
sense syllables (coded using alphanumeric designations) as gesting how additional training might remediate in this
follows: A1→B1, A1→C1, A2→B2, A2→C2, A3→B3, respect.
A3→C3, A4→B4, A4→C4, and then tested for derivation In summary, analogy is an important type of emergent
of the following four untrained relations: B1↔C1, B2↔C2, intellectual ability. RFT has examined and modeled analogy
B3↔C3, B4↔C4. After participants passed these equiva- in young children and provided an important contribution to
lence tests, they were then tested for the derivation of equiv- the understanding of the acquisition of this ability. Thus, RFT
alence relations between equivalence (and nonequivalence) has offered a substantive model of analogy, and has identified
relations themselves (i.e., equivalence–equivalence analogy as an important emergent repertoire. Hence, we
responding). This involved using compound stimuli com- wanted to focus on this particular ability, and placed an em-
prised of either two nonsense syllables that were equivalent phasis on analogy in our relational assessment protocol. One
or two that were nonequivalent. Participants were required to way in which we wished to extend the previous work was by
choose an equivalent pair in the presence of an equivalent pair also investigating nonarbitrary analogy, or relational
(i.e., equivalence–equivalence) and a nonequivalent pair in the matching, which RFT research has not examined before. We
presence of a non-equivalent pair (i.e., nonequivalence–non- included this in addition to arbitrary analogy because as sug-
equivalence). For example, given B3C3 and B3C4 as com- gested above in the introduction, nonarbitrary relational
parisons, if the sample was B1C1 then they had to choose responding is an important foundation for arbitrary relational
B3C3, whereas if B1C2 was the sample, they had to choose responding.
B3C4. All participants related equivalence relations to other In the present study, we looked at relational framing across
equivalence relations, and nonequivalence relations to other a number of different frames (including coordination, compar-
nonequivalence relations, and thus this constituted a basic ison, opposition, temporality, and hierarchy), across four dif-
model of analogical reasoning. ferent levels (including nonarbitrary relations, nonarbitrary
In the wake of Barnes et al. (1997), one stream of research analogy, arbitrary relations, and arbitrary analogy), and across
used their equivalence-equivalence model to investigate the a number of different age groups (in particular, 3- to 4-, 4- to
emergence of analogy in young children (Carpentier et al., 5-, 5- to 6-, and 6- to 7-year-old children). As previously
2002, 2003). Carpentier et al. (2002) used the equivalence– suggested, one key purpose of the present study was to extend
equivalence paradigm to investigate analogy in a range of age previous work on the development of relational framing
groups including adults and 9- and 5-year-old children. As in broadly by examining the development of a range of different
the original Barnes et al. study, they found that adults and 9- relational frames in young children. Previous RFT studies
year-old participants readily showed equivalence–equivalence have used a number of different methodologies to examine
responding. In contrast, the 5-year-old children, although and train relational framing in young children. One of the most
readily passing equivalence testing, initially failed to show efficient methods is the relational evaluation procedure (REP).
equivalence–equivalence responding and required additional This methodology allows participants to report on or evaluate
training before doing so. In particular, they required training sets of arbitrarily applicable relations defined by various sets
 Psychol Rec

of contextual cues. In one of the most impressive examples of Demographic information for the participants is summarized
the applied educational utility of this technology, Cassidy et al. in Table 1. Participant ages ranged from 36 to 84 months (M =
(2011) and various follow-up studies (e.g., Cassidy et al., 2016; 59.96, SD = 13.76), including five 3- to 4-year-old (M = 39.8
Hayes & Stewart, 2016) have used the REP to assess and train months, SD = 3.9), six 4- to 5-year-old (M = 53.33 months, SD
AADRR by presenting statements involving nonsense words = 2.73), six 5- to 6-year-old (M = 65.67 months, SD = 3.08),
juxtaposed with contextual cues (e.g., “CUG is the SAME as and seven 6- to 7-year-old participants (M = 75.14 months, SD
DAX,” “DAX is the SAME as YIM”), and asking participants = 4.18). None of the participants had any known developmen-
(typically children from age 10 to adolescence) questions that tal or intellectual disabilities. Full-scale IQ scores, which were
required them to derive relations based on those presented state- obtained for the purposes of this study, ranged from 94 to 132
ments (e.g., “Is DAX the SAME as CUG?” “Is CUG the same as (M = 113.25, SD = 10.59).
YIM?”). Training with this methodology is extremely efficient The assessment measures were administered by the re-
because multiple exemplars of relational patterns can be easily searcher (the first author of the present study) at the partici-
generated and readily presented, and indeed training with this pants’ school in a separate, quiet classroom within the school
variation of the REP protocol has been shown to substantially building. When conducting the assessments, the researcher
boost children’s intellectual performance. and the participant were seated at a child-sized school desk;
Given that a key goal of the present study was to assess rela- the participant sat facing the desk and the researcher sat at the
tional framing across various levels, it was decided to employ a side of the desk facing the participant and with a full view of
“relational statement” format similar in some respects to that used the testing materials in front of the participant.
by Cassidy et al. (2011). There was one critical difference, how-
ever. In Cassidy et al., participants were required to read sentences Materials and Apparatus
specifying and/or querying the relations between nonsense words
(e.g., “CUG is the same as DAX”). Considering the range of ages The materials used in this study included a standardized mea-
in the present study (i.e., from 3 to 7 years), it seemed probable that sure of intelligence, the Stanford-Binet Intelligence Scales-5th
some participants, especially the younger ones, were not yet Edition for Early Childhood (SB5; Roid, 2003), and the rela-
readers or had minimal reading skills. Hence, in order to allow tional assessment. The relational assessment was presented on
for minimal reading skills, and thus ensure that all participants a computer; Stages 1 and 2 of the assessment were presented
could be assessed and trained equally effectively on AADRR, on an Apple iPad Air 2 using Apple Keynote software, and
the stages in the relational assessment focused on this repertoire Stages 3 and 4 were presented on an Acer Chromebook R 11
utilized colored shapes, single letters, and audio options. It was using Microsoft PowerPoint software.
envisaged that use of this format would allow us to draw on the
advantages of the REP (facilitating more efficient and extensive The Stanford-Binet Intelligence Scales-5th Edition for Early
testing of AADRR than MTS) with a younger set of participants. Childhood
The training and testing format utilized in the current study
was employed in the context of the multistage relational as- The SB5 is an assessment of intelligence and cognitive abili-
sessment that allowed testing of a range of different types of ties and is considered the standard measure of global intellec-
relations. Also, in accordance with the broader aims of exam- tual ability for children and adults aged 2–85+ (Roid, 2003).
ining the development of relational framing more broadly, it In order to provide a reliable profile of differential abilities, the
did so at a number of different levels of complexity, including SB5 yields three composite scores, including Full Scale IQ
(1) nonarbitrary relations, (2) relating of nonarbitrary rela- (FSIQ), Verbal IQ (VIQ), and Nonverbal IQ (NVIQ). Full
tions, (3) arbitrary relations, and (4) relating of arbitrary rela- Scale IQ, VIQ, and NVIQ composite scores have high reli-
tions. In addition, participants’ relational performance across ability coefficients ranging from .95 for VIQ to .98 for FSIQ
and within all levels and frames was correlated with their age
and intellectual performance, as assessed on a standardized
test of intellectual functioning, namely the Stanford-Binet Table 1 Participant demographic information
Intelligence Scales (5th edition) for Early Childhood (SB5).
N % FSIQ SD

Gender Male 10 42
Method Female 14 58
Age (months) 36–47 5 21 118.60 12.76
Participants and Settings 48–59 6 25 110.83 9.00
60–71 6 25 114.67 4.37
Participants included 24 students (14 females, 10 males) at- 72–84 7 29 110.29 14.01
tending a private, nonsecular U.S. school in New Jersey.
Psychol Rec

(Roid, 2003). The validity of the SB5 is supported based on its coordination, comparison, opposition, and temporal sub-
correlation with a number of alternative intelligence tests in- stages, and boxes of varied sizes and colors in the hierarchy
cluding the Wechsler Preschool and Primary Scale of substage. The coordination substage included MTS trials of a
Intelligence-Revised (WPPSI-R), and the Woodcock- sample and three comparison stimuli. The comparison sub-
Johnson III Tests of Cognitive Abilities-General Intellectual stage included trials in which the participant was shown a
Ability-Standard (WJ III GIA; Garred & Gilmore, 2009). In sample and three comparisons; the comparisons were identical
the present study, in order to calculate correlations between to the sample except for size, and included one bigger, one
the nonnorm-referenced relational assessment scores and SB5 smaller, and one identical comparison. The opposition sub-
scores, raw scores were used for SB5 total, Verbal, and stage included trials in which the participant was shown a
Nonverbal scores. sample and three comparisons identical to the sample except
for variance on one dimension on a gradient scale. For exam-
The Relational Assessment ple, if the sample was a black square, the comparisons includ-
ed a white, gray, and black square. The temporality substage
The relational assessment examined different patterns of rela- included trials in which the participant was instructed to watch
tional responding across four stages, including Stage 1: non- the iPad screen for sequentially appearing shapes. The first
arbitrary (physical) relations; Stage 2: nonarbitrary analogical stimulus appeared 0.5 s after the onset of a new trial, and the
relations (relations between physical relations); Stage 3: arbi- second stimulus appeared 1.0 s after the first stimulus. The
trary (abstract) relations (relational frames); and Stage 4: arbi- hierarchy substage included trials in which the participant was
trary analogical relations (relations between abstract rela- shown two or three different sized and different colored boxes.
tions). Within each stage were five substages focused on par-
ticular relations, including coordination, comparison, opposi- Stage 2: Nonarbitrary Analogical Relations In Stage 2, all
tion, temporality, and hierarchy. Each substage included 10 stimuli presented were compound stimuli. In all substages, a
trials (see Figure 1 for a schematic presentation of the relation- sample compound stimulus was presented at the top of the
al assessment). In what follows, we present the general layout screen, and two comparison compound stimuli were presented
of the stimuli for each of the stages and substages in the as- below the sample. The stimuli used in the sample compound
sessment, and in the procedure section we will describe the were always different from the stimuli used in the comparison
administration of the assessment. In Stages 1 and 2 of the compounds. In the coordination substage, the sample was
relational assessment, the sample stimulus (in Stage 2, a com- composed of either two identical or two nonidentical pictures,
pound stimulus) was presented at the top of the computer and the comparisons included compound stimuli composed of
screen, and two comparison stimuli (in Stage 2, compound identical stimuli and nonidentical stimuli. In the comparison
stimuli) were presented at the bottom left and right of the substage, each of the sample and comparison compounds in-
screen (for an illustrative example, see Appendices A1 and cluded three stimuli that were similar except for size. Two of
A2). In Stage 3, a relational network of stimuli was presented the three stimuli in each compound were outlined in red, and
in the middle of the screen (see Appendix A3), whereas in an arrow indicated which stimulus to compare with another
Stage 4 a relational network of stimuli was presented at the stimulus in the compound (e.g., from smaller to bigger or
top of the screen, a sample compound stimulus was in the bigger to smaller). In the opposition substage, each of the
middle of the screen, and the comparison compound stimuli sample and comparison compounds included three stimuli
were at the bottom left and right of the screen (see Appendix that varied on a gradient scale along one dimension (e.g., size,
A4). For all trials, stimuli were delineated with a solid black color, quantity). In each compound, two of the stimuli had a
line including a horizontal black line underneath the sample solid red line underneath and were either nonarbitrarily oppo-
and a vertical line between the comparison stimuli. site or not opposite. In the temporality substage, both the sam-
ple and comparison compounds included two stimuli which
Stage 1: Nonarbitrary Relations The relational stimuli includ- appeared onscreen either simultaneously or 0.5 s apart. In the
ed simple, monochromic pictures or shapes in the hierarchy substage, each trial included sample and comparison

Fig. 1 Schematic representation

of the relational assessment stages Stage 1: Nonarbitrary Relations Substages in Each Stage
and substages. Note. This figure
illustrates the elements and Stage 2: Nonarbitrary Analogical Relations •Coordination
sequence of the relational •Comparison
assessment •Opposition
Stage 3: Arbitrary Relations
•Temporality
•Hierarchy
Stage 4: Arbitrary Analogical Relations
 Psychol Rec

stimuli consisting of a square inside another square, and small, child who participated, and verbal consent was also obtained
blue dots located in either the innermost or outermost square from each of the participants.
or outside the squares.
Stage 1: Nonarbitrary Relations
Stage 3: Arbitrary Relations An adaptation of the REP format
was employed in all substages. The relational stimuli were In the coordination substage, participants were to match the
simple, monochromic shapes (circles, triangles, squares) sep- sample to a comparison upon hearing the instruction, “Which
arated by a single Latin letter indicating the contextual cue (S one of these at the bottom is like this one at the top?” In the
for Same, D for Different, M for More, L for Less, O for comparison substage, participants were asked to identify the
Opposite, B for Before, A for After, C for Contains, and I for correct comparison in relation to the sample upon hearing the
Inside), plus corresponding audio icons (which, when instruction, “Which one of these is bigger or smaller than the
touched, produced an audio recording of the contextual cue one at the top?” In the opposition substage, participants were
(e.g., “same”) through the laptop or computer speaker). asked to identify the correct comparison in relation to the
sample upon hearing the instruction, “Which one of these is
Stage 4: Arbitrary Analogical Relations The REP format was opposite to the one at the top?” In the temporality substage,
also used in Stage 4, including the use of monochromic circles participants were to identify the order in which the stimuli
for the relational stimuli plus visual and audio signals appeared upon hearing instructions such as, “Which one was
representing the contextual cues. The sample and comparison before or after the other one/was stimulus 1 before or after
compounds were combinations of the relata in the relational stimulus 2?” In the hierarchy substage, participants were to
network. identify where a box was in relation to another box upon
hearing instructions such as, “Which box is inside/which
box contains the other box?”
Procedure
Stage 2: Nonarbitrary Analogical Relations
At the start of the session, the researcher placed the iPad or
computer in front of the participant. Participants were provid- In Stage 2, all stimuli presented were compound stimuli; in all
ed with detailed instructions before they commenced with substages, the participants’ task was to match the sample
each substage in every stage. Responses were scored as either (compound stimulus) relation to the correct comparison (com-
correct or incorrect. A correct response was defined as touch- pound stimulus) relation. For example, in the coordination
ing or pointing to the correct comparison or providing a cor- substage, the sample was composed of either two identical
rect vocal response. An incorrect response was defined as or two nonidentical pictures, and the comparisons included a
touching or pointing to the incorrect comparison, touching compound stimulus composed of two identical stimuli and a
both correct and incorrect comparisons, providing an incorrect compound composed of two nonidentical stimuli. Participants
vocal response, not responding, or engaging in other behavior were required to match the sample to one of the comparisons
that could not be categorized as correct. Participants did not upon hearing the instruction, “Look at these at the top. Which
receive feedback for any responses during the relational as- one of these (point to the comparison stimuli) is like the ones
sessment or SB5 testing. Generalized reinforcement was pro- at the top?” A pretrial sample was presented before adminis-
vided for compliance and participation throughout the tering each substage in order to familiarize the participant with
assessments. the testing format (for an illustrative example, see Appendix
The sequence in which the two assessments were adminis- A2).
tered was randomized so that participants completed either the
relational assessment or the SB5 first. The SB5 was completed Stage 3: Arbitrary Relations
in one session, and the relational assessment was completed in
one to two sessions. For the relational assessment, each ses- Prior to starting Stage 3, a pretest was administered to famil-
sion lasted between 20 and 60 min, including scheduled iarize participants with the test format. The participant was
breaks every 10–15 min, or if the participant requested a shown a computer screen with a relational network, for exam-
break. For all participants, both assessments were conducted ple: [Red Circle] [S] [Blue Circle]. The assessor instructed the
within a maximum of 4 weeks of each other. participant to look at the screen and said, “The S means same.
Prior to conducting the study, ethical approval for recruit- If you can’t remember what the S means, you can tap on the S
ment of participants was obtained from the research ethics and the computer will tell you.” Once the participant was
committee of the host (research) institution. Consent for comfortable with the visual and audio stimuli the assessor
conducting the study was obtained from the principal of the provided the instruction, “Let’s start. We are going to pretend
New Jersey school. Caregiver consent was obtained for each these shapes like food, and we are going to talk about whether
Psychol Rec

they like or do not like the same food. Look at the screen in either contained each other or were inside each other. For
front of you.” A relational network was presented on the example, the participant saw: [Red Circle] [I] [Blue Circle]
screen, for example: [Red Circle] [S] [Blue Circle]; [Blue and the assessor said, “Red is inside Blue.” The assessor
Circle] [S] [Yellow Circle], and the assessor read, “Red likes would then present one of the following questions depending
the same food as Blue, and Blue likes the same food as on the particular trial-type: Is Blue inside Red? Does Blue
Yellow.” The assessor asked questions about the relational contain Red? Does Red contain Blue? Which one is inside?
network, including questions about directly given relations Which one contains the other one?
(e.g., “Does Red like the same food as Blue?”), questions
requiring mutual entailment (e.g., “Does Red like different Stage 4: Arbitrary Analogical Relations
food to Blue?”), and questions requiring combinatorial entail-
ment (e.g., “Does red like the same food as yellow?”). The The same relational network composed of circles and single
pretest included 10 yes/no questions. letters was used for all trials in each substage. The sample and
For all substages in Stage 3, the assessor first read the comparison compounds were combinations of the relata in the
relational network to the participant and then asked the trial relational network. For example, in the coordination substage,
questions. Questions became increasingly difficult and re- the relational network would be read as, “Blue is the same as
quired responses including directly trained, mutually entailed, yellow, yellow is the same as red, and red is different to
and combinatorially entailed relations. The coordination sub- green.” The sample compound may be [Blue : Red] and the
stage was like the pretest but introduced novel stimuli. comparison compounds may be [Yellow : Green] and [Red :
In the comparison substage, the relational network includ- Yellow] (see Appendix A4 for an illustrative example for each
ed colored circles and the contextual cues more (M) and less of coordination, comparison, opposition, temporality, and
(L). Participants were told that they were going to pretend-buy hierarchy). On each trial, the researcher read the relational
their favorite food or candy, “These circles are like coins and network to the participant and then delivered the instruction,
we are going to pretend that you can use them to buy your “Look at this one at the top (pointing to the sample com-
favorite food/candy.”1 For example, the participant saw: [Blue pound). Which one of these (pointing to each of the compar-
Circle] [L] [Red Circle] and the assessor said: “Blue buys less ison compounds in turn) is like this one at the top?” For ex-
than Red, so which coin should you take to the store to buy ample, given the relational network mentioned above, [Blue S
[insert participant’s favorite food]?” The correct choice, in this Yellow], [Yellow S Red], and [Red D Green], participants
case, would have been to select the red coin because red buys could derive the relation of sameness for a sample stimulus
more. [Blue : Red], a relation of sameness for a comparison stimulus
In the opposition substage, the relational network included [Red : Yellow], and a relation of difference for a comparison
the contextual cue (O) for opposite and colored circles stimulus [Yellow : Green]. This example was for the coordi-
representing coins. Participants were told that they were going nation substage; for each of the other substages, the partici-
to pretend-buy their favorite food or candy: “These circles are pants had to match the correct comparison relation to the ap-
like coins and we are going to pretend that you can use them to propriate sample relation, but the nature of the relations that
buy your favorite food/candy.” For example, the participant had to be matched depended on which substage it was. There
saw: [Red Circle] [O] [Blue Circle] and the assessor said, were 10 trials in each substage, including trials for directly
“Red buys many/a lot/a few/a little. Red is opposite to Blue, trained, mutually entailed, and combinatorially entailed rela-
which coin should you take to the store to buy [insert partic- tions in that order (i.e., increasing difficulty).
ipant’s favorite food]?” The correct choice on every trial was
the selection of the color worth a lot/many. Interobserver Agreement
In the temporal substage, the relational network included
colored squares and the contextual cues before (B) and after Interobserver agreement was calculated for 20% of all com-
(A). The assessor told the participant that the colored squares pleted relational assessment substages by a trained research
were in a race: “These silly little squares are racing, which one assistant. The research assistant was trained in data collection
reaches the finish line before/after the other one?” For exam- until they reached 100% accuracy prior to collecting IOA data.
ple, the participant saw: [Red Square] [B] [Blue Square] and Trial-by-trial IOA was calculated and ranged from 97% to
the assessor said, “Red was before Blue. Was Blue before/ 100% (M = 98.66%). Agreement across all measures was
after Red?” counted if both observers provided the same score for each
In the hierarchy substage, the relational network included item assessed, and disagreement was counted if one observer
colored circles and the contextual cues inside (I) and contains provided a different score compared to the other observer.
(C). The assessor told the participant that the colored circles Interobserver agreement was calculated for 24% of the verbal
(Vocabulary) and nonverbal (Object Series/Matrices) routing
1
Adaptation of the procedure used in Barnes-Holmes et al. (2004b). substages of the SB5. Trial-by-trial IOA was calculated and
 Psychol Rec

ranged from 94% to 100% (M = 97.83%). Treatment integrity for the 3- to 4-year-old cohort, from 40% to 58% correct (M =
was evaluated on 20% of all completed relational assessment 48%, SD = 12.15) for the 4- to 5-year-old cohort, from 50.5%
substages, and 23.5% of all SB5 routing substages by a trained to 71.5% correct (M = 60.4%, SD = 16.62) for the 5- to 6-year-
research assistant. The observer scored a + on each trial that old cohort, and from 68% to 78% correct (M = 71%, SD = 7.3)
the trainer was 1) observed to gain attention prior to the trial, for the 6- to 7-year-old cohort.
2) accurately presented the question, and 3) consequated ap- Mean scores within each stage showed improvements
propriately. Mean procedural integrity ranged from 97% to across age cohorts. The biggest improvement in relational
100% (M = 99%) for the relational assessment substages, assessment scores based on age occurred in the later stages,
and from 88% to 100% (M = 98.1%) for the SB5 routing that is, Stages 3 (arbitrary relations) and 4 (arbitrary analogy).
substages. For example, comparing the youngest with the oldest cohorts,
we see that in Stage 3, the 3- to 4-year-old cohort scored on
average 20% correct across all substages whereas the 6- to 7-
Results year-old cohort scored 75% correct, a difference of 55%. In
Stage 4, the 3- to 4-year-old cohort scored on average 10%
In the results section we will examine the relational assess- correct whereas the 6- to 7-year-old cohort scored 54% cor-
ment protocol scores, correlations between relational assess- rect, a difference of 44%. In contrast, in Stages 1 and 2, there
ment scores and age, correlations between relational assess- was less of an age-based gap in performance. In Stage 1, the 3-
ment scores and raw IQ, correlations within the protocol, and to 4-year-old cohort scored an average of 72% correct whereas
finally, the acquisition of relations across age groups. Results the 6- to 7-year-old cohort scored 86% correct, a difference of
for Stages 2 (nonarbitrary analogy) and 4 (arbitrary analogy) 14%, whereas in Stage 2, the 3- to 4-year-old cohort scored an
will receive particular attention in each subsection. In addi- average of 50% correct whereas the 6- to 7-year-old cohort
tion, because percentage of correct responding is provided as a scored 68% correct, a difference of 18%.
performance outcome across different trial types, in Appendix Apart from considering the general improvement based
2 we provide a table showing chance level of responding on age, we can also examine each stage to check for
(based on possible number of outcomes) for different trial particular discontinuities between specific age cohorts.
types. In the section below, in which we examine performance For example, in Stage 2 (nonarbitrary analogy), there is
across relational frames, we compare performance of the dif- a large gap in performance between the 4- to 5-year-old
ferent cohorts on different stages and for different relations cohort and 5- to 6-year-old cohort (M = 54% to 68%,
with chance level performance. respectively). These data show that the transition from
the 4- to 5-year-old to the 5- to 6-year-old age range
Relational Assessment Protocol Scores results in particular improvement in nonarbitrary analogy.
In Stage 3 (arbitrary relations), the biggest improvement
Table 2 shows scores on both the relational assessment (over- in performance occurred between the 3- to 4-year-old and
all and by stage) and the SB5 (full scale and subscale) both as 4- to 5-year-old participants (M = 20% to 56%, respec-
a function of the entire cohort as well as by age group. Total tively). These data suggest substantial development in ar-
relational assessment scores across all participants ranged bitrary relational framing between these ages. In Stage 4
from 26.5% to 78% correct (M = 56%, SD = 28.58). Stage 1 (arbitrary analogy), the biggest improvement in perfor-
(nonarbitrary relations) scores across all substages and all par- mance occurred between the 4- to 5-year-old and 5- to
ticipants ranged from 58% to 94% (M = 80%, SD = 5.19); 6-year-old cohorts (M = 8% and 34%, respectively).
Stage 2 (nonarbitrary analogy) scores ranged from 40% to This finding for arbitrary analogy is similar to that for
84% (M = 61%, SD = 5.46); Stage 3 (arbitrary relations) nonarbitrary analogy in Stage 2. The fact that there ap-
scores ranged from 0% to 88% (M = 54%, SD = 10.70); and pears to be substantive improvement in analogical ability
Stage 4 (arbitrary analogy) scores ranged from 0% to 66% (M between the 4- to 5-year-old and 5- to 6-year-old partici-
= 29%, SD = 12.53). pants at both the nonarbitrary and arbitrary levels supports
earlier research findings (e.g., Carpentier et al., 2002) that
Relational Assessment Protocol Scores and Age analogical ability begins to emerge around 5 years of age.
One final point concerns the difference in variability in
With regard to breakdown by age group, we can see relatively performance in arbitrary analogy between different co-
predictable patterns wherein relational assessment scores in- horts. In particular, the standard deviation decreased from
creased as a function of age and mean relational assessment 11.37 for the participants aged 5- to 6-years-old to 4.3 for
scores increased by approximately 10% for each additional participants aged 6- to 7-years-old. This suggests less var-
year in age. In particular, total relational assessment scores iable, more stable responding in the older cohort follow-
ranged from 26.5% to 52% correct (M = 38%, SD = 19.84) ing the initial emergence of analogy.
Psychol Rec

Table 2 Participant scores on the relational assessment and SB5

ID M/ Age (m) SB5- SB5- SB5- SB5- RA- RA1 RA2 RA3 RA4
F FSIQ NV V T T NAR NAA AR AA

3- to 4-year-old cohort
S4 F 36 127 46 57 103 62 29 24 9 0
S26 F 37 132 54 59 113 104 35 26 17 26
S3 F 40 100 37 40 77 53 33 20 0 0
S5 F 40 122 55 48 103 77 39 28 10 0
S17 F 46 112 55 50 105 84 43 27 14 0
Mean 39.80 118.60 49.40 50.80 100.20 76.00 35.80 25.00 10.00 5.20
SD 3.90 12.76 7.89 7.60 13.61 19.84 5.40 3.16 6.44 11.63
4- to 5-year-old cohort
S11 M 52 110 59 60 119 80 32 26 22 0
S22 M 55 102 50 62 112 102 43 26 29 4
S23 M 49 126 62 68 130 116 42 34 31 9
S27 F 53 103 51 57 108 93 37 24 28 4
S34 F 54 108 54 66 120 95 38 27 30 0
S60 F 57 116 74 64 138 90 31 24 28 7
Mean 53.33 110.83 58.33 62.83 121.17 96.00 37.17 26.83 28.00 4.00
SD 2.73 9.00 8.96 4.02 111.18 12.15 4.96 3.71 3.16 3.63
5- to 6-year-old cohort
S39 M 68 118 82 80 162 134 41 42 34 17
S40 F 61 110 64 76 140 105 38 32 31 4
S46 F 63 121 78 76 154 143 46 37 27 33
S47 M 67 115 79 69 148 101 41 31 25 4
S49 M 66 110 73 67 140 115 43 29 22 21
S59 M 69 114 81 77 158 127 46 32 26 23
Mean 65.67 114.67 76.17 74.17 150.33 120.83 42.50 33.83 27.50 17.00
SD 3.08 4.37 6.74 5.04 9.24 16.62 3.15 4.79 4.32 11.37
6- to 7-year-old cohort
S52 F 75 94 76 69 145 136 40 32 27 27
S53 M 76 123 87 101 188 156 47 40 41 28
S54 M 72 128 86 99 185 138 37 33 35 33
S55 F 74 100 74 70 144 138 42 32 38 26
S58 F 72 95 74 63 137 136 47 35 35 19
S56 M 73 120 80 92 172 144 43 30 44 27
S61 F 84 112 93 92 185 146 46 37 33 30
Mean 75.14 110.29 81.43 83.71 165.14 142.00 43.14 34.14 37.57 27.14
SD 4.18 14.01 7.39 15.83 22.37 7.30 3.80 3.44 3.82 4.30
All participants
Mean 59.96 113.25 67.67 69.25 136.92 111.46 39.96 30.33 26.92 14.25
SD 13.76 10.59 14.97 15.46 29.25 28.58 5.19 5.46 10.70 12.53

SB5-FSIQ Stanford-Binet Intelligence Scales-5th Edition for Early Childhood Full Scale IQ; NV nonverbal; V verbal, T total, RA relational assessment,
NAR nonarbitrary relations, NAA nonarbitrary analogical relations, AR arbitrary relations, AA arbitrary analogical relations

Correlating Assessment Protocol Performance and data suggest strong correlations between each of the stages
Age and age. Table 3 shows a matrix of Spearman’s rank correla-
tions between variables including age, SB5 total and substage
Figure 2 shows graphs of the relationships between scores on raw scores, and relational assessment total and substage
each of the four relational assessment stages and age. These scores. There is a strong correlation between relational
 Psychol Rec

assessment total score and age (r = .859, p < 0.01). Further comparison and hierarchy may be particularly important foun-
analyses also show strong, significant correlations between dational intellectual skills.
total scores for each stage and age (Stage 1: r = .604, p <
0.01; Stage 2: r = .709, p < 0.01; Stage 3: r = .818, p < Correlating Intraprotocol Relations
0.01; and Stage 4: r = .746, p < 0.01). All correlations across
stages and age were significant at the .01 level. These corre- Spearman’s rank analysis in Table 3 shows strong, significant
lations suggest, consistent with RFT, that the capacities to relationships within and across many of the stages and sub-
engage in relational responding and analogical responding stages. Stage 1 comparison and hierarchy scores show signif-
(both overall as well as across different frames) are established icant correlations across substage total scores for all stages;
and strengthened via ongoing exposure to the typical these data may suggest that nonarbitrary comparison and hi-
socioverbal environment. erarchy are important prerequisite relations for more complex
Within-stage analyses show a significant correlation be- language. Stage 2 coordination scores show significant corre-
tween age and comparison relations in Stage 1, and age and lations across substage total scores for all stages; these data
coordination, and hierarchy in Stage 2. These analyses show indicate that performance on nonarbitrary coordinate analogy
significant correlations between age and all substages in may be a good predictor of relational responding more gener-
Stages 3 and 4. ally. Stage 3 coordination and comparison scores show signif-
icant correlations across substage total scores for all stages
Relational Assessment Protocol Performance and Raw whereas Stage 4 coordination, comparison, and opposition
IQ scores show significant correlations across substage total
scores for all stages. Again, this seems to indicate that arbi-
Figure 3 shows graphs of the relationships between scores on trary relational framing may be particularly important for in-
each of the four relational assessment substages and raw IQ tellectual potential in general.
scores. Spearman’s rank analysis reveals a strong correlation Only the comparison substage in Stage 1 shows significant
between relational assessment total scores and raw IQ scores correlations with age, IQ, and total relational assessment
(r = .874, p < 0.01). Further analyses show strong correlations score. The coordination and hierarchy substages in Stage 2
between each stage of the relational assessment scores and show significant correlations with age, IQ, and total relational
raw IQ scores (Stage 1: r = .512, p < 0.05; Stage 2: r = .782, assessment score. All Stage 3 and Stage 4 substages show
p < 0.01; Stage 3: r = .717, p < 0.01; and Stage 4: r = .815, p < strong, significant correlations with age, raw IQ score, and
0.01). Furthermore, each stage of the relational assessment total relational assessment score. These findings further sup-
also correlated with both SB5-NV and SB5-V subscale raw port the key importance of arbitrary relational responding (i.e.,
scores (SB5-NV and Stage 1: r = .534, p < 0.01; Stage 2: r = relational framing) as a core intellectual ability that is acquired
.806, p < 0.01; Stage 3: r = .637, p < 0.01; and Stage 4: r = across childhood.
.760, p < 0.01; and SB5-V and Stage 1: r = .484, p < 0.05; In both Stages 1 and 3, the comparison substage is strongly
Stage 2: r = .788, p < 0.01; Stage 3: r = .753, p < 0.01; and correlated with all of the substages in Stage 4. These data
Stage 4: r = .774, p < 0.01). Overall, these correlations sug- suggest nonarbitrary and arbitrary comparative relational rep-
gest, consistent with RFT, a strong link between relational ertoires are important intellectual skills, and are strong predic-
responding (both overall as well as across different frames) tors of analogical ability. Stage 1 comparison scores also show
and intellectual potential, as reflected in the raw IQ score. a significant correlation with Stage 2 coordination scores,
Within-stage analyses show a significant correlation be- which in turn, as previously noted, shows significant correla-
tween Stage 1 comparison and hierarchy scores, and SB5 tions with age, IQ, and total relational assessment score. Thus,
raw score. These findings are consistent with previous re- it may seem that having a strong nonarbitrary comparative
search showing the importance of comparative relational repertoire facilitates nonarbitrary coordinate analogy, which
framing for intellectual potential (Cassidy et al., 2011, supports nonarbitrary and arbitrary relational responding.
2016). There are also significant correlations between Stage
2 coordination, comparison, and hierarchy scores, and SB5 Examining Performance Across Relational Frames
raw score. Finally, in Stages 3 and 4, there are significant
correlations between all substage scores and SB5 raw scores. One final analysis looked at the acquisition of relational fram-
This pattern is consistent with the idea that arbitrary relational ing in each stage. Figure 4 shows the mean percent correct for
framing is particularly important for intellectual potential. each relational frame across stages, and for each age cohort.
Upon further examination across all four stages, comparison In general, in Stage 1 (nonarbitrary relations), there is an
and hierarchy are the only substages that show a consistent improvement with age across cohorts. Performance
correlation with raw IQ as revealed by both SB5 total score as across cohorts shows little variation in coordination
well as SB5 subscale scores. This pattern suggests that (mean scores range from 98% to 100% correct).
Table 3 Matrix of Spearman’s Rho correlations for all measures administered

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Psychol Rec

1. Age –
2. SB5-T .879** –
3. SB5-NV .856** .958** –
4. SB5-V .832** .973** .894** –
5. RA-T .859** .874** .810** .871** –
6. S1-T .604** .512* .534** .484* .688** –
7. S2-T .709** .782** .806** .788** .831** .684** –
8. S3-T .818** .717** .637** .753** .817** .430* .649** –
9. S4-T .746** .815** .760** .774** .915** .504* .672** .639** –
Stage 1 Substages: 1 = Coordination; 2 = Comparison, 3 = Opposition; 4 = Temporality; 5 = Hierarchy
10. SS-1 0.316 0.349 0.327 0.371 .436* 0.394 0.394 0.381 0.396 –
11. SS-2 .548** .595** .578** .534** .625** .516** .477* .439* .631** .416* –
12. SS-3 0.332 0.225 0.242 0.206 .429* .821** .459* 0.28 0.224 0.265 .474* –
13. SS-4 0.393 0.313 0.355 0.277 0.357 .528** 0.252 0.122 0.321 0.121 –0.084 0.178 –
14. SS-5 0.384 .431* .430* .487* .577** .564** .767** .457* .470* .483* 0.165 0.294 0.211 –
Stage 2 Substages: 1 = Coordination; 2 = Comparison, 3 = Opposition; 4 = Temporality; 5 = Hierarchy
15. SS-1 .605** .702** .693** .680** .648** .422* .714** .429* .657** 0.333 .590** 0.151 0.212 .475* –
16. SS-2 0.326 .499* .469* .541** .477* .558** .592** 0.318 0.31 0.269 0.299 .434* 0.156 .527** 0.235 –
17. SS-3 0.211 0.289 0.295 0.318 0.288 .446* .556** 0.154 0.085 0.238 –0.125 0.322 0.259 .563** 0.17 0.343 –
18. SS-4 0.329 0.154 0.171 0.134 0.262 0.389 0.355 0.226 0.114 0.123 0.199 .418* 0.069 0.262 0.089 –0.041 0.335 –
19. SS-5 .568** .582** .637** .557** .683** 0.374 .690** .641** .657** 0.154 0.376 0.173 0.185 .456* .612** 0.221 0.049 –0.047 –
Stage 3 Substages: 1 = Coordination; 2 = Comparison, 3 = Opposition; 4 = Temporality; 5 = Hierarchy
20. SS-1 .658** .605** .555** .590** .682** .643** .621** .687** .521** .419* .526** .625** 0.121 .432* .427* 0.346 0.359 0.237 .551** –
21. SS-2 .730** .715** .631** .693** .837** .421* .525** .754** .826** .419* .687** 0.175 0.135 0.286 .474* 0.217 0.008 0.196 .513* .455* –
22. SS-3 .568** .455* .458* .470* .554** 0.298 .579** .718** .428* 0.274 0.17 0.116 –0.004 .571** 0.312 0.242 0.338 0.246 .493* 0.362 .533** –
23. SS-4 .676** .684** .606** .704** .632** 0.185 .546** .781** .480* 0.033 0.209 0.031 0.065 0.28 0.356 0.392 0.144 0.101 .530** .416* .506* .569** –
24. SS-5 .640** .560** .436* .620** .567** 0.229 0.31 .812** .442* 0.378 0.345 0.194 0.147 0.19 0.318 0.098 –0.083 0.122 0.353 .586** .512* 0.336 584** –
Stage 4 Substages: 1 = Coordination; 2 = Comparison, 3 = Opposition; 4 = Temporality; 5 = Hierarchy
25. SS-1 .703** .749** .667** .729** .829** .551** .548** .557** .865** 0.399 .496* 0.262 .453* 0.347 .576** 0.341 0.114 –0.01 .530** .444* .741** 0.375 .450* .432* –
26. SS-2 .702** .685** .678** .618** .850** .579** .661* .595** .885** 0.339 .703** .409* 0.23 0.348 .612** 0.273 0.055 0.124 .731** .629** .734** .405* .428* 0.315 .751** –
27. SS-3 .657** .776** .759** .737** .864** .445* .737** .613** .911** 0.317 .546** 0.221 0.222 .458* .610** 0.403 0.146 0.025 .759** .489* .774** .468* .480* 0.305 .782** .837** –
28. SS-4 .648** .658** .616** .598** .769** 0.403 .491* .476* .893** 0.262 .599** 0.158 0.296 0.289 .501* 0.127 –0.073 0.22 .505* 0.392 .738** 0.184 0.306 0.356 .708** .792** .768** –
29. SS-5 .583** .620** .573** .560** .747** 0.389 .486** 0.403 .894** 0.262 .599** 0.14 0.278 0.308 .529** 0.132 –0.059 0.225 .479* 0.322 .733** 0.172 0.258 0.263 .726** .779** .782** 982** –

Note. ** Correlation is significant at the 0.01 level (2-tailed); * Correlation is significant at the 0.05 level (2-tailed); SB5 Stanford-Binet Intelligence Scales - 5th Edition for Early Childhood Full Scale IQ;
NV non-verbal; V verbal; T total; RA relational assessment; S stage; SS substages
 Psychol Rec

Stage 1: Nonarbitrary Relations vs. Age Stage 2: Nonarbitrary Analogical Relations vs. Age
R² = 0.4152 R² = 0.5022
50 50

40 40

Stage 2 Score
Stage 1 Score

30 30

20 20

10 10

0 0
36 48 60 72 84 36 48 60 72 84
Age in Months Age in Months

Stage 3: Arbitrary Relations vs. Age Stage 4: Arbitrary Analogical Relations vs. Age
R² = 0.6713
R² = 0.4843
50 50

40 40
Stage 3 Score

Stage 4 Score
30 30

20 20

10 10

0 0
36 48 60 72 84 36 48 60 72 84
Age in Months Age in Months
Fig. 2 Relational assessment vs. age across stages. Note. Regression slopes are shown for participant scores at each stage and age

Comparative and hierarchical relational performance these relational repertoires, the latter pattern perhaps suggests their
start to emerge and show steady improvement as age potential difficulty. Comparing performance across Stage 3 rela-
increases, especially between the 4- to 5-year-old and tions with chance level responding, we note first that the youngest
5- to 6-year-old cohorts. Participants across all cohorts group fails to meet a criterion of chance level of responding on any
perform lower in temporality and opposition, and per- relations whereas the oldest group does so for all relations. The
formance in both these frames gradually improves two other groups are somewhere in between with both showing
across cohorts. Comparing performance across Stage 1 strong performance relative to chance for sameness, comparison,
relations with chance level responding (see Figure 4) we and opposition while showing near chance level responding for
can see that the two older cohorts are responding above both temporality and hierarchy.
chance levels on all relations whereas the two younger As regards analogy in Stage 4 (arbitrary analogy), the 3- to
cohorts are performing above chance levels in all rela- 4-year-old and 4- to 5-year-old cohort scores are low and well
tions except temporality. below chance level performance. Performance shows substan-
In Stages 2 (nonarbitrary analogy) and 3 (arbitrary relations), tial improvement in the 5- to 6-year-old cohort and this im-
performance across all relational frames generally improves with provement continues in the 6- to 7-year-old cohort.
age. The most obvious improvement across all frames with age Furthermore, in the latter group analogical performance is
occurs in Stage 3. In particular, there is a clear increase in perfor- improved not just in coordination but also in various other
mance between the 3- to 4-year-old and 4- to 5-year-old cohorts. frames. In particular, the 5- to 6-year-old cohort shows obvi-
Comparing performance across Stage 2 relations with chance level ous improvement in performance in coordinate analogy.
responding we can see that the only relation on which all cohorts However, participants are still not doing that well (even the
are responding above chance is sameness. In the case of other coordination performance is only just above chance level),
relations, the two younger cohorts are at or below chance in the indicating that analogy emerges at this age but it is still a
case of all relations,2 whereas the two older cohorts both show fragile repertoire. The data show gradual improvement by
above chance level responding for all other relations except tem- the 6- to 7-year-old cohort, and frames beyond coordination
porality (5- to 6-year-olds) and opposition (6- to 7-year-olds). By are emerging and improving. It might be noted from the graph
showing how, in a slightly unusual context (i.e., nonarbitrary anal- that even the oldest cohort are still not responding much above
ogy), even older children can still show weak performance on chance level for any of the relations. However, a more detailed
Psychol Rec

Stage 1: Nonarbitrary Relations vs. SB5 Raw Stage 2: Nonarbitrary Analogical Relations vs.
Score SB5 Raw Score
R² = 0.2982
50 50 R² = 0.6333

40 40
Stage 1 Score

Stage 2 Score
30 30

20 20

10 10

0 0
70 82 94 106 118 130 142 154 166 178 190 70 82 94 106 118 130 142 154 166 178 190
SB5 Raw Score SB5 Raw Score

Stage 3: Arbitrary Relations vs. SB5 Raw Score Stage 4: Arbitrary Analogical Relations vs. SB5
Raw Score
50 R² = 0.6197 R² = 0.5971
50
40
Stage 3 Score

40

Stage 4 Score
30
30
20 20

10 10

0 0
70 82 94 106 118 130 142 154 166 178 190 70 82 94 106 118 130 142 154 166 178 190
SB5 Raw Score SB5 Raw Score
Fig. 3 Relational assessment scores vs. raw IQ score across stages. Note. Regression slopes are shown for participant scores at each stage and SB5 raw scores

Stage 1: Nonarbitrary Relations Stage 2: Nonarbitrary Analogical Relations

100 100

80 80
% Correct

% Correct

60 60

40 40

20 20

0 0
3–41years 4–52years 5–6 3years 6–7 4years 3–41years 4–52years 5–63years 6–74years
Age cohorts Age cohorts

Coordination Comparison Opposition Temporality Hierarchy Coordination Comparison Opposition Temporality Hierarchy

Stage 3: Arbitrary Relations Stage 4: Arbitrary Analogical Relations

100 100

80 80
% Correct

% Correct

60 60

40 40

20 20

0 0
3–41years 4–52years 5–6 3years 6–7 4years 3–41years 4–5 2years 5–6 3years 6–7 4years
Age cohorts Age cohorts

Coordination Comparison Opposition Temporality Hierarchy Coordination Comparison Opposition Temporality Hierarchy

Fig. 4 Mean normative acquisition across stages and relations. Note. Mean relational assessment scores across substages and age cohorts for each stage
in the relational assessment. The black bars represent chance level responding for each frame across each level
 Psychol Rec

analysis of our data in this respect showed that children in the form of relational framing with respect to intellectual potential
group were not responding randomly on trials as they tended (Sternberg, 1977; Sternberg et al., 2001).
to answer simpler trials correctly while failing more difficult The correlations observed in this study between relational per-
ones. This pattern was also observed more generally across all formance and IQ score support and extend the growing body of
stages and all relations, suggesting that the comparison with research showing that relational framing supports linguistic and
chance level responding should be seen as a guide rather than cognitive performance (e.g., Cassidy et al., 2011, 2016; Dixon
indicative of whether children were actually responding at et al., 2018; Gore et al., 2010; Hayes & Stewart, 2016; Moran
random or not. et al., 2014; Mulhern et al., 2017; O’Hora et al., 2005, 2008;
It was not a focus of the present study to analyze response O’Toole & Barnes-Holmes, 2009). The results in this study also
differences by gender. However, we performed a cursory and showed significant correlations between relational assessment
tentative analysis (given the small numbers involved) by gen- scores and age, adding to the extant research showing that relational
der and found no noticeable differences between females and responding strengthens as a function of age in typically developing
males on relational assessment scores either within stages or children (e.g., McHugh et al., 2004; Mulhern et al., 2017). The
for the assessment as a whole. strong correlations observed in this study indicate that the relational
assessment employed here could be a reliable, relatively compre-
Summary hensive tool for assessing relational skills in young children. Lastly,
an important function of this study was to focus in particular on
The normative data suggest, consistent with RFT, that the capacity analogical reasoning, and the significant correlations found between
to engage in relational responding and analogical responding (both analogy and IQ performance provide further evidence that analogy
overall as well as across different frames) are established and is a particularly relevant pattern of relational framing with regard to
strengthened via ongoing exposure to the typical socioverbal en- intellectual potential (e.g., Sternberg, 1977).
vironment. Relational performance is correlated with measured IQ
which supports the RFT concept that relational framing is critical Relational Responding and IQ
to language and cognition. Furthermore, the arbitrary stages are
more highly correlated with IQ than the nonarbitrary stages. Considering these data, first, we see strong correlations between
Comparison framing seems to be a particularly important relation relational ability and IQ (see also Cassidy et al., 2011; Dixon et al.,
as it shows significant correlations with age, IQ, and total relational 2018; Gore et al., 2010; Moran et al., 2014; Mulhern et al., 2017;
assessment score. Considering the five relational frames tested in O’Hora et al., 2005, 2008; O’Toole & Barnes-Holmes, 2009). The
the assessment, several patterns are evident. Nonarbitrary and ar- present study represents an advance in one respect over previous
bitrary coordination emerges first, and temporality emerges last. studies that have correlated relational framing and IQ in terms of
Also, there is a difference between nonarbitrary and arbitrary re- its scope; we examined derived relational responding across mul-
lational responding for frames of opposition and hierarchy; oppo- tiple frames and included typically developing children across a
sition scores are lower in the nonarbitrary stages and higher in the number of different age cohorts.
arbitrary stages, and the hierarchy scores are higher in the nonar- The significant correlations observed in this study between
bitrary stages but lowest in the arbitrary stages. relational performance and IQ support and extend the growing
body of research finding that framing relationally is a requisite
for linguistic and cognitive performance. In the present study,
Discussion the correlations between IQ and responding in the arbitrary rela-
tional stages were more significant than those between IQ and
There is now considerable evidence that arbitrary relational responding in the nonarbitrary stages. Furthermore, within-stage
responding is the principal behavior characterizing human lan- analyses also showed significant correlations between all substage
guage and cognition (Dymond & Roche, 2013; Fryling et al., scores and IQ in arbitrary Stages 3 and 4. These correlations
2020; Zettle et al., 2016). However, despite the evidence, re- suggest, consistent with previous RFT research (e.g., Cassidy
search on the normative development of relational responding, et al., 2011, 2016), a strong link between relational framing and
including the sequence of relational frame acquisition, is as yet intellectual potential. Additional within-stage analyses showed
limited. This study is one of the first attempts to provide a cross- significant correlations between IQ and both nonarbitrary compar-
sectional assessment of the acquisition of a selection of frames in ison and hierarchy scores in Stage 1. These findings are consistent
young children. Relational responding of coordination, compar- with previous research noting the importance of comparative re-
ison, opposition, temporality, and hierarchy was assessed in 3- to lational responding for intellectual potential (Cassidy et al., 2011,
7-year-old children across four levels of complexity, and perfor- 2016). There were also significant correlations between Stage 2
mance was correlated with age and intellectual skill as assessed coordination, comparison, and hierarchy scores, and IQ; findings
on a standardized IQ test. In addition, a key focus of the relational consistent with the literature on analogy and intelligence (Christie
assessment was analogical responding, a particularly important & Gentner, 2014; Gentner & Christie, 2010; Goswami & Brown,
Psychol Rec

1989; Sternberg, 1977). Upon further examination, comparison Mulhern et al. found that participants in the 3- to 4-year-old
and hierarchy were the only substages across all four stages that cohort did not show arbitrarily applicable responding in accor-
showed a consistent correlation with raw IQ as revealed by both dance with hierarchy whereas participants in the 5- to 6-year-old
SB5 total score as well as SB5 subscale scores. This pattern sug- cohort showed arbitrary hierarchical framing relatively robustly.
gests that comparison and hierarchy may be particularly important These patterns of development across frames should be consid-
foundational intellectual skills and should be considered for further ered when designing an RFT-based curriculum focused on train-
investigation. ing relational framing, a point we discuss further below.
With respect to analogy (relating of relations) in the present
Relational Responding and Age study, data from Stage 4 (arbitrary analogy), similar to that
from Stage 2 (nonarbitrary analogy), showed that the biggest
The relational assessment developed in the present study ex- age-based improvement in relational assessment scores oc-
amined five relational frames, including coordination, com- curred between the 4- to 5-year-old and 5- to 6-year-old co-
parison, opposition, temporality, and hierarchy at both nonar- horts, and there was almost no difference between the 3- to 4-
bitrary and arbitrary levels. Performance on each frame was year-old and 4- to 5-year-old cohorts. The fact that there ap-
examined independently within and across stages in an effort pears to be substantive improvement in analogical ability be-
to elucidate frame development across age. Results showed tween the 4- to 5-year-old and 5- to 6-year-old participants at
relatively predictable patterns wherein total relational assess- both the nonarbitrary and arbitrary levels supports earlier re-
ment scores increased as a function of age, and average scores search (e.g., Carpentier et al., 2002, 2003), which also found
for each stage showed clear, stable improvements across age an emergence of analogical reasoning around 5 years of age.
cohorts. A strong correlation was found between total assess- Within stage analyses also showed some noteworthy cor-
ment score and age, and further analyses showed significant relations with respect to particular relational frames and age.
correlations between total scores for each stage and age. These For example, in Stage 1, the only significant correlation with
correlations suggest, consistent with RFT, that the capacity to age was the comparative relation; in Stage 2, coordination and
engage in relational responding (both overall as well as across hierarchy correlated with age; and, in Stages 3 and 4, all sub-
different frames) is established and strengthened via ongoing stages correlated with age. These analyses suggest some inter-
exposure to the typical socioverbal environment. esting findings to which we will return later.
The biggest age-based improvement in relational assessment As previously suggested, despite substantial evidence showing
scores between the youngest and oldest cohorts occurred in Stage that relational framing is a core skill underlying cognitive ability,
3 (arbitrary relations) and Stage 4 (arbitrary analogy). In these research on the normative development of relational responding
two stages, the gap in terms of overall correct responding be- and sequence of frame acquisition is as yet limited. In this study,
tween the 3- to 4-year-old and the 6- to 7-year-old cohorts was we investigated the normative development of multiple relational
substantial, whereas the difference in scores was not as profound frames in participants aged 3- to 7-years-old. This is one of the first
in Stages 1 and 2. These data suggest that there is already sub- attempts to provide a cross-sectional, comprehensive look at the
stantial development in nonarbitrary relations (as in Stage 1) acquisition of frames in young children.
before the age of 3 years, resulting in relatively less room for The relational assessment traced the acquisition of nonarbi-
improvement after age 3. Apart from finding a general improve- trary to arbitrary relational responding and nonarbitrary to arbi-
ment based on age, we also found that there were particular trary analogical responding across frames of coordination, com-
discontinuities between specific age cohorts within stages. For parison, opposition, temporality, and hierarchy. The normative
example, in Stage 2 (nonarbitrary analogy), we noted a large gap data suggest, in line with RFT, that the ability to engage in
in performance between the 4- to 5-year-old and 5- to 6-year-old relational responding (in general and across different frames) is
children. These data may indicate that the transition from the 4- acquired gradually throughout childhood (at least with respect to
to 5-year-old to the 5- to 6-year-old age range results in particular typically developing children). The cross-sectional data suggest
improvement in nonarbitrary analogy. In Stage 3 (arbitrary rela- that typically developing children acquire the coordination rela-
tions), the biggest age-based improvement in relational assess- tion first. This further supports previous suggestions that the
ment scores occurred between the 3- to 4-year-old and 4- to 5- coordination relation is the most common and ubiquitous pattern
year-old cohorts, and there was almost no change between the 4- of relational responding, and that its acquisition may facilitate the
to 5-year-old and 5-to 6-year-old cohort. These data might sug- development of other relational patterns in a child’s developing
gest considerable development in arbitrary relational framing verbal repertoire (Barnes-Holmes, Barnes-Holmes, & Smeets,
between the 3- to 4-year-old and 4- to 5-year-old cohorts. This 2004; Hughes & Barnes-Holmes, 2016).
is an important finding because the development of relational The current data also support the importance of comparative
framing as a generalized operant across frames has not previous- responding in accordance with age, IQ, and total relational
ly received much attention. One previous study (Mulhern et al., assessment score. Nonarbitrary comparison was the only
2017) investigated the acquisition of hierarchical relations. nonarbitrary relation that showed significant correlations with
 Psychol Rec

age, IQ score, total relational assessment score, and total scores Thus, conceptualizing the physical dimension along which tem-
for each stage. These findings suggest that nonarbitrary poral comparatives are arranged requires a more complex verbal
comparative relations may be an especially relevant relational repertoire and metaphorical understanding. From an RFT point
pattern for intellectual potential, the emergence of arbitrary of view, this might explain why temporal framing appears to be
relational framing, and analogical responding. Furthermore, acquired later than other frames. Given the importance of this
these data also suggest that the nonarbitrary comparative repertoire, however, further research in testing and training of
relation may play an important role in additional language both nonarbitrary and arbitrary temporal relations is warranted.
development. Future researchers could further investigate In the present study, with the exception of opposite rela-
training nonarbitrary comparative relations if they are weak or tional responding, participants performed better on the nonar-
missing in a child's verbal repertoire in order to facilitate bitrary stages than the arbitrary stages, thus supporting RFT
additional language development. The present data also suggest and previous research (Berens & Hayes, 2007; Gorham et al.,
that arbitrary comparison is particularly important; for example, 2009). In general, it is accepted that nonarbitrary responding is
there were significant correlations between age, IQ, and total required for the acquisition of arbitrary relational responding
assessment score and comparison at every stage with the (Rehfeldt & Barnes-Holmes, 2009). Previous research has
exception of Stage 2 comparison and age. In addition, the found that participants who did not learn to derive a particular
comparison data show that arbitrary comparative relations had pattern of arbitrary relations were aided in doing so via addi-
significant correlations with arbitrary coordination, opposition, tional training in the corresponding pattern of nonarbitrary
temporality, and hierarchy; with total scores for each stage in relations (Berens & Hayes, 2007; Gorham et al., 2009).
the assessment; and with all frames in Stage 4, arbitrary Likewise, in the PEAK assessment and curriculum model,
analogical responding. These findings provide further evidence Dixon et al. (2014) train simple and complex verbal relations
that the comparative relation may be especially important for using nonarbitrary stimuli first, and then progress to equiva-
more complex language performance. This also echoes the lence and transformation training.
finding from the Cassidy et al. (2011, 2016) studies that compar- Given the foregoing, one perhaps surprising outcome of the
ative relations play a key role in intellectual potential. Cassidy present study was that participants scored higher in the arbitrary
et al. found significant increases in mean IQ score after training opposition stage than in the nonarbitrary opposition stage. We
multiple stimulus relations including comparison. Finally, in both think it is possible that children scored poorly on nonarbitrary
Stages 1 and 3 of the present study, the comparison substage was opposition because opposite relations are not typically systemat-
strongly correlated with all substages in Stage 4. These data ically trained as two dichotomous stimuli on a specified physical
suggest that nonarbitrary and arbitrary comparative relational dimension on a continuum (Hughes & Barnes-Holmes, 2016).
repertoires are important intellectual skills and may be predictive Instead, opposite relations are more likely to be taught as an
of analogical ability. Future researchers may want to further in- intraverbal response (e.g., the opposite of tall is short, big/small,
vestigate the effects of training nonarbitrary and arbitrary com- full/empty), and thus participants may be more familiar with the
parative relations in students both with and without language arbitrary format (i.e., what is the opposite of many/few?) than the
delays. nonarbitrary format as in the present study. However, it remains
The present cross-sectional data also showed that scores were unclear whether participant performance was idiosyncratic to the
lowest for temporal relations across all age cohorts, suggesting test or idiosyncratic to their education. In any case, opposite
that the temporal relation is acquired after the other tested frames. relations are empirically important as shown by Cassidy et al.
These findings are consistent with cognitive research which has (2011, 2016), for example, who found that training opposite and
found that temporal understanding emerges relatively late in de- comparative relations led to improvement in IQ scores. Future
velopment (McCormack & Hoerl, 2017; Pyykkönen & Järvikivi, researchers may want to investigate systematically training
2012). A noted advantage of the RFT approach adopted in the opposite relations as functional units, as defined by RFT, at
present study, however, is the analysis of temporal understanding both nonarbitrary and arbitrary levels.
in terms of functional units. The fact that we can conceptualize Considering the potential for improving language and cog-
temporal understanding in terms of relational frames means that nition, further research into relational assessment and training
we can target this repertoire for training and assess for potential for the purpose of curriculum development is clearly warrant-
improvements on this basis. Within the RFT literature, temporal ed. Up until recently, most empirical work on verbal behavior
framing has previously received less empirical attention than has largely been influenced by Skinner's (1957) analysis of
other relations (Hughes & Barnes-Holmes, 2016). It shares the verbal behavior (Dixon et al., 2017; Dymond et al., 2010). As
same basic pattern as comparative relations in that it entails a result, commonly used language assessments such as the
responding to events in terms of their directional displacement Assessment of Basic Language and Learning Skills-Revised
along a specified dimension. However, time is inherently more (ABLLS-R; Partington, 2008) and the Verbal Behavior
abstract than other comparatives, such as size (Hayes et al., Milestones and Placement Program (VBMAPP; Sundberg,
2001b), and requires discriminating successive changes in time. 2008) are based on Skinner's analysis of verbal behavior; thus,
Psychol Rec

training focuses on the basic verbal operants (i.e., echoics, correlated with age, and in particular, the data showed at which
mands, tacts, intraverbals) with little attention to more com- age particular frames were acquired. In Stage 3, the biggest age-
plex verbal behavior. Dixon et al. (2018) provided experimen- based improvement in arbitrary relational scores occurred be-
tal work assessing and training more complex language. The tween the 3- to 4-year-old and 4- to 5-year-old cohorts. These
present study contributes to the work on relational language data are consistent with the Mulhern et al. (2017) study, which
assessment. found that 3-to 4-year-old participants did not respond correctly
Previous relational training studies (e.g., Barnes-Holmes, on tests of hierarchical framing, whereas participants aged 5 and
Barnes-Holmes, & Smeets, 2004; Belisle et al., 2016; Berens older performed better on these tests. Based on these data, con-
& Hayes, 2007) showed that derived relational responding can siderable development in arbitrary relational framing occurs be-
be brought under operant control. A functional analysis of tween ages 4 and 5. In addition, the data from Stage 3 in the
young or developmentally delayed children's existing rela- present study indicate that coordination is acquired first, and
tional abilities would provide the framework for a robust, temporality is acquired last. Thus, a relational language program
flexible, and individualized RFT-based curriculum. The pres- could introduce relational training around the developmental age
ent study is one of the first studies to look at the sequence of of 4, and start with training the relational frame of coordination
acquisition of multiple frames in young children; these find- first. The previously mentioned VB-MAPP (Sundberg, 2008) is
ings could be a valuable reference in curriculum design. based on the verbal repertoires of typically developing children
Related to this point, one possible limitation in the current up to 4 years old; thus, the development of a robust relational
study is the representativeness of the sample population. All assessment and curriculum suitable for the developmental age of
participants recruited for this study attended the same private 4+ is warranted.
school in New Jersey. As mentioned, the average IQ score for
participants in this study without the outlier was 112 (113 with Analogy
the outlier participant), which is slightly above the average range
of 90–109 (Roid, 2003). Future research using the present pro- Apart from examining the acquisition of relational framing in
tocol to assess relational framing in young children should in- general with age, the present study also focused considerable
clude substantially larger numbers both overall as well as within attention on relating relations, that is, analogy. The study
each of the age cohorts. It should also recruit from a wider variety assessed analogy in young children at both nonarbitrary and
of educational institutions such as public schools and disadvan- arbitrary relational levels. It is commonly accepted in mainstream
taged schools. Such extensions of the present work would argu- cognitive research (e.g., Bod, 2009; Gentner, 1983) that analogy
ably constitute more representative testing and provide more is an important pattern of relational framing with regard to intel-
generalizable results. lectual potential. Moreover, analogical reasoning is frequently
Despite this point, the present data are informative in terms of applied as a metric of intelligence (Sternberg, 1977), and as a
the development of a possible RFT-based curriculum. For exam- measure to predict academic success; for example, the Law
ple, the present data show correlations between age and both non- School Admissions Test (LSAT; Lapiana, 2004). The theoretical
arbitrary and arbitrary relational ability. These normative data importance of analogy has also been recognized explicitly within
could help inform a developmentally sequenced relational curric- the RFT literature. For example, the relating of relations is in-
ulum; for example, a logical training sequence would include cluded as one of the five levels (including mutual entailing, rela-
training increasingly difficult or complex levels of nonarbitrary tional framing, relational networking, relating relations, and re-
relational frames before training arbitrary relational frames. lating relational networks) in Barnes-Holmes et al.’s (2017,
Another area of warranted research includes testing and training 2020) recently proposed hyperlevel multidimensional (HDML)
the transition from nonarbitrary to arbitrary relations; a compre- framework for examining and discussing relational framing.
hensive relational curriculum should include training-for-transition However, despite the ubiquity and utility of analogy, functional
procedures. assessments targeting relations among relations as analogy have
Additional within-stage analyses show some interesting re- not yet been examined.
sults with respect to particular relational frames which could be The present data extend previous work on assessment of anal-
considered when designing relational programs. For example, in ogy by Carpentier et al. (2002, 2003, 2004). However, the pres-
Stage 1, there was a significant correlation between comparison ent study arguably advances the Carpentier et al. studies by using
and age, IQ, total relational assessment scores, and total substage a more time-efficient format than the match-to-sample (MTS)
scores for each substage. Future researchers could investigate procedure used by Carpentier et al. (this will be discussed at
training nonarbitrary comparative relations in order to facilitate greater length further ahead). In the present study, two of the
the development of other frames. In Stage 2, nonarbitrary ana- stages (Stages 2 and 4), addressed the relating of relations;
logical frames of coordination and hierarchy correlated with age; Stage 4 demonstrated the RFT definition of analogy, or deriving
therefore, it may be worth training these frames first in nonarbi- relations between relations, whereas Stage 2 tested the nonarbi-
trary analogy programs. In Stages 3 and 4, all substages trary precursor to analogical responding.
 Psychol Rec

In Stage 4, the total score as well as individual substage further evaluate the effects of training nonarbitrary analogy if it is
scores show significant correlations with age, IQ score, and a weak or missing skill in young children.
total assessment scores. Total score for arbitrary analogical Finally, an additional point that might be made regarding the
responding showed a slightly stronger correlation with IQ importance of analogy concerns the data from Participant S26, an
performance than basic arbitrary relations (Stage 3), whereas outlier in the 3- to 4-year-old cohort. Participant S26’s full-scale
basic arbitrary relations showed a slightly higher correlation IQ score was 132; without his data, the mean IQ score for all
with age compared to arbitrary analogical relations. These participants was 112. It is interesting that Participant S26’s data
data provide further evidence that analogical relations are un- are similar to those of his cohort for the nonarbitrary relations and
doubtedly tied with intellectual potential. nonarbitrary analogy stages. However, his scores were above the
The results for Stage 4 showed a marked difference in scores mean in arbitrary relations and arbitrary analogy; he was the only
between the 4-to 5-year-old cohort and the 5-to 6-year-old co- participant under 4 years of age to respond correctly to arbitrary
hort. These findings contribute to the extant RFT research sug- analogy tasks, thus positively skewing the results of the 3- to 4-
gesting a developmental divide in the acquisition of analogical year-old cohort. He also outperformed the 4-to 5-year-old partic-
ability at around 5 years of age (Carpentier et al., 2002, 2003). ipants in the arbitrary analogy stage. His data provide further
The 5-year-old participants in the Carpentier et al. studies were evidence that arbitrary analogical responding is strongly tied with
able to demonstrate arbitrary equivalence–equivalence and intellectual potential.
nonequivalence–nonequivalence only after additional
compound-matching training with the trained relations between
elements (e.g., A1B1-A3B3). The mean scores for the 6- to 7- Relational Assessment Format
year-old cohort were better again and were more evenly distrib-
uted across frames. In light of the strong correlation between One potentially important aspect of the present study was the
basic arbitrary relations and age mentioned above, it is possible assessment format. Previous RFT studies have used a number
that an additional year of practicing deriving relations (Stage 3) of different methodologies to examine and train relational
facilitated relating relations within relations in Stage 4. framing in young children, including, for example, the MTS
Unlike the Carpentier et al. studies, however, the present study (see Rehfeldt & Barnes-Holmes, 2009) procedure and the re-
examined both nonarbitrary (Stage 2) and arbitrary (Stage 4) lational evaluation procedure (REP; see Barnes-Holmes et al.,
analogical responding across five frames, providing further in- 2001; Stewart et al., 2004). MTS procedures require extensive
sight into the developmental sequence and acquisition of analog- baseline training prior to starting any testing or training. For
ical responding. Total relational assessment scores for Stage 2 example, in the Carpentier et al. (2002) study, the 5-year-old
(nonarbitrary analogical relations) showed strong, significant cor- participants required an average of 234 baseline trials in
relations with age, IQ score, and total assessment scores. Data Experiment 1 before training and testing could begin. The
were further analyzed for patterns of responding by age group in REP, on the other hand, is a more efficient method in which
order to identify at which age analogical responding is acquired. multiple exemplars of relational patterns can be easily gener-
Stage 2 results showed a gradual improvement in scores by age, ated and readily presented without requiring any prerequisite
and suggested the acquisition of analogy, at least in coordinate MTS training. This methodology allows participants to report
and difference relations, at 5 years of age. This is the first RFT on or evaluate sets of arbitrarily applicable relations defined
study to focus on nonarbitrary analogy in addition to arbitrary by various sets of contextual cues. The testing format utilized
analogy. In the domain of comparative psychology more work in the current study was employed in the context of the mul-
has been done testing relating of nonarbitrary relations in tistage relational assessment that allowed testing of a range of
nonhumans; indeed, various species have been found to pass different types of relations across different levels of complex-
such tests. For example, chimpanzees (Gillan et al., 1981), crows ity using the REP. The relational networks in the present study
(Smirnova et al., 2015), and baboons (Fagot & Maugard, 2013) were displayed on the screen during each trial in order for
have passed nonarbitrary analogical (i.e., relating relations) tasks. participants to derive relations and relate relations among re-
In addition, such work has also been conducted by cognitive- lations in the arbitrary stages, thus omitting the need for time-
developmental researchers (e.g., Christie & Gentner, 2014). The consuming MTS pretraining.
present study begins to bridge that gap in the RFT-based study of In order to make the present relational assessment equally
nonarbitrary analogy in young children. From an RFT perspec- accessible to all participants, including younger cohorts un-
tive of course, nonarbitrary analogy is not full analogy (i.e., de- able to read, a novel procedure was implemented, which ex-
riving relations among arbitrary relations). However, it is an im- cluded any textual instructions or tasks. Instead, this measure
portant repertoire and should be trained as a prerequisite skill used simple, monochromic shapes and single, alphabetic let-
before arbitrary analogy. The strong correlations between nonar- ters to indicate the contextual cue (i.e., S/D, M/L, S/O, B/A,
bitrary and arbitrary analogy total scores in the present study and C/I for same/different, more/less, opposite, before/after,
provide evidence for this analysis, and future researchers could and contains/inside, respectively). In addition, the letter
Psychol Rec

identifying the contextual cue was paired with optional audio distinction is warranted and future researchers may consider
stimuli, for example, “same” for S. testing coordination and distinction more systematically and
A functional analysis of intelligence as relational skills, providing independent analyses for both frames.
such as the analysis provided by the relational assessment in A third potential limitation of the relational assessment may be
this study, provides useful information on the relational skills the linear versus nonlinear, and same versus mixed presentation of
that contribute to intelligent behavior as assessed on IQ tests. the arbitrary relational networks in Stages 3 and 4. In a linear-same
Considering the predictive validity of IQ scores (Sternberg series, stimuli are presented sequentially (e.g., A–B–C) and the
et al., 2001), understanding the variables that influence intel- contextual cue between relata is the same. For example, in a linear-
ligent behavior is a socially valid endeavor. Furthermore, the same comparative relational network, A is more than B, and B is
extant RFT research on relational training and intellectual per- more than C, or A > B, B > C (Fienup & Brodsky, 2020; Vitale
formance demonstrates that relational framing can be trained et al., 2008). In a linear-mixed series, stimuli are presented sequen-
and strengthened (e.g., Barnes-Holmes, Barnes-Holmes, & tially but the contextual cue is not the same between relata, for
Smeets, 2004; Barnes-Holmes, Barnes-Holmes, Smeets, example, A > B, B < C. In nonlinear series, stimuli are not pre-
Strand, & Friman, 2004; Belisle et al., 2016; Berens & sented linearly (e.g., A–B, C–A); in a nonlinear-same series the
Hayes, 2007). Thus, identifying weaknesses in relational abil- contextual cues are the same between relata, and in a nonlinear-
ity could provide the scaffolding needed for a relational cur- mixed series the cues are not the same. For example, A > B, C > A
riculum (discussed in more detail below). The relational as- and B > A, C < A, respectively (Vitale et al., 2008). In Stage 3 in
sessment protocol used in the present study can provide a the present study, the coordination substage included linear-same,
useful instrument in such work going forward. linear-mixed, and nonlinear-same relational networks, and all oth-
Despite its advantages, the relational assessment measure used er substages (comparison, opposition, temporality, and hierarchy)
herein may also need some further refinement. Analyses of the included only linear-same relational networks. In order to distin-
present data reveal certain trends that are interesting; nevertheless, guish between sameness and difference, the difference relation
we have to bear in mind that the format of the assessment may had to be included in the coordination relational networks, thus
have made some relations more difficult than they would be if we resulting in mixed networks. For all substages including coordi-
had given participants a symmetry test or a MTS task, for exam- nation, however, questions regarding relations in the network al-
ple. In addition, because the assessment is constrained in what it ways progressed from directly trained, to mutually entailed, to
tests in each particular frame, we can make some comparisons combinatorially entailed relations.
between individual frames, but it is hard to be absolute or defin- In Stage 4, the coordination and opposition substages included
itive about relational ability. For example, 1) we can clearly see linear-mixed networks, and the comparative, temporal, and hier-
that across the four stages, coordination is a simpler relation than archical substages included nonlinear-mixed networks. Deriving
the other relations; 2) in Stage 3, we can visibly see an improve- relations from nonlinear networks is more difficult than deriving
ment in substages across age; and 3) in Stage 4, we can see the from linear networks (Hunter, 1957; Vitale et al., 2008). However,
acquisition of analogical responding at age 5, but other compari- all substages included mixed contextual cues, and as in Stage 3,
sons are not as conclusive. However, the primary purpose of the trials progressed from directly trained, to mutually entailed, to
assessment is to provide a general overview of a child's nonarbi- combinatorially entailed analogical relations. Future replications
trary and arbitrary relational repertoire. In order get a more com- of Stages 3 and 4 should include systematic transitions from
prehensive measure of a child's relational abilities, future re- linear-same-mixed to nonlinear-same-mixed relational networks
searchers could develop a relational protocol that systematically in order to gain more detailed information regarding participants’
assesses direct relations, entailed relations, and transformation of relational skills.
stimulus function with more scope and depth. For example, this
could be done by adding more detailed tasks to each substage, one
stage at a time, or by focusing on the evolution of nonarbitrary to Conclusion
arbitrary relational framing across age groups, one frame at a time.
Furthermore, additional relations such as distinction and deixis Research in RFT has produced considerable proof that rela-
could be included in future relational assessments. Approaching tional framing is a core skill underlying language and cogni-
relational training in such a functionally specified way will enable tion. However, despite the evidence, research into the assess-
language protocols to assess and train more generative, flexible, ment and training of relational responding remains limited.
and complex language. The present data contribute to the exiguous literature on rela-
A second potential limitation of the relational assessment tional testing and training, and initiate the RFT-research on the
may be the omission of distinction as an independent frame. In normative development of frames. The results from the pres-
the present study, distinction was not tested in Stage 1; and in ent study could be referred to as a starting point for further
Stages 2–4 distinction was included in the coordination sub- research in the normative development and sequence of rela-
stage in order to provide a comparison option. More work on tional frames, and for relational skills assessment and training.
 Psychol Rec

Appendix A1

Relational Assessment Stage 1: Nonarbitrary Relations

Coordination

Example: Which one is like the one at the top (instructor points to sample)?

Comparison

Example: Which one is bigger/smaller than the one at the top (instructor points to sample)?

Opposition

Example: Which one is opposite to the one at the top (instructor points to sample)?

Temporality

Example: Which x is before/after the other y? Was the x before/after the y?

Hierarchy

Example: Which one is inside? Which one contains the other one? Does x contain y? Is x inside y?
Psychol Rec

Appendix A2

Relational Assessment Stage 2: Nonarbitrary Analogical

Relations

Coordination

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?

Comparison

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?

Opposition

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?

Temporality

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?

Hierarchy

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)
 Psychol Rec

Appendix A3

Relational Assessment Stage 3: Arbitrary Relations

Coordination

S
S
D

Example: Does x like the same food as y? Does x like different food than y?

Comparison

M
M
M

Example: Which coin would you take to the store?

Opposition

O
O
O

Example: Which coin would you take to the store?

Temporality

B
A
B

Example: Is x before/after y?

Hierarchy

C
I
C

Example: Is x inside y? Does x contain y?

Psychol Rec

Appendix A4

Relational Assessment Stage 4: Arbitrary Analogical

Relations

Coordination

1 4 8

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?

Comparison

M L M M L M M L M

1 5 10

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?

Opposition

S O S S O S S O S

1 5 10

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)

Temporality

B B A B B A B B A

1 4 10

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?

Hierarchy

I C I I C I I C I

`
`

1 5 10

Example: Which one of these (points to comparisons) is like the one at the top (points to sample)?
 Psychol Rec

Appendix 2 process-based therapy. The Psychological Record. Advance online

publication. https://doi.org/10.1007/s40732-019-00372-3.
Barnes-Holmes, D., Hayes, S. C., Dymond, S., & O’Hora, D. (2001).
Multiple stimulus relations and the transformation of stimulus func-
tions. In S. C. Hayes, D. Barnes-Holmes, & B. Roche (Eds.),
Table 4 Chance level responding across stages and substages Relational frame theory: A post-Skinnerian account of human lan-
guage and cognition (pp. 51–71). Kluwer Academic/Plenum
Stage 1 Stage 2 Stage 3 Stage 4 Barnes-Holmes, Y., Barnes-Holmes, D., & Smeets, P. M. (2004).
Establishing relational responding in accordance with opposite as
Coord 33% 50% 38% 50% generalized operant behavior in young children. International
Comp 33% 50% 40% 50% Journal of Psychology & Psychological Therapy, 4, 559–586
Opp 33% 50% 40% 50% Barnes-Holmes, Y., Barnes-Holmes, D., Smeets, P. M., Strand, P., &
Friman, P. (2004). Establishing relational responding in accordance
Temp 50% 50% 42% 50%
with more-than and less-than as a generalized operant behavior in
Hier 45% 50% 40% 50% young children. International Journal of Psychology &
Psychological Therapy, 4, 531–558
This table represents chance level responding, or what you might expect a Belisle, J., Dixon, M. R., Stanley, C. R., Munoz, B., & Daar, J. H. (2016).
child to get correct purely on the basis of chance or consistently guessing. Teaching foundational perspective-taking skills to children with au-
These data can be used to compare with participant relational assessment tism using the PEAK-T curriculum: Single-reversal “I-You” deictic
scores frames. Journal of Applied Behavior Analysis, 49(4), 965–969.
https://doi.org/10.1002/jaba.324.
Berens, N. M., & Hayes, S. C. (2007). Arbitrarily applicable comparative
relations: Experimental evidence for a relational operant. Journal of
Availability of data and material The datasets generated and analyzed Applied Behavior Analysis, 40(1), 45–71. https://doi.org/10.1901/
during the current study are available from the corresponding author on jaba.2007.7-06.
reasonable request. Bod, R. (2009). From exemplar to grammar: A probabilistic analogy-
Code availability Not applicable based model of language learning. Cognitive Science, 33(5), 752–
793. https://doi.org/10.1111/j.1551-6709.2009.01031.x.
Funding Not applicable Carpentier, F., Smeets, P. M., & Barnes-Holmes, D. (2002). Matching
functionally same relations: Implications for equivalence-
equivalence as a model for analogical reasoning. The
Declaration Psychological Record, 52, 351–370. https://doi.org/10.1007/
BF03395435.
Conflicts of interest The authors declare that they have no conflict of Carpentier, F., Smeets, P. M., & Barnes-Holmes, D. (2003). Equivalence-
interest. equivalence as a model of analogy: Further analyses. The
Psychological Record, 53, 349–371
Ethical Approval The study was approved by the NUI Galway Research Carpentier, F., Smeets, P. M., Barnes-Holmes, D., & Stewart, I. (2004).
Ethics Committee. All procedures performed in the studies involving Matching derived functionally-same stimulus relations:
human participants were in accordance with the ethical standards of the Equivalence-equivalence and classical analogies. The
institutional research committee and with the 1964 Helsinki declaration Psychological Record, 54, 255–273. https://doi.org/10.1007/
and its later amendments or comparable ethical standards. BF03395473.
Cassidy, S., Roche, B., Colbert, D., Stewart, I., & Grey, I. M. (2016). A
Consent to participate Informed consent was obtained from all individ- relational frame skills training intervention to increase general intel-
ual participants included in the study. ligence and scholastic aptitude. Learning & Individual Differences,
47, 222–235. https://doi.org/10.1016/j.lindif.2016.03.001.
Cassidy, S., Roche, B., & Hayes, S. C. (2011). A relational frame training
intervention to raise intelligence quotients: A pilot study. The
References Psychological Record, 61(2), 173–198. https://doi.org/10.1007/
BF03395755.
Alexander, P.A., Willson, V. L., White, C. S., Fuqua, J. D., Clark, G. D., Christie, S., & Gentner, D. (2014). Language helps children succeed on a
Wilson, A. F., & Kulikowich, J. M. (1989). Development of ana- classic analogy task. Cognitive Science, 38(2), 383–397. https://doi.
logical reasoning in 4- and 5-year-old children. Cognitive org/10.1111/cogs.12099.
Development, 4(1), 65–88. https://doi.org/10.1016/0885-2014(89) Dixon, M. R., Belisle, J., McKeel, A., Whiting, S., Speelman, R., Daar, J.
90005-1. H., & Rowsey, K. (2017). An internal and critical review of the
Barnes, D., Hegarty, N., & Smeets, P. M. (1997). Relating equivalence PEAK relational training system for children with autism and related
relations to equivalence relations: A relational framing model of intellectual disabilities: 2014–2017. The Behavior Analyst, 40(2),
complex human functioning. Analysis of Verbal Behavior, 14(1), 493–521. https://doi.org/10.1007/s40614-017-0119-4.
57–83. https://doi.org/10.1007/BF03392916. Dixon, M. R., Belisle, J., & Stanley, C. R. (2018). Derived relational
Barnes-Holmes, D., Barnes-Holmes, Y., Luciano, C., & McEnteggart, C. responding and intelligence: Assessing the relationship between
(2017). From the IRAP and REC model to a multi-dimensional the PEAK-E pre-assessment and IQ with individuals with autism
multi-level framework for analyzing the dynamics of arbitrarily ap- and related disabilities. The Psychological Record, 68(4), 419–
plicable relational responding. Journal of Contextual Behavioral 430. https://doi.org/10.1007/s40732-018-0284-1.
Science, 6(4), 434–445. https://doi.org/10.1016/j.jcbs.2017.08.001. Dixon, M. R., Belisle, J., Whiting, S. W., & Rowsey, K. E. (2014).
Barnes-Holmes, D., Barnes-Holmes, Y., & McEnteggart, C. (2020). Normative sample of the PEAK relational training system: Direct
Updating RFT (more field than frame) and its implications for training module and subsequent comparisons to individuals with
Psychol Rec

autism. Research in Autism Spectrum Disorders, 8(11), 1597–1606. Hayes, J., & Stewart, I. (2016). Comparing the effects of derived rela-
https://doi.org/10.1016/j.rasd.2014.07.020. tional training and computer coding on intellectual potential in
Dougher, J. M., Hamilton, D. A., Fink, B. C., & Harrington, J. (2007). school-age children. British Journal of Educational Psychology,
Transformation of the discriminative and eliciting functions of gen- 86(3), 397–411. https://doi.org/10.1111/bjep.12114.
eralized relational stimuli. Journal of the Experimental Analysis of Hayes, S. C., Barnes-Holmes, D., & Roche, B. (eds.). (2001a). Relational
Behavior, 88(2), 179–197. https://doi.org/10.1901/jeab.2007.45-05. frame theory: A post-Skinnerian account of human language and
Dymond, S., & Barnes, D. (1995). A transformation of self- cognition. Kluwer Academic/Plenum
discrimination response functions in accordance with the arbitrarily Hayes, S. C., Gifford, E. V., Wilson, K. G., Barnes-Holmes, D., & Healy,
applicable relations of sameness, more than, and less than. Journal O. (2001b). Derived relational responding as learned behavior. In S.
of the Experimental Analysis of Behavior, 64(2), 163–184. https:// C. Hayes, D. Barnes-Holmes, & B. Roche (Eds.), Relational frame
doi.org/10.1901/jeab.1995.64-163. theory: A post-Skinnerian account of human language and
Dymond, S., May, R. J., Munnelly, A., & Hoon, A. E. (2010). Evaluating cognition (pp. 21–49). Kluwer Academic/Plenum
the evidence base for relational frame theory: A citation analysis. Hughes, S., & Barnes-Holmes, D. (2016). Relational frame theory: The
The Behavior Analyst, 33(1), 97–117. https://doi.org/10.1007/ basic account. In R. D. Zettle, S. C. Hayes, D. Barnes-Holmes, & A.
bf03392206. Biglan (Eds.), The Wiley handbook of contextual behavioral science
Dymond, S., & Roche, B. (eds.). (2013). Advances in relational frame (pp. 129–178). Wiley Blackwell
theory: Research and application. Context Press/New Harbinger. Hunter, I. M. L. (1957). The solving of three term series problems. British
Fagot, J., & Maugard, A. (2013). Analogical reasoning in baboons (papio Journal of Psychology, 48(4), 286–298. https://doi.org/10.1111/j.
papio): Flexible reencoding of the source relation depending on the 2044-8295.1957.tb00627.x.
target relation. Learning & Behavior, 41, 229–237. https://doi.org/ Lapiana, W. P. (2004). Merit and diversity: The origins of the Law School
10.3758/s13420-012-0101-7. Admissions Test. Saint Louis University Law Journal, 48, 955–990
Fienup, D. M., & Brodsky, J. (2020). Equivalence-based instruction: Levinson, P. J., & Carpenter, R. L. (1974). An analysis of analogical
Designing instruction using stimulus equivalence. In M. Fryling, reasoning in children. Child Development, 45(3), 857–861. https://
R. A. Rehfeldt, J. Tarbox & L. J. Hayes (Eds.), Applied behavior doi.org/10.2307/1127862.
analysis of language & cognition (pp. 157–173). Context Press/ Lipkens, R., Hayes, S. C., & Hayes, L. J. (1993). Longitudinal study of
New Harbinger the development of derived relations in an infant. Journal of
Fryling, M., Rehfeldt, R. A., Tarbox, J., & Hayes, L. J. (eds.). (2020). Experimental Child Psychology, 56, 201–239
Applied behavior analysis of language and cognition. Context Luciano, C., Rodriguez, M., Manas, I., Ruiz, F., Berens, N. M., &
Press/New Harbinger Valdivia-Salas, S. (2009). Acquiring the earliest relational operants:
Coordination, distinction, opposition, comparison, and hierarchy. In
Garred, M., & Gilmore, L. (2009). To WPPSI or to Binet, that is the
R. A. Rehfeldt & Y. Barnes-Holmes (Eds.), Derived relational
question: A comparison of the WPPSI-III and SB5 with typically
responding applications for learners with autism and other devel-
developing preschoolers. Australian Journal of Guidance &
opmental disabilities (pp. 149–172). Context Press/New Harbinger
Counselling, 19(2), 104–115. https://doi.org/10.1375/ajgc.19.2.104.
Lunzer, E. A. (1965). Problems of formal reasoning in test situations.
Gentner, D. (1983). Structure mapping: A theoretical framework for anal-
Monographs of the Society for Research in Child Development,
ogy. Cognitive Science, 7(2), 155–170. https://doi.org/10.1016/
30(2), 19–46. https://doi.org/10.2307/1165774.
S0364-0213(83)80009-3.
McCormack, T., & Hoerl, C. (2017). The development of temporal con-
Gentner, D., & Christie, S. (2010). Mutual bootstrapping between lan- cepts: Learning to locate events in time. Timing & Time Perception,
guage and analogical processing. Language & Cognition, 2(2), 261– 5(3–4), 297–327. https://doi.org/10.1163/22134468-00002094.
283. https://doi.org/10.1515/LANGCOG.2010.011. McHugh, L., Barnes-Holmes, Y., & Barnes-Holmes, D. (2004). A rela-
Gil, E., Luciano, C., Ruiz, F. J., & Valdivia-Salas, S. (2014). A further tional frame account of the development of complex cognitive phe-
experimental step in the analysis of hierarchical responding. nomena: Perspective taking, false belief understanding, and decep-
International Journal of Psychology & Psychological Therapy, tion. International Journal of Psychology & Psychological Therapy,
14(2), 137–153 4, 303–323
Gillan, D. J., Preman, D., & Woodruff, G. (1981). Reasoning in the Moran, L., Stewart, I., McElwee, J., & Ming, S. (2014). Relational ability
chimpanzee: I. Analogical reasoning. Journal of Experimental and language performance in children with autism spectrum disor-
Psychology: Animal Behavior Processes, 7(1), 1–17. https://doi. ders and typically developing children: A further test of the TARPA
org/10.1037/0097-7403.7.1.1. protocol. The Psychological Record, 64, 233–251. https://doi.org/
Gore, N. J., Barnes-Holmes, Y., & Murphy, G. (2010). The relationship 10.1007/s40732-014-0032-0.
between intellectual functioning and relational perspective-taking. Morsanyi, K., & Holyoak, K. J. (2010). Analogical reasoning ability in
International Journal of Psychology & Psychological Therapy, autistic and typically developing children. Developmental Science,
10(1), 1–17 13(4), 578–587. https://doi.org/10.1111/j.1467-7687.2009.00915.x.
Gorham, M., Barnes-Holmes, Y., Barnes-Holmes, D., & Berens, N. Mulhern, T., Stewart, I., & McElwee, J. (2017). Investigating relational
(2009). Derived comparative and transitive relations in young chil- framing of categorization in young children. The Psychological
dren with and without autism. The Psychological Record, 59, 221– Record, 67, 519–536. https://doi.org/10.1007/s40732-017-0255-y.
246. https://doi.org/10.1007/BF03395660. Mulhern, T., Stewart, I., & McElwee, J. (2018). Facilitating relational
Goswami, U. (1989). Relational complexity and the development of an- framing of classification in young children. Journal of Contextual
alogical reasoning. Cognitive Development, 4(3), 251–268. https:// Behavioral Science, 8, 55–68. https://doi.org/10.1016/j.jcbs.2018.
doi.org/10.1016/0885-2014(89)90008-7. 04.001.
Goswami, U., & Brown, A. L. (1989). Melting chocolate and melting O’Hora, D., Pelaez, M., & Barnes-Holmes, D. (2005). Derived relational
snowmen: Analogical reasoning and causal relations. Cognition, responding and performance on verbal subtests of the WAIS-III. The
35(1), 69–95. https://doi.org/10.1016/0010-0277(90)90037-K. Psychological Record, 55, 155–175. https://doi.org/10.1007/
Goswami, U., & Brown, A. L. (1990). Higher-order structure and rela- BF03395504.
tional reasoning: Contrasting analogical and thematic relations. O’Hora, D., Pelaez, M., Barnes-Holmes, D., Raw, G., Robinson, K., &
Cognition, 36(3), 207–226. https://doi.org/10.1016/0010-0277(90) Chaudhary, T. (2008). Temporal relations and intelligence: correlat-
90057-Q. ing relational performance with performance on the WAIS-III. The
 Psychol Rec

Psychological Record, 58, 569–584. https://doi.org/10.1007/ Sternberg, R. J., & Nigro, G. (1980). Developmental patterns in the so-
BF03395638. lution of verbal analogies. Child Development, 51(1), 27–38. https://
O’Toole, C., & Barnes-Holmes, D. (2009). Three chronometric indices of doi.org/10.2307/1129586.
relational responding as predictors of performance on a brief intel- Stewart, I. (2016). The fruits of a functional approach for psychological
ligence test: The importance of relational flexibility. The science. International Journal of Psychology, 51(1), 15–27. https://
Psychological Record, 59, 119–132. https://doi.org/10.1007/ doi.org/10.1002/ijop.12184.
BF03395652. Stewart, I., Barnes-Holmes, D., & Roche, B. (2004). A functional-
Partington, J. W. (2008). The assessment of basic language and learning analytic model of analogy using the relational evaluation procedure.
skills-revised (the ABLLS-R). Behavior Analysts The Psychological Record, 54(4), 531–552. https://doi.org/10.1007/
Piaget, J., Montangero, J., & Billeter, J. B. (2001). The formation of BF03395491.
analogies (R. L. Campbell, Trans.). In J. Piaget, Studies in reflecting Stewart, I., McElwee, J., & Ming, S. (2013). Language generativity,
abstraction (pp. 139–152). Psychology Press. (Original work pub- response generalization, and derived relational responding.
lished 1977) Analysis of Verbal Behavior, 29, 137–155. https://doi.org/10.1007/
Pyykkönen, P., & Järvikivi, J. (2012). Children and situation models of BF03393131.
multiple events. Developmental Psychology, 48(2), 521–529. Stewart, I., McLoughlin, S., Mulhern, T., Ming, S., & Kirsten, E. B.
https://doi.org/10.1037/a0025526. (2020). Assessing and teaching complex relational operants:
Analogy and hierarchy. In M. Fryling, R. A. Rehfeldt, J. Tarbox,
Rehfeldt, R. A., & Barnes-Holmes, Y. (eds.). (2009). Derived relational
M. Fryling, & L. J. Hayes (Eds.), Applied behavior analysis of
responding: Applications for learners with autism and other devel-
language and cognition: Core concepts & principles for
opmental disabilities. Context Press/New Harbinger
practitioners. Context Press/New Harbinger
Roid, G. H. (2003). Stanford-Binet intelligence scales (5th ed.). Riverside Stewart, I., & Roche, B. (2013). Relational frame theory: An overview. In
Publishing S. Dymond & B. Roche (Eds.), Advances in relational frame theory:
Sidman, M. (1971). Reading and auditory-visual equivalences. Journal of Research and application (pp. 51–71). Context Press/New
Speech & Hearing Research, 14(1), 5–13. https://doi.org/10.1044/ Harbinger
jshr.1401.05. Sundberg, M. L. (2008). VB-MAPP: Verbal behavior milestones assess-
Skinner, B. F. (1957). Verbal behavior. Prentice Hall ment and placement program. AVB Press
Smirnova, A., Zorina, Z., Obozova, T., & Wasserman, E. (2015). Crows Vitale, A., Barnes-Holmes, Y., Barnes-Holmes, D., & Campbell, C.
spontaneously exhibit analogical reasoning. Current Biology, 25, (2008). Facilitating responding in accordance with the relational
256–260. https://doi.org/10.1016/j.cub.2014.11.063. frame of comparison: Systematic empirical analyses. The
Steele, D., & Hayes, S. C. (1991). Stimulus equivalence and arbitrarily Psychological Record, 58, 365–390. https://doi.org/10.1007/
applicable relational responding. Journal of the Experimental BF03395624.
Analysis of Behavior, 56(3), 519–555. https://doi.org/10.1901/jeab. Zettle, R. D., Hayes, S. C., Barnes-Holmes, D., & Biglan, A. (eds.).
1991.56-519. (2016). The Wiley handbook of contextual behavioral science.
Sternberg, R. J. (1977). Component processes in analogical reasoning. Wiley Blackwell
Psychological Review, 84(4), 353–378
Sternberg, R. J., Grigorenko, E. L., & Bundy, D. A. (2001). The predic- Publisher’s Note Springer Nature remains neutral with regard to jurisdic-
tive value of IQ. Merrill-Palmer Quarterly, 47(1), 1–41 tional claims in published maps and institutional affiliations.