ISO/TR 21959-2:2020

TECHNICAL ISO/TR
REPORT 21959-2
First edition
2020-02
Road vehicles — Human performance
and state in the context of automated
driving —
Part 2:
Considerations in designing
experiments to investigate transition
processes
Véhicules routiers — Etat et performance humaine dans le contexte
de la conduite automatisée —
Partie 2: Principes expérimentaux pour etudier les processus de
transition
Reference number
ISO/TR 21959-2:2020(E)
©
ISO 2020

---------------------- Page: 1 ----------------------
ISO/TR 21959-2:2020(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/TR 21959-2:2020(E)

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 List of Acronyms . 1
5 Purpose . 2
6 Transition process models . 2
6.1 General . 2
6.2 Transition process model for system-initiated transitions . 2
6.3 Transition process model for human-initiated transitions . 3
7 Human factors that influence takeover performance . 4
7.1 General . 4
7.2 Driver attributes . 4
7.2.1 Knowledge . 4
7.2.2 Experience and trust . 5
7.2.3 Demographic attributes . 5
7.3 Driver readiness/availability . 5
7.3.1 Sitting position and posture . 6
7.3.2 Engagement in non-driving related activities . 6
7.3.3 Drowsiness . 7
7.3.4 Mind wandering . 8
7.3.5 Situation awareness . 8
7.3.6 Operating state/mode awareness . 9
7.3.7 Attentiveness . 9
7.3.8 Receptivity . 9
8 System factors that influence takeover performance . 9
8.1 General . 9
8.2 System behaviour .10
8.2.1 Type of transition .10
8.2.2 System behaviour within takeover mode .10
8.2.3 System-initiated risk mitigation strategy .10
8.2.4 System limitations and failures .11
8.2.5 Stability and reliability of the system functions .11
8.2.6 Level of automated driving to which the system shifts in transition . .12
8.3 Human machine interfaces for RtI .12
8.3.1 Design parameters for HMI .12
8.3.2 Total time budget.13
8.3.3 Other human machine interfaces to improve drivers’ takeover performance .13
9 Test scenarios .13
9.1 General .13
9.2 Parameters for specifying test scenarios .14
9.3 Considerations for selecting/designing adequate test scenarios .16
9.3.1 Investigating driver state transitions during automated driving .16
9.3.2 Investigating takeover performance in non-critical transitions .16
9.3.3 Assessing takeover performance at system limits .16
10 Takeover performance .17
10.1 Introduction .17
10.2 Taxonomy of human performance measures .18
10.2.1 Addressed phases of transition .18
© ISO 2020 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/TR 21959-2:2020(E)

10.2.2 Suitability for transition type .19
10.2.3 Scope of assessment .20
10.2.4 Data considerations . . .21
10.3 Overview of measures and characteristics .22
11 Testing environments .24
11.1 General .24
11.2 Types.24
11.2.1 Simulator studies .24
11.2.2 Roadway studies .26
11.3 Advantages and disadvantages .27
11.3.1 Realism-to-safety trade-off .27
11.4 Considerations for test environment selections .29
Annex A (informative) Human machine interfaces/interactions for automated vehicles .30
Bibliography .34
iv © ISO 2020 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/TR 21959-2:2020(E)

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 39,
Ergonomics.
A list of all parts in the ISO 21959 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO 2020 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO/TR 21959-2:2020(E)

Introduction
Although automation technology is advancing at a rapid pace, the majority of automated driving levels
[1]
(as defined by SAE J3016, 2016 ) still require a human to fulfil specific remaining (driving related)
tasks. The safety-critical human’s task is the takeover task in transition from a higher level to a lower
level of automated driving. Researchers and developers continue to seek system design and human
machine interface improvements for better takeover performance. Researchers face a challenge
in understanding the limitations of a human’s ability to perform the takeover task, which involves
different human factors. Developers work to evaluate systems to see whether the takeover process is
effective at minimum risk in specific scenarios. There are a wide variety of experiments to evaluate
takeover performance in transition for many different purposes. This document contains information
to consider in the takeover scenario, some of which is still under investigation, in order to help readers
design experiments to evaluate takeover performance and design appropriate experiments.
vi © ISO 2020 – All rights reserved

---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/TR 21959-2:2020(E)
Road vehicles — Human performance and state in the
context of automated driving —
Part 2:
Considerations in designing experiments to investigate
transition processes
1 Scope
This document focuses on system-initiated and human-initiated transitions (Clause 6) from a higher
level to a lower level of automated driving. Human factors and system factors that can influence
takeover performance are included (Clauses 7 and 8). Although some are still under investigation, there
is a need to appropriately set these factors as variables to better understand their effects or to better
control/eliminate their influence. This approach will aid research design by ensuring that important
factors are considered and support consistency across studies enabling meaningful comparisons of
findings. This document also includes information on considerations in test scenario design (Clause 9),
common measures for human takeover performance (Clause 10) and considerations in choosing a
testing environment (Clause 11) to help readers design experiments comparable to other studies.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
No terms and definitions are listed in this document.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
4 List of Acronyms
DDT Dynamic Driving Task
DMS Driver Monitor System
ECG Electrocardiogram
EEG Electroencephalogram
HMI Human-Machine Interface
KSS Karolinska Sleepiness Scale
MRM Minimal Risk Manoeuvre
NDRT Non-driving Related Task
© ISO 2020 – All rights reserved 1

---------------------- Page: 7 ----------------------
ISO/TR 21959-2:2020(E)

NDRA Non-driving Related Activities
OEDR Object and Event Detection and Response
ODD Operational Design Domain
RtI Request to Intervene
SAGAT Situation Awareness Global Assessment Technique
SDLP Standard Deviation of Lateral Position
SIMS Situational Motivation Scale
SuRT Surrogate Reference Task
TLC Time to Lane Crossing
TTC Time-to-Contact/Collision
5 Purpose
The purpose of this document is to provide considerations in designing experiments to measure human
takeover performance in transition situations in order to better understand human limitations, evaluate
systems, and improve systems, including human machine interfaces. This document is expected to help
users design appropriate experiments for their purposes. This document does not provide any design
principles to restrict or direct the system design.
6 Transition process models
6.1 General
Transition processes included in this document are generally based on the models defined in
[2]
ISO/TR 21959-1 . A human’s safety-critical task is the takeover task in transition from a higher
level to a lower level of automated driving both for system-initiated and human-initiated transitions.
[2]
ISO/TR 21959-1 defines typical transition process models from automated to manual driving (i.e.
level 0). However, the models can be adapted for transitions between different levels (e.g. 4→2 or 3→1).
This clause reminds readers of the relevant transition process models.
6.2 Transition process model for system-initiated transitions
In a system-initiated transition, the system may issue a request to intervene (RtI) when it finds a
dynamic driving task (DDT) performance-relevant system failure or an object/event which cannot be
handled by the system for levels 1–4. The system also may issue an RtI when exiting the operational
design domain (ODD) for which it was designed, (e.g. exiting a motorway, exiting assumed environmental
conditions such as weather and traffic). The driver is expected to take over the DDT in response to an
RtI to continue driving. The system may terminate immediately after issuing an RtI for level 2 while
it shifts to the takeover mode following an RtI before termination for levels 3 and 4 (Figure 1). There
can also be other types of transitions after an RtI, such as transitions from level 2 to level 1 and from
level 3 to level 2. The driver’s task model can be adjusted depending on the level after the RtI (i.e. object
and event detection and response [OEDR] task for transition to level 2, OEDR task +lateral control or
OEDR task +longitudinal control for transition to level 1). When the driver does not initiate intervention
within the takeover mode, the system may shift to the minimal risk manoeuvre (MRM) to stop the
vehicle safely for level 3 and level 4 (see 8.2.2 for details).
2 © ISO 2020 – All rights reserved

---------------------- Page: 8 ----------------------
ISO/TR 21959-2:2020(E)

Figure 1 — Transition process model for system-initiated transitions from automated to
manual driving
6.3 Transition process model for human-initiated transitions
The driver is authorized to take over the DDT at any point during operation of the automated driving
functions, except for some level 4 and level 5 features some or all of the time. The human-initiated
transition may be either optional or mandatory. The optional case is the transition where the user
wishes to drive manually without being in a safety critical situation. The mandatory case is the
transition where the level 2 system fails to avoid an undetected object/event due to the system’s
functional limitations or where the system suddenly terminates without issuing an RtI due to a DDT
performance-relevant system failure. In such mandatory transitions, the driver is expected to detect
the object/event or the failure and initiate transition (Figure 2). This type of transition is mainly from
level 2 to manual driving but can be from level 2 to level 1. The driver’s task model can be adjusted
depending on the level after the initiation.
© ISO 2020 – All rights reserved 3

---------------------- Page: 9 ----------------------
ISO/TR 21959-2:2020(E)

Figure 2 — Transition process model for mandatory human-initiated transitions from
automated (level 2) to manual driving due to driver’s detection of a safety-critical object/event
or a DDT performance-relevant system failure
7 Human factors that influence takeover performance
7.1 General
It is known that a driver’s takeover performance varies with the influences of multiple factors. In
experiments, there is a need to appropriately set factors as variables for investigating their effects or
better controlling/eliminating their influence. This will allow for the design of experiments that are
easier to compare with other studies. This clause presents information about “internal” human factors
that may influence a driver’s takeover performance. Driver’s takeover performance includes time in the
driver state transition phase (i.e. response time of significant driver intervention to RtI) and quality in
the post transition control phase (i.e. how well the driver controls the vehicle right after the significant
driver intervention; see also Clause 10).
7.2 Driver attributes
7.2.1 Knowledge
Drivers’ knowledge about system functions, limitations and the required driver’s role, has been found
[3][4][5]
to influence takeover performance in some studies . Other studies have found that instructions
[6][7]
have limited effects . In general, the sources of a driver’s knowledge are diverse and may include
mass-media, instruction manuals, instructions given at a car-dealership and other various sources.
In experiments, such knowledge can be controlled, to some extent, by screening subjects using
questionnaires investigating their level of a-priori knowledge and by providing them with controlled
information about the functions, limitations and driver’s role for the specific system of study. It is to be
noted that difficulty in forming a detailed picture of subjects’ exact knowledge obtained from various
sources for various systems may lead to some variation in the results of takeover performance. In
some instances, participants may have incorrect knowledge about system function, limitations and the
required driver’s role leading to misbehaviour or misuse.
4 © ISO 2020 – All rights reserved

---------------------- Page: 10 ----------------------
ISO/TR 21959-2:2020(E)

7.2.2 Experience and trust
A driver’s experience with using the system has been found to influence takeover performance. How
the driver has previously interacted with the system may influence performance in different ways as
a result of different levels of understanding and trust. Short term system interaction experiences may
lead to a better understanding of the system’s functions and limitations and better trust calibration,
[8][9][10][11][4][5]
resulting in better driver takeover performance . Longer-term experience with no to a
[12]
few system disengagements may lead to a driver’s over-trust of the system and complacency , which
[6]
can degrade takeover performance . In contrast, longer-term experience with too many system
[13]
disengagements may lead to driver under-trust, which might improve takeover performance but
[14]
which also may lead to disuse of the system . In experiments, such experiences can be controlled, to
some extent, by screening subjects using questionnaires investigating their experiences with specific
systems featured in the study and the frequencies of the interactions they have experienced. After
screening, new subject experience can be introduced by providing subjects specific driving conditions
after providing controlled interactions. It is to be noted that similar systems with the same level of
automated driving can differ in functions and limitations (i.e. detection targets, ODD, reliability and
others) by brand or even by different models within one brand. Also, different users of the same brand
system may use the system in different ways in different traffic environments. Therefore, experience
still may lead to some dispersion in the results.
7.2.3 Demographic attributes
7.2.3.1 Age
A driver’s age-related perceptual, cognitive and physical limitations may influence takeover
performance. Visual impairments of older drivers are diverse and are often accompanied by eye
[15]
diseases . Such impairments may degrade perception of traffic environment in the OEDR task or
in the process of transition. Visual impairments may also cause difficulty in reading system status
[16]
information displayed in the cockpit . Cognitive impairments may degrade understanding of
“complicated” system functions/limitations and the driver’s role. These impairments may also lead to
[17]
problems with divided attention and slow down task switching in transitions. Physical impairment
may degrade speed and accuracy of the response behaviour in transitions. Although, as mentioned
above, there are several hypotheses for older drivers’ degraded takeover performance, the effects of
[18][19]
age are still under discussion. Some researchers have found significant negative effects of age
[5][20][21] [22][23]
, whereas other researchers found only limited effects . In experiments, subjects can
be screened not only based on age but also based on the results of perceptual, cognitive and physical
response tests. However, it is to be noted that the effects of age have large inter/intra-individual
variability and still may lead to some dispersion in the results of takeover performance of subjects who
were screened via tests.
7.2.3.2 Other demographic attributes
There are other driver demographic attributes that may influence takeover performance, such as
experience and skill of manual driving, style of manual driving with individual and cultural differences,
technology-sensitivity and general trust of technology. However, these factors have not yet been well
studied.
7.3 Driver readiness/availability
Conceptually, readiness/availability is a driver’s dynamic state during automated driving, which
influences their takeover performance. Readiness/availability can be continuous; lower readiness/
[24][25][26][27]
availability than a required level may lead to degraded and unsafe takeover performance
[28]
(see also A.2.2). Considering the definitions of the driver’s role for each level of automation, the
required level of readiness/availability generally increases with decreasing levels of automation. The
readiness/availability is considered to include several components related to motoric/physical and
cognitive states (Table 1). Each component can have different metrics and different effects on takeover
performance. The required level for each component of readiness/availability can be experimentally
determined as the level that leads to a successful takeover by comparing the metrics and the takeover
© ISO 2020 – All rights reserved 5

---------------------- Page: 11 ----------------------
ISO/TR 21959-2:2020(E)

performance in time and quality in certain traffic conditions for a specific system transition design (see
also A.2.2).
Table 1 — Components of readiness/availability
Components of readiness/ Motoric/physical state Cognitive state
availability
Sitting position —
Posture —
Hands/arms, feet/legs, trunk
Engagement in NDRAs
Hands/arms, manual operation Visual
Cognitive
Drowsiness —
Mind wandering —
Situation awareness —
Operating state/m
...