ISO/TS 14198:2019

TECHNICAL ISO/TS
SPECIFICATION 14198
Second edition
2019-04
Road vehicles — Ergonomic aspects
of transport information and control
systems — Calibration tasks for
methods which assess driver demand
due to the use of in-vehicle systems
Véhicules routiers — Aspects ergonomiques des systèmes
d'information et de contrôle du transport — Tâches de calibration
pour méthodes qui évaluent la distraction du conducteur due à
l'utilisation des systèmes embarqués
Reference number
ISO/TS 14198:2019(E)
©
ISO 2019

---------------------- Page: 1 ----------------------
ISO/TS 14198:2019(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2019 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/TS 14198:2019(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 3
5 Calibration tasks . 4
5.1 Principle and overview . 4
5.2 Types of calibration tasks . 4
5.3 Critical Tracking Task (CTT). 4
5.3.1 Description . . . 4
5.3.2 Operation of the CTT . 5
5.3.3 Setup for CTT . 6
5.3.4 Test conditions for CTT . 6
5.3.5 Participants for CTT . 6
5.3.6 Participant instruction for CTT . 6
5.3.7 Practice trials . 7
5.3.8 Test metrics . 7
5.4 Surrogate Reference Task (SURT) . 7
5.4.1 Description . . . 7
5.4.2 Setup for SURT . 8
5.4.3 Test conditions for SURT . 9
5.4.4 Participants for SURT . . 9
5.4.5 Participant instructions for SURT . 9
5.4.6 Practice trials .10
5.4.7 Test metrics .10
5.5 N-back .10
5.5.1 Description . . .10
5.5.2 Operation of the n-back .10
5.5.3 Setup for n-back .11
5.5.4 Test conditions for n-back .11
5.5.5 Participants for n-back .11
5.5.6 Participant instruction for n-back .11
5.5.7 Practice trials .12
5.5.8 Test metrics .12
6 Calibration criterion .12
6.1 Calibration criterion procedure .12
6.2 General calibration considerations .14
Annex A (informative) Calibration task setup details using the LCT .15
Annex B (informative) Multi-lab reference data for LCT .17
Annex C (informative) Calibration task setup details using the DRT .19
Annex D (informative) Multi-lab reference data for DRT .20
Bibliography .22
© ISO 2019 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/TS 14198:2019(E)

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso
.org/iso/foreword .html.
This document was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 39,
Ergonomics.
This second edition cancels and replaces the first edition (ISO/TS 14198:2012), which has been
technically revised. The main changes compared to the previous edition are as follows:
— in addition to the Lane Change Test (LCT), the Detection Response Task (DRT) is added as a
primary task;
— in addition to the Critical Tracking Task (CTT) and Surrogate Reference Task (SURT), the n-back
task is added as calibration task.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.
iv © ISO 2019 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/TS 14198:2019(E)

Introduction
The number of standardized methods to assess driver attentional demand due to the use of in-vehicle
information and communication devices is continuing to increase. In applying these methodologies, it is
important to understand and document variability in participants’ performance of standard calibration
tasks and procedures across laboratories and/or time.
A suitable calibration task should have the following attributes:
— It should be robust against the variations in cultural background of participants.
— Properly applied, the task should give repeatable quantitative results. It should be sensitive to
inappropriate variations in participants, equipment, location, experimenter and instruction.
— It should use durable and readily available equipment for conducting the task.
— It should apply to the driver population and be usable in a driving-like context.
A standardized calibration task can be used to produce a range of statistically stable, repeatable and
comparable secondary task demands for a participant in an experimental setting. This setting can be
used to assess the effect on driving performance of the attentional demand due to driver interaction
with an information, entertainment, and control or communication system while a vehicle is in motion.
Different calibration tasks are specified in this document to cover calibration manual, visual, and
cognitive aspects of various secondary task characteristics.
© ISO 2019 – All rights reserved v

---------------------- Page: 5 ----------------------
TECHNICAL SPECIFICATION ISO/TS 14198:2019(E)
Road vehicles — Ergonomic aspects of transport
information and control systems — Calibration tasks for
methods which assess driver demand due to the use of in-
vehicle systems
1 Scope
This document provides procedures that can be used as a secondary task in a dual task setting to
determine whether that evaluation setting is standardized and valid for purposes of assessing driver
attentional demand due to the use of an in-vehicle system. This document does not define calibration
procedures for other evaluation activities that a laboratory might undertake.
This document provides guidance on selecting a calibration task given a specific primary task. The
primary tasks of interest include those that would be used in the evaluation of attentional demand.
Such primary tasks are defined in other documents.
The description of a calibration task includes its application, experimental setup, data collection, and
procedures for analysis of results.
The purpose of this document is not to define a reference criterion as to whether a given secondary task
is suitable for use while driving. Although specific settings of parameters of a calibration task might be
used to realize such a predefined pass/fail criterion, this document does not provide such a criterion for
a given level of attentional demand.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 26022:2010, Road vehicles — Ergonomic aspects of transport information and control systems —
Simulated lane change test to assess in-vehicle secondary task demand
ISO 17488, Road vehicles — Transport information and control systems — detection-response-task (DRT)
for assessing attentional effects of cognitive load in driving
ISO/IEC 7498-1, Information technology — Open Systems Interconnection — Basic Reference Model: The
Basic Model
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 7498-1 and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
© ISO 2019 – All rights reserved 1

---------------------- Page: 6 ----------------------
ISO/TS 14198:2019(E)

3.1
calibration task
type of reference task used for the purpose of comparing different tests or test results between sites, or
over time at a given site
3.2
criterion
threshold or value of a variable to be met
3.3
demand
total visual, auditory, cognitive, or physical resources required of the driver to accomplish the primary
task and interact with a Transport Information and Control System (TICS) in a dual task setting
3.4
dual task
two tasks concurrently performed, typically the primary task plus the secondary task
3.5
environment
physical surroundings in which data are captured and collected, consequently, the level of control over
the independent variables in a study
EXAMPLE Laboratory, simulator, test track, real road.
3.6
evaluation
procedure in which the effect of a Transport Information and Control System (TICS) or another device
is assessed
Note 1 to entry: It may be undertaken retrospectively after the TICS has been in use for a considerable time as a
product.
Note 2 to entry: The results of the evaluation will depend on the HMI, but also on the equipment reliability and
subsequent behavioural changes which may affect driving performance.
3.7
method
high-level approach to an assessment, based on theory and principles, which implies an underlying
rationale in the choice of assessment techniques
EXAMPLE Behaviour analysis, workload assessment, and analysis of psycho-physiological responses.
3.8
metric
quantitative measure of driver behaviour independent of the tool used to measure it
EXAMPLE Eye glance duration and vehicle speed.
3.9
performance
result of skill application demonstrated by a participant in performing a driving task or Transport
Information and Control System (TICS) related task
3.10
primary task
task to be calibrated
Note 1 to entry: The Lane Change Test in ISO 26022 is one example a task that simulates or approximates a
primary driving task.
Note 2 to entry: This document considers the Detection Response Task (ISO 17488) is the primary task to be
calibrated even if performed with or without driving.
2 © ISO 2019 – All rights reserved

---------------------- Page: 7 ----------------------
ISO/TS 14198:2019(E)

3.11
primary driving task
activities that the driver must undertake while driving including navigating, path following,
manoeuvring, avoiding obstacles, and controlling speed; and which a participant may perform through
the duration of a test (simulated substitute for driving)
3.12
secondary task
non-driving related additional task
Note 1 to entry: A calibration task for the purpose of this document.
3.13
secondary task demand
sum of visual, auditory, cognitive, motor, and speech resource demands required by a non-driving
related task
3.14
system paced secondary task
activity in which the change from the current to the next state in the interaction between user and
system is initiated by the system
Note 1 to entry: The pace can be fixed or variable.
3.15
target bar
moving line on the critical tracking task display which indicates the task error
3.16
task
process of achieving a specific and measureable goal using a prescribed method
3.17
user paced secondary task
activity in which the change from the current to the next state in the interaction between user and
system is initiated by the user
4 Abbreviated terms
CI Confidence Interval
CTT Critical Tracking Task
DRT Detection Response Task
LCT Lane Change Test
MDEV Mean Deviation (According to ISO 26022)
SURT Surrogate Reference Task
a
TICS Transport Information and Control System
a
A list of TICS fundamental services has been defined by ISO/TC 204/WG 1.
© ISO 2019 – All rights reserved 3

---------------------- Page: 8 ----------------------
ISO/TS 14198:2019(E)

5 Calibration tasks
5.1 Principle and overview
For calibration purposes, a standardized calibration task shall be used as a secondary task in a dual
task setting in combination with a method to assess attentional demand due to the use of an in-vehicle
system. The dual task setting shall include a primary task and the secondary calibration task.
Examples for driving-like dual task settings may include the operation of a TICS secondary task in the
Lane Change Test (LCT), in a driving simulator environment, or the Detection Response Task (DRT).
The calibration shall be performed in a setting that is intended for the assessment of secondary tasks
and follow the training and experimental procedures of that method for assessing attentional demand.
Development of the calibration tasks and associated procedures have used the ISO 26022 LCT, and
ISO 17488 DRT to represent the primary task. While the calibration tasks described herein are intended
to be applicable to other primary task implementations and dual task settings, care shall be taken
to ensure that the conditions are sufficiently similar to those of ISO 26022 or ISO 17488 considering
equipment and instructions to ensure a valid application of this document and its procedures.
5.2 Types of calibration tasks
There are various possibilities to realize a calibration task. In the following subclauses, three example
alternatives are specified in detail. These alternatives include a system-paced secondary task (critical
tracking task, CTT), a user-paced secondary task (surrogate reference task, SURT), and an auditory-
vocal delayed digit recall task (n-back). Both the CTT and SURT alternatives represent visual-manual
tasks that can be used in a dual task setting and are recommended as calibration tasks while using the
LCT as the primary task. Whereas the n-back is a cognitive task that can be used in a dual task setting
and is recommended as a calibration task while using the DRT as the primary task.
5.3 Critical Tracking Task (CTT)
5.3.1 Description
The CTT is a visual-manual task, which requires continuous control activity by the participant.
The participant controls the position of a vertically moving target bar with respect to a fiducial line
(centreline) within a target area by manipulating up and down arrow keys. The arrow key control gives
discrete commands to the target bar which moves it up and down. The up key moves the target bar up,
and the down key moves it down.
The dynamics of the motion of the target bar are a first order instability. If the participant does nothing,
the target bar moves (divergently) towards the edge of the display. The participant then has to make
suitable corrective arrow key inputs to bring the target bar back towards the centre (the red dashed
line in Figure 1).
A control system block diagram of the CTT is shown in Figure 2.
4 © ISO 2019 – All rights reserved

---------------------- Page: 9 ----------------------
ISO/TS 14198:2019(E)

Key
1 target bar
2 centreline
Figure 1 — Typical screen of the CTT with target bar above the centreline
Key
λ Level of instability or rate of divergence (adjustable).
s Laplace transform variable.
Figure 2 — Control system block diagram of the CTT
5.3.2 Operation of the CTT
When the program is first started, the screen will look like Figure 1. Nothing will happen initially.
The centreline is displayed as a red dashed horizontal line in the centre of the display. The target bar
is displayed as a black line. The target bar will start to move away from the centreline showing an
increasing error. Two short blue reference lines are shown above and below the centre to subdivide the
screen for better orientation. The instruction to the participant is to control the position of the target
bar with the arrow keys (arrow up key and arrow down key) and keep it as close to the centreline as
possible so as to minimize the errors.
© ISO 2019 – All rights reserved 5

---------------------- Page: 10 ----------------------
ISO/TS 14198:2019(E)

5.3.3 Setup for CTT
The setup consists of a screen with SVGA resolution plus keyboard should be as follows:
— The subtended vision angle (width) of the display area relative to participant's eyes shall be 13° ± 1°
horizontally.
— 19 inch (483 mm) screen for a width to height ratio of display area of 4:3. For a display with a width
to height ratio of 16:9 (and equivalent to a 19 inch screen), one should use a setup of with a 23 inch
(584 mm) screen.
— The centre of the secondary display shall be positioned 28° ± 2° horizontally (right or left depending
on the intended display position in the vehicle, LHD or RHD) and 20° ± 2° vertically from participant's
straight ahead line of sight. For further details see Annex A.
— To control the target bar movement, the arrow key of a standard PC keyboard (or an equivalent
arrangement of keys, for example Figure 3) shall be used. Participants are allowed to place the
keyboard in a comfortable position on the same side of the steering wheel as the CTT display on a
table or console aside but not connected to their body.
Figure 3 — Example of key-pad to move the highlight and confirm target location result
5.3.4 Test conditions for CTT
The test in the dual task setting shall include one specified level of λ, 0,5, the CTT easy condition. The λ
level shall be set prior to each run and kept constant during the run. In each run the primary task and
the CTT task shall be performed for at least 2 min.
5.3.5 Participants for CTT
The CTT as a calibration task shall be performed in a dual task setting by at least 16 [n = 16] participants
who are licensed drivers. Participants should be familiar with the primary task as well as operation
of the CTT. The level of participant familiarity shall be documented in the protocol. In the case of
calibration using an LCT setting, it is recommended to select the participants according to ISO 26022,
and following the sample description regarding age, gender and familiarity with primary and secondary
[2]
task in Bengler, K., Mattes, S., Hamm, O., Hensel, M., 2010 . Typically, this was an average age of 32 to
45, gender balanced, and instructed and practiced in both the primary and secondary tasks.
5.3.6 Participant instruction for CTT
The following verbal/written instructions for CTT shall be given to the participant:
“A horizontal black target bar is displayed on the secondary task screen. There is also a red dashed
horizontal line in the centre of the screen. When the task is started, the black target bar will tend to
6 © ISO 2019 – All rights reserved

---------------------- Page: 11 ----------------------
ISO/TS 14198:2019(E)

move away from the red centreline. The motion of the black target bar can be controlled by the arrow-
up- and arrow-down keys in order to bring the target bar back to the centre. These keys give discrete
commands to the target. (The down arrow key moves the target bar down; the up arrow key moves the
target bar up.) The goal is to control the motion of the black target bar to keep it as close as possible to
the red dashed centre line.”
The λ level is set to a fixed value at the beginning of a run and defines the secondary task difficulty.
If the participant does nothing, the target bar moves towards the edge of the display. The further the
black target bar is away from the centre line, the faster it will move, so a larger (longer duration) input
is needed, to bring it back to the centreline.
The difficulty of controlling the black target bar can vary with the size of the error. The participant will
need to work quickly and correctly in order to keep the target bar near the centreline.
5.3.7 Practice trials
Participants shall practice the CTT secondary task alone at λ levels of 0,5 and 1,0. Before proceeding to
the dual task conditions, participants shall be able to perform the CTT alone at a level of λ = 1,0 for one
minute successfully (i.e. without the target bar hitting the upper or lower limit of the display). Only the
λ value of 0,5 is used in the dual task trials.
Primary task and dual task practice trials shall be conducted as prescribed in the corresponding
standard or other procedures.
5.3.8 Test metrics
The primary task measures shall be those prescribed for the primary task. If the Lane Change Test is
used, these are in ISO 26022.
Participants’ performance in the CTT is measured using:
— CTT root mean square deviation of the target bar from the centreline in pixels.
— CTT percentage of time with target bar at the upper or lower limit.
5.4 Surrogate Reference Task (SURT)
5.4.1 Description
The Surrogate Reference Task (SURT) is both visually and manually demanding. The level of demand
can be varied.
The SURT is a visual search paradigm. In a typical visual search paradigm, participants are asked to
report whether or not a pre-specified target is embedded in a multi-item display, usually comprised of
an array of alphanumeric symbols, forms, colours or words. The non-target items are called distractors.
To vary the time the participant will need to identify the target and to answer (reaction time), the
similarity between target and distractors can be manipulated. The more similar the distractors are
to the target the more time is typically needed to identify the target. Unless these distractors have
no unique feature distinguishing them from the target (feature with a visual “pop out” character),
the number of distractors can also increase the time participants will need for search (Treisman and
[36]
Gelade, 1980 ). An example for the task is shown in Figure 4.
For the SURT, participants shall locate a pre-specified target circle which is either
...