ISO 39003:2023

DRAFT INTERNATIONAL STANDARD
ISO/DIS 39003
ISO/TC 241 Secretariat: SIS
Voting begins on: Voting terminates on:
2022-07-11 2022-10-03
Road Traffic Safety (RTS) — Guidance on ethical
considerations relating to safety for autonomous vehicles
ICS: 03.220.20
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
This document is circulated as received from the committee secretariat.
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/DIS 39003:2022(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. © ISO 2022

---------------------- Page: 1 ----------------------
ISO/DIS 39003:2022(E)
DRAFT INTERNATIONAL STANDARD
ISO/DIS 39003
ISO/TC 241 Secretariat: SIS
Voting begins on: Voting terminates on:

Road Traffic Safety (RTS) — Guidance on ethical
considerations relating to safety for autonomous vehicles
ICS: 03.220.20
COPYRIGHT PROTECTED DOCUMENT
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
© ISO 2022
THEREFORE SUBJECT TO CHANGE AND MAY
This document is circulated as received from the committee secretariat.
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
NOT BE REFERRED TO AS AN INTERNATIONAL
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on STANDARD UNTIL PUBLISHED AS SUCH.
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
IN ADDITION TO THEIR EVALUATION AS
or ISO’s member body in the country of the requester. BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
ISO copyright office
USER PURPOSES, DRAFT INTERNATIONAL
CP 401 • Ch. de Blandonnet 8
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
CH-1214 Vernier, Geneva
POTENTIAL TO BECOME STANDARDS TO
Phone: +41 22 749 01 11
WHICH REFERENCE MAY BE MADE IN
Reference number
Email: copyright@iso.org
NATIONAL REGULATIONS.
Website: www.iso.org ISO/DIS 39003:2022(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
Published in Switzerland
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
ii
  © ISO 2022 – All rights reserved
PROVIDE SUPPORTING DOCUMENTATION. © ISO 2022

---------------------- Page: 2 ----------------------
ISO/DIS 39003:2022(E)
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 External factors affecting autonomous vehicle safety . 2
4.1 General . 2
4.2 The road environment . 2
5 Stakeholders in AV design and operations . 3
5.1 Producers – Manufacturers; designers and their suppliers . 3
5.2 Distribution chain – Distributors; Sellers . 3
5.3 Purchasers, owners, and operators . 4
5.4 Government agencies and interested parties . 4
6 Governance, assessment, and evaluation . 4
6.1 Ethical reference for assessment . 4
6.2 Additional Standards . 4
6.3 General . 5
6.3.1 Higher organizational level . 5
6.3.2 Development Organizational level. 5
6.3.3 Specific Development and implementation processes. 6
6.3.4 Post Implementation Checking Against Ethical Criteria . 7
6.4 Conducting the assessment . 7
6.5 Expression of results and conclusion . 8
7 Operationalization of Ethics - Discussion on values and ethics to consider .8
7.1 Ethical Framework for the design of AVs (driving rules and ethical design) . 8
7.1.1 Purpose . 9
7.1.2 Values . 10
7.1.3 Principles . 10
7.1.4 Methods for Construction and Evaluation of Maxims .13
7.2 The Background of Maxims .13
7.2.1 Maxim Design and Construction . 13
7.2.2 Evaluations of Maxims . 14
7.3 Driving Rules . 15
7.3.1 Need vs. desire Driving Rule . 16
7.3.2 Once on the road space . 16
7.3.3 In the lane behaviour (includes braking). 17
7.3.4 Lane switching . 17
7.3.5 In the presence of the other . 18
7.3.6 Road/Infrastructure Use Cases . 19
7.3.7 Resolving Conflict .20
7.3.8 Negotiations . 20
7.3.9 AV unable to function as intended . 21
7.3.10 Yielding to first responders and emergency response vehicles . 21
7.3.11 Protecting other road users . 22
7.3.12 Unavoidable collision with other road users . 23
7.3.13 Other Issues . 24
8 A Framework for rule construction and dealing with violations and deviations .24
8.1 Framework . 24
8.2 Goals . 25
8.3 Primary rules . 25
8.4 Supporting Rules . 26
iii
© ISO 2022 – All rights reserved

---------------------- Page: 3 ----------------------
ISO/DIS 39003:2022(E)
8.5 Precautionary and Disabling Rules (prevention) . 26
8.6 Reinforcing and Enabling Rules (performance) . 26
8.7 Counter rules . 26
8.8 Exceptions (prevention and performance) . 26
8.9 Discretionary and Compensatory rules . 27
8.10 Misconduct . 27
8.11 Violation . 27
8.12 Deviations . 27
8.13 Breakdowns .28
8.14 RULE STRATEGY .28
8.15 Boundaries (for prevention of unwanted behavior) .28
8.15.1 Margins .28
8.15.2 Barriers .28
8.15.3 Buffers .28
8.16 Promoters (for performance) .29
8.17 Further considerations . 29
9 External/Internal Design .29
10 Sustainability .30
11 Review and Re-evaluation following controls system updates .31
Annex A .32
Annex B Sustainability Issues .34
Annex C .35
Annex D Action plan – an example .41
Bibliography .43
iv
  © ISO 2022 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/DIS 39003:2022(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 241, Road traffic safety management
systems.
A list of all parts in the ISO 39000 series can be found on the ISO website.
v
© ISO 2022 – All rights reserved

---------------------- Page: 5 ----------------------
ISO/DIS 39003:2022(E)
Introduction
0.1  General
A long established and commonly held view is that the single most significant factor in road traffic
safety are the actions of the driver. It must however be emphasized that the road transport system is
a complex socio-technical system which places high demands on humans to negotiate. Crashes occur
since human beings due to finite cognitive capacity and physiological limitations cannot always cope
with these demands. To increase road safety the road transport system therefore has to be designed
to support the road user to cope with this complexity and to mitigate the effects of crashes when this
is not possible. When technology exceeds human capability autonomous vehicles have the potential to
replace the human driver and to further increase road safety
Autonomous vehicles have the potential to replace the human driver and to increase road safety by
reducing the opportunity of poor decision making and improper conduct in daily operations. This will
probably take a long time and in the meantime the capabilities of technology and humans should be
combined and integrated in such a way that the strengths of both are utilized efficiently. Important
safety improvements were made over the last century, but AD technology provides new opportunities
to improve safety even further.
Autonomous vehicles (AV) are unlikely to gain widespread acceptance until the travelling public feels
assured of their safety and security, not only of passengers but also other vehicles and vulnerable
road users. This includes a behaviour of the AV according to the desires and requirements of society.
However, despite contemporary expectations and a very optimistic view on technology, humans have
an extremely valuable faculty that machines will probably never possess -- ethical decision making and
judgement. For real traffic situations the human does not always have the prerequisites to take a rational
decision based on ethics since the time frame is often too narrow. For that reason, many “decisions”
are made instinctively without the possibility of making a well-reasoned and balanced ethical decision.
Defining this element and imparting it on machines is critical for the success of autonomous vehicles.
This can only be achieved by ensuring that autonomous vehicles are equipped with driving rules that
align with the general ethical beliefs, needs, and desires of humanity on a global level, subject to local
specific nuances.
To achieve the imprint of global and local ethical considerations in AV design, there is a need for a
framework of ethics involving the necessary stakeholders of different areas. To that objective’s end,
it is important to develop standards for AV behaviour. The objective is to ensure that these vehicles
are designed to minimize risk and behave in all circumstances in a way that is aligned with universal
expectations of a fair society. While there are few standards available, or under development, that
address the engineering and technological aspects of autonomous vehicles, there are no international
standards that address aspects concerning the general topics of driving policy and ethical behaviour,
which are also important. By driving policy we mean a general approach of how an AV makes a decision
and performs manoeuvres. Ethical relevant behaviour represents positive or potentially negative
impact on road users and especially the vulnerable ones as well as the public space at large.
The objective of this work is to lay out a framework for the development of a standard for ethical and
societally accepted driving policy (e.g., those relating to fairness and equity with respect to other road
users, the public space, and the environment).
0.2  The Concept of Autonomous Vehicles
Autonomous vehicles have the objective to substitute driver, including tasks, decisions, and
responsibilities. Hence the driver behaviour model proposed by Michon (1985) and applied to
autonomous vehicles by Ulbrich and colleagues (2013) (Figure 1) can be a start to design and operate
autonomous vehicles. The driving task consists of three levels: the strategic level concerned with the
higher-level trip goals (e.g., route choice), the tactical level concerned with the maneuvering decisions,
including negotiations and interactions with other vehicles, and operational level concerned with the
execution of these tactical and operational behaviours at the level of vehicle control. There is no strictly
hierarchical relationship among these levels.
vi
  © ISO 2022 – All rights reserved

---------------------- Page: 6 ----------------------
ISO/DIS 39003:2022(E)
Figure 1 — Hierarchical model of driving task (source Ulbrich et al., 2013)
Beside this consideration which has a timely sequence, the autonomous vehicle must be designed
thoroughly to ensure safe and secure operation, the decision base for the desired behavior need to be
established. Therefore, all necessary information needs to be available, situation needs to be captured
and understood while different perspectives will support the final decision.
Hence a functional decomposition into six layers (Amersbach, Winner, 2017) (Figure 2) can provide the
first step into solution space.
vii
© ISO 2022 – All rights reserved

---------------------- Page: 7 ----------------------
ISO/DIS 39003:2022(E)
Figure 2 — Decomposition Layers (source Amersbach, Winner, 2017)
— Information access
Information needs to be available and accessible
— Information reception
All necessary information can be captured
— Information processing
Captured information need to contain all required classification and identification for further
process
— Situational understanding
Based on information, situation need to be captured and understood
— Behavioral decision
Based on designed or trained situational awareness, the desired behavior needs to be chosen
— Action
Vehicle transforms whished behavior into action
Within this framework, technical realization can differ while the focus of the situation decision will
be comprehensible, accountable, and comparable among different designs. A thoroughly development
will benefit transparency for action and taken decision. The work described in this standard focuses
primarily on Layer 4 – behavioral decision
0.3  The Concept of Ethical considerations for autonomous vehicles
viii
  © ISO 2022 – All rights reserved

---------------------- Page: 8 ----------------------
ISO/DIS 39003:2022(E)
Philosophy helps people question, understand and make sense of the world so they can act properly in
it. This means that decisions and actions are not only intrinsically fair, but that they are also performed
in a way that is balanced with respect with other’s needs, the societal needs and humanistic values,
as well as with respect to the physical world around us. It is widely accepted that philosophy can be
divided into theoretical philosophy and practical philosophy. Theoretical philosophy is concerned with
understanding and making sense of the world and includes topics such as ontology, epistemology, and
meta-ethics amongst others. Practical philosophy focuses on topics such as human agency, practices,
values, as well as ethical dimensions of (human) behaviour and conduct.
Within practical philosophy there are two main branches: Normative and applied ethics. Normative
ethics is the study of ethical action and determining standards for decision making and conduct (e.g.
deontology, virtue ethics, etc. - see Annex B). Applied ethics is the application of standards of ethics to
real life situations (e.g. biomedical ethics, AI ethics, political ethics, etc. - see Annex B). In this document,
ethics refers to both normative and applied ethics but not to meta-ethics (which is more concerned
with the nature and basis of ethical concepts and the underlying reasoning and assumptions behind
moral theories).
Applied ethics is a flexible and practical way to address ethical considerations in the development of
new technologies because an applied ethics field, such as AV ethics, can borrow from more than one
normative school of ethics. Therefore, the framework offered in this document, although based on “a
principle-based approach, can use ideas from deontology and virtue ethics, for example, to help solve
problems. This is a balanced approach; it does not condone one type of normative ethics over another,
it offers a range of perspectives that will help the designer/developer in choosing the best (or better)
decision possible for specific situations.
In summary, ethics is the study of how to choose to act in situations. To make decisions, we need clarity
on what matters and to understand what is considered good, bad, right, and wrong. Therefore, ethics
may be viewed as a tool that helps us create the difference between a “good” decision and a “bad” one.
This is of great importance in the development of autonomous vehicles because choices made during
the design and development of AV systems determine its “driving behaviour” and how it caters for its
passengers and interacts with other road users (e.g., vulnerable road users). “Driving behaviour” is
what was designed for and programmed into the machine, “conduct” is what actually transpires as a
consequence of applying the driving behaviour to the real world.
This document offers a particular framework for AV ethics which is intended to support the practical
integration of ethics into the AV development process. The framework we suggest builds upon
Socrates conception of the hierarchical nature of philosophy for practical use which has three levels
(See Figure 3). The base level is the conduct that transpires as a consequence of decision and actions
(behaviour), the second level is how these behaviours are governed. Namely what kind of policy and
arrangements are in place to make decisions. The top level is Purpose, which Behaviour and Conduct
must align with, in order to achieve the goals and objectives of the entire endeavour. Finally, knowledge
is the “tool” for making sure that the purpose is reasonable and balanced and that the governance and
behaviour/conduct are indeed feasible and appropriate.
Figure 3 — Socrates’ hierarchical nature of philosophy
ix
© ISO 2022 – All rights reserved

---------------------- Page: 9 ----------------------
ISO/DIS 39003:2022(E)
Figure 4 is a graphical depiction of the framework advocated here for AV ethics. It shares similarities
with other ethical approaches for emerging technologies, it is also unique as it has been created
specifically for the AV context. Essentially, the AV ethics framework provides guidance for reflective
and critical decision making and is composed of four main elements: purpose, values, principles,
and process. The framework approach provides structure and guidance, yet it is flexible enough to
a
...