MECH5495M Home Assignment v2, 2020-03-30
MECH5495M Engineering Psychology Human Factors
Home Assignment (40% of the total module mark)
General instructions
Please submit via Minerva no later than 12.00 noon Wednesday May 13, 2020. See the Deadlines
and Feedback section of the MechEng Minerva organisation (link) for general submission guidance.
The maximum mark on the home assignment is 100. For full marks, all four tasks need to be carried
out. The estimated total workload is about 40 hours (but note that if you do not have a good prior
grasp of MATLAB, especially tasks 1(e), 4(f), and 4(h) may not be feasible within this time.)
Do not plagiarise! It will be detected and will have severe consequences for you. If you are copying
text or code from another student, even with the intent to completely edit it, you are plagiarising.
Your submission should consist of:
A single report with your answers for the four tasks, in a format compatible with Turnitin (e.g.,
Word .docx, Acrobat .pdf), using your student ID as the “title” of your Turnitin submission.
o Use the cover sheet available via the Minerva link above. Since the submission is
electronic, a date stamp is not needed, and you can sign with just “I confirm the above”.
o On the first page of your report after the cover sheet, clearly indicate the total word count,
excluding cover page and reference list (if you have one), but including everything else.
o Your answers are marked on content, not length. The maximum total word count is
3500. Any parts of the report extending beyond this word count limit will not be marked.
MATLAB files as requested in Task 1 and 4, with understandable variable names and code
comments, provided in a single .zip file using your student ID as file name:
o For Task 1: do_1_CalculateMetricValues.m, MetricValues.mat, and your files for task 1(e).
o For Task 4: SimulateRearEndCollisionScenario.m.
For some further guidance on MATLAB, including variable naming conventions, see the Appendix.
Task 1: Analysing pilot control data [20 marks]
(The data for this task were kindly provided by Dr Andreas Haslbeck,
from a study with Lufthansa First Officers in a flight simulator at the
Technical University of Munich.)
Let us assume that you are part of a team investigating the manual
flying behaviour of pilots, as an input to further development of a fly-
by-wire system. As a first step you run a small flight simulator
experiment with eleven pilots, and record their use of the side-stick
control, in both pitch (forward/backward) and roll (left/right) directions
during an approach to landing, from an initial altitude of about 4500
feet above ground. You want to study the behaviour both in an early
phase, between 4500 and 1500 feet, and a late phase, from 1500 to
50 feet. Based on literature and your own past experience, you
hypothesise that pilots will be more active in their side-stick input in
the late phase (i.e., at lower altitudes) than in the early phase (i.e., at high altitudes).
The eleven recordings are provided in the file PilotData.mat, which can be loaded into MATLAB
using the load command. This loads a structure SPilotData into the MATLAB workspace, and
An Airbus side-stick.
(https://commons.wikimedia.o
rg/wiki/File:Airbus_A380_coc
kpit_zoom_on_sidestick.JPG)
MECH5495M Home Assignment v2, 2020-03-30
2
SPilotData(1) provides the data signals recorded for the first pilot, and so on. For each pilot, there
are data vectors holding time stamps in seconds, the altitude in feet, and side-stick deflection in
pitch and roll directions, with ±1 indicating full deflection and 0 indicating no side-stick input.
(a) Inspect the recorded data by plotting the various signals as a function of time. (For example,
plot(SPilotData(1).VTimeStamp_s, SPilotData(1).VAltitude_ft) will plot the altitude signal for the
first pilot as a function of time.) Looking at the data, do you see any indications that there may be
support for the hypothesis suggested above? Provide one or two example plots supporting your
answer. [3 marks]
Let us denote the pitch and roll deflections of the side-stick by and respectively, and let us
assume that you decide to test two different metrics quantifying how active the pilots are in their
control: (1) , the fraction (i.e., a value between 0 and 1) of time during an approach phase
when the pilot provided non-zero input, defined as at least one of || or || exceeding 0.01 (1% of
full side-stick deflection). (2) , the average, during a phase of the approach, of the combined
side-stick deflection = √2 + 2.
The provided MATLAB script do_1_CalculateMetricValues.m loads the data, loops through all of the
recordings, identifies the early and late phases, calculates metric values, stores them in two
matrices and saves these in a file MetricValues.mat. The metric is already implemented, but
not the metric.
(b) Implement the calculation of in do_1_CalculateMetricValues.m, and provide, as part of
your submission, your version of this file as well as the generated MetricValues.mat. [4 marks]
The provided MATLAB script do_2_StatisticalAnalysis.m loads the MetricValues.mat file, generates
overview plots of the metric data, applies Wilcoxon’s signed-rank test (testing the null hypothesis
that the early and late phase data have the same median), and calculates effect size (Cohen’s d).
(c) Run do_2_StatisticalAnalysis.m, and provide in your report the generated figure (with box
plots and statistical analysis results), together with a discussion of the obtained results. Is there
support for the hypothesis that pilots are more active in their control in the later phase of the
landing approach? Motivate your answer, and discuss why you think pilots behave the way
suggested by the results, ideally with reference to some concepts you have learned in this
module. [5 marks]
(d) In general, if a statistical test indicates, with statistical significance, that the pilots are more
active in the later phase, does this mean that all pilots are more active in the later phase? Justify
your answer. [2 marks]
(e) Write MATLAB code to carry out one further piece of analysis of this dataset. This can be
anything you find interesting or useful, such as a different way of visualising the data, or
calculation of some other objective metric (e.g., how common is it that pilots move the side-stick
in just one of the two directions at a time, versus applying both pitch and roll inputs at the same
time?), or a different way of structuring the analysis (e.g., divide the approach into more than just
two phases, or look separately at individual pilots to see if they differ in their overall control
strategies, …), or even further statistical testing, if you know what you are doing. If you want to,
you can consult for example (Haslbeck et al., 2016) or (Haslbeck et al., 2018) for inspiration. In
your report, explain what question you are trying to answer with your analysis, describe what you
have done, provide one or more figures illustrating your results, and provide some text briefly
discussing what you have found. Also provide your MATLAB script file(s) with your submission.
[6 marks]
MECH5495M Home Assignment v2, 2020-03-30
3
Task 2: Evaluating external HMI design for automated vehicles [20 marks]
In the most advanced levels of road vehicle automation, it is envisioned that human occupants will
not at all need to monitor the driving, or that there may not even need to be a human in the vehicle
(SAE Level 4 or 5; Society of Automotive Engineers International, 2018). One concern with respect
to such automated vehicles (AVs), is how to make them capable of interacting effectively with
human road users, for example pedestrians. In current traffic, a human driver may for example
make eye contact with a pedestrian that is waiting to cross the road, apply deceleration and maybe
signal with a hand gesture or by flashing the headlights that the pedestrian can cross.
An illustration of pedestrians crossing the road
(Source: https://www.highwaycodeuk.co.uk/rules-for-pedestrians-crossing-the-road.html)
(a) Imagine a pedestrian who is wanting to cross a road (like in the photo above), and who is
looking at an approaching, human-driven vehicle, trying to determine whether or not the driver is
yielding to let him or her cross. Using the information processing model of cognition, and any
other concepts from the module you find useful, describe roughly what you think may be
happening in the pedestrian’s brain while he or she is making the decision to cross or not in front
of the vehicle. (Note: There is no clear right or wrong answer here; what is important is that you
are able to use the module’s taught concepts in a reasonable way.) [3 marks]
In various studies, participants have reported that putting external displays on the outsides of
vehicles “would be helpful when autonomous vehicles become available” (Clamann et al., 2017).
Therefore, researchers are now trying to design external human-machine interfaces (eHMIs), to
enable AVs to communicate with other road users. Two potential designs are proposed below.
Design 1 suggests that a green light will be presented to indicate that the AV is accelerating and a
red light will be presented to indicate that the AV is decelerating.
Design 1
(AV is accelerating) (AV is decelerating)
MECH5495M Home Assignment v2, 2020-03-30
4
(b) Describe and explain two different ways in which Design 1 may be problematic from an
information processing point of view. You should consider at least two of the following aspects:
perception; working memory; long-term memory. [5 marks]
(c) Consider the perceptual and memory principles of display design, as described in the course
book. Choose one of these principles, describe the principle, and explain how Design 1 is not
properly taking the principle into account. [2 marks]
Design 2 instead shows a text message ‘Please Cross’ to indicate that the pedestrian is safe to
cross, and ‘Please Do Not Cross’ to indicate that it is not safe for the pedestrian to cross.
Design 2
(It is safe to cross in front of the AV) (It is not safe to cross in front of the AV)
(d) Describe and explain two different ways in which Design 2 may be problematic from an
information processing point of view. You should consider both of the following aspects: bottom-
up processing and top-down processing. [5 marks]
(e) Consider the perceptual principles of display design, as described in the course book.
Choose one of these principles, describe the principle, and explain how Design 2 is not properly
taking the principle into account. [2 marks]
(f) Taking into the account the principles of display design you have considered in (c) and (e)
above, suggest and briefly describe an eHMI that you think may be better than Design 1 and 2.
You can provide a graphical illustration of your suggested eHMI if you want, but it is also ok to
just describe it in text. [3 marks]
MECH5495M Home Assignment v2, 2020-03-30
5
Task 3: Human Factors evaluation of a Tesla Autopilot accident [20 marks]
On May 7, 2016, a 2015 Tesla Model S collided with a tractor trailer crossing an uncontrolled
intersection on a highway west of Williston, Florida, resulting in fatal injuries to the Tesla driver. Data
obtained from the Model S indicated that: 1) the Tesla was being operated in Autopilot mode (SAE
Level 2) at the time of the collision; 2) the Automatic Emergency Braking (AEB) system did not
provide any warning or automated braking for the collision event; 3) the driver took no braking,
steering or other actions to avoid the collision; and 4) the last recorded driver action was increasing
the cruise control set speed to 74 miles/h = 119 km/h about two minutes prior to impact. The crash
occurred on a clear day with dry road conditions. On June 21, 2016, NHTSA deployed a Special
Crash Investigations team to the crash site to evaluate the vehicle and study the crash environment.
NHTSA’s crash reconstruction concluded that the Tesla vehicle’s sensors did not detect the tractor
trailer, but that the tractor trailer should have been visible to the Tesla driver for at least seven
seconds prior to impact.
(a) Drawing on your knowledge of the levels of automation, describe the expected roles of the
vehicle and driver while the Tesla autopilot system was engaged. [2 marks]
(b) Leaving aside the technical failings of the Autopilot system, there are a number of “Problems
with Automation”, from a human-factors perspective. Drawing on your knowledge of these, select
and describe two that you think best account for why the above accident occurred. [4 marks]
(c) Make a fault tree analysis of this accident, showing how you think technical errors and human
errors may have contributed to causing this crash. You will probably want to use at least five
(probably more) events besides the crash itself, and at least one or two AND or OR gates. You
do not need to write text in addition to the fault tree, as long as you make sure to label each
event in the tree with enough detail to make it easily understandable. [4 marks]
(d) Discuss how your fault tree analysis maps on to the different parts of either the accident
causation model in Fig 16.1 of the main course book by Lee et al., or the accident causation
model in Fig 1 of Chapter 38 in the alternative course book edited by Salvendy. Please state
clearly which accident causation model you are using. [4 marks]
(e) Drawing on your fault tree analysis and the principles of human-centred automation, discuss
how you would design the Autopilot system to avoid the two “Problems of Automation” you listed
above as possibly contributing to the accident. [6 marks]
MECH5495M Home Assignment v2, 2020-03-30
6
Task 4: Modelling driver responses during automation failures [40 marks]
Let us assume that you are part of a team
developing automated vehicle functionality,
and that you are tasked with developing a
driver model that can help you better
understand situations such as the Tesla
accident described in Task 3 above. As a
starting point, you make use of a model by
Markkula et al (2016), describing driver
braking, as observed in real critical rear-
end collision situations with manually driven
vehicles. As illustrated to the right, this
model’s behaviour is dependent on the
urgency of the situation, quantified in terms
of the time left to collision, assuming
constant velocities of both involved vehicles
(i.e., assuming the driver does not take any
action). 0 is the initial time to collision, at
the time = 0 when the collision threat
becomes physically visible to the driver, e.g., 0 = 7 s for the Tesla crash described in Task 3.
The model suggests that in critical rear-end situations, drivers respond with deceleration as a
piecewise linear function (the blue curve above), with the deceleration onset time obtained as:
= 0.2 + ( − 0.2), (1)
where 0.2 is the first time at which the driver sees a of 5 seconds or less, is the time at which
collision occurs if the driver maintains constant speed, and is a random value, drawn from one of
two distributions as shown below. The “eyes-on-threat” normal distribution is used if 0 > 5 s (the
type of situation shown in the plots above), and the “eyes-off-threat” lognormal distribution is used if
0 ≤ 5 s. Note that in this latter case, 0.2 = 0 by definition. Also note that in both types of situations,
a randomly drawn ≥ 1 means that the model does not apply any deceleration before crashing.
The black curves show probability distributions for the Markkula et al (2016) braking model, with
values for means (m) and standard deviations (s). The blue histograms show observed human data.
The other two random variables in the figure above are , the final acceleration level applied by the
model, and , which is a scale factor for the rate of change of acceleration rate , obtained as:
= ⋅
−1, (2)
where is at the time of brake onset.
MECH5495M Home Assignment v2, 2020-03-30
7
(a) Explain in your own words (i.e., not just restating the above) how the Markkula et al (2016)
model describes driver braking in critical rear-end situations, including how the urgency of the
situation affects the braking. [4 marks]
The provided MATLAB script do_SimulateRearEndCollisionScenario.m calls a function
SimulateRearEndCollisionScenario, which implements and simulates this driver braking model for a
collision obstacle with zero speed in the forward road direction. As provided, the script runs the
scenario once, configured as a reconstruction of the Tesla crash, with a 12 m wide obstacle
(representing the tractor trailer), initial speed 0 = 119 km/h, and 0 = 7 s. Each time you run
do_SimulateRearEndCollisionScenario.m, you will get a figure window showing a bird’s eye view of
the road with the obstacle at longitudinal position = 0, and trajectory and final position for the
modelled car, blue or red for non-collisions and collisions respectively. The figure window also
shows time series plots of a number of useful quantities. Due to the probabilistic nature of the
model, each time you run it you will get a different result. You can also run many simulations at a
time, by changing the setting of the variable c_nMonteCarloSimulations in the script.
(b) Recall that the human driver in the real Tesla crash described in Task 3 did not respond at all
before the crash. How common is this type of behaviour among drivers of manually driven
vehicles in critical rear-end scenarios, according to the Markkula et al (2016) model? Justify your
answer mathematically and/or with simulation results. [4 marks]
(c) Based on your result in (b), would you say that the Tesla driver’s behaviour is likely to be
representative of “drivers of manually driven vehicles in critical rear-end scenarios”, as described
by the Markkula et al (2016) model? Justify your answer, and briefly discuss possible
explanations for this finding (it is ok to refer back to answers you provided in Task 3). [4 marks]
Obstacles at zero speed along the road’s forward direction are difficult to reliably detect with current
automotive sensors, and many on-market automated driving systems therefore come with
disclaimers saying that that the system may not respond at all if, for example, a vehicle that you are
following in automated mode changes lanes to reveal a stationary obstacle further ahead.
Let us assume that you are tasked with using the driver model to provide rough, simulation-based
predictions about this type of “reveal” scenario. You wish to investigate whether serious accidents
can be avoided by ensuring that the automated vehicle always keeps a long enough time gap
to the lead vehicle. As a first step, you assume that the driver monitoring the automation is
reasonably attentive, and that their braking behaviour can therefore be approximated by the
Markkula et al (2016) model. You wish to study a scenario at 70 mph = 113 km/h, where the lead
vehicle changes lanes to reveal a stationary obstacle of width 2 m, just 0.5 s before reaching it,
such that 0 = + 0.5 s.
(d) Modify the parameter settings in do_SimulateRearEndCollisionScenario.m to run many
simulations of this “reveal” scenario. Please note that you do not need to include the original,
lane-changing lead vehicle in any way, since it is assumed to already have changed lanes when
the simulation starts. A time gap setting that would be considered relatively typical for manual
driving is = 1.5 s. With this time gap, what is the model’s predicted frequency of collisions in
general, and of collisions at more than 60 km/h speed? [3 marks]
Let us assume that your design goal is to keep the risk of > 60 km/h collisions lower than 25%, but
that you also want to keep ≤ 3 s, because drivers are unlikely to accept larger settings.
(e) According to the model, is it possible to achieve this design goal? As part of your answer,
provide a plot showing the frequency of collisions at more than 60 km/h as a function of
MECH5495M Home Assignment v2, 2020-03-30
8
between 1 s and 5 s. (Tip: the function SimulateRearEndCollisionScenario returns number of
collisions at > 60 km/h as an output nHighSpeedCrashes, so if you want to you can modify
do_SimulateRearEndCollisionScenario.m to loop through different time gap settings
programmatically, instead of going through them manually.) [5 marks]
Next, you wish to model also the possibility of steering avoidance (to the left, in this right-hand
driving scenario). Maybe shorter time gap settings are possible when the adjacent lane is free? Let
us assume that you have data indicating that avoidance steering can be modelled independently
from the braking, using the same type of piecewise linear function as the braking model, to describe
the vehicle’s curvature (roughly proportional to steering wheel angle) over time, with parameters:
– eyes-on-road:
m = 0.88; s = 0.87
– eyes-off-road:
m = 1.27; s = 0.77
k:
m = 0.031; s = 0.042
A:
m = 0.010; s = 0.0025
The function SimulateRearEndCollisionScenario is prepared for handling such a model, which will
be activated when the input parameter c_SModelParameters.bSimulateAvoidanceSteering is set to
true, but the model itself is not implemented in the provided version of the file.
(f) Implement, in SimulateRearEndCollisionScenario.m, the avoidance steering model, and
provide your edited file with your submission. Leave the braking model untouched, and note that
you will need to copy and adapt code from the braking model in two places: to set up a structure
c_SSteeringModelParameters (or similar name), e.g, somewhere around line 48, and to add a
new call to the function PiecewiseLinearAvoidanceModel, at line 81. (Line numbers will obviously
change as you edit the originally provided function.) Provide, in your report, a figure showing
results for a set of simulations at some . Does the presence of steering avoidance change
the conclusion you made in (e) with respect to your design goal? [7 marks]
Finally, you also want to consider the “stabilisation” steering that
happens after steering avoidance. Maybe drivers are likely to lose
control of the vehicle at this stage? Let us assume that you have data
indicating that the model proposed by Reñski (1998) acceptably
reproduces stabilisation steering in this scenario:
() = ⋅ ( − ),
where is a control gain, is a control delay, and () is the
momentary angle between the vehicle’s heading and a sight point in
the middle of the target lane, a preview time further down the road. Again, the function
SimulateRearEndCollisionScenario is prepared for handling such a model, activated using
c_SModelParameters.bSimulateStabilisationSteering, but the model is not implemented.
(g) Drawing on what you have learned in this module about control-theoretic models of human
behaviour, explain how the Reñski (1998) model describes driver steering. [4 marks]
(h) Implement the stabilisation steering model in SimulateRearEndCollisionScenario.m, adding
your code from line 117 (line numbering of originally provided file), using = 0.1, = 0.2 s (tip:
exactly two of the 0.1 s simulation time steps), and =1.5 s. You will need to understand and
use (at least) the variables iTimeStep, VY, VLongitudinalSpeed, VHeading, and
c_SScenarioParameters.laneWidth. Include a code excerpt showing your implementation in your
report, and explain clearly in writing what each line of your code is doing. Also, provide a figure in
your report showing simulation results at some . Assuming that this model is reliable, does
stabilisation steering seem to be a concern in this scenario? Justify your answer. [9 marks]
MECH5495M Home Assignment v2, 2020-03-30
9
References
Clamann, M., Aubert, M., Cummings, M. L. (2017). Evaluation of vehicle-to-pedestrian
communication displays for autonomous vehicles. Proceedings of the 96th TRB Annual Meeting.
Haslbeck, A., Hoermann, H. J. (2016). Flying the needles: flight deck automation erodes fine-
motor flying skills among airline pilots. Human factors, 58(4), 533-545.
Haslbeck, A., Hoermann, H. J., Gontar, P. (2018). Stirring the pot: comparing stick input patterns
and flight-path control strategies in airline pilots. The International Journal of Aerospace
Psychology, 28(1-2), 15-30.
Markkula, G., Engström, J., Lodin, J., Bärgman, J., Victor, T. (2016). A farewell to brake reaction
times? Kinematics-dependent brake response in naturalistic rear-end emergencies. Accident
Analysis Prevention, 95, 209-226.
Reñski, A. (1998). The driver model and identification of its parameters. SAE Technical Paper No.
980011.
Society of Automotive Engineers International (2016). Taxonomy and definitions for terms related to
driving automation systems for on-road motor vehicles. Standard J3016_201806.
Appendix: Further guidance on MATLAB
Advice for beginners
If you need help getting started with MATLAB, there are lots of good resources for beginners online,
not least MathWorks own “Getting Started” guide, which is available here, or by entering the
command doc matlab at the MATLAB command prompt. This module’s own MATLAB skills self-
assessment test can also help you get started.
Naming convention for variables in the provided scripts
MATLAB is a “weakly typed” programming language, i.e., it does not require declaring variables
before use, and any variable can hold data of any type. This can make coding faster, but also
increases the risk of generating code that is incorrect or hard to understand. To mitigate against
those risks, the code supplied with this home assignment uses some simple naming conventions:
Use of prefixes to indicate type, e.g. iTimeStep:
o i : integer
o n : integer used to count something
o b : Boolean
o s : character string
o No prefix indicates floating point numbers
Use of additional prefixes to indicate data structures; can be combined with the prefixes
above, e.g. VbValidTimeSteps for a vector of Booleans:
o V : vector
o M : matrix
o C : cell array
o S : structure
Use of the additional prefix c_ to indicate a constant variable, i.e., one that is not intended to
change during program execution.
You are welcome to adopt these conventions in your solutions if you want to, but it is not required.