ISYS3401 IT Evaluation (2023)
Individual Assignment 1 (30%)
This IT Evaluation assignment follows a 3-phase assessment plan introduced in lecture 5. The aim of
this assignment is to develop an integrated multi-phase evaluation plan from the IT consumer (not IT
provider) perspective for the adoption of IT (with a longer-term view) translate into business benefits.
The technology chosen for this assignment is the “use of Smartphone Pulse Oximeter” for remote
health monitoring. There are 3 sequential parts to assess the technology as well as how the technology
is used to achieve desired outcome(s). In this assignment, your tasks include appraisal of evidence
from existing evaluation studies, to develop 3 detailed plans to evaluate the technology and it’s
usability, and anticipated impacts for the business operation.
Plan 1 (Technical Performance): Reviewing evidence from existing evaluation of the
technology (10%).
In this part 1 you would first perform a quick “review” on the technology with these questions in mind:
(1) Description of the Technology, (2) What does this technology set out to do? (3) Metric and
measurements used to evaluate the technical performance of the technology, (4) Your review of
existing evidence; (5) What are the challenges in evaluating the technical performance of the
technology?
Deliverables: Summarise your findings based on the suggested questions above and develop an
evaluation plan to collect primary data to assess suitable performance metrics for the context of use
in remote monitoring.
Plan 2 (Usability focus): Evaluation Plan for the use of Smartphone Pulse Oximetry
device in remote health monitoring (10%).
In this part 2, you are interested in the users’ behavioural responses based on their use experience of
the technology. You may consider the Usability or Acceptance Models as a starting point, then
modify/adjust the instruments for your usability evaluation with these questions in mind: (1)
constructs and metrics you plan to measure? (2) How would usability data be collected? Then develop
an evaluation plan to collect data for both formative and summative assessments.
Deliverables: Develop an UX evaluation plan to produce evidence to inform about the acceptability
of using Smartphone Pulse Oximetry by the potential users. Note: there are many “usability” models
and instruments (some from lectures and readings) for you to consider.
Plan 3 (Impact focus): Evaluation Plan for the Impact from adopting use of Smartphone
Pulse Oximetry device in remote health monitoring (10%).
In this part 3, you are interested in the benefits derived from the integration of technology in the
operations. Your evaluation goal is to look for evidence to inform about the investment decision of
using Smartphone Pulse Oximetry in the remote monitoring operations with these questions in mind:
(1) Considering be the total cost of ownership if using the technology will become business as usual?
(2) Any other non-users could be directly or indirectly affected? (3) What Key Performance Indicators
(KPI) can be used to monitor and improve on over time?
Deliverables: The goal of this evaluation plan is to inform business executives about the anticipated
benefits realisation. You may consider suitable process, organisational and business indicators. Hence
KPI used for data collection could be reflective of efficiency, errors reduction, revenue improvement
and competitive advantage, etc…
Submission Details:
1. Assignment description to be made available on Canvas before 24th March. You have 3 weeks to
complete this assignment.
2. This assignment is done individually.
3. Submit the following files to Canvas before 11:59pm Sunday, 16th April 2023 (end of mid-
semester break)
4. A written evaluation proposal should be written in word document (up to 2000 words) with 3
clear sections addressing each part. Suggested structure of each part should contain:
Summarise and any relevant background and previous evidence as the basis of your
evaluation plan.
Elaborate on your motivation and objectives behind the evaluation. State the purpose of
your evaluation plan, what would you like to measure? Why?
Discuss and articulate the variables used for each construct in your model, including a
question instruments you plan to use for data collection. Justification might be needed.
Detail your data collection plan including methods to collect data.
In conclusion of each part, it is important to re-iterate your evaluation goal(s) as well as what
you have excluded in the plan (things you do not plan to measure).
References: make sure professionally done
5. Useful References:
Study finds smartphones can double as pulse oximeters (2022-09-12):
https://digitalhealth.modernhealthcare.com/technology/study-finds-smartphones-can-
double-pulse-oximeters
Smartphone camera oximetry in an induced hypoxemia study:
https://www.nature.com/articles/s41746-022-00665-y
Patient and clinician use characteristics and perceptions of pulse oximeter use: A scoping
review: https://www.sciencedirect.com/science/article/pii/S1386505622000491
Several case studies can be found in Chapter 10 of the Measuring the User Experience book
could be of value.
A Canvas Discussion Board will be setup for sharing concerns and issues related to the
assignment.