Smarter Solutions MCA Tool Note February 2017
Smarter Solutions MCA Tool Note February 2017
Smarter Solutions MCA Tool Note February 2017
http://creativecommons.org.licences/by/4.0/
This work is licensed under a Creative Commons Attribution 4.0 Licence. You are free to copy, communicate and adapt
the work, as long as you attribute the authors.
The Queensland Government supports and encourages the dissemination and exchange of information. However,
copyright protects this publication. The State of Queensland has no objection to this material being reproduced, made
available online or electronically but only if its recognised as the owner of the copyright and this material remains
unaltered.
The Queensland Government is committed to providing accessible services to Queenslanders of all cultural
and linguistic backgrounds. If you have difficulty understanding this publication and need a translator,
please call the Translating and Interpreting Service (TIS National) on 13 14 50 and ask them to telephone
the Queensland Department of Transport and Main Roads on 13 74 68.
Disclaimer: While every care has been taken in preparing this publication, the State of Queensland accepts no
responsibility for decisions or actions taken as a result of any data, information, statement or advice, expressed or
implied, contained within. To the best of our knowledge, the content was correct at the time of publishing.
2. Project description 1
3. Define options 2
4. Criteria selection 2
5. Weighting approach 3
5.1 Ranking method 3
5.1.1 Rank sum 4
5.1.2 Rank reciprocal 4
5.1.3 Rank order centroid 5
5.1.4 Pairwise 5
6. Weighting assignment 6
7. Scoring 7
8. Sensitivity 7
9. Results 7
Table of Figures
Figure 1 Smarter solutions MCA Tool overview 1
Table of Tables
Table 1 Pairwise scoring approach 6
Table 2 Smarter solutions MCA Tool scoring 7
Table 2 Weighting methodology comparison 11
Criteria Selection
2. Project description
The project must be clearly defined within the MCA to ensure that appropriate options are short- listed for evaluation and
that the criteria selected for assessment reflect the nature of the service requirement or opportunity. Accordingly, the
project should be defined in terms of:
• Decision context - what is the overarching service need or opportunity that this project response seeks to
address? What is the timeframe for required impacts?
• Network context – what is the current transport infrastructure environment? What is the age of current assets at
project site? Is there any complimentary infrastructure investment occurring in close proximity within the transport
network?
• Key objectives - what is the key objective the project is being implemented to achieve? What are the segmented
objectives (ultimate, intermediate, immediate) and corresponding timeframes?
4. Criteria selection
Selecting appropriate and relevant criteria is a critical component of the MCA process. Criteria must accurately reflect the
potential impact of all short-listed options and facilitate the appropriate evaluation and comparison of these alternative
options in addressing the identified transport need/ service requirement. Within this MCA Tool a comprehensive range of
criteria have been detailed for selection. This criteria has classified into 5 broad categories:
1. Economic Data
2. Transport Performance and Logistics
3. Construction and Constructability
4. Environmental Impact
5. Social Factors
When NOS are identified in the short-listed options, the following criteria must be selected.
• Implementation Cost
• Whole-of-Life Operating and Maintenance Costs
• Level of Service (LOS)
• Implementation Risk
• Value for money
• Performance horizon (benefitted years)
• Stageability
• Regional Development
• Rapid Benefit Cost Ratio
5. Weighting approach
There are various different preference weighting methodologies that can be utilised for assigning relative weighting to the
criterion in multi-criteria assessment. The key objective in assigning weights is to determine the relative importance of the
decision criteria (against other criteria under consideration) to the specific transport problem and/or project objective.
Four primary weighting techniques are included within this MCA Tool. These techniques can be categorised into two
broad methodologies - ranking method and analytical hierarchy approach. Both weighting methodologies rely on the
subjective judgements of decision-makers to determine priority rankings, before applying objective formulae to obtain
normalised weightings.
Based on the mixed nature (qualitative and quantitative) of the MCA criteria, subjective ranking is required due to the
difficulty experienced in eliciting relative weightings for decision criteria. This difficulty can be attributed to various factors
including the qualitative nature of some criteria, lack of appropriate expertise, incomplete/partial information, stage in the
investment decision making lifecycle, information processing capability and potential difficulty in reaching consensus
among participants/stakeholders.
Accordingly, priority ranking is easier to elicit, however care must be taken to ensure that these rankings are not distorted
by bias. It is further recommended that the ranking of options is facilitated in an open forum to eliminate this potential bias
and to ensure that sufficient discussion and documentation can ensue to support decision-making.
Where:
Where:
Where:
• rj is the rank of the jth objective
• K is the total number of objectives
Results summary
Comparable to the rank reciprocal methodology, the rank order centroid methodology returns weights that are more
dispersed than the rank sum methodology. However, rather than placing additional importance on the first rank only, rank
order centroid exaggerates both the first and last ranked criterion (see Appendix 1). As above, the weighting method
selected should reflect the relative importance of specific criteria to the project need / opportunity.
5.1.4 Pairwise
Pairwise comparisons allow decision makers to weight coefficients and compare alternatives with relative ease. As a
process, the pairwise comparison technique involves comparing and prioritising each of the selected decision criteria
against each other sequentially. For example, criterion A is compared against criterion B and assessed in terms of which
criterion is ‘preferred’ over the other. This process results in a pair-wise matrix, as illustrated in Table 1 (over), and
generates a series of weightings for each criterion.
Results summary
The elicitation of pairwise ranking provides decision-makers with oversight into the relative degree of importance of each
criterion with respect to each other. However, care must be taken as the nature and design of the pairwise comparison
can result in logical flaws.
6. Weighting assignment
The weighting technique selected above will determine the method undertaken to assign weightings to the selected
criteria. As mentioned, the selection of weighting technique should reflect the nature of the criteria selected to evaluate an
options’ suitability in satisfying a transport opportunity or service need.
This MCA Tool is designed to automate the assignment of weightings according to either the prioritised ranking of criteria
or the pairwise comparison of criteria, depending on the weighting methodology selected.
5 Very Positive Option achieves a significantly better outcome than the Base Case.
1 Very Negative Option achieves a significantly poorer outcome than the Base Case.
Note: The MCA Tool prepopulates the score for the Base Case option, which will equal 3 × ݊݊, where ݊݊ equals the
number of criteria selected. The Base Case score is important for project prioritisation and decision-making; options that
score near or below the Base Case should be further evaluated to assess the viability of these options to deliver value for
money and material improvements in network performance (or meet future service requirements).
The Base Case is defined in Section 4.
Further details regarding the explicit description of each scale rating to the criteria can be found in Appendix 2.
8. Sensitivity
A sensitivity analysis should be conducted in order to assess the robustness of the final ranking against the assigned
criterion weights. Various methods of sensitivity testing can be applied to MCA; this Tool incorporates a form of
‘thresholding’.
This sensitivity analysis approach assesses the robustness of selected options by changing the proportional weightings of
each criteria by ±25% in the first instance and then by ±50%. This approach exposes the individual criteria that are most
sensitive to changes in weighting and accordingly facilitates the narrowing of options to finalise the project decision.
9. Results
The results of the MCA, inclusive of the sensitivity analysis, must be assessed to determine which option is most suitable,
within the defined MCA criteria, at addressing the identified service need / requirement.
11
Appendix 2: Criteria for selection
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
Economic Data
Estimated cost is Estimated cost is Estimated cost is Estimated cost is Estimated cost is
Estimated cost of moderately less significantly less
significantly moderately equivalent to the
Implementation construction and than NOS option than NOS option
$ higher than NOS higher than NOS cost of the
Costs* procurement (outturn
option (> $100 option ($0 - $100 identified NOS ($0 - $100 million (> $100 million
estimate)
million more) million more) option less) less)
Estimated cost is Estimated cost is Estimated cost is Estimated cost is Estimated cost is
Whole-of-life Estimated cost of whole- significantly moderately equivalent to the moderately less significantly less
Operation and of-life asset operation higher than NOS higher than NOS cost of the than NOS option than NOS option
Maintenance* and maintenance $
option (> $100 option ($0 - $100 identified NOS ($0 - $100 million (> $100 million
million more) million more) option less) less)
12
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
Significant Significant
Estimated change in increase in Moderate Moderate
vehicle operating costs ( increase in No change to reduction in reduction in
vehicle
Vehicle/Bus operating vehicle operating vehicle operating vehicle operating vehicle operating
Road User $ operating costs
costs: fuel, tyre wear, costs (0% - 50% costs relative to cost (0% - 50% cost (50% -
Vehicle (50% - 100%
lubricants, repairs, increase on increase on Base Base Case reduction on 100% reduction
Operating Costs maintenance) Case) Base Case) on Base Case)
Base Case)
BCR Rapid Benefit Cost Ratio Ratio 0 to 0.5 0.5 to 1 N/A 1 to 1.5 > 1.5
13
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
No impact on
Change in user % of public Significant Moderate public transport Moderate Significant
behaviour to increase transport mode reduction in reduction in patronage increase in public increase in public
Public Transport public transport share public transport public transport transport transport
relative to Base
Patronage patronage patronage patronage patronage patronage
Case
14
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
Safety
Significant
Impact on safety incl. negative impact Moderate No impact on Moderate Significant
Safety accidents, injuries, # crashes on safety negative impact safety relative to positive impact positive impact
casualties and property per million relative to Base on safety relative Base Case on safety relative on safety relative
damage VKT Case to Base Case to Base Case to Base Case
Freight
15
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
Significant
# reduction in Moderate No change in Moderate Significant
Frequency of Impact on the frequency (measured freight reduction in freight frequency increase in increase in
Service of freight services relative to Base frequency freight frequency relative to Base freight frequency freight frequency
Case) relative to Base relative to Base Case relative to Base relative to Base
Case Case Case Case
Impact of construction
on the local community, Significant Moderate No impact on
Community including visual amenity, Descriptive negative impact negative impact local community N/A N/A
Disruption safety risk, increased on local on local as a result of
traffic and additional community community construction
parking demand
16
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
Significant Moderate
Potential engineering or evidence of evidence of No evidence of
Engineering / construction challenges potential potential potential
Constructability - during construction or Descriptive engineering or engineering or engineering or N/A N/A
across lifecycle construction construction construction
challenges challenges challenges
Ability for the option to No ability for the Ability for the
Stageability be implemented in Binary N/A option to be N/A option to be N/A
discrete stages over implemented in implemented in
time stages stages
Environmental Impact
17
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
Social Factors
Significant Moderate
Are there any significant barriers to barriers to No barriers to
barriers to implementation implementation implementation
Barriers to development? E.g. Descriptive arising from arising from arising from N/A N/A
development existing land use or existing land use existing land us existing land
cultural significance relative to Base relative to Base use relative to
Case Case e Base Case
Significant Moderate
misalignment of misalignment of No change to Well aligned to Perfectly aligned
Degree of alignment to option to option to future land use strategic land use to strategic land
Future land use strategic land use and Descriptive strategic land strategic land use relative to Base and planning use and
planning objectives use and planning and planning Case objectives planning
objectives objectives objectives
18
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
Socioeconomic
Significant
negative impact Moderate Moderate Significant
on user negative impact No impact on positive impact positive impact
Impact on user behaviour and on user user behaviour on user on user
Mode Shift behaviour and influence Descriptive mode shift and behaviour and and mode shift behaviour and behaviour and
on mode shift mode shift mode shift relative to Base mode shift mode shift
relative to Base relative to Base Case relative to Base relative to Base
Case Case Case Case
Additional Additional
Impact to local land, properties properties
property and Quantity of affected relative affected relative No addition
Impact on businesses resulting affected to Base Case – to Base Case – N/A N/A properties
property owners from disruption during properties impact across a impact across a affected relative
construction and corridor/route localised spatial to Base Case
operation spatial area area
Impact on visual
amenity and urban Significant Moderate No impact on
quality as a result of reduction in reduction in visual amenity Moderate Significant
Visual Amenity changes in bikeways, Descriptive visual amenity visual amenity and urban quality increase in increase in
and Urban Quality walking paths, noise and urban and urban relative to visual amenity visual amenity
during construction and quality quality Base Case and urban and urban
design/aesthetic quality quality
No impact on
Significant Moderate community Moderate Significant
Severance Impact on community Descriptive increase in increase in severance reduction in reduction in
severance community community relative to Base community community
severence severence Case severence severence
19
Scoring Range
Criterion Indicator Measure
1 2 3 4 5
* Where an NOS option has not been short-listed for assessment, please refine these criteria to reflect the options identified and the key project objectives.
20
2