EBU Summative Assessments

in Urology
UEMS Debate: „Maintaining Professional Development
in the Hands of the Medical Profession”,
Brussels 19 Oct 2017

Artur A. Antoniewicz M.D., Ph.D., FEBU

EBU Examination Committee Chairman 2011-2017
Polish Urological Assiociation Delegate in EBU 2002-2017
Disclosures
• NONE
Personal CME-CPD acheivements
2002-2017: 3187 credits
(212 credits/year)
25th Anniversary of EBU Examinations
• Introduced in 1992 by a
group of visionary,
ambitious urologists.
• Among ‘founding fathers’
are: F. Schröder (NL), H.
Madersbacher (AT), J.
Fitzpatrick (IR), R. Nijman
(NL), J. Mattelaer (BE), A.
Matos-Fereira (P).
• Mission is to set a
European standard in
urology training.
• Provides a pillar for better
patients’ care.
 Development EBU Examinations
• 1st FEBU exam held in conjunction with EAU
Congress in Genova (Italy) in July 1992
• Provides an opportunity for urological
societies in those countries where there is no
formal exit examinations
• Recognised as a Mark of Excellence and an
essential assets to a urologists’ portfolio
 Objective Test of Knowledge

Miller GE. The assessment of clinical skills/competence/performance. Academic

Medicine (Supplement) 1990; 65: S63-S7.
 A fair exam’s characteristics
• Reliability: We get the same results if the same group is
examined a second time
• Invariance: The results from the exam of one group can
be meaningfully compared with the other group’s.
• Discrimination: Can differentiate among students on
the basis of how well they know the material being tested.
• Validity: It measures what we think it measures.
Structure EBU Examinations
FEBU Oral Examination
Tests a candidate’s ability to evaluate and manage
common urological cases.

FEBU Online Written Examination

Assesses if the candidate has the level of knowledge
defined by EBU Examination Committee.

In-Service Assessment
Identifies a candidate’s deficiencies and
provides feedback for future training.
 Online In-Service Assessment
• Formative assessment
• Allows candidates to evaluate their current
knowledge base against current standards.
• Encourages teacher & student to reflect on their
training scheme.
• 100 MCQs covering all fields of urology.
• Web-based
• Easy to take
• 50 Euro
 10
30
40
50
60
70

20

0
Austria
Croatia
Denmark
France
Germany
Hungary
Italy
Malaysia
Netherlands
Pakistan
Portugal
Other
Slovakia
Spain
Switzerland
735 candidates in 31 countries

United Kingdom
EBU In-Service Assessment 2017
FEBU Online Written Examination
• Summative assessment
• Assess if the candidate demonstrates the
minimum level of knowledge recognised by
the EBU (min. 62-66%)
• 100 MCQs covering all fields of urology
(oncology 35%, functional urology 20%,
stones 15%, BPH 10%, andrology & infertility
10%, infections 10%)
 Relative formula
Pass/fail mark =

P/F Mark = (Average score – SD) +1

Practically 2016: 68,70-6,62+1= 63,08

Pass/fail 2016: 64
 FEBU Written Examination
• 4 languages: English, German, Polish,
Hungarian, Turkish
• Approx. 450 candidates per year - average
pass mark 80%
• Taken at secure Pearson test centres
throughout Europe
 FEBU Oral Examination
• Summative assessment
• Test the candidate’s ability to evaluate and
manage common cases in every day practice
• Candidate is examined by 2 urologists
and will be given three clinical cases
• 1-day examination
• Pass rate: 95%
• Date: 02 June 2018
 FEBU Oral Examination
• 12 languages: Dutch, Danish, English, French,
Hungarian, German, Greek, Italian, Polish,
Portuguese, Spanish, Turkish
• Brussels, Warsaw, Budapest, Ankara
• Board of Examiners involved (about 100
experts)
• Successful candidates obtain FEBU Diploma
and FEBU title
 EBU Oral examination
• 17 minutes per case
• All cases with 1-5 visuals (pictures, videos, graphs,
laboratory results etc.)
• Structured debate with Q&A based on written protocol
• Area to be assessed: 1. Diff.diagnosis 2.Clinical
judgment 3.Therapy and Follow-up
• Pre-defined „absolute failures” - failure of the exam
• Performed by Examinaer in the presence of Trustee
• Protocol in written
• Examiantion Board debate before announcing results
EBU Oral Examination Scoring System
• 3 real cases presented; scoring per case is
min. 3 and max .12
• Overall score is maximum 36; passing score is
24 (OVER 66%)
• Marks are transcribed on a 1-10 scale
Score ≤ 17 18 - 20 21 - 23 24 - 26 27 - 29 30 - 32 ≥ 33
Mark 3 4 5 6 7 8 9
Mark Oral Mark Written Result
• 4
Result
5
based on 8 or 9
mark
7 or 8
Written Re-exam
Examination
Re-exam
5 9 Compensation = Pass
6-9 Pass
Tablet assisted oral examination 2017
All visuals are displayed on tablet
(to scroll function for CT images)
 EBU Assessments 2012-2017
800
700
600
500
ISA
400
Written Exam
300 Oral Exam
200
100
0
2012 2013 2014 2015 2016 2017
 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

UEMS/EBU countries A A A A A A A A A A

Austria 54 53 45 42 53 60 63 47 43 54
Belgium 11 2 16 23 20 9 11 15 24 18
Bulgaria
Czech Republic 30 28 38 43 49 34 54 24 29 37
Denmark 12 12 14 13 14 17 15 18 19
Estonia 1 2 2 2
Finland
France 3 2 1 1 2 1
Georgia
Germany 32 74 57 48 42 29 49 31 76 64
Greece 43 49 35 36 36 29 33 11 50 36
Hungary 30 27 28 - 19 17 15 12 23
Ireland 1
Italy 32 14 19 59 8 13 17 9 5
Latvia
Lithuania
Luxembourg
Malta 2 1 2 4 3 3
The Netherlands 47 52 54 56 46 41 37 38 42 27
Norway 1 4 7 4 5 6
Poland 44 54 35 39 34 42 43 56 46 59
Portugal 7 4 6 2 15 15 13
Romania 23 2 1 1
Slovakia 2
Slovenia
Spain 9 12 12 11 14 11 9 20 25 24
Sweden 1 10 9 14 15 14 14 18 20
Switzerland 44 38 37 38 39 54 34 36 44 52
Turkey 86 59 73 71 49 38 42 28 11 16
United Kingdom 72 81 104 93 99 116 125 118 116 80
Total (incl. ROW) 699 738 722 725 741 740 760 702 700 658
 Worldwide 5.209 FEBU’s
Europe’s Top 10

Switzerland 195

Portugal 199

United Kingdom 248

Austria 257

France 265

Greece 392

Italy 409

Spain 494

Germany 656

Poland 662

0 100 200 300 400 500 600 700

Data on EBU files

 EBU & ABU
• EBU Examinations were initially inspired by
the American Board Examinations
• Our visit at ABU Oral Examination 2013 in
Dallas provided confirmation that EBU and
ABU Oral Examinations are on-par
Summary Examination 2011-2017
• Written exam: number of MCQ’s limited to
100
• Written exam: ONLINE (Pearson Vue)
• Data Management System (Examfolio)
• Oral exam: visuals obligatory
• Oral exam: iPad incorporated (Softwebo)
• Turkey onboard as of 2017 (+100 candidates)
• MEDBOOK incorporated (logbook online)
 Urology in Europe 2017:
EBU – EAU – National Society
 European Training in Urology (curriculum
urology)
 European formative assessment in Urology
(In-Service)
 European summative assessment (FEBU) in
Urology within 25 years
 CME/CPD in place: EU-ACME PROJECT working
 EBU Accredited Training Centers (82)
 European recertification in Urology accessible
EBU ORAL EXAM 2018 Warsaw

2 June 2018
Examfolio: Management System

QUESTION
AUTHORISING

EXAM &
QUESTION
QUESTION
EDITING
ANALYSING
 3-in-1 Role-based solution

Questions Exams Analysis

1. Authoring 4. Administration 7. Question Statistics
2. Approval 5. Monitoring 8. Exam Consistency
3. Selection 6. Feedback 6. Candidate scores
Examfolio- Exam Management System
• 1.800 items in the database
• Efficient platform for
item bank & analysis
• In future Examfolio used
for the online
In-Service Assessment

UNION EUROPÉENNE DES MÉDECINS SPÉCIALISTES (U.E.M.S.)

 Online exams include visuals
• 1.800 items in the database
• Efficient platform used for
item bank & analysis
• In future Examfolio to be
used for the online
In-Service Assessment

UNION EUROPÉENNE DES MÉDECINS SPÉCIALISTES (U.E.M.S.)

Questions
APPROVAL

Work on questions
is organised into
clear cycles of

Reviewing
Revising
Approving

For every exam held,

feedback helps to
improve questions
Questions
AUTHORING

Writing committee
sign in and author
questions

Questions can be
evaluated pre and
post exam
Exam analysis
Consistency

KR20 - Consistency

Mean and SD

Question
performance

Candidate
performance
Demographic analysis
Results

Results from exams

can be used to
compare candidates at
different institutions or
in different countries…

… allowing organisers
to deploy educational
resources improve
overall standards.
Item analysis
Item
Questions are analysed
using:

Point Bi-Serial
correlation

Discrimination
index,

Difficulty index

Distractor
efficiencies
Examfolio: Management System
Examfolio: Management System
What is Medbook for EBU
• E-Portfolio

• Working Portfolio

• Assessment Portfolio

• Development Portfolio

• Showcase Portfolio

• Direct link to EBU EBU accreditation

Showcase Portfolio
Working Portfolio
Assessment Portfolio
Development Portfolio
14 Core Procedures
European Boad of Urology
EBU Examinations (FEBU Title)
What procedures are done by residents in Europe?

Survey on 359 EUREP participants in 2013

Presented in Stackholm in 2014

88 % of the residents are confident to
perform a TURB-T without supervision

• 93% in Southern Europe

• 89% in Western Europe
• 71% in Eastern Europe
1 out of 3 residents is not
confident to perform a
TUR-P unassisted!!!
40% of the Senior Residents
have NEVER performed a
major procedure!!!
 E-BLUS (ESUT)
European Training
in Basic Laparoscopical Skills
Next step: simulation based assessment ?
Simulation of the Examination
Revalidation: UK only
 Potential Options for the Future

• FEBU title recertification every 5 years.

• Based on CME-CPD activity (250 credits/5 years)
• Written Exam online (60% correct answers
needed)
• EBU should become a partner in development of
pan-European system of re-assessment, re-
certification and re-validation in urology