22 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
1 Laboratory of 
Biocybernetics, Faculty 
of electrical engineering, 
university of Ljubljana, 
Ljubljana, slovenia
2 school of engineering, 
university of Warwick, 
Coventry, united Kingdom
3 institute of oncology 
Ljubljana, Ljubljana, 
slovenia
4 Faculty of Medicine, 
university of Ljubljana, 
Ljubljana, slovenia
5 department of 
otorhinolaryngology and 
Cervicofacial surgery, 
university Medical Centre 
Ljubljana, Ljubljana, 
slovenia
Korespondenca/
Correspondence:
eva pirc, e: eva.pirc@
fe.uni-lj.si
Ključne besede:
vrednotenje tehnologij v 
zdravstvu (Hta); analiza 
stroškovne učinkovitosti 
(Cea); model Markova; 
elektroporacija; 
elektrokemoterapija (eCt)
Key words:
health technology 
assessment (Hta); 
cost-effectiveness 
analysis (Cea); Markov 
model; electroporation; 
electrochemotherapy 
(eCt)
Study design of a medical device  
pre-market assessment: a case study on 
electrochemotherapy
Zasnova študije vrednotenja novih tehnologij v 
medicini: študija na primeru elektrokemoterapije
Eva Pirc,1 Leandro Pecchia,2 Matej Reberšek,1 Gregor Serša,3 
Marko Snoj,4 Aleš Grošelj,5 Damijan Miklavčič1
Abstract
Final goal of the study is to estimate the cost-effectiveness of electrochemotherapy for the tre-
atment of basal-cell carcinoma and skin melanoma. Paper consists of two parts: the first part 
presents basic principles and concepts of health technology assessment and cost-effectiveness 
analysis, and the second part reports an early cost-effectiveness analysis of electrochemothe-
rapy for the treatment of basal-cell carcinoma and skin melanoma that we are developing. Few 
cost-effectiveness analyses of electrochemotherapy have already been done, but with a lack of 
information about intervention effectiveness in terms of quality of life, which may result in inac-
curate or even inadequate conclusions. In order to obtain the most realistic results, two general 
Markov models and their reduced versions for initial calculations are presented. The models 
were designed specifically to assess electrochemotherapy of basal-cell carcinoma and skin me-
lanoma. Also, data required for successful calculations have been identified, some of which are 
missing and will be collected within different studies which are still under way, including rando-
mized control trials. Additionally, recommendations for data collection process and follow-up 
reporting are made. With this paper we want to raise awareness about the importance of nume-
ric quality of life reporting and usefulness/meaning of EQ-5D questionnaire that might not be 
self-evident at first sight, but are crucial for cost-effectiveness analysis.
Izvleček
Končni cilj projekta je izdelava analize stroškovne učinkovitosti (angl. Cost-Effectiveness 
Analysis) elektrokemoterapije za zdravljenje bazalnoceličnega karcinoma in kožnega melano-
ma. Prispevek je sestavljen iz dveh delov. V prvem delu prispevka so predstavljeni osnovni kon-
cepti in postopki vrednotenja tehnologij v zdravstvu (angl. Health Technology Assessment) in 
analize stroškovne učinkovitosti, v drugem delu pa poročamo o zgodnji stroškovni analizi (angl. 
early Cost-Effectiveness Analysis) elektrokemoterapije, kot terapije za zdravljenje bazalnocelič-
nega karcinoma in kožnega melanoma, kar je predmet naše študije. Nekaj stroškovnih analiz je 
že nastalo vendar zaradi pomanjkanja podatkov o učinkovitosti zdravljenja, v smislu povečanja 
kakovosti življenja bolnikov (angl. Quality of Life ), zaključki ne morejo biti točni. Da bi izdelali 
čimbolj relevantno analizo, v članku predstavljamo dva splošna Markova modela in njuni poe-
nostavljeni različici, ki ju bomo uporabili za začetne izračune. Modela sta bila zasnovana posebej 
za analizo elektrokemoterapije bazalnoceličnega karcinoma in kožnega melanoma. Dodatno so 
opredeljeni tudi potrebni podatki za uspešne izračune. Manjkajoče podatke bomo zbrali v okviru 
različnih študij, ki še vedno tečejo, vključno z randomiziranimi kliničnimi študijami. Predstavlje-
no je tudi priporočilo za poročanje, ki bi olajšalo zbiranje podatkov. S tem prispevkom predvsem 
želimo dvigniti splošno ozaveščenost o pomembnosti številčnega poročanja o kakovosti življe-
nja in uporabnosti oziroma pomenu vprašalnikov EQ-5D, ki na prvi pogled morda nista samou-
mevna, vendar sta bistvenega pomena za analizo stroškovne učinkovitosti.
study design of a medical device pre-market assessment 23
Pregledni znanstveni članek
prispelo: 12. 1. 2017 
sprejeto: 14. 11. 2017
Citirajte kot/Cite as: Pirc e, Pecchia e, reberšek M, serša g, snoj M, grošelj a, Miklvavčič d. study design of 
a medical device pre-market assessment. Zdrav Vestn. 2018;87(1–2):22–40.
DOI: 10.6016/ZdravVestn.2482
1.  Introduction
Electroporation is an evolving tech-
nique with many applications (1-4); in 
this article the focus is on one of the 
most successful, i.e. electrochemothe-
rapy (ECT). ECT is an antitumor the-
rapy that increases the cytotoxicity of 
chemotherapeutic drugs such as bleo-
mycin and cisplatin by the help of appli-
ed electric pulses. Electroporator is a de-
vice that generates high-voltage electric 
pulses, which are delivered to the tissue 
trough electrodes (5). If the electric fi-
eld generated between the electrodes is 
sufficient, the permeabilization of the 
target (tumor) cells is triggered and en-
trance of the previously injected che-
motherapeutic drug into cells (within 
the tumor) is enabled (6). As a result, 
electrochemotherapy is a highly efficient 
treatment, with complete response rates, 
based on a single treatment between 60 
to 70 % and objective response rates up 
to 80 % (7).
Basal-cell carcinoma (BCC) is the 
most common form of skin cancer 
and its incidence is still increasing 
mainly due to the population ageing. 
Worldwide, every year two to three 
million patients are diagnosed with 
non-melanoma skin cancer (8). The 
current gold standard for the treatment 
of BCC (and other skin malignancies) 
is surgical excision (9), but for patients 
that are unsuitable for conventional tre-
atments, ECT offers a good alternative. 
Specifically, in the treatment of BCC, 
the objective response rates after a sin-
gle ECT session are close to 100 % (75). 
Upon that, the surrounding tissue re-
mains undamaged and, consequently, a 
good cosmetic outcome is obtained. In 
most cases, ECT can be performed as an 
outpatient procedure under sedation or 
local anaesthesia. Therefore, treatment 
with ECT results in a considerably 
shorter hospital stay, faster recovery 
and reduced health care costs (10). ECT 
diminishes the need for surgery, it can 
be a feasible treatment option for cuta-
neous lesions resistant to other therapi-
es or can serve as an adjunct to other 
therapies. The UK National Institute for 
Health and Care Excellence (NICE) de-
emed that ECT was a safe treatment for 
primary BCC, however it also warned 
about the limited evidence for its effica-
cy (11).
In western countries the incidence 
of melanoma has been increasing for 
as long as recorded (12). Melanoma 
spreads by lymphogenous and also by 
hematogenous route. ECT is a stan-
dardised procedure for the treatment 
of superficial metastases of melanoma 
resistant to other treatments. Skin me-
tastases of melanoma occur in 2–20 % 
of melanoma patients (13). ECT has se-
veral advantages: the treatment can be 
administered in an outpatient setting 
under local anaesthesia and deep seda-
tion; it can also be scheduled as a day-
-care or day-hospital procedure under 
general sedation; repeated sessions can 
be performed with a minimum inter-
val of one to two weeks (14-17). Overall 
ECT can be considered as “patient 
friendly” procedure with effective-
ness consistently reported throughout 
24 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
the reports. Its effect can be localised, 
thus reducing side effects. Later in the 
article, the focus is mainly on the co-
st-effectiveness of ECT, which will fa-
cilitate a comparison of ECT to other 
treatment options, not only with re-
spect to its effectiveness but also its 
cost effectiveness.
Screening and evaluation of medical 
technologies is becoming crucial, as in 
most of the developed countries health 
care expenditures are constantly increa-
sing, while at the same time budgets are 
getting tighter (18). In Slovenia, health 
care spending is currently estimated 
below the European average (19,20). 
Between years 2002 and 2008, health 
care spending in Slovenia amounted to 
between 7.49 % and 8.08 % of GDP. It 
spiked in year 2009 to 8.56 % of GDP 
and has been between 8.5–8.73 % of 
GDP ever since (19,21). Unfortunately, 
based on demographic data, the fo-
recast is not reassuring. In year 2017, 
18.7 % of Slovenian population is older 
than 65 years (22). The projections show 
that this number is going to be approxi-
mately 33.6 % by 2060, what will una-
voidably result in a significant increase 
in health care expenditure. Simulations 
show that health care expenditure 
will increase from 0.5 % up to 2.8 % of 
GDP per year, by the year 2060 (23). 
Independently of the relative or abso-
lute expenditure per year, the health 
care national budgets are in any case li-
mited. Therefore, decision makers have 
to consider carefully all public money 
spending, especially since, in limited 
budget scenarios, the introduction of a 
new technology may result in the exclu-
sion of an old one. There are however 
cases where innovations were inclu-
ded into the medical practice, without 
any previous economical evaluations 
and had to be abandoned later due to 
economic inefficiency (73). The intro-
duction of new technologies in the 
Slovenian public health care system is 
still relatively arbitrary. Each innovati-
on is evaluated according to three cri-
teria: i) health effectiveness (the more 
severe the disease is the more points it 
gets); ii) professional justification of the 
program (meta-analyses and systematic 
literature reviews get the most points, 
regardless of the results); iii) economic 
efficiency (only price is considered here, 
the cheaper the innovation is, the more 
points it gets). All innovations are than 
ranked on a priority list according to 
the number of points collected (24).
Therefore, an efficiency improve-
ment of health care systems is necessary. 
Choices about providing health care in-
terventions, based on available evidence 
regarding safety, effectiveness and cost 
effectiveness thus have to be made (18). 
Widely used in most developed coun-
tries is a Health Technology Assessment 
(HTA) evaluation process. HTA is a 
structured process aiming to inform 
decision makers about choices which 
can be used to allocate healthcare bud-
get (25). In the majority of cases, HTA 
provides an estimate of the incremental 
cost-efficacy ratio (ICER), which infor-
ms decision makers about the gain in 
terms of quality of life per cost that the 
introduction of a new technology will 
produce (77). This article reports on an 
early cost-effectiveness analysis (eCEA) 
of ECT for the treatment of BCC and 
skin melanoma that we are developing. 
The first part of the paper introduces 
basic concepts of HTA and CEA, while 
the second part reports on a case study 
of electroporation based treatments, in 
particular ECT for BCC and skin mela-
noma.
study design of a medical device pre-market assessment 25
Pregledni znanstveni članek
2.  Health technology 
assessment
According to the World Health 
Organization (WHO), Health technology 
assessment (HTA) refers to the systema-
tic evaluation of properties, effects, and/
or impacts of health technology. It is a 
multidisciplinary process to evaluate the 
social, economic, organizational and et-
hical issues of a health intervention or 
health technology. The main purpose of 
conducting an assessment is to inform a 
policy/decision maker (26). The evalua-
tion can be applied to systems, general 
equipment, instruments, hardware and 
software as well as to procedures, stan-
dards, norms, staff skills, professional 
knowledge, drugs, public health pro-
grams, etc. However, HTA is currently 
manly employed for the pharmaceuti-
cals. Slowly it is being claimed also for 
biomedical devices, where slightly di-
fferent methods or at least some modi-
fications of currently used methods are 
required (27). Main outcome of such 
evaluation should provide information 
about the costs /economic effectiveness, 
savings, performance, safety, ethical and 
social impacts and improvements of the 
investigated treatment/drug. Finally, the 
question that needs to be answered is: 
“Do we really need this technology and 
why?”.
The formal proof of HTA can be fo-
und in systematic reviews, meta-analy-
ses and randomized controlled clini-
cal trials (28). A multi criteria decision 
analysis is suggested by which one can 
evaluate the cost-effectiveness through 
the benefits of quality adjusted life ye-
ars (QALY). One of these analyses is the 
Cost-Effectiveness Analysis (CEA); its 
basics concepts and principles are pre-
sented in the following section.
Unlike drugs, which cannot be 
commercialised if sufficient evidence 
on their effectiveness is not collected, 
medical devices can be marketed after 
their safety has been proven. As a con-
sequence, decision makers are called to 
make choices while cost-efficacy evi-
dence is not fully available, especially 
for innovative technologies. Therefore, 
in the past years there has been a signi-
ficant diffusion of early HTA (eHTA) or 
early economic analyses (78,79). In those 
cases, the limited clinical evidence that 
is available is projected using statistical 
methods, the costs are estimated assu-
ming the worst case (i.e., max costs) and 
the uncertainty is quantified using sta-
tistical techniques. As a result, an eHTA 
informs the decision makers about the 
incremental risk-opportunity ratio, whe-
re risk is considered as a potential cost 
and the opportunity is considered as po-
tential effectiveness. An eHTA assumes 
that proper HTA analyses are preformed 
when sufficient clinical data are availa-
ble. The alternative would be to just wait 
and postpone the introduction of medi-
cal devices that can potentially save lives 
or increase significantly life quality, but 
this is not feasible with medical devices 
because their lifecycle is much shorter 
than that of drugs.
2.1.  Cost-Effectiveness 
Analysis
There is a wide variety of approaches 
to economic evaluations of health tech-
nologies, one of them being cost-effecti-
veness analysis (CEA), which measures 
the incremental resources required for a 
new intervention in monetary units and 
the technology impact on patient health 
using different scales. In this paper, the 
focus is on the CEA, which measures 
26 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
impact on health in terms of quality 
of life, which is known as cost - utility 
analysis (29).
2.1.1. Quality of life 
(QoL) and QALY
In HTA it is assumed that people in 
their life move through different he-
alth states, each of those states has a 
specific value attached to it - Quality 
of life (QoL). QoL describes a quality 
of individual’s daily life, including so-
cial, emotional and physical aspects. 
For evaluating QoL, health care indi-
ces that provide information how he-
alth care influences patients, known as 
Health Adjusted life Years (HAYs), are 
used. One of them is Quality Adjusted 
Life Years (QALY), which is a gene-
ral index (i.e., not disease specific), 
and a unit of health care outcome 
that merges length of life with quali-
ty of life. QALYs are used in the Cost 
Effectiveness Analysis (CEA) to deter-
mine the ratio of incremental costs (i.e., 
new technology versus a benchmark) 
to QALY gained. The index is defined 
as the value-weighted time-life-years, 
weighted by their quality-accumulated 
over the time (30). QoL is normalized 
to a standardized scale with ranges 
from 0.0 (death) to 1.0 (perfect health). 
According to some authors, negative 
values can also be reached, describing 
states worse than dead (e.g., dementia 
and coma are often considered equal to 
or worse than death (31)). Widely used 
and translated into most languages is 
the EuroQol (EQ-5D) evaluation tool 
(questionnaire), which is a standardi-
zed instrument for use as a measure of 
health outcome (The study/trial/project 
that use EQ-5D should be registered; 
see also: https://euroqol.org/support/
how-to-obtain-eq-5d/) (32,33). It is also 
one of the measures recommended for 
use in cost-effectiveness analyses by the 
Washington Panel on Cost Effectiveness 
in Health and Medicine (34) and NICE 
(National Institute for Health and Care 
Excellence) (35). The health status is 
measured in five dimensions: mobility 
(walking ability); self-care (ability to 
wash or dress); usual activities (such as 
“work, study, housework, family or le-
isure activities”); pain/discomfort; and 
anxiety/depression. The respondents 
rate each dimension. There are two 
versions with a three-level (EQ-5D-3L, 
which rate each dimension using three 
values: ‘0’, indicating no problems, ‘1’ in-
dicating some problems, or ‘2’ indicating 
huge problems) and a five-level (EQ-
5D-5L) scale. On the basis of patients’ 
evaluation of their physical, social and 
cognitive functions final index – value 
is calculated. Therefore, the EQ-5D-3L 
defines 243 potential health states (i.e., 
35), which together with two additional 
states for dead or unconscious give a to-
tal of 245 health states. For example, a 
year of perfect health is worth 1 and all 
non-perfect health years are worth less 
than 1, depending on the burden of the 
disease to the patient (36). Thus an in-
tervention that generates six additional 
years in a health state valued at 0.65 will 
generate more QALYs than compara-
ble technique that generate seven years 
in health state estimated at 0.5. There 
are also some other measurement in-
struments available that are frequently 
used in parallel or alternatively, such as 
Nottingham health profile, Quality of 
life Scale (QUOLS) and others (37-40). 
Since healthcare budget is always limi-
ted (sometimes scarce) and the intro-
duction of a new technology may result 
in the exclusion of another, it is impor-
tant to use a general index (e.g. QALY) 
in order to quantify the benefit on the 
whole national population (41,73).
study design of a medical device pre-market assessment 27
Pregledni znanstveni članek
Figure 1: Incremental cost-effectiveness plane/diagram (70). The 
resulting ICERs are presented graphically as a ratio between costs and 
the effectiveness/utility or as a distribution with uncertainty in cost-
effectiveness plane. Four quadrants represent all combinations of 
possible outcomes. The more effective outcomes are located further 
right on the x-axis, and with the rise of y-axis the cost of the outcome 
rises (69). An ICER of an innovation that is more costly and more efficient 
then the benchmark is located in the first quadrant; in case of a more 
costly and less efficient technology, ICER is in second quadrant. Other 
options can be derived from the figure.
2.1.2. Incremental 
cost ratio (ICER)
The result of CEA is presented with 
incremental cost ratio (ICER). ICER is 
defined as the additional cost per addi-
tional benefit/utility that is measured 
in QALY. Because QALY has a normal 
distribution and a sum of two normal 
variables is also normally distributed, 
ICER can be calculated as a ratio of two 
asymptotically normal variables (42). 
In the following equation (Eq. 1) tre-
atment B represents a gold standard or 
benchmark to which new treatment X, 
e.g. electrochemotherapy, is compared.
 (1)
The visualization of ICER is a dis-
tribution over a sample population, it 
is presented with a four quadrant cost-
-effectiveness plane that illustrates the 
relation between an incremental levels 
of effectiveness (utility gained) of an 
outcome and additional total cost of 
implementing this outcome (Figure 1). 
In case when new technology is more 
effective and cost efficient (fourth qu-
adrant in Figure 1), compared to the 
benchmark, i.e. treatment B, than the 
innovation, i.e. treatment X, is for sure 
more suitable and worth of adopting. 
More commonly the outcomes of CEA 
of new technologies are divergent, for 
example the innovation is more effecti-
ve but also more costly (case 1 and 2 in 
Figure 1). It should be decided whether 
the incremental utility is worth the cost 
and sustainable (43).
Maximum acceptable ceiling ra-
tio (max ICER) is the threshold or the 
maximum amount that a decision-ma-
ker is prepared to pay for one incremen-
tal QALY. It is drawn in the cost-effecti-
veness plane, as a threshold line (Figure 
1, red line). If the calculated ICER falls 
below this line, the new technology is 
considered cost effective and is adopted 
(cases 1 and 4 in Figure 1), otherwise it 
is rejected based on its cost-ineffecti-
veness (cases 3 and 2 in Figure 1) (44). 
Decisions, which apply to the public 
health care thresholds and acceptable 
outcomes of maximum ICER, are sug-
gested by the WHO, however the pro-
blem remains how to apply these thre-
sholds to each specific case. As analyses 
are often made by commercial entities 
with vested interests, the results may 
favor the new technology. It thus ne-
eds to be taken into account that the 
evaluator’s subjective impact is always 
present. Another question that is also 
present is, how much the society shou-
28 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
ld pay for a QALY. According to the ge-
nerally present opinion, e.g. in the UK, 
a QALY is worth somewhere between 
£20,000 and £30,000 (44). In Slovenia, 
rather than having a maximum value 
defined, each innovation is placed on 
a priority list of admission based on 
the evaluation of the following crite-
ria: health effectiveness; quality of ju-
stification of recommended program; 
economic effectiveness (high scores 
correlate with lower costs); population 
perspective (more patients more po-
ints); and organizational efficiency. The 
quality of life is not considered in the 
evaluation.
CEA always includes a comparison 
of a new technology to the benchmark 
technology, if available. The innovati-
on cannot be cost effective by itself, it 
has to be cost effective compared to the 
benchmark. In a CEA, a treatment pro-
ducing an additional 0.5 QALY at an in-
cremental cost of €3,000 per patient, is 
considered having a cost of €6,000 per 
QALY (i.e. €3,000 /0.5 QALY = €6,000 /
QALY) (36).
2.1.3. Discounting
It should always be taken into acco-
unt that values of costs and outcomes 
change with time (29). Because the CEA 
are projected through a certain period of 
time discounting is necessary. Cost and 
outcomes should be discounted relative 
to their present value at the rate of 3 % 
or up to 5 %, per year (29). The cost di-
scount rate can be estimated from the 
equation (Eq. 2). Some authors suggest 
a common rate for costs and outcomes 
and others prefer a lower rate for ou-
tcomes. NICE discounting guidance for 
cost effectiveness analysis (45) requires 
that both costs and health outcomes are 
discounted at 3.5 % (46).
 (2)
For example, if the present value of 
cost is €2,500 and a 5 % discount rate 
is used, one year in the future, the cost 
will be €2,375 and 5 years in the future 
the expense will be only €1,875. The same 
procedure is used for the discounting of 
health care outcomes.
2.1.4. Sensitivity analysis
The degree of uncertainty is a subject 
to variables used in CEA. To estimate 
plausible variations, a sensitivity analysis 
should always be performed. Such sensi-
tivity analysis provides information on 
how a variation of a certain variable 
affects the result of the CEA. We distin-
guish between deterministic and proba-
bilistic sensitivity analysis. Deterministic 
sensitivity analysis uses an approach in 
which one or more parameters are chan-
ged manually across a pre-specified ran-
ge (range should correspond to the un-
certainty defined in literature). Results 
are than analysed and the extent of the 
impact of input parameter variation on 
the output values is defined. Probabilistic 
sensitivity analysis captures uncertainty 
of all input variables, it is characterized 
through the use of probability distributi-
ons, translated to uncertainty and results 
in means and standard error for the in-
cremental costs (47). If a correlation of 
input variables is present and available, it 
should also be considered and incorpo-
rated into the model (48).
2.1.5. Data collection and 
decision-making models
Most often, data collection metho-
ds used in HTA are: systematic litera-
ture review; meta-analysis; modelling; 
group judgment; unstructured litera-
study design of a medical device pre-market assessment 29
Pregledni znanstveni članek
ture review; and expert’s opinions (29). 
Sometimes it happens that for a speci-
fic new medical device or an innovati-
ve technique there is not yet sufficient 
evidence available, apart from data pro-
ving its safety. When only few or low 
quality studies are available, potential 
source of bias must always be conside-
red and documented (29). Additionally, 
in order to incorporate conditionality 
and uncertainty of data collected, de-
cision models are used to simulate ad-
justments of projections of the existing 
primary data. With modeling patients 
conditions, treatment efficacy, treatment 
and maintenance costs and incidence 
of the disease, projections to a future 
costs and outcomes of the treatments 
or innovations can be made. Decision 
making trees, fuzzy logic and state tran-
sition modeling, such as Markov model 
and Monte Carlo simulation, are most 
often used (49). Markov models are wi-
dely used in HTA and health-economy 
studies, and recently Craven et al. (50) 
showed that this approach can also be 
used in pre-market HTA studies for me-
dical devices.
In the CEA, each state in the model 
represents a specific state of health, or 
stage of disease between which the pa-
tient migrates. There is always one state 
that represents death, which is consi-
dered an absorbing state as it does not 
have a return path. Each state is associa-
ted with an average QALY, reflecting the 
quality of life of patients in this health 
status, a cost that is reflecting the cost 
needed to maintain the patient in this 
state (e.g., avoid deteriorations) and the 
probability to move from the current 
health state to another, or the stay in 
the same state in the following period 
of time (usually one year). Health sta-
tes are defined according to the disea-
se being studied. The model’s structure 
should be as simple as possible, as it is 
not necessary to model the full com-
plexity of the disease. Moreover, since 
the CEA focuses on incremental costs 
and QALY, only those states in which 
the technology under assessment is ma-
king real differences are relevant for the 
model. Model complexity (i.e. number 
of parameters and model order) should 
always be limited by the quantity and 
quality of data available (51). However, 
more aggregated structure that still 
includes fundamental disease process 
and interventions is often the best so-
lution (47).
3.  Case study on 
electrochemotherapy (ECT)
Electroporation is an evolving tech-
nique with many applications (1-4). 
In our study the focus is on electro-
chemotherapy of basal-cell carcinoma 
(BCC) and skin melanoma, as these two 
therapies are used for the longest time 
and have a very high success rate. Few 
cost-effectiveness analyses have already 
been done (52,53), but due the lack of 
information obtained, especially about 
QoL increase, the results are incomp-
lete, inadequate and/or inaccurate. 
Numerical data about QoL increase are 
not available, QoL is reported only as 
”better”, ”highly improved”, ”significan-
tly better”, or using similar descriptors. 
Most probably the main reason that 
QoL was not acquired in previous stu-
dies is due to poor reporting. Our study 
originates from a previous one (52) and 
uses innovative eCEA methods (54). 
In order to facilitate result compari-
sons with benchmark technologies, 
cancer type specific analyses are done. 
For the scope of this paper, models for 
30 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
both studied cancer types are designed 
and presented. In addition, required 
and missing data are defined and pre-
sented (Table 1). In the near future we 
plan to collect all missing data trough 
randomized control trials and different 
studies which are still under way. With 
this paper we also want to raise gene-
ral awareness about the importance of 
numeric QoL reporting and useful-
ness/meaning of EQ-5D questionnaires 
that might not be self-evident at first 
sight. If despite our effort the data will 
still not be available, simulations will be 
done to acquire the missing data. Our 
main aim is to build a general model 
that will allow us – with minimal chan-
ges – to also simulate/project the cost 
efficiency for patients with better initial 
survival options and to also stimulate 
similar studies for other cancer types 
treated with ECT.
Cancer patients that are treated with 
innovative technologies (which have not 
yet been introduced into clinical pra-
ctice) in most cases have severe cancers 
and are not suitable for any other alre-
ady established treatment. In fact they 
are usually without other treatment 
options. However, the goal of several 
emerging applications in oncology, such 
as ECT, is to be used in early-diagnosed 
patients with better survival options. 
Measurement of QoL increase in pati-
ents with severe cancer is more deman-
ding and biased. QoL of a patient with 
severe cancer and poor survival opti-
ons after the therapy cannot be equally 
compared with the QoL of patients, with 
better survival options before the pro-
cedure. Their quality of life before tre-
atment was not the same, therefore we 
can assume that only a minimum incre-
ase in the quality of life could be obser-
ved (56).
Regarding the benchmark, there are 
different options. Some authors compare 
the innovative technology with the most 
recent cost-effective technology while 
others compare it with the gold standard/
benchmark in order to make the analysis 
more replicable. For skin cancer, compe-
titive technologies are: surgical excision, 
topical intralesional therapy, photoche-
motherapy and radiotherapy (14,55). In 
our study, the ECT treatment of BCC 
and skin melanoma will be compared 
to surgery and radiotherapy. We assume 
ECT is considerably cheaper than radi-
otherapy while having at least compara-
ble effect (10,14,52).
3.1.  Constructing the model
While the structure of the mo-
del should reflect the essential featu-
res of the disease and its interventi-
ons irrespective of data availability, 
it is expected that in some cases data 
availability may affect choices regar-
ding model structure (29). In order to 
Figure 2: A general eight state Markov model of skin melanoma. The 
model should always reflect the essential features of the disease and its 
outcomes (40).
study design of a medical device pre-market assessment 31
Pregledni znanstveni članek
Figure 3: A reduced – initial Markov model for skin melanoma, only 
the states that are relevant for the new technology, in this case ECT, are 
included.
use the most appropriate models that 
would give the most realistic results, 
models are designed in advance and 
the data needed will be obtained sub-
sequently in the near future (56). Two 
general models, for each cancer type, 
are presented in this paper (Figure 2, 
4), covering whole complexity of the 
disease and its essential features. Those 
two models will be used for final simu-
lations that could provide the cost-e-
ffectiveness evaluation of electroche-
motherapy as a treatment also used in 
patients with less severe cancer stages. 
For the initial analysis, however, models 
are simplified/reduced (e.g. aggregating 
states) (Figure 3, 5), in order to faci-
litate the calculations and focus only 
on states that are relevant for the new 
technology (i.e. where ECT is currently 
used). Models for different cancer types 
differ from each other with respect to 
the number of states; their definition 
and flow through the model are con-
ditioned by disease characteristics and 
possible outcomes.
Because cancer may be a recurrent 
disease, Markov model approach is 
used for modeling (57). The first gene-
ral model was made for BCC (Figure 4). 
It is simple, it has only seven states and 
much less possible state transitions 
than the second general model that was 
made for skin melanoma (Figure 2) 
which has nine states. The initial states 
in both general models represent popu-
lation without cancer. (These two states 
will be eliminated at the initial calcula-
tions, because they are not relevant for 
the new technology - ECT.) The fol-
lowing states represent different cancer 
stages, such as primary tumor, distant 
metastases, metastases in the lymph 
nodes, distant metastases with predo-
minantly cutaneous symptoms and di-
stant metastases without predominan-
tly cutaneous symptoms in case of skin 
melanoma. Because all patients are not 
diagnosed with cancer at its initial sta-
te, some might enter the model with the 
disease that combines several states in 
the model. Clinical stages of skin mela-
noma are defined as follows: at stages I 
and II patients only have a primary tu-
mor; stage IIIa,b means that a patient 
has a primary tumor and metastases in 
the lymph nodes; stage IIIc combines 
a primary tumor, local metastases and 
metastases with predominantly cutane-
ous symptoms. The last and most severe 
stage IV includes a primary tumor and 
distant metastases (76).
Electrochemotherapy is at the time 
of writing manly used in stage IIIc and 
IV patients (10,14), therefore a reduced 
- initial models for initial calculations 
consist only of states relevant for those 
two cancer stages. A reduced BCC model 
is slightly different and simpler. There are 
only three possible outcomes: complete 
response, residual disease and progressi-
ve disease. In the case of BCC, electro-
32 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
chemotherapy is used only in patients 
with bigger and recurrent tumors (10,14). 
Both general models also have the same 
two absorbing states, one is death due to 
cancer and the other is death from other 
causes; for initial calculations these two 
states will be merged into one state (pa-
tients can move to these two states from 
any other state, arrows are not drawn 
in the models for clarity (Figure 2, 4)). 
The probability of staying in dead state 
is always one and the cost zero. As seen 
from the model, all states are numera-
ted and have two additional parameters: 
QALY and cost. The time step at which 
patients migrate between states is set in 
accordance with disease characteristi-
cs (58). Because significant changes in 
QoL of patients treated with electroche-
motherapy are detected already within 
few months after the treatment (34,59), 
the time step for the study is set to three 
months. In all models (Figure 2–5) num-
bers in the circles enumerate states, for 
example the state without cancer in both 
general models designates the state 1. The 
letter P in both general models (Figure 2, 
4) represents a probability of transition 
between two states. The first undersigned 
number represents the state from which 
a transition is made and the second un-
dersigned number provides the informa-
tion to which state it is moving. Finally, 
in order to correctly include data in the 
model, a conditioned probability calcu-
lation will be performed (58).
3.2.  Data collection approach
For each state represented in the mo-
del, three data are needed for a success-
ful cost-effectiveness analysis:
1. The probability of transition between 
different states in a defined time step 
(ExampleBCC g. model: P23 represents the 
probability of a complete response 
three months after the treatment.);
2. The cost of staying for three months 
in each state (ExampleBCC g. model: 
cost2 represents the sum of all expen-
ses a health care provider has in the 
selected time step, such as medical 
examinations, medications, therapies, 
etc. in case patient has an untreated 
BCC);
3. The utility of patients in each 
state, expressed in QALY 
(ExampleBCC g. model: QALY3 represents 
the average result of an EQ-5D qu-
estionnaire filled out by the patients 
with complete response).
For the initial calculations, initial mo-
dels that reflect only the impact of the 
technology under investigation (ECT) 
will be used. In case any of above listed 
information is not available, those data 
will have to be estimated using the best 
possible approach, including:
1. The transition probability between 
states or QoL data can be adapted 
from randomized control trials te-
Figure 4: A seven state general Markov model of basal-cell carcinoma 
(BCC).
study design of a medical device pre-market assessment 33
Pregledni znanstveni članek
sting the most similar technology 
in the most possible similar cancer 
(Figure 6: Case 1 and 2). The missing 
values can be estimated by fitting 
probabilistic distribution and then 
calculating mean and standard devi-
ation, which are also useful for sensi-
tivity analysis that will be performed 
later.
2. Alternatively, the missing data can be 
derived from interviews with expert 
clinicians and fitting the results with 
statistical distribution (Figure 6: 
Case 3).
3. The missing data for a general model 
can be simulated or predicted from 
initial calculations, made on reduced 
models that include only states rele-
vant for the new technology. (The si-
mulation will also provide the CEA 
estimation in case ECT would be 
used in early-diagnosed patients with 
greater survival options.)
Once the data for initial models will 
be collected, an incremental analysis will 
be run in order to determine the incre-
mental cost-effectiveness ratio and its 
confidence interval. Finally, two steps 
will conclude the first stage of this study:
1. Sensitivity analysis (the model will be 
run using a deterministic approach 
by changing the input parameter va-
lue)
2. Statistical analysis (i.e. using Monte 
Carlo simulation, each subject will 
move from one state to another 
according to the given transition pro-
bability.)
3.3.  Field of interest
1. Probability of transition between 
states: Response rates for initial calcula-
tions / probabilities for the electroche-
motherapy treatments and other com-
parable technologies will be collected at 
first from different oncological trials in 
Slovenia. In case these data will have a 
high standard deviation, or there will not 
be enough of them for a representative 
pattern, we will expand our study requ-
esting data and support from InspECT 
database (60). (Probability values will be 
defined from study reports; the use of a 
table (Table1) is suggested for easier data 
collection). Finally, we would also like to 
make some possible outcome predicti-
ons, in case that the technique will/wou-
ld also be used in patients with earlier 
cancer stages and better survival options; 
for these purposes general models were 
developed. Cancer incidence data that 
will define the probability P12 (in both 
general models) for skin melanoma will 
be obtained from the Cancer Registry of 
the Republic of Slovenia (61). In case of 
BCC, the incidence should be multiplied 
by 1.3, because non-melanoma skin can-
cers are usually not properly included in 
the cancer registries (62). Also, the fact 
that the incidence of BCC continues to 
increase should be taken into account in 
general calculations.
Figure 5: A reduced – initial Markov model for BCC, only the states that 
are relevant for the new technology, in this case ECT, are included.
34 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
Because most databases are made on 
an annual basis, the collected data will 
be projected on a three-month scale, if 
needed. Other probabilities will be si-
mulated from initial calculations and 
expert’s opinions. In case of skin me-
lanoma, patients can be diagnosed at 
different cancer stages or might be in a 
state that is a combination of two sta-
tes in the model; for example, they can 
have a primary tumor and distant me-
tastases with predominantly cutaneous 
symptoms. In order to properly fit tho-
se probabilities in the model, statistical 
methods will be used.
2. Quality of life evaluation: alre-
ady after a brief literature survey, it has 
become obvious that most of the re-
levant reports are missing data about 
EQ-5D results even though these shou-
ld be collected as stated at clinical trial 
specifications (63,64). The quality of life 
is reported only as ”better”, ”highly im-
proved”, ”significantly better”, or using 
similar descriptors (65,66,71,72). For the 
Markov model analysis, however, nume-
ric data are required. This is one of the 
main reasons why we have decided to 
postpone the analysis, as we want to co-
llect real data instead of implementing 
expert’s estimations of QALY increase. It 
seems more rational to collect EQ-5D re-
sults, as only numerical values will pro-
vide us with adequate/useful results. At 
first we will try to collect the test results 
from already completed clinical trials. 
Most likely, if EQ-D5 were done, the re-
sults were collected at 3 and 8 months af-
ter the treatment, as specified in clinical 
trial documentation, so the rescaling on 
a time step basis will be necessary to fit 
into the model. Then we will ask physici-
ans who are involved in trials including 
electroporation treatments to include 
the EQ-5D evaluation tool into their stu-
dies if they are not already included, and 
use the developed table template (Table 
1) for reporting.
3. Incremental cost evaluation: co-
sts depend on the specific application 
protocol, disease stages and severity. 
The first necessary and inevitable cost, 
which is common to all applications, 
is staff (e.g., nurses, doctors). Local or 
general anesthesia is also used in some 
procedures. In case of electrochemothe-
rapy, the cost of intratumoral or intra-
venous application of chemotherapeu-
tics cistplatinum or bleomycin must be 
considered. The cost of patient’s hospita-
lization, the necessary medical examina-
tions before the procedure and any other 
specific treatment that may entail side 
effects must also be taken into account.
The second necessary cost is the 
electroporator and electrodes. IGEA 
S.p.A. (Italy) produces a Cliniporator 
device, appropriate for electrochemothe-
rapy of smaller cutaneous or subcutane-
ous tumors. Cliniporator VITAE is an 
upgrade, which has higher pulse ampli-
tude and can also be used for electro-
chemotherapy of deep-seeded tumors 
or irreversible electroporation (IRE). 
Figure 6: Visualization of data collecting and data simulating process.
study design of a medical device pre-market assessment 35
Pregledni znanstveni članek
In the USA, AngioDynamics produces 
NanoKnife, which is mainly used for 
IRE. For successful application, electro-
des are needed that may be for single or 
multiple use. IGEA offers many different 
electrode types. NanoKnife has only ne-
edle electrodes; from one up to six per 
treatment can be used. The amortiza-
tion expense or annual lease of the de-
vice must also be considered. The cost 
of electrochemotherapy has already 
been evaluated for Italy by Colombo et. 
al. (52), a cost-effectiveness analysis has 
also been done, but without the quality 
of life consideration. All the collected 
data thus must be rechecked and upda-
ted for further evaluations. The authors 
calculated an average use of 1.3 electro-
de per procedure. In a further study, 
the number of available electrode types 
may significantly change because there 
are several new electrode configurations 
available. Maintenance was estimated 
biannually at €6,000 and annual lease 
of the device at that time was €12,000. 
The device lifespan is 8 years, and for all 
further calculations Colombo et. al. (52) 
assumed that at least 100 patients are tre-
ated per year with a single device, which 
may be an overly optimistic assumption. 
The cost is then calculated per patient. 
Electroporator and electrode price must 
be divided by its lifespan and the num-
ber of patients treated on a yearly basis. 
The cost of being in each state needs to 
be defined. The cost evaluation of being 
in initial state (having cancer) will be ta-
ken from the literature; for the USA, UK 
estimations of specific cancer types on a 
yearly basis are available (67,68). In case 
of expanding the data collection to other 
countries, we should consider the issue 
of different living standards, as the stan-
dard of living and money value is not the 
same everywhere. To align the costs, data 
collected from all around the world will 
be calibrated if needed. The adjustment 
will be made on the basis of living co-
sts (74).
3.4.  Data collection proposal
To facilitate initial data collection, a 
table template (Table 1) is proposed. The 
table refers to reduced models; in case of 
BCC there are five possible states and in 
case of skin melanoma only three. For 
each patient, every three months the cli-
nician is supposed to:
• define the medical interventions;
• assess patient state by circling the 
appropriate number in the table;
• obtain EQ-5D questionnaire and re-
port numerical value;
• record all medical interventions or 
examinations that patient had in 
the last period, including the type of 
anesthesia during ECT, days spend in 
hospital, biopsy, blood analysis and 
radiological assessment;
• report about all drugs prescribed to 
the patient for his/her condition, also 
for the management of side effects, 
analgesics, wound care dressing…
• report any other potential costs.
4.  Conclusion
In the scope of this paper two gene-
ral Markov models for CEA have been 
developed, one for skin melanoma and 
the other for BCC. Initially, both mo-
dels are reduced in such a manner that 
they consists only of states that are re-
levant for the current use of new tech-
nology, i.e. electrochemotherapy. But for 
a cost-effectiveness analysis, the data on 
quality of life increase on a time step 
scale are also needed, which in our case 
represents a problem. Researchers are 
currently only reporting that quality of 
life is increased after the treatment, but 
36 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
the numerical data that are crucial for 
successful cost-effectiveness analysis 
are missing (65,66,71,72). In this paper, 
the data needed for the evaluation of 
cost-effectiveness of electrochemothe-
rapy treatment of skin melanoma and 
BCC are clearly identified. Additionally, 
recommendations for data collection 
process and follow-up reporting are 
made.
Because electrochemotherapy is an 
emerging technology, it is still not used 
in all patients, but as the proof of con-
cept rises, it is gradually moving up the 
cancer staging scale. A realistic cost-e-
ffectiveness analysis for a specific cancer 
type will facilitate equipment purcha-
se and clinical practice implementati-
on. Additionally, the prediction of cost 
effectiveness can also influence the next 
round of fund raising. In our opinion, an 
additional incentive from a good cost-
-effectiveness analysis would definitely 
also benefit other electroporation-based 
therapies. The biggest obstacle is Quality 
of life data, which will be overcome by 
obtaining the EQ-5D questionnaires 
in the near future. Because the time 
step is set to three months, we believe a 
two-year data will give a representative 
pattern. Furthermore, specific models 
should be developed for each disease 
separately in order to include all speci-
fic phenomena in a specific cancer type. 
Electrochemotherapy must be compa-
red to well established procedures that 
are cancer-type, location and size depen-
dent.
Table 1: A table template proposed for the initial data collection.
patient code: 
0001/2017
Pre-operative 
evaluation
Procedure Follow-up at 3 months Follow-up at 6 months
BCC/ skin melanoma
Medical interventions 1 visit at the 
specialist
Ect in general 
anesthesia, chest 
x-ray
2 visits at the 
specialist
3 visits at the 
specialist
state (reduced model) 1 , 2 , 3 , 4 , 5 1 , 2 , 3 , 4 , 5 1 , 2 , 3 , 4 , 5 1 , 2 , 3 , 4 , 5
eQ_5d 0.45 0.43 0.38 0.33
Biopsy Y / n Y / n Y / n Y / n
Blood analysis Y / n Y / n Y / n Y / n
radiological assessment no / us / Ct / Mr /
pet-Ct
no / us / Ct / Mr /
pet-Ct
no / us / Ct / Mr /
pet-Ct
no / us / Ct / Mr /
pet-Ct
days of hospitalization 0 2 0 2
Wound care dressing
(product name)
NO NO YES: XY© NO
prescribed drugs NO Paracetamol 
500 mg tablets 
3 X 1
NO NO
other potential costs
(describe)
NO NO NO NO
study design of a medical device pre-market assessment 37
Pregledni znanstveni članek
5.  Acknowledgment
This study was supported by COST 
Action BM1309 (COST EMF-MED) 
within the STSM (ECOST-STSM-
BM1309–110416–073660) and the 
Slovenian Research Agency (ARRS), 
conducted within the scope of the 
European Associated Laboratory on the 
Electroporation in Biology and Medicine 
(LEA- EBAM)
References
1. Haberl s, Miklavcic d, sersa g, Frey W, rubinsky B. Cell membrane electroporation-Part 2: the applications. 
ieee electrical insulation Magazine. 2013;29(1):29–37.
2. kotnik t, Frey W, sack M, Haberl Meglič s, Peterka M, Miklavčič d. electroporation-based applications in 
biotechnology. trends in Biotechnology. 2015;33(8):480–8.
3. zorec B, Préat v, Miklavčič d, Pavšelj n. active enhancement methods for intra- and transdermal drug deli-
very: a review. zdrav vestn. 2013;82(5):339–356.
4. golberg a, sack M, teissie J, Pataro g, Pliquett U, saulis g, et al. energy-efficient biomass processing with 
pulsed electric fields for bioeconomy and sustainable development. Biotechnology for Biofuels. 2016;9(1).
5. Pirc e, reberšek M, Miklavčič d. 12 dosimetry in electroporation-Based technologies and treatments. dosi-
metry in Bioelectromagnetics: CrC Press; 2017. p. 233–68.
6. stepišnik t, Jarm t, grošelj a, edhemović i, djokić M, ivanecz a, et al. elektrokemoterapija – učinkovita 
metoda zdravljenja tumorjev s kombinacijo kemoterapevtika in električnega polja. zdrav vestn. 2016;85(1).
7. Mali B, Jarm t, snoj M, sersa g, Miklavcic d. antitumor effectiveness of electrochemotherapy: a systematic 
review and meta-analysis. european Journal of surgical Oncology (eJsO). 2013;39(1):4–16.
8. narayanan dL, saladi rn, Fox JL. review: ultraviolet radiation and skin cancer. international Journal of 
dermatology. 2010;49(9):978–86.
9. telfer nr, Colver GB, Morton Ca. Guidelines for the management of basal cell carcinoma. British Journal of 
dermatology. 2008;159(1):35–48.
10. Clover aJp, o’Mahony J, soden d. electrochemotherapy of Basal Cell Carcinoma. Handbook of electropo-
ration: springer international Publishing; 2016. p. 1–12.
11. niCe. electrochemotherapy for primary basal cell carcinoma and primary squamous cell carcinoma. [Mar-
ch 22, 2017] available from: https://www.nice.org.uk/guidance/ipg478/resources/electrochemotherapy-
-for-primary-basal-cell-carcinoma-and-primary-squamous-cell-carcinoma-1899869938127557.
12. siegel rl, Miller kd, Jemal a. Cancer statistics, 2016. Ca: a cancer journal for clinicians. 2016;66(1):7–30.
13. savoia p, Fava p, nardò t, osella-abate s, Quaglino p, Bernengo MG. skin metastases of malignant melano-
ma: a clinical and prognostic survey. Melanoma research. 2009;19(5):321–6.
14. (14) Campana LG, sepulcri M, Valpione s, Corti L, rossi Cr. electrochemotherapy for superficially Metastatic 
Melanoma. Handbook of electroporation: springer international Publishing; 2017. p. 2095–112.
15. Quaglino p, Mortera C, osella-abate s, Barberis M, illengo M, rissone M, et al. electrochemotherapy with 
intravenous Bleomycin in the Local treatment of skin Melanoma Metastases. annals of surgical oncology. 
2008;15(8):2215–22.
16. Campana LG, Mocellin s, Basso M, puccetti o, de salvo GL, Chiarion-sileni V, et al. Bleomycin-Based 
electrochemotherapy: Clinical Outcome from a single institution’s experience with 52 Patients. annals of 
surgical Oncology. 2009;16(1):191–9.
17. Campana LG, Valpione s, Mocellin s, sundararajan r, Granziera e, sartore L, et al. electrochemotherapy for 
disseminated superficial metastases from malignant melanoma. British Journal of surgery. 2012;99(6):821–
30.
18. van tulder M. Health technology assessment (Hta) increasingly important in spine research. european spi-
ne Journal. 2011;20(7):999–1000
19. the organisation for economic Co-operation and development (oeCd)“Health resources - Health spen-
ding - OeCd data.” theOeCd. [March 22, 2017] available from: http://data.oecd.org/healthres/health-spen-
ding.htm.
20. pallikarakis n, Moore r. Health technology in europe - regulatory Framework and industry perspectives of 
the »new approach«. ieee engineering in Medicine and Biology Magazine. 2007;26(3):14–7.
21. analiza zdravstavenega sistema v sloveniji; Pregled izdatkov v zdravstvu; končno poročilo, oktober 
2015. [ June 30, 2016] available from: http://www.mz.gov.si/fileadmin/mz.gov.si/pageuploads/anali-
za/04022016porocila sl/report expenditure review slovenia Final FOrMatted si 4.pdf.
38 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
22. statistični urad republike slovenije- starejše prebivalstvo. [March 22, 2017] available from: http://www.stat.
si/doc/starejseprebivalstvo.pdf.
23. analiza zdravstvenega sistema v sloveniji Povzetek in ključne ugotovitve. [March 22,2017] available from: 
http://www.mz.gov.si/fileadmin/mz.gov.si/pageuploads/analiza/analiza_Zs_povzetek_in_kljucne_ugoto-
vitve_lektorirana_verzija.pdf.
24. Ministrstvo za zdravje rs. Postopek obravnave vlog za nove zdravstvene programme 2015. available from: 
http://www.mz.gov.si/si/o_ministrstvu/zdravstveni_svet_in_ostala_posvetovalna_telesa/zdravstveni_
svet/postopek_za_vloge/.
25. turchetti G, spadoni e, Geisler e. Health technology assessment. ieee engineering in Medicine and Biology 
Magazine. 2010;29(3):70–6.
26. World Health Organization [June 30, 2016]. available from: http://www.who. int/medical devices/as-
sessment/en/.
27. pecchia L, Craven Mp. early stage Health technology assessment (Hta) of biomedical devices. the MatCH 
experience. iFMBe Proceedings: springer Berlin Heidelberg; 2013. p. 1525–8.
28. thokala p, duenas a. Multiple Criteria decision analysis for Health technology assessment. Value in Health. 
2012;15(8):1172–81.
29. Clifford s, goodman. Hta 101 introduction to Health technology assessment. Bethesda,Md: national li-
brary of Medicine (Us); 2014. [July 4, 2016] available from: https://www.nlm.nih.gov/nichsr/ hta101/Hta 101 
Final 7–23–14.pdf.
30. Weinstein MC, torrance G, McGuire a. QaLYs: the Basics. Value in Health: the Journal of the international 
society for Pharmacoeconomics and Outcomes research 12 suppl 1:s5–9.
31. patrick dL, starks He, Cain KC, uhlmann rF, pearlman ra. Measuring preferences for Health states Worse 
than death. Medical decision Making. 1994;14(1):9–18.
32. euroQol official pag. [July 4, 2016] available from: http://www.euroqol.org/.
33. rabin r, Charro Fd. eQ-sd: a measure of health status from the euroQol Group. annals of Medicine. 
2001;33(5):337–43.
34. Weinstein MC, siegel Je, gold Mr, kamlet Ms, russell lB. recommendations of the Panel on Cost-effective-
ness in Health and Medicine. Jama. 1996;276(15):1253–8.
35. guide to the methods of technology appraisal 2013. [april 4, 2013] available from: https://www.nice.org.uk/
process/pmg9.
36. Phillips C, thompson g. What is a QalY Health economics 2nd ed. newmarket: Hayward Medical Commu-
nications; 2009.
37. Burckhardt Cs, anderson kl. Health and Quality of life Outcomes. 2003;1(1):60.
38. Hunt sM, McKenna sp, Mcewen J, Williams J, papp e. the nottingham health profile: subjective health sta-
tus and medical consultations. social science & Medicine part a: Medical psychology & Medical sociology. 
1981;15(3):221–9.
39. stavrianou K, pallikarakis n. Quality of life of end-stage renal disease patients and study on the implemen-
tation of nocturnal home hemodialysis in greece. Hemodialysis international. 2007;11(2):204–9.
40. longworth l, rowen d. Mapping to Obtain eQ-5d Utility values for Use in niCe Health technology as-
sessments. value in Health. 2013;16(1):202–10.
41. torrance GW, Feeny d. utilities and Quality-adjusted Life Years. international Journal of technology as-
sessment in Health Care. 1989;5(04):559–75.
42. Briggs aH, Wonderling de, Mooney Cz. Pulling cost-effectiveness analysis up by its bootstraps: a non-para-
metric approach to confidence interval estimation. Health economics. 1997;6(4):327–40.
43. Hill sr. Cost-effectiveness analysis for clinicians. BMC Medicine. 2012;10(1).
44. Fenwick e, Marshall da, levy ar, nichol g. Using and interpreting cost-effectiveness acceptability curves: 
an example using data from a trial of management strategies for atrial fibrillation. BMC Health services 
research. 2006;6(1).
45. O’Mahony JF, Paulden M. niCe’s selective application of differential discounting: ambiguous, inconsistent, 
and Unjustified. value in Health. 2014;17(5):493–6.
46. Claxton K, paulden M, Gravelle H, Brouwer W, Culyer aJ. discounting and decision making in the economic 
evaluation of health-care technologies. Health economics. 2010;20(1):2–15.
47. Weinstein MC, o'Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, et al. principles of Good 
practice for decision analytic Modeling in Health-Care evaluation: report of the ispor task Force on Good 
research Practices—Modeling studies. value in Health. 2003;6(1):9–17.
48. andronis L, Barton p, Bryan s. sensitivity analysis in economic evaluation: an audit of niCe current practice 
and a review of its use and value in decision-making. Health technology assessment. 2009;13(29).
study design of a medical device pre-market assessment 39
Pregledni znanstveni članek
49. rodina-theocharaki a, Bliznakova K, pallikarakis n. Markov Chain Monte Carlo simulation for projecti-
on of end stage renal disease patients in Greece. Computer Methods and programs in Biomedicine. 
2012;107(1):90–6.
50. Craven Mp, Morgan sp. early stage economic evaluation with a small Medical device start-up Company 
Using a Markov Model. Journal of Medical devices. 2011;5(2):027516.
51. Foster Kr, Koprowski r, skufca Jd. Machine learning, medical diagnosis, and biomedical engineering rese-
arch - commentary. BioMedical engineering Online. 2014;13(1):94.
52. Colombo g. Cost-effectiveness analysis of electrochemotherapy with the Cliniporator™ vs other methods 
for the control and treatment of cutaneous and subcutaneous tumors. therapeutics and Clinical risk Ma-
nagement. 2008;4:541–8.
53. CadtH rapid response reports. irreversible electroporation for tumors of the pancreas or Liver: a review 
of Clinical and Cost-effectiveness. Ottawa (On): Canadian agency for drugs and technologies in Health; 
2016.
54. pecchia L, Craven Mp. early stage Health technology assessment (Hta) of biomedical devices. the MatCH 
experience. iFMBe Proceedings: springer Berlin Heidelberg; 2013. p. 1525–8.
55. spratt de, gordon spratt ea, Wu s, derosa a, lee nY, lacouture Me, et al. efficacy of skin-directed the-
rapy for Cutaneous Metastases From advanced Cancer: a Meta-analysis. Journal of Clinical oncology. 
2014;32(28):3144–55.
56. Briggs aH. Handling Uncertainty in Cost-effectiveness Models. Pharmacoeconomics. 2000;17(5):479–500.
57. komorowski M, raffa J. Markov Models and Cost effectiveness analysis: applications in Medical research. 
secondary analysis of electronic Health records: springer international Publishing; 2016. p. 351–67.
58. o’Mahony JF, newall at, van rosmalen J. dealing with time in Health economic evaluation: Methodologi-
cal issues and recommendations for Practice. Pharmacoeconomics. 2015;33(12):1255–68.
59. Bianchi G, Campanacci L, ronchetti M, donati d. electrochemotherapy in the treatment of Bone Metasta-
ses: a Phase ii trial. World Journal of surgery. 2016;40(12):3088–94.
60. Brizio M, ribero s, Campana LG, Clover aJp, Gehl J, Kunte C, et al. international network for sharing pra-
ctices on electrochemotherapy (inspeCt): an integrative patients treatment Consortium. Handbook of 
electroporation: springer international Publishing; 2016. p. 1–18.
61. the Cancer registry of republic of slovenia. available from: http://www.onko-i.si/rrs/.
62. trakatelli M, Morton C, nagore e, ulrich C, del Marmol V, peris K. update of the european guidelines for 
basal cell carcinoma management. european Journal of dermatology 2014;24(3):312–29.
63. Marty M, sersa g, garbay Jr, gehl J, Collins Cg, snoj M, et al. electrochemotherapy – an easy, highly effecti-
ve and safe treatment of cutaneous and subcutaneous metastases: results of esope (european stan-
dard operating procedures of electrochemotherapy) study. european Journal of Cancer supplements. 
2006;4(11):3–13.
64. Campana LG, Clover aJ, Valpione s, Quaglino p, Gehl J, Kunte C, et al. recommendations for improving the 
quality of reporting clinical electrochemotherapy studies based on qualitative systematic review. radio-
logy and oncology. 2016;50(1):1–13.
65. Larkin Jo, Collins CG, aarons s, tangney M, Whelan M, o??reily s, et al. electrochemotherapy. annals of 
surgery. 2007;245(3):469–79.
66. sersa G, Cufer t, paulin sM, Cemazar M, snoj M. electrochemotherapy of chest wall breast cancer recurren-
ce. Cancer treatment reviews. 2012;38(5):379–86.
67. Mariotto aB, robin Yabroff k, shao Y, Feuer eJ, Brown Ml. Projections of the Cost of Cancer Care in the Uni-
ted states: 2010–2020. JnCi Journal of the national Cancer institute. 2011;103(2):117–28.
68. national Cancer institute, Cancer prevalence and Cost of Care projec- tions. available from: https://costpro-
jections.cancer.gov/.
69. david Parkin. techniques of economic appraisal (including Cost- effectiveness analysis and Modelling, 
Cost-utility analysis, option ap- praisal and Cost-Benefit analysis, the Measurement of Health Benefits in 
terms of QalYs and related Measures e.g. dalYs). Health economics 4d. available from: https://www.he-
althknowledge.org.uk/public-health-textbook/medical-sociology-policy-economics/4d-health-economi-
cs/economic-appraisal.
70. Black WC. the Ce Plane. Medical decision Making. 1990;10(3):212–4.
71. Lido p, paolino G, Feliziani a, santurro L, Montuori M, sanctis Fd, et al. Cutaneous metastasis of unknown 
primary presenting as massive and invasive abdominal lesion: an elective approach. anais Brasileiros de 
dermatologia. 2015;90(6):879–82.
72. ribero s, Balagna e, sportoletti Baduel e, Picciotto F, sanlorenzo M, Fierro Mt, et al. efficacy of electroche-
motherapy for eruptive legs keratoacanthomas. dermatologic therapy. 2016;29(5):345–8.
40 Zdrav Vestn | januar – februar 2018 | Letnik 87
JaVno ZdraVstVo (VarstVo pri deLu)
73. refaat t, Choi M, gaber g, kiel k, Mehta M, gradishar W, et al. Markov Model and Cost-effectiveness analysis 
of Bevacizumab in Her2-negative Metastatic Breast Cancer. american Journal of Clinical Oncology. 
2014;37(5):480–5.
74. Cost of living index. [July 4, 2016] available from: https://www.expatistan.com/cost-of-living/index/europe.
75. Bertino G, sersa G, de terlizzi F, occhini a, plaschke CC, Groselj a, et al. european research on electroche-
motherapy in Head and neck Cancer (eureCa) project: results of the treatment of skin cancer. european 
Journal of Cancer. 2016;63:41–52.
76. amin MB, edge sB, Greene FL, Byrd dr, Brookland rK, Washington MK, et al. organization of the aJCC Can-
cer staging Manual. aJCC Cancer staging Manual: springer international Publishing; 2016. p. 31–7.
77. pecchia L, Bracale u, Bracale M. Health technology assessment of Home Monitoring for the Continuity 
of Care of patient suffering from congestive heart failure. iFMBe Proceedings: springer Berlin Heidelberg; 
2009. p. 184–7.
78. ijzerman MJ, koffijberg H, Fenwick e, krahn M. emerging Use of early Health technology assessment in Me-
dical Product development: a scoping review of the literature. Pharmacoeconomics. 2017;35(7):727–40.
79. Fasterholdt i, Krahn M, Kidholm K, Yderstræde KB, pedersen KM. review of early assessment models of 
innovative medical technologies. Health Policy. 2017;121(8):870–9.