42Gac Sanit. 2006;20(Supl 3):42-51
Accuracy of cancer death certificates in Spain: 
a summary of available information
Beatriz Pérez-Gómeza / Nuria Aragonésa / Marina Pollána / Berta Suáreza / Virginia Lopea / Alicia Llácerb / 
Gonzalo López-Abentea
aÁrea de Epidemiología Ambiental y Cáncer. Centro Nacional de Epidemiología. Instituto de Salud Carlos III. Madrid. España.
bÁrea de Análisis Epidemiológico y Situación de Salud. Centro Nacional de Epidemiología. Instituto de Salud Carlos III. 
Madrid. España.
(Validez de la certificación de la muerte por cáncer 
en España: resumen de la evidencia disponible)
Correspondencia: Beatriz Pérez-Gómez.
Área de Epidemiología Ambiental y Cáncer.
Centro Nacional de Epidemiología.
Instituto de Salud Carlos III.
Sinesio Delgado, 6. 28029 Madrid. España.
Correo electrónico: bperez@isciii.es
Recibido: 25 de noviembre de 2005.
Aceptado: 22 de febrero de 2006.
Abstract
Objectives: Differences in mortality rates within Europe might
be partly due to the quality of mortality statistics. The present
article summarizes the available data on the quality of can-
cer death certification in Spain. A short description of the tem-
poral distribution of the proportion of deaths due to ill-defined
tumors in Spain –an indirect indicator of the quality of cancer
death certification– is also provided.
Methods: Relevant studies were identified from electronic da-
tabases (MEDLINE, EMBASE, IME and IBECS) and from ma-
nual searches of the references contained in the articles re-
trieved. Quality data on death certificates for all tumors and
for each specific cancer location were summarized, and all main
cancer sites were classified according to their pooled accu-
racy indicators. Trends for the percentage of deaths due to ill-
defined tumors and conditions were studied for the period from
1980 to 2002.
Results: In Spain, deaths from cancer as a whole and lea-
ding cancer sites (lung, colon-rectum, prostate, stomach, pan-
creas, female breast, uterus, brain, leukemia, lymphomas and
myeloma) were well-certified. However, other frequent loca-
tions, such as the larynx, esophagus and liver were overcer-
tified, while deaths from bladder, kidney and ovarian cancer
were undercertified. The percentage of deaths due to ill-de-
fined tumors and causes was regularly higher in females and
decreased in both sexes during the study period. However,
the recent introduction of the International Classification of Di-
seases (ICD)-10 has reversed this trend.
Conclusions: Spanish death certificates can be considered
as accurate and useful to estimate the burden of cancer, though
certification of some frequent sites should be improved. The
possible effect of the introduction of the ICD-10 requires ca-
reful surveillance.
Key words: Death certificates. Mortality. Cancer. Quality con-
trol. Spain.
Resumen
Objetivos: Parte de las diferencias en tasas de mortalidad
por cáncer entre países europeos podrían deberse a diferencias
de calidad en las estadísticas de mortalidad. Nuestro objeti-
vo es sintetizar la información cuantitativa que hay acerca de
la calidad de los certificados de defunción de cáncer en Es-
paña, y se añade una somera descripción de la evolución tem-
poral de la proporción de defunciones por tumores mal defi-
nidos, indicador indirecto de calidad.
Métodos: Se identificaron los estudios relevantes mediante
búsquedas en bases de datos electrónicas (MEDLINE, IME,
EMBASE e IBECS), y posteriormente se añadieron referen-
cias presentes en los artículos encontrados. Se extrajo la in-
formación acerca de calidad de certificación para cáncer en
conjunto y para las principales localizaciones tumorales, y se
clasificaron los tumores según sus indicadores de calidad. Se
estudió también la tendencia del porcentaje de muertes mal
definidas o tumores mal definidos entre 1980-2002.
Resultados: En España, el cáncer en conjunto y las princi-
pales localizaciones –pulmón, colon-recto, próstata, estóma-
go, páncreas, mama, útero, cerebro, leucemia, linfomas y mie-
loma– están bien certificados. Sin embargo, otras localizaciones
como laringe, hígado y esófago están sobrecertificadas, mien-
tras que el cáncer de vejiga, riñón y ovario están infracertifi-
cados. Los porcentajes de muertes por tumores o condicio-
nes mal definidas, mayores en mujeres, han disminuido en
el período estudiado, aunque la introducción de la CIE-10 ha
invertido esta tendencia.
Conclusiones: En general, los certificados de cáncer pue-
den considerarse válidos y útiles para estimar el impacto del
cáncer en España, aunque la certificación de algunas locali-
zaciones importantes tendría que mejorar. Debería estudiar-
se el posible efecto de la introducción de la CIE-10.
Palabras clave: Certificados de defunción. Mortalidad. Cán-
cer. Control de calidad. España.
Background
One of the most usual approaches to studying the
situation of cancer worldwide is to analyse the geographic
distribution of mortality rates and their trends. Informa-
tion on the quality of cancer mortality data is thus es-
sential for interpreting differences in mortality statistics.
In Spain, mortality represents the only comprehen-
sive and homogeneous source of information on can-
cer for the whole country. The source of mortality sta-
tistics is the medical death certificate (DC), a compulsory
administrative document completed by the practitioner
who certifies the death. This certificate is subsequently
transcribed onto a second document, the Statistical Bu-
lletin of Death (SBD), and both are sent to the Munici-
pal Civil Registry. Usually, the Civil Registry forwards
the SBDs to the regional offices of the National Statis-
tic Institute (Instituto Nacional de Estadística [INE]) on
a monthly basis, where all items except cause of death
are digitally recorded. When the data have been duly
screened to detect errors and ensure quality control, the
underlying causes of death (Causa básica de defunción)
are coded at the Regional Authority Mortality Registries
by trained teams, applying common criteria in accor-
dance with International Classification of Diseases (ICD)
guidelines. National coding protocols have been esta-
blished to guarantee homogeneity of data1, while spe-
cific methods have also been implemented to validate
accuracy systematically. Digital data files are then sent
to the INE head office, which releases them once they
have been rendered anonymous.
Quantitative data that would enable to asses the qua-
lity of cancer death certificates in Spain are relatively
scarce. Several studies have addressed this topic, but
all refer to areas of the country or to specific popula-
tions, and some have been published in local epide-
miological bulletins or in symposium proceedings,
which are not easily accessible. In this paper, we sought
to synthesize information derived from these studies and
to complement such data with a short description of the
temporal distribution of the proportion of deaths due to
ill-defined tumours in Spain, as an indirect indicator of
the quality of cancer death certification.
Methods
For review purposes, DC and SBD were deemed to
be death certificates, as a very high concordance bet-
ween both documents has been reported2. Studies into
the quality of cancer death certification in Spain were
identified through: 1) a MEDLINE and EMBASE search
using broad search criteria (January 1966 to January
2006); 2) a similar search in Spanish bibliographical da-
tabases, the Spanish Medical Index (Índice Médico Es-
pañol [IME]), and IBECS; and 3) references in identi-
fied papers. In two studies3,4, only abstracts of poster
presentations at scientific meetings were published. In
these cases, quality-indicator data presented in the pos-
ter were specifically searched for.
Studies were considered eligible if they reported
quantitative estimates of the accuracy of death certifi-
cates containing any mention of cancer. In these stu-
dies, cancer death certificates were compared against
a second source of information, mainly comprising cli-
nical or anatomo-pathological reports (which were
taken as the «gold standard»). Agreement between both
sources was measured using detection and confirma-
tion rates.
According to Percy et al5, the detection rate (DR) or
sensitivity for a specific site is defined as the propor-
tion of hospital diagnoses (available clinical/anatomo-
pathological information) with cancer of a certain site,
with a death certificate where this disease is conside-
red to be the basic cause of death, whereas the con-
firmation rate (CR) or positive predictive value is the pro-
portion of cancer deaths in which the underlying cause
specified in the death certificate is confirmed by hos-
pital diagnosis.
DRs and CRs can be computed as: a) site-speci-
fic cancer rates at three digits of the ICD, that is con-
sidering an indicator per cancer-site; b) all-site three-
digit rates, an overall indicator for cancer where the
rates’ numerator contains all cancer cases (ICD-9 codes
140-208) in which the site specified in the death cer-
tificate and clinical information agree; and c) all-tumour
rates, also an overall indicator where the rates’ nu-
merator includes all cases that just mention «cancer»
in both the death certificate and gold standard, even
though there might be site misclassification. These fi-
gures are logically expected to be higher than overall
comparisons at a three-digit level. A graphical sche-
me that might help understand these concepts is shown
in figure 1.
Identified papers were classified into three main ca-
tegories:
I. Studies focusing on all-cause death certificates,
which provide accurate detection and confirmation rates
(fig. 1).
II. Studies based solely on death certificates that
mention cancer. These could be subdivided into two
groups:
a. Studies with an additional source of informa-
tion on cg or bg (fig. 1) that allows them to esti-
mate accurate detection and confirmation rates.
In some cases, these data6 were not considered
for the purpose of calculating site-specific can-
cer rates, thereby leading to overestimated
rates.
43
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
b. Studies without information on cancer deaths
not certified as such, or on non-cancer cases erro-
neously certified as cancer (cg or bg). This rules
out the possibility of computing all-tumour or
three-digit agreement detection rates, and means
that site-specific detection rates and confirma-
tion rates are likely to be overestimated (fig. 1).
In some cases4,7,8, it was possible to find infor-
mation on global false positives within the text of
the paper. We used these data to calculate un-
biased all-tumour confirmation rates.
III. Necropsy-based studies. Since necropsies are
relatively scarce in this country9, the main problem with
these studies is external validity, as they tend to focus
on very specific populations.
Many of the detection and /or confirmation rates of-
fered in the summary tables were directly taken from
the selected studies, though, where possible, the tables
were completed by computing DRs and CRs using data
provided in the papers.
In a second stage, a pooled CR and DR was cal-
culated for each specific cancer location using data from
all studies that had covered that location, and then all
main cancer sites were classified according to Percy’s
criteria, the bench-mark for these types of studies5. Fi-
nally, as a complementary approach, we calculated the
percentage of deaths due to ill-defined tumour versus
all-tumour deaths and the percentage of deaths of ill-
defined conditions versus all-cause deaths for the pe-
riod 1980-2002, using whole country mortality figures
supplied by the INE.
Results
Published studies on quality of cancer death certificates in Spain
A total of 14 studies providing quantitative data on
quality of cancer death certification in Spain were found.
The main characteristics of these studies, their classi-
fication according to the above criteria and, where avai-
lable, all-tumour (ICD-9 codes 140-208) and all-site three-
digit detection and confirmation rates are shown in table
1. In addition, figure 2 depicts the geographical loca-
tion of the respective study populations.
Six of these studies belonged to categories I and
II(a)3,6,10-14, those furnishing the most accurate estima-
tors, with detection rates ranging from 75.2-100% for
all tumours as a whole and 64.8-100% for all-site three-
digit rates. On average, confirmation rates were higher
than detection rates, ranging from 91.5-99.3% for all-
tumour and 68.2-80.8% for all-site three-digit rates.
Another five studies were classified as category
II(b)7,8,15-18. In general, they supplied data on all-tumour
and all-site three-digit confirmation rates. Yet, with two
exceptions4,8, these studies failed to consider global false
positives when estimating three-digit confirmation rates,
an approach that implies a certain overestimation of the
agreement. Where possible, therefore, we calculated a
44
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
Site-specific detection rate = a / (a + cg + cs)
Site-specific confirmation rate = a / (a + bg + bs)
All-site three-digit detection rate =  ∑a / ∑(a+ cg + cs)
All-site three-digit confirmation rate =  ∑a /∑ (a + bg + bs)
All-tumour detection rate =  a + bs + cs / a + bs + cs + cg
All-tumour confirmation rate =  a + bs + cs / a + bs + cs + bg
= site-concordant cases
= global false positive (corresponds to a non-cancer cause of death)
= site false positive (corresponds to cancer at other location
= global false negative (cancer case certified outside ICD cancer codes)
= site false negative (cancer case certified as cancer at other location)
= true negative cases
a
bg
bs
cg
cs
d
DC/SBD
+
_
+
a
cg + cs
–
bg + bs
d
DC/SBD
+
_
+
a + bs + cs
cg
–
bg
d
Clinical/anatomo-pathological
information
Clinical/anatomo-pathological
information
a) Three-digit agreement rates b) All-tumour agreement rates
Figure 1. Cross-tabulation of cause of death according to clinical diagnoses and medical death certificates and quality-indicator
formulae.
corrected confirmation rate by including global false po-
sitives in the denominator. Only one study8 had adop-
ted this approach and, in this case, the «biased» con-
firmation rate (excluding global false positives) was
reckoned for comparison purposes. Both «biased» –de-
noted by «*»– and corrected confirmation rates are shown
in table 1. On average, biased confirmation rates were
three points higher than corrected rates. In these stu-
45
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
Euskadi (Basque Country) (Izarzurgaza, 1994)
Soria (Ruiz Liso, 1989)
Zaragoza (DGA, 1988)
Girona (Sánchez Garrido, 1996)
Terrasa (Nava, 1985, 1986)
Barcelona (Bosch, 1981, 1983; Pañella, 1989)
Mallorca (Cáffaro, 1995)
Valencia (García Benavides, 1986,1989)
Murcia (Navarro, 1984; Cirera, 2002)
Granada (Martínez, 2000)
Galicia (Carballeira, 1989)
Figure 2. Geographical location of published studies furnishing data on quality of cancer death certification in Spain.
Table 1A. Quality of cancer death certification in Spain. Characteristics and classification of published studies sorted by publication year
Death certificates 
Study category Author Geographic location Institution Period analysed or SBDa
All-cause/cancer
I
I
I
I
II(a)
II(a)
II(b)
II(b)
II(b)
II(b)
II(b)
II(b)
III
III
García-Benavides, 198610, 198911
Pañella, 198912
Ruiz Liso, 198913
Giménez, 20023
Cáffaro, 19956
Sánchez-Garrido, 199614
Bosch, 198115, 198316
Navarro, 198417
DGA, 198818
Izarzurgaza, 19944
Martínez, 20008
Cirera, 20027
Nava, 1985, 198619;35
Carballeira, 19899
Valencia (city)
Barcelona (city)
Soria (province)
a) National
b) National
Island of Majorca
Girona (province)
Barcelona (city)
Murcia (province)
Zaragoza (city)
Euskadi (Basque Country)
(region)
Granada (province)
Murcia (region)
Terrassa (city)
Galicia (region)
University of Alicante
Municipal Institute of Health
Provincial Hospital
CISATER
CISATER
Cancer Registry
Catalonian Institute of Oncology
Provincial Hospital
Cancer Registry
Hospital, Municipal Institute of Health
Public Health Authority, Cancer
Registry
Cancer Registry
Cancer Registry
Cancer Registry
Cancer Registry
Comarcal Private Hospital
Public Health Authority
1984
1985
1985
1981-1995
1987-1998
1989
1985-1989
1979
May 1981-
Oct 1983
1983
1989
1991-1994
1992
1980-1981
1987
1,068/279
1,480/197a
993/260a
1,465/284
773/224a
–/1,255
–/244a
–/2,945
–/2,928a
–/1,366a
–/3,945a
–/4,772
–/1,658
49/7
90/19a
SBD: Statistical Bulletin of Death.
aThe study uses SBD.
46
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
dies, all-tumour and corrected all-site three-digit con-
firmation rates were comparable to those obtained for
categories I and II(a).
Only two papers came within category III (necropsy-
based studies)9,19. Their results were slightly worse than
others, but it should be borne in mind here that in Spain
most autopsies are restricted to cases with uncertain
diagnosis.
Table 2 summarises available information in Spain
on detection and confirmation rates for malignant neo-
plasms by the main specific locations. The data, sor-
ted by the period analysed, revealed that in the interim
between the early 1980’s and the most recent study
(1992 data), the quality of certification had improved.
Highest indices were found for cancer of the stomach,
colon and rectum, pancreas, lung, melanoma, female
breast, brain and haematological tumours. In contrast,
other sites, such as ill-defined tumours and non-mela-
noma skin cancer, displayed lower rates of agreement.
Furthermore, in order to have a brief overview of the
quality of cancer certification for specific sites, pooled
estimators were calculated to classify the accuracy of
death certification for specific cancers according to
Percy’s criteria5, which depend on detection and con-
firmation rates (table 3).
Ill-defined tumours and ill-defined causes
Figure 3 shows the trend in the percentage which
ill-defined tumours and ill-defined causes represent of
all-cancer and all-cause deaths, respectively, over the
calendar period 1980-2002, in both sexes. During the
1980’s, Regional Mortality Registries became respon-
Table 1B. Quality of cancer death certification in Spain. Characteristics and classification of published studies sorted by publication year
Cancer DC with Cancer deaths detected only DR (%)
Author Savailable information by validating source (global false CR (%) Notes
All-cause/cancer from validating source positives)
279
154
233
1,173
1,557
644
685
3,298
4,231
1,371
7
19
30 (revised DC without mention 
of cancer)
36 (revised SBD without mention 
of cancer)
0
91 (clinical follow-up)
29 (clinical follow-up)
65 (Cancer registry)
61 (Cancer registry)
Not searched
Not searched
Not searched
Not searched
Not searched
Not searched
6 (cancer deaths only detected by 
necropsy)
5 (cancer deaths only detected by 
necropsy)
89.9(G) 
71.7(3D)
79.9(G) 
64.8(3D)
100(G)
100(3D)
75.2(G)
88.1(G)
94.7(G)
93.3(3D)
50.0(G)
38.5(3D)
73.7(G)
42.1(3D)
95.3(G)
76.9(3D)
92.9(G) 
68.2(3D)
91.5(G)
80.8 (3D)
97.5(G)
96.0(G)
99.3(G)
73.0(3D)
77.2(3D)a
81%(3D)*
64.4(3D)*
73.1(3D)*
99.1(G)
79.5(3D)
0.80.1(3D)*
96.2(G)
74.0(3D)
76.9(3D)*
98.5(G)
76.7(3D)
79.4(3D)*
85.7(G)
71.4(3D)
73.7(G)
42.1(3D)
Studies all DC of this period
Studies all DC of this period.
Validating source: histopathological 
records. Unconfirmed cases are
considered ill certified
No additional cases found
Toxic Oil cohort
Excludes multiple tumours
Only gynaecological cancer
Excludes ill-defined & multiple tumours 
and non-residents in the province
Exclude multiple tumours, in situ, skin 
non-melanoma and non-residents in
this region
Excludes multiple tumours 
Necropsy-based
Necropsy-based
CR: Confirmation rate; DR: Detection rate; G: all-tumour agreement rates (ICD-9 codes 140-208); 3D: all-site three-digit agreement rates.
3D*: all-site three-digit agreement rates with only those death certificate (DC) / Statistical Bulletin of Death (SBD) that really corresponded to malignant tumours used as
denominator.
García-Benavides, 
198610, 198911
Pañella, 198912
Ruiz-Liso, 198913
Giménez, 20023
Cáffaro, 19956
Sánchez-Garrido,
199614
Bosch, 198115, 198316
Navarro, 198417
DGA, 198818
Izarzurgaza, 19944
Martínez, 20008
Cirera, 20027
Nava, 1985, 198619;35
Carballeira, 19899
47
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
Ta
bl
e 
2.
 S
pa
ni
sh
 c
an
ce
r-d
ea
th
-c
er
tif
ic
at
e 
de
te
ct
io
n 
an
d 
co
nf
irm
at
io
n 
ra
te
s 
by
 s
ite
 
M
ur
ci
a
G
ra
na
da
 
Eu
sk
ad
i 
M
aj
or
ca
 
G
ir
on
a
B
ar
ce
lo
na
 
S
or
ia
Va
le
nc
ia
 1
98
4
Za
ra
go
za
M
ur
ci
a 
19
92
91
/9
4
19
89
19
89
85
-8
9
19
85
19
85
(G
ar
cí
a
83
19
81
(C
ir
er
a,
(M
ar
tín
ez
,
B
as
qu
e 
(C
áf
fa
ro
,
(S
án
ch
ez
,
(P
añ
el
la
, 
(R
ui
z 
Li
so
, 
B
en
av
id
es
,
(D
G
A
, 
(N
av
ar
ro
,
20
02
)6
20
00
)1
7
C
ou
nt
ry
)
19
95
)6
96
)1
4
19
89
)1
2
19
89
)1
3
19
86
,1
98
9)
9,
12
19
88
)1
8
19
84
)1
7
Lo
ca
tio
n
IC
D
-9
 c
od
es
D
R
C
R
D
R
C
R
D
R
C
R
D
R
C
R
D
R
C
R
D
R
C
R
D
R
C
R
D
R
C
R
D
R
C
R
D
R
C
R
M
ou
th
 &
 p
ha
ry
nx
14
0-
14
9
56
.6
81
.1
55
.5
89
.2
67
.5
87
.7
57
.8
92
.9
10
0.
0
10
0.
0
60
.0
75
.0
57
.1
80
41
.4
57
.1
O
es
op
ha
gu
s
15
0
95
.2
69
.0
79
.4
72
.5
90
.1
85
.5
92
.9
76
.5
75
.0
10
0.
0
85
.7
75
.0
84
.6
68
.7
86
.4
76
.0
S
to
m
ac
h
15
1
82
.0
88
.5
80
.7
87
.8
88
.2
90
.8
75
.0
84
.2
79
.3
85
.0
89
.2
90
.6
71
.4
86
.9
83
.1
90
.2
75
.3
84
.1
C
ol
on
-r
ec
tu
m
15
3-
15
4
87
.0
96
.2
75
.6
93
.4
91
.6
83
.9
86
.8
92
.3
90
.3
93
.3
82
.6
82
.6
82
.7
90
.5
70
.5
77
.1
C
ol
on
15
3
90
.5
69
.8
69
.3
71
.1
70
.1
70
.1
81
.2
68
.3
75
.0
71
.4
63
.9
79
.3
52
.0
60
.5
R
ec
tu
m
15
4
48
.6
97
.2
48
.8
84
.4
62
.6
77
.7
51
.5
82
.9
33
.3
50
.0
69
.6
66
.7
55
.6
83
.3
Li
ve
r
15
5
82
.1
50
89
.9
47
.5
90
.6
47
.5
80
.7
41
.2
66
.6
14
.3
53
.8
38
.9
85
.7
37
.5
53
.3
29
.6
G
al
lb
la
dd
er
15
6
73
.3
91
.7
59
.7
93
.6
67
.5
81
.3
52
.2
85
.7
75
.0
85
.7
37
.5
10
0
70
.0
66
.7
40
.0
84
.6
P
an
cr
ea
s
15
7
97
.4
78
.7
83
.0
86
.6
79
.5
77
.4
82
.9
87
.2
10
0.
0
85
.7
10
0
66
.7
10
0.
0
60
.0
71
.4
52
.6
La
ry
nx
16
1
83
.3
62
.5
81
.4
64
.2
83
.5
68
.6
85
.0
63
.0
10
0.
0
10
0.
0
80
.0
80
.0
96
.4
79
.4
75
.0
65
.9
Lu
ng
16
2
93
.8
89
.8
91
.6
89
.9
93
.7
92
.4
95
.4
95
.4
67
.6
88
.5
10
0.
0
94
.9
84
.6
91
.7
93
.3
91
.2
92
.3
86
.6
S
ki
n
17
2-
17
3
66
.7
80
.0
59
.3
90
.0
22
.2
10
0
50
.0
50
.0
44
.4
85
.7
M
el
an
om
a
17
2
90
.9
83
.3
76
.9
10
0
76
.7
88
.5
50
.0
50
.0
O
th
er
s
17
3
46
.2
75
.0
46
.2
80
.0
44
.0
91
.7
15
.4
50
.0
B
re
as
t-


17
4
92
.1
97
.5
87
.9
98
.6
92
.2
98
.7
91
.7
10
0
94
.1
88
.9
10
0.
0
10
0.
0
88
.0
95
.7
86
.7
98
.6
78
.8
10
0.
0
U
te
ru
s
17
9,
18
0,
18
2
78
.0
86
.7
78
.0
81
.4
73
.7
90
.3
86
.0
88
.0
93
.0
82
.0
83
.3
b
62
.5
62
.5
68
.2
76
.9
62
.5
C
er
vi
x
18
0
51
.7
88
.2
57
.6
97
.1
50
.0
93
.8
68
.0
94
.4
43
.2
86
.4
10
0.
0
50
.0
75
.0
10
0.
0
0.
0
0.
0
20
.0
50
.0
C
or
pu
s
18
2
50
.0
83
.3
41
.9
88
.6
52
.6
45
.5
37
.5
85
.7
43
.9
82
.8
10
0.
0
10
0.
0
7.
1
33
.3
14
.3
33
.3
O
va
ry
18
3
82
.1
85
.2
70
.1
90
.1
71
.2
78
.7
71
.4
71
.4
83
.7
77
.9
10
0.
0
10
0.
0
36
.4
44
.4
58
.3
87
.5
P
ro
st
at
e
18
5
93
.2
84
.6
87
.6
81
.3
90
.8
91
.4
91
.4
85
.5
90
.9
76
.9
68
.8
84
.6
88
.2
65
.2
90
.0
33
.3
O
th
er
 g
en
ita
l-

18
6-
18
7
0.
0
0.
0
70
.0
70
.0
77
.8
10
0.
0
50
.0
10
0
10
0.
0
10
0.
0
Te
st
ic
ul
ar
18
6
0.
0
10
0.
0
10
0.
0
60
.0
10
0.
0
B
la
dd
er
18
8
77
.9
91
.4
76
.4
90
.2
87
.3
88
.7
74
.4
91
.4
10
0.
0
77
.8
50
.0
70
.0
63
.8
95
.0
58
.8
10
0.
0
K
id
ne
y
18
9
75
.0
80
.0
79
.0
84
.5
80
.6
92
.1
58
.8
83
.3
10
0
66
.7
70
.0
77
.8
66
.7
25
.0
B
ra
in
19
1
96
.2
92
.6
95
.5
83
.6
94
.4
87
.9
10
0
84
.4
10
0
77
.7
91
.7
78
.6
En
do
cr
in
e 
G
la
nd
s
19
3-
19
4
10
0.
0
71
.4
75
.0
75
.0
73
.7
10
0.
0
10
0
80
10
0.
0
80
.0
Th
yr
oi
d
19
3
10
0.
0
75
.0
64
.7
78
.6
80
.0
10
0.
0
75
.0
10
0
10
0.
0
10
0.
0
Ill
-d
ef
in
ed
 tu
m
ou
rs
19
5-
19
9
58
.2
55
.4
50
.6
30
.0
56
.5
51
.7
52
.6
42
.4
31
.6
30
.0
5.
8
23
.1
Ly
m
ph
om
as
20
0-
20
2
90
.2
83
.0
87
.0
77
.7
85
.7
91
.3
10
0.
0
10
0.
0
86
.7
a
92
.9
a
62
.5
76
.9
50
.0
50
.0
H
od
gk
in
20
1
50
.0
50
.0
75
.0
85
.7
10
0.
0
40
.0
60
.0
75
.0
O
th
er
s
20
0,
20
2
81
.6
75
.6
71
.9
85
.3
84
.8
90
.3
63
.6
77
.8
M
ul
tip
le
 M
ye
lo
m
a
20
3
10
0.
0
94
.4
10
0.
0
92
.3
81
.3
92
.9
10
0.
0
10
0.
0
10
0.
0
10
0.
0
Le
uk
ae
m
ia
20
4-
20
8
92
.6
b
96
.2
b
96
.4
c
96
.4
c
89
.1
88
.2
10
0.
0
10
0.
0
10
0.
0
10
0.
0
10
0.
0
87
.5
95
.0
86
.3
85
.7
c
80
.0
c
C
R
: c
on
fir
m
at
io
n 
ra
te
; D
R
I: 
de
te
ct
io
n 
ra
te
. a
In
cl
ud
es
 IC
D
 2
00
-2
03
; b
O
nl
y 
ly
m
ph
oi
d 
ca
se
s 
w
er
e 
fo
un
d 
(I
C
D
 2
04
);
 c H
ae
m
at
op
oi
et
ic
 a
nd
 r
et
ic
ul
ar
 s
ys
te
m
 (
IC
D
-O
 1
69
).
R
at
es
 fr
om
 r
ev
ie
w
ed
 s
tu
di
es
, t
ho
ug
h 
in
 s
om
e 
ca
se
s 
D
et
ec
tio
n 
R
at
es
 (
D
R
) 
an
d 
C
on
fir
m
at
io
n 
R
at
es
 (
C
R
) 
w
er
e 
ca
lc
ul
at
ed
 w
ith
 d
at
a 
su
pp
lie
d 
in
 th
e 
pa
pe
rs
.
48
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
Table 3. Accuracy of death certification for specific cancers in Spain according to Percy’s criteria (Percy 1981). Pooled analysis
Well-certified Over-certified Under-certified Ill-certified
(DR 
 80 and CR 
 80) (DR 
 80 and CR  80) (DR  80 and CR 
 80) (DR  80 and CR  80)
Location ICD-9 DR CR Location ICD-9 DR CR Location ICD-9 DR CR Location ICD-9 DR CR
Stomach 151 83 89 Oesophagus 150 87 78 Mouth & pharynx 140-149 59 85 Colon 153 72 70
Colon-rectum 153-154 83 90 Liver 155 85 45 Rectum 154 54 82 Gallbladder 156 58 79
Pancreas 157 84 80 Larynx 161 83 67 Skin 172-173 54 87 Corpus uterus 182 42 76
Lung 162 92 91 Melanoma 172 78 91 Ill-defined tumours 195-199 53 39
Breast-
 174 90 98 Skin (non-melanoma) 173 42 80 L. Hodgkin 201 69 69
Uterus 179,180,182 82 83 Cervix uterus 180 51 91
Prostate 185 89 82 Ovary 183 74 81
Brain 191 96 85 Other genital- 186-187 69 82
Lymphomas 200-202 86 80 Testicular 186 78 88
Multiple Myeloma 203 96 94 Bladder 188 76 91
Leukaemia 204-208 93 93 Kidney 189 76 83
Endocrine Glands 193-194 79 83
Thyroid gland 193 76 89
Lymphomas, others 200,202 76 83
CR: confirmation rate; DR: detection rate. 
sible for the coding process. This change led to an im-
provement in the quality of the information, which is re-
flected in the downward trend in the proportion of ill-de-
fined causes. The initial decline in the percentage of
ill-defined causes was accompanied by an increase in
the proportion of ill-defined tumours, which registered
a less clear pattern. Nevertheless, it should be noted
that, coinciding with the introduction of ICD-10 in
Spain, 1999 witnessed an increase in these indicators,
with the percentage of ill defined causes and ill defined
tumours rising by 31% and 10% respectively over the
previous year’s figures, followed by an apparent leve-
lling-off. Interestingly, women registered higher values
for these two indicators of bad certification, in all cases.
When proportional mortality was computed using ad-
justed rates, women continued to have worse results
(data not shown).
Discussion
Though quality at a national level has not been stu-
died, available data suggest that, overall, cancer death
certificates in Spain possess an accuracy comparable
to that reported for other industrialised countries5,20,21.
Indirect estimations such as the proportion of ill-defined
causes in Spain show similar percentages to those re-
gistered by other developed countries22.
The first Spanish study to address death certifica-
te reliability was published in 198115. Specifically focu-
sed on cancer death certificates, this study solely co-
vered the Barcelona metropolitan area. Several authors
10
5
0
Pe
rc
e
n
ta
ge
1980 1985 1990 1995 2000
% of ill-defined tumours/all tumours
Women
Both sexes
Men
% of ill-defined causes/all causes
Women
Both sexes
Men
Figure 3. Percentages of deaths due to ill-defined tumours and
ill-defined conditions versus all-tumour and all-cause mortality.
Trends for the period 1980-2002 by sex and for both sexes.
subsequently studied the quality of death certificates in
other parts of the country. In this paper, we summari-
sed all available information to provide a global view of
the quality of Spanish cancer-mortality statistics.
In Spain, published quality estimators are basically
drawn from regional studies, many of which are spon-
sored or undertaken by Cancer Registries. Accordingly,
it should be borne in mind that, despite the existence of
national coding protocols, inferring quality indicators for
the country, such as our pooled estimators, might also
be problematic, since decentralisation of the coding pro-
cess could cause inter-regional variability, and there are
huge areas of the country where death certification qua-
lity studies have not been conducted (fig. 2). Only Giménez
et al3, in their study on a toxic-oil poisoned cohort, pro-
vide national data, though their results could also be mis-
leading as they refer to a cohort of sick people, subjec-
ted to a thorough follow-up over time. The progressive
increase in the number of Cancer Registries in the country
might go some way towards having more representati-
ve data about quality of cancer death certification in Spain
in the future. Nevertheless, results from the different stu-
dies were quite similar for most cancer sites.
Compared to other causes of death, cancer (ICD 140-
208) seems to be well certified in Spain, with detection
rates being as much as 9 points higher for all tumours than
for all causes together, and confirmation rates over 20
points higher than for all causes11,12, which could be due
to the fact that cancer is usually a well-characterised diag-
nosis, and in most cases has histological confirmation.
All-tumour detection rates ranged from 79.9 to 100
and the CRs exceeded 90%. Depending upon their re-
sults, some studies have classified this broad category
as well-certified3,11,13, while others have viewed tumours
as being slightly underreported3,12. However, the per-
centage of underreporting in Spain, as estimated by Gar-
cía-Benavides et al11 and Cáffaro et al6, seems to be
around 5-6%, which is comparable to international fi-
gures20,23,24. Hence, global cancer mortality figures can
be considered accurate and useful for estimating the bur-
den of this group of diseases.
As expected, when site misclassification was taken
into account, agreement estimates were lower. All-site
three-digit detection rates from categories I and II(a) stu-
dies range from 64.8 to 100 and confirmation rates from
68.2 to 80.8. Based on these indicators, cancer could
be deemed to be ill-certified according to Percy’s cri-
teria. It is remarkable that, in general, Ruiz-Liso et al13
obtained better results than other studies. The CR range
in category II(b) studies was similar (64.4-81%). The de-
sign used in such studies excludes global false positi-
ves, thus slightly overestimating this indicator. Accor-
ding to three studies6-8, in which both correct and biased
estimators were available, biased data were on avera-
ge three points higher. In the USA5 and France20, CRs
were slightly higher than in Spain (82.7 and 86%, res-
pectively). Nevertheless, it should be noted that those
studies belong to category II(b) and are also biased.
If some problematic categories are grouped (this is
the case of cervical –ICD 180–, corpus –ICD 182– and
unspecified uterine cancer –ICD 179–; or colon –ICD
153– and rectal neoplasms –ICD 154–), all-site three-
digit agreement rates improve. This can be seen in Ci-
rera and Navarro7, where the proportion of agreement
using three-digit ICD was close on 80%, and aggrega-
tion of problematic locations raised it to 83%. A com-
parable increase was previously described in the USA5
(4%) and in Ontario21 (6%).
Analysis of specific anatomic locations shows that,
in general, the main leading cancer sites are well cer-
tificated. Thus, lung, colon-rectum (ICD 153-154), pros-
tate, stomach, pancreas, female breast, uterus (ICD 179-
180,182) and brain cancer, as well as leukaemia,
lymphomas, myeloma belong to this category. Together,
they represented around 69% of all cancer deaths re-
gistered in Spain in 200225. Nonetheless, it should be
noted that colon and rectal cancers are respectively ill
and undercertified unless they are considered together,
since mutual misclassification of the two sites has been
reported7,21. A similar situation can be observed with res-
pect to uterus. Overall, this location has good accuracy
rates, but cervix and corpus uterus are under –and ill–
certified, respectively. Mortality due to uterus cancer in
Spain registered a steady decrease since 1976, con-
trasting with the slightly increasing trend in cervix mor-
tality26, which has been explained mainly as a conse-
quence of a reassignment of cases previously coded
as «uterus non-specified»14. Nevertheless, if uterus is
regarded as a single category, its certification has im-
proved with time and, in more recent studies, achieves
acceptable figures.
Some other frequent locations, such as larynx, blad-
der or ovary, which rank among the ten leading causes
of cancer death in Spain, evince problems in certifica-
tion. A certain degree of overreporting has been descri-
bed for laryngeal cancer, due to misclassification of head
and neck tumours6,8,18. Similarly, there is certain measu-
re of overreporting for oesophageal cancer, mainly attri-
butable to the inclusion of stomach cases6-8,18, as well as
for liver, largely due to misclassification of gallbladder and
ill-defined neoplasms6-8,13,18. Percy et al27 warned that in-
clusion in ICD-9 code 155 of liver cancer that was not
specified as primary or secondary, was a possible
cause of misclassification for these tumours, as it might
lead to some coders registering secondary liver cancers
under this category.
Furthermore, there have been reports of underes-
timation of urinary bladder cancer, erroneously certified
as prostate neoplasm6-8,13,17,18. It should, however, be
noted that, even with this underestimation, Spain had
the second highest bladder mortality rate within the Eu-
ropean Union in 200026. In addition, some Spanish aut-
49
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
hors have reported undernotification of skin cancers due
to lack of information on their anatomic location6, the-
reby implying their inclusion as ill-defined and unspe-
cified tumours (ICD-9 195, 199).
Accuracy of ovary-neoplasm certification seems to
vary widely among studies, though it can be regar-
ded as undercertified according to pooled estima-
tors. While some reported cases of death due to ova-
rian cancer were really due to abdominal or uterine ne-
oplasms, in some studies7,8,14, «unspecified uterus tu-
mours» would appear to include some ovarian cancers
as well as tumours of the cervix and endome-
trium6,8,14,17,18.
Ill-defined tumours were badly certified according to
Percy’s criteria. Cirera and Navarro7 reported there was
clinical information that would allow for almost half of
these tumours to be included in other categories. The
selfsame problem has been described in other coun-
tries, such as Brazil28.
Our results show that all cancer sites classified as
well certified in Spain have been reported as such in
the USA by Percy et al5. However, this author also en-
countered good agreement rates for oesophagus, blad-
der, gallbladder, thyroid gland and kidney, locations that
did not display a good standard in Spanish studies. In
Ontario, Canada, Reynolds21 similarly observed good
DRs and CRs for all those neoplasms classified as well-
certified in Spain, with the exception of pancreas, and
also found high rates for cervix uterus, bladder, kidney,
ovary and endocrine glands. In France, Laplanche et
al20 reported CRs of over 80% for breast, colon-rectum,
lung and pancreas, values similar to Spanish figures.
In contrast, they found CRs of under 80% for head and
neck and cervix, tumours which in Spain register good
CRs. Stomach cancer was also well classified in Bra-
zil29 and Italy30. In international comparisons, however,
a relevant factor to be considered are differences in in-
ternational coding practice, since divergences up to 30%
have been described by Percy and Muir31 among wes-
tern countries using the ICD-9.
With respect to accuracy of certification, Percy men-
tioned the influence of several factors such as age, sex,
geographic area, presence of an autopsy or place of
death5. In Spanish studies, a lower quality has been re-
ported for older ages and for women6-8,18. These sex-re-
lated differences are reflected in the percentage of ill-de-
fined tumours and ill-defined causes, which are regularly
higher in females and could in part be due, both to 
gynaecological neoplasms18 and to the older 
age of women. Insofar as place of death is concerned,
a lower quality of death certification has been associa-
ted with death at home6,7, though other authors have fai-
led to find any difference11. Finally, the quality of certi-
fication has been shown to be slightly lower in rural areas18.
As Navarro et al17 points out, clinical information is ne-
eded to validate death certificates, thus implying the ex-
clusion of those cases where this information is not avai-
lable. Death certificates lacking complementary clinical or
anatomo-pathological data could be of worse quality, as
they probably include more home deaths. Navarro found
that death certificates excluded for this reason belonged
to subjects who were, on average, seven years older than
those included in her study. All this may well lead to ove-
restimation of the quality reported in many studies.
Several strategies have been proposed to motivate
and improve physicians’ certification28 such as a periodic
assessment of coding practices along with the educa-
tion and motivation of medical students and physicians.
In Spain, several Regional Health Authorities imple-
mented specific workshops that showed their efficacy
in enhancing death certificate quality indicators32. Yet,
these interventions are questioned by Swift’s study33,
which failed to find significant changes in the state of
certification after the introduction of formal education into
the medical syllabus.
A further point of discussion is the effect of the in-
troduction of the ICD-10 on the quality of mortality data.
To date, we have been unable to find any validation study
in Spain covering the ICD-10 coding period. In 1999,
Ruiz et al34 compared ICD-9 and ICD-10 coding in a huge
sample of Spanish death certificates. They reported that,
whereas ill-defined condition figures increased almost
a 14% with the use of this latest version, neoplasms se-
emed quite stable. In contrast, our data indicate that the
ICD-10 effect might be greater than thought, and that
it has also affected tumours coding. This could suggest
a worsening in the quality of data, and careful survei-
llance is thus called for.
In conclusion, the quality of cancer death certifica-
tion in Spain for all tumours and all main sites has im-
proved over the last two decades and can be conside-
red comparable to internationally published data. Thus,
mortality data constitute a valid indicator to estimate the
burden of cancer. However, for some locations, such as
the oesophagus or bladder, death certificate informa-
tion should be approached with caution. Misclassifica-
tion may generate problems for studying mortality
trends and planning future needs. It should be noted
that, in general, most available information on the qua-
lity of death certification reflects the situation from 1970
to 1990, when the ICD-9 was in use. The relatively re-
cent introduction of the ICD-10 may have affected qua-
lity indicators and should thus be carefully monitored.
Finally, our results point to the need to improve death
certification in the case of Spanish women.
Acknowledgements
The work of V. Carvajal was supported by the Carlos III
Institute of Health (Grant No. 03/0007). The work of B. Suá-
rez was funded by the Network of Epidemiology and Public
50
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
51
Pérez-Gómez B et al. Accuracy of cancer death certificates in Spain: a summary of available information
Gac Sanit. 2006;20(Supl 3):42-51
Health Research Centres, International Health & Co-opera-
tion Foundation (Grant No. C03/09).
The authors wish to thank Michael Benedict for his help
with the English, and Isabel Izarzugaza for supplying Euska-
di’s information.
References
1. Instituto Nacional de Estadística. Manual de causas de de-
función. Madrid: INE; 1996.
2. Garrucho G, Almazán M, Madrazo M, Sánchez J, Villalobos
H, Infiesta JA. Análisis de la concordancia de los datos re-
cogidos en el certificado médico de defunción y el boletín 
estadístico de defunción. Rev Sanid Hig Publica. 1990;64:
63-72.
3. Giménez O, Abaitua I, Sánchez-Porro P, Posada M. Análisis
de las causas de muerte en la cohorte del síndrome del acei-
te tóxico. Validación de los certificados oficiales de defunción.
Gac Sanit. 2002;16:118.
4. Izarzurgaza I, Sastre B, Ibarrola T. Validez del boletín esta-
dístico de defunción en la Comunidad Autónoma del País
Vasco. Resultados preliminares. Granada: XIX Réunion du
groupe pou l’Epidemiologie et l’enregistrement du cancer dans
les pays du langue latine; 1994.
5. Percy C, Stanek E III, Gloeckler L. Accuracy of cancer death
certificates and its effect on cancer mortality statistics. Am J
Public Health. 1981;71:242-50.
6. Cáffaro M, Garau I, Cabeza E, Franch P, Obrador A. Validez
de los certificados de defunción por cáncer en Mallorca. Gac
Sanit. 1995;9:166-73.
7. Cirera L, Navarro C. Validez de la certificación de la muerte
por cáncer en la Comunidad de Murcia. Oncología. 2002;
25:264-72.
8. Martínez C, Sánchez MJ, Rodríguez M, Alaminos FJ, Medi-
na MJ. Exactitud del diagnóstico de cáncer en los certifica-
dos de defunción de la provincia de Granada. Revista de On-
cología. 2000;2:245-52.
9. Carballeira Roca C, Vázquez Fernández E, Brana Rey N,
López Rois F, Loureiro Santamaria C, Hervada Vidal J. Apro-
ximación a calidade das estadísticas de mortalidade. Gali-
cia 1987. Gac Sanit. 1989;3:566-72.
10. García-Benavides F. Fiabilidad de las estadísticas de mor-
talidad. Valencia: Conselleria de Sanitat i Consum. Genera-
litat Valenciana. Monografies Sanitàries. Serie A. N.o 2; 1986.
11. García-Benavides F, Bolumar F, Peris R. Quality of death cer-
tificates in Valencia, Spain. Am J Public Health. 1989;79:
1352-4.
12. Pañella H, Borrell C, Rodríguez C, Roca J. Validación de la
causa básica de defunción en Barcelona, 1985. Med Clin.
1989;92:129-34.
13. Ruiz Liso JM, Sanz Anquela JM, Alfaro Torres J, Dodero de
Solano S, García Pérez MA. Valoración histopatológica de
los boletines de defunción en Soria en 1985. Rev Sanid Hig
Publica. 1989;63:41-51.
14. Sánchez Garrido MV, Izquierdo Font A, Beltrán Fabregat M,
Bosch José FX, Viladiu Quemada P. Tendencias tempora-
les de la mortalidad por cáncer de cérvix en Cataluña 
1975-1992: Análisis del Boletín Estadístico de Defunción y
del Registro de Cáncer de Girona. Gac Sanit. 1996;10:
67-72.
15. Bosch Jose FJ, García González A, Orta Buj J. Mortalidad
por tumores malignos en la ciudad de Barcelona. Rev Sanid
Hig Publica. 1981;55:31-67.
16. Bosch FX, García A, Orta J, Juvanet J, Camprodon A, Pu-
marola A. The accuracy of medical certifications of cancer
deaths and of cancer diagnosis in the municipal area of Bar-
celona in 1979. Rev Esp Oncol. 1983;30:17-24.
17. Navarro Sánchez C, Sánchez Sánchez JA, Molina Illán JA.
Validez del Boletín Estadístico de Defunción como fuente de
datos en las estadísticas sobre el cáncer. Un estudio preli-
minar. Bol Salud Región Murciana; 1984;4:177-80.
18. Diputación General de Aragón. Exactitud de los certificados
de defunción por cáncer en Zaragoza. Bol Epidemiol Aragón;
1988;5:199-206.
19. Nava Caballero JM, Huguet Recasens M, Carrasco MA. Cer-
tificado médico de defunción: su fiabilidad. Rev Sanid Hig Pu-
blica. 1986;60:85-94.
20. Laplanche A. Quality of death certificates in cases of cancer
death in France. Bull Cancer. 1998;85:967-9.
21. Reynolds DL, Nguyen VC, Clarke EA. Reliability of cancer mor-
tality statistics in Ontario: a comparison of incident and death
diagnoses, 1979-1983. Can J Public Health. 1991;82: 120-6.
22. Regidor E, Rodríguez C, Ronda E, Gutiérrez JL, Redondo JL.
La calidad de la causa básica de muerte del Boletín Estadís-
tico de Defunción. España, 1985. Gac Sanit. 1993;7:12-20.
23. Griffith GW. Cancer surveillance with particular reference to
the uses of mortality data. Int J Epidemiol. 1976;5:69-76.
24. Puffer RR, Griffith GW. Patterns of urban mortality. Report of
the Inter-American investigation of mortality. Pan American
Health Organization. Scientific Publication. 1967;151.
25. López-Abente G, Pollán M, Aragonés N, Pérez-Gómez B.
Mortalidad por cáncer y otras causas en España en 2002.
Área de Epidemiología Ambiental y Cáncer. Centro Nacio-
nal de Epidemiología. ISCIII. [Actualizado 2005; citado 9-
8-2005]. Disponible en: http://cne.isciii.es/htdocs/can-
cer/mort2002.txt
26. López-Abente G, Pollán M, Aragonés N, Pérez-Gómez B, Her-
nández V, Lope V, et al. Plan Integral del Cáncer. Situación
del cáncer en España. Madrid: Ministerio de Sanidad y Con-
sumo; 2003.
27. Percy C, Ries LG, Van Holten VD. The accuracy of liver can-
cer as the underlying cause of death on death certificates.
Public Health Rep. 1990;105:361-7.
28. Monteiro GT, Koifman RJ, Koifman S. Reliability and accu-
racy of reported causes of death from cancer. I. Reliability of
all cancer reported in the State of Rio de Janeiro, Brazil. Cad
Saude Publica. 1997;13 Suppl 1:39-52.
29. Monteiro GT, Koifman RJ, Koifman S. Reliability and accuracy
of reported causes of death from cancer. II. Accuracy of 
stomach cancer reported in the municipality of Rio de Ja-
neiro County, Brazil. Cad Saude Publica. 1997;13 Suppl 1:
53-65.
30. Barchielli A, De Angelis R, Frova L. Use of mortality statis-
tics for the study of the distribution of digestive system tumors:
characteristics and quality of the data. Ann Ist Super Sani-
ta. 1996;32:433-42.
31. Percy C, Muir C. The international comparability of cancer mor-
tality data. Results of an international death certificate study.
Am J Epidemiol. 1989;129:934-46.
32. Cirera L, Martínez C, Contreras J, Navarro C. Aprendizaje 
y satisfacción de los talleres de pre y posgrado de medi-
cina para la mejora en la certificación de las causas de 
defunción, 1992-1996. Rev Esp Salud Pública. 1998;72:185-
95.
33. Swift B, West K. Death certification: an audit of practice en-
tering the 21st century. J Clin Pathol. 2002;55:275-9.
34. Ruiz M, Cirera Suárez L, Pérez G, Borrell C, Audica C, Mo-
reno C, et al. Comparabilidad entre la novena y la décima
revisión de la Clasificación Internacional de Enfermedades
aplicada a la codificación de la causa de muerte en Espa-
ña. Gac Sanit. 2002;16:526-32.