Alcuni estratti da
Citation, Citation, Citation : Bibliometrics, the web and the Social Sciences and Humanities
Citation, Citation, Citation : la bibliométrie, Internet et les sciences humaines et sociales
Christine Kosmopoulos et Denise Pumain http://cybergeo.revues.org/15463#text
16Each
of the three existing large databases of citations of scientific
literature has substantial gaps, which make it impossible to establish
a pertinent assessment of researchers or journals (the impact factor)
by means of bibliometric tools.
17Although
considered by some to be the standard citation database, the Web of
Science has been criticized for many years for its bias against the
SSH. It inventories some 8,700 international journals (there are an
estimated 20,000 scientific journals in the world), but only 3,000
6
for the SSH, almost all of which are of Anglo-Saxon origin. By way of
example, the SSH journals supported by the CNRS do not appear in the
WOS. In 2004, the CNRS tried to negotiate improvements with Thomson/ISI
that would take into account specific European needs and
particularities, but no progress has been made. A report by Philippe
Jeannin on the evaluation of research in the SSH submitted in 2003 to
the Ministry of Research and New Technologies confirms that French
journals in these fields are not covered.
- 7 http://www.esf.org
- 8 http://www.obs-ost.fr
18With
a view to providing a better analysis of European scientific production
in the WOS, the European Scientific Foundation (ESF) launched a program
to evaluate journals in the SSH, publishing in June 2007 an initial
list called the European Reference Index for the Humanities (ERIH)
7,
which classes journals by rank (A, B, C). This list does not include
certain research areas in the social sciences such as geography, which
will become the subject of another classification. The ESF provides
rulings concerning the integration of journals in this reference list,
which explains why not all the journals supported by the CNRS figure in
it. But if the ERIH does not have the vocation to be a bibliometric
tool, and is not offered as an alternative to WOS, the Observateur des
Sciences et des Techniques (OST), which has the specific mission to
conceive and produce indicators about research and development, has
announced that it will be working on impact indexes based on the
journals inventoried by the ESF
8.
19The
Scopus database (Elsevier) could be an alternative to the monopoly of
Thomson. In fact, Scopus indexes around 17,000 titles, of which 2,850
are in the SSH, that is, twice as many as the WOS, and is not limited
to Anglo-Saxon journals. The geographical distribution of the titles is
25% for the United Kingdom, (4,157 journals), only 25% for the rest of
Europe/Middle-East/Africa, 37% for North America, 12% for Asia/Pacific,
and 1% for South America. There is therefore much broader coverage than
that offered by the WOS. Furthermore, the inquiries made to this Scopus
base associated with the search engine Scirus yield a much wider range
of results, because Scopus also includes other documents than the
articles in the journals. The list of the sources of Scirus is
indicated on the site. Unfortunately, the period covered by Scopus
remains very limited (11 years).
A comparative study of the WOS and Scopus9 for the disciplines in the SSH is currently underway under the direction of Christine Kosmopoulos.
20When
all is said and done, with both databases the coverage of the
publications in the journals, both from the perspective of the journals
covered and of the length of time covered, is so incomplete as to give
a biased representation of scientific production in the SSH, and
therefore, cannot provide solid results for a bibliometric evaluation.
- 10 http://urfist.univ-lyon1.fr/GoogleScholar.pdf
- 11 http://www2.hawaii.edu/~jacso/extra/gs/
21Another
advantage of Google Scholar, in addition to the free access it
provides, is that it inventories all scientific literature without
distinction: articles in journals, whether or not they have an
editorial board, but also theses, books, extracts of books, reports,
pre-prints, etc. Nevertheless, this specialized search engine presents
other problems. Unlike the WOS or Scopus, it provides no information
about the resources it uses. As Jean-Pierre Lardy indicates in his 2007
publication on the URFIST
10
site, there is no list of commercial editors or the servers of the
indexed archives, no information about the period covered, the volume,
or even the countries involved. Certain known sources of Google
Scholar, for example the bibliographical database Francis of the INIST,
are not in conformity with bibliometric norms. There are also
significant gaps in the coverage of publishers’ archives. A test with
reference to important figures in the sciences confirms this
observation, as Peter Jacso demonstrates
11. For this database to become an incontestable reference for citations, it would need a significant and systematic overhaul.
- 12 http://www.harzing.com/resources.htm#/pop_gs.htm
- 13 http://www.harzing.com/
22Because
of the opacity of the sources and the incomplete coverage (not all of
the publications necessarily appear in the same journal), the
information extracted by the tool Google Scholar and analysed by
Anne-Will Harzing on her website
12 cannot yet guarantee a reliable evaluation. Yet Harzing’s work does make a critical use of this database possible
13.
The truth is that at this stage, the tool known as “Google
Scholar-Harzing” gives a much better account of the various forms of
communication in the SSH than the WOS or Scopus. Furthermore, its chief
advantage is that it makes it possible to intervene in calculations
about the index, in selecting the publications that are considered to
be truly indicative of the scientific activity of an author,
eliminating publications by authors with the same name, and
duplications or references that are not really pertinent, and in
proposing a whole variety of indexes that correct the h index. In this
way, the g index (Egghe’s g-index) is calculated on the same principle
as index h, but gives more weight to frequently-cited articles. Another
index (Individual h-index) corrects index h with the average number of
authors per article, in order to make it possible to compare output
between the disciplines in which the procedures for identifying the
authors of an article are very different. A further feature of index h
is its proper measuring according to the number of years of publishing
activity of the author (Age-weighted citation rate), which entitles it
to compare people who are in different phases of their scientific
careers. Index h can also be calculated by assigning more weight to
recent articles, giving a higher score to people who are still
productive (Contemporary h-index).
23To
the extent that the Google Scholar database provides better coverage
for publications in the human and social sciences, especially books, it
would undoubtedly be useful to undertake for each discipline and
sub-discipline a systematic test of its different measurements. For
that matter, A. W. Harzing encourages this on her site. The value of
the indexes can be considerably enhanced by a better knowledge of the
quality of their presentation of scientific values habitually
recognized by peers. Indeed, in spite of reticence about their use and
the imperfection of the existing databases, it is highly probable that
the expansion of online publications and the ease with which they can
be consulted will soon lead to the adoption of citation indexes in
institutional procedures for the recruitment and assessment of
researchers. Instead of allowing themselves to be obliged to use the
considerably biased instruments of the Web of Science that are
presently used for a number of the natural and life sciences, it would
be in the interests of researchers in the human and social sciences to
appropriate tools developed from a database that is more open to their
publication practices. Certainly all these quantified measurements
cannot provide a complete substitute for the more qualitative
evaluation, partly subjective, but much more subtle and sure,
represented by the classic assessment by peers. Even in experimental
disciplines, in which, it is generally admitted, the number of
citations reflects quite well the quality of the production of a
scientist, and in which the classification of journals is
systematically taken into account in publication strategies, the tools
of bibliometry are objects of recurring criticism. In fact, it is the
social practice of scientific citation that warrants a critical
examination.
37The
position of the social sciences in the systems of bibliometric analysis
is not very secure, for reasons linked to differences in scientific
practices, which are sometimes attributed to a presumed ‘delay’ on the
part of these disciplines in relation to what has been set in place in
the disciplines known as the ‘hard’ sciences, but which probably also
derive from other factors, in particular, cultural and linguistic
diversity, and the more rapid historical development of research
subjects. These subjects are also related to ‘society’, implying strong
interaction between the object of scientific investigation and the
social context in which the object is investigated (Latour, 1996). For
that matter, some base their rejection of all bibliometrics in the
human sciences on this supposedly insurmountable difference between
them and the ‘historical sciences’ (the expression is Jean-Claude
Passeron’s, 1991).
38The
development of collaborative tools, thanks to the second generation of
the Internet (Web 2.0) and the movement of Free Access, and advances in
the area of the internationalisation of the norms of exchange, offer
new elements for reflection on bibliometrics in general, and open up
perspectives for increasing the uses of bibliometrics in the human and
social sciences. From now on, the technical means will make
distribution possible, along with the sharing of scientific work at a
low cost, and greater and greater access to resources, in particular
via the OAI-PMH protocol or the RSS feed, with, as a result, an
increase in the pool of citations, indispensable for arriving at solid
analytical results. We can even imagine the formation of large new
repositories or new databases that would combine existing databases
(scientific search engines, open archives, libraries, etc.), as well as
bibliographical references provided on the web pages of researchers. To
this database of statistical information could even be added a
directory based on peer opinion (Raan, 2003).
- 34 http://opcit.eprints.org/
- 35 http://citec.repec.org/
- 36 http://repec.org/
- 37 http://eprints.ecs.soton.ac.uk/13804/
39An experiment conducted between 1999 and 2002 by the Universities of Southampton, Cornell and arXiv.org, Open Citation Project
34,
came to the conclusion that research on rows of citations and the links
between the references is one of the examples of the OAI services,
encouraging researchers to post their work on institutional sites. The
HAL database launched by the CCSD of the CNRS is clearly a part of this
project, and could serve as a bibliometric tool. In the science of
economics, the bibliometric database CitEc
35
could also serve as an example to other disciplines in the social
sciences. It is based on RePec (Research Papers in Economics)
36, in which 68 countries participate. The interesting contribution of Stevan Harnad in the
11 Annual Meeting of the International Society ofr Scientometrics and Informetrics, in Madrid, analyses the advantages of Open Access Scientometrics
37.
- 38 http://cordis.europa.eu/l
40Since
the beginning of 2006, the CNRS has been conducting a large information
campaign along these lines. The ESFRI (European Strategy Forum on
Research Infrastructures) reflects on the new forms of evaluation of
the SSH in the perspective of the 7th program-frame (7e
Programme-Cadre) of European Union research
38.
The European Foundation for Science undertook the indexing and
validation of lists of journals for each area of the human and social
sciences, with a classification based on their reputation.
41It
remains true that in the field of bibliometrics, as also for evaluation
in general, trusting in a single indicator, however sophisticated, is
an objective totally unsuited to what we know of the complexity of
social systems, and that it would be advisable to set up not only
batteries of indicators, but also multivarious methods for analyzing
them and for preparing all decisions. Placing research in the network
on a global scale thanks to electronic support and communication should
cause the emergence of new forms of scientific evaluation, better
harmonized, and of which the tools of bibliometry are only one aspect.
E da un altro lavoro
JournalBase - A Comparative International Study of Scientific Journal Databases in the Social Sciences and the Humanities (SSH)
JournalBase. Une comparaison des bases de données scientifiques internationales en sciences humaines et sociales (SHS)
Michèle Dassa, Christine Kosmopoulos et Denise Pumain
53As
with all the scientific disciplines, the SSH are drawing on more and
more digital tools, whether it be for publishing, archiving,
documentary research, or the analysis and processing of information
that they produce and use. The visibility of scientific output, like
the intra- and inter-disciplinary exchanges, are certainly facilitated
and amplified by the documentary databases. The temptation is great for
certain institutions also to use these for purposes of evaluation, even
if this practice remains highly open to criticism, particularly in the
area of the SSH. But whether we wish it or not, the bibliometric
databases exist. It is therefore important to know their contents well,
to be able to clarify the contours and the limits, even evaluate their
use for evaluation!
54What
emerges from our study is that no tool reflects appropriately the
totality of the important journals of reference in SSH. The broadest
coverage in terms of the number of journals, and in particular European
journals, is without doubt that of ERIH, but ERIH is
only concerned with the “Humanities”. The AERES list also offers a much
wider range than the big commercial databases. It is worth noting that Scopus
has announced the inclusion of several hundred journals in the
Humanities in 2009 that are not taken into account in the present
study, which was stopped at the end of 2008. An updating of the
totality of the data will be carried out in the course of 2010.
Nevertheless, from now on, the list established in this article, which
we call JournalBase, constitutes the broadest list, because
it lists without double counts and with exact identifiers the journals
mentioned in each of the other databases.
55For
lack of an exhaustive list of existing journals, it is not possible to
measure exactly the degree to which the bibliometric databases for the
journals of the SSH are representative. We do however obtain very
different results according to the disciplines. Obviously, whatever the
tool under consideration, the degrees of representation calculated
according to the language and the publishing country reveal very strong
biases, which reflect not so much gradients of scientific quality as
relations of domination in the present world, of a political and
cultural, even economic, order.
56These
results lead us, therefore, to stress the importance of demonstrating
more transparence as to the procedures and criteria of selection of the
journals in the different tools, whether or not they are commercial,
especially to limit the geographical and disciplinary biases that we
have been able to observe in the course of this catalogue of the
contents of the databases.
57Our objective from now on is to go forward in updating the 2009 JournalBase,
and in the development of a collaborative platform that will include
dynamic access to the database, in order to make possible an
international watchfulness over this information. The publication in an
open access journal that allows interactivity should likewise make it
possible to create a space for debate backed by information, and a
bibliography validated by the scientific community.
