Citation graph, weighted impact factors and performance indices

Abstract

A scheme of evaluating an impact of a given scientific paper based on importance of papers quoting it is investigated. Introducing a weight of a given citation, dependent on the previous scientific achievements of the author of the citing paper, we define the weighting factor of a given scientist. Technically the weighting factors are defined by the components of the normalized leading eigenvector of the matrix describing the citation graph. The weighting factor of a given scientist, reflecting the scientific output of other researchers quoting his work, allows us to define weighted number of citation of a given paper, weighted impact factor of a journal and weighted Hirsch index of an individual scientist or of an entire scientific institution.

Appendices

Appendix A: Some exemplary graph matrices and their leading eigenvectors

In this appendix we provide examples of some simple matrices and analyze properties of their leading eigenvector. Although a matrix of a small size N directly represents only a small citation graph which describes a small group of N scientists, it can be also applied to model a huge graph with a sub-graph structure: each vertex may represent a given field or subfield of science. Therefore studying even such oversimplified cases can be helpful in understanding the properties of the connectivity matrix of a citation graph and its leading eigenvector.

Let us start with the simplest case of N = 2,

$$ C_2=\left[ \begin{array}{cc} 0 & a\\ b & 0 \end{array} \right], \quad x=\left[ \begin{array}{c} \sqrt{a} \\ \sqrt{b} \end{array} \right]. $$

(17)

The leading eigenvalue reads $\lambda=\sqrt{ab},$ and in this case the weights x _i given be the corresponding eigenvector are proportional to the square root of the flow between the vertexes. Obviously this is not longer the case for larger graphs,

$$ C_3=\left[ \begin{array}{ccc} 0 & a & 0\\ 0 & 0 & b \\ c & 0 & 0 \end{array} \right] \quad \quad x=\left[ \begin{array}{c} a^{2/3}b^{1/3}\\ b^{2/3}c^{1/3}\\ a^{1/3}c^{2/3} \end{array} \right]. $$

(18)

The following numerical example shows that the weights given by the leading eigenvector grow slower than linearly with the average entry in each row,

$$ C_4 =\left[ \begin{array}{cccc} 0 & 1 & 1 & 1\\ 2 & 0 & 2 & 2\\ 3 & 3 & 0 & 3\\ 4 & 4 & 4 & 0 \end{array} \right], \quad \quad x \approx \left[ \begin{array}{c} 0.3223\\ 0.5738\\ 0.7755\\ 0.9409 \end{array} \right]. $$

(19)

Consider now some other numerical examples of size four

$$ C =\left[ \begin{array}{cccc} 0 & 4 & 0 & 0\\ 6 & 0 & 2 & 1\\ 0 & 0 & 0 & 0\\ 0 & 0 & 3 & 0 \end{array} \right], \quad \quad x \approx \left[ \begin{array}{c} 0.6325 \\ 0.7746 \\ 0 \\ 0 \end{array} \right]. $$

(20)

Observe that quotations by authors, the papers of which were never cited do not contribute at all to the weighting index!

$$ C = \left[ \begin{array}{cccc} 0 & 6 & 6 & 6\\ 2 & 0 & 1 & 1\\ 0 & 1 & 0 & 1\\ 1 & 0 & 1 & 0\\ \end{array} \right], \quad \quad x \approx \left[ \begin{array}{c} 0.8975\\ 0.4228\\ 0.1247\\ 0.2036 \end{array} \right]. $$

(21)

Similarly, quotation by a junior scientist, the papers of which received a little attention of the scientific community, are much less important than a citation by an accomplished author. This is seen by comparing the third and the fourth component of the eigenvector of the above citation matrix, in which the first two rows represent a renowed researcher and a less experienced author, respectively.

It is illustrative to analyze the case of two weakly connected subgraphs, represented below by the first and the second pair of nodes. If the coupling between the subgraphs is symmetric, C _2,4 = C _4,2 the leading eigenvector lives in both subspaces,

$$ C = \left[ \begin{array}{cccc} 0 & 4 & 0 & 0\\ 6 & 0 & 0 & 1\\ 0 & 0 & 0 & 4\\ 0 & 1 & 6 & 0 \end{array} \right], \quad \quad x \approx \left[ \begin{array}{c} 0.4197\\ 0.5691\\ 0.4197\\ 0.5691 \end{array} \right], $$

(22)

However, if there is more fluxes between both subgraphs start to differentiate, the weight of the leading vector moves toward the distinguished subsystem,

$$ C = \left[ \begin{array}{cccc} 0 & 4 & 0 & 0\\ 6 & 0 & 0 & 1\\ 0 & 0 & 0 & 4\\ 0 & 0.5 & 6 & 0 \end{array} \right], \quad \quad x \approx \left[ \begin{array}{c} 0.4939\\ 0.6502\\ 0.3493\\ 0.4597 \end{array} \right], $$

(23)

$$ C =\left[ \begin{array}{cccc} 0 & 4 & 0 & 0\\ 6 & 0 & 0 & 1\\ 0 & 0 & 0 & 4\\ 0 & 0.1 & 6 & 0 \end{array} \right], \quad \quad x \approx \left[ \begin{array}{c} 0.5913\\ 0.7480\\ 0.1870\\ 0.2365 \end{array} \right]. $$

(24)

If two graphs are not connected, the leading eigenvalue is degenerated and one finds a corresponding eigenvector localized exclusively in the more populated subspace,

$$ C =\left[ \begin{array}{cccc} 0 & 4 & 0 & 0\\ 6 & 0 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0 & 0 & 3 & 0 \end{array} \right], \quad \quad x \approx \left[ \begin{array}{c} 0.6325\\ 0.7746\\ 0\\ 0 \end{array} \right]. $$

(25)

To lift such a degeneracy one may modify the analyzed matrix C by forming its convex combination with the flat matrix S such that S _ij  = 1/N. In this way one assures (Brin et al. 1998) that the leading eigenvector of C(p) = (1 − p)C + pS can be obtained by iterating sufficiently long the flat vector with all entries equal, w _i  = 1/N, by the matrix C(p).

Appendix B: Practical remarks on evaluating the weighting vector

Selection of the data

The key issue by constructing the citation graph is an access to a reliable data base containing the scientific literature. For instance one may rely on the data provided by the ISI Web of Science, although some experts claim that it is biased toward the scientific journals published in English only and it does not cover uniformly the entire literature. Alternatively one may chose to use some publicly open web search engines, like Google Scholar. In this case it is believed that Google does not cover systematically earlier scientific literature. Furthermore it is not clear how to set simple criteria, which web documents should be taken into account. On one hand one might restrict the attention to the papers published by a scientific journal, which is first found in an earlier compiled list of all sources accepted. On the other hand, due to popularity of various web archives and preprint depositories (like arxiv.org) one might also accept formally unpublished preprints posted there. In such a case a special care has to be taken in order to avoid double counting the same article, first deposited in an archive, and later published in a journal, often under a slightly changed title.

Different fields of science

As illustrated with some simple matrix examples, if two fields of science are not coupled by any cross-citations, the leading vector describes only scientists working in the larger field. Similarly, if two fields of science are coupled only weakly by a few cross-citations, the leading eigenvector tends to be localized in the subgraph with more scientists, papers and citations, so the weighting factors handicap researchers working in a less popular subfield. The splitting of the entire graph into subgraphs can be defined in an objective way by applying the recent method of Newman (2006) to find community structure in the citation graph. Since it is well known that the citation patterns depend on the branch of science (Batista et al. 2006, Althouse et al. 2009), one should rather analyze two subgraphs separately, or renormalize the leading eigenvector separately for a given subfield. This is consistent with a rather general ’rule of thumb’: the bibliometric data should be normalized against the average computed for scientists working in the similar field of science in the corresponding window of time (Radicchi et al. 2008).

Degeneracy in names

It might not be easy to distinguish papers written by various scientist, who publish under the very same name (Aksens 2008). In principle one may try to distinguish them by the scientific discipline, the affiliations and the time window of their publishing activity, but is its unlikely to expect that the success rate will tend to unity. On the other hand, it is reasonable to conjecture that not distinguishing between the scientists with the same name will not impact much the weighting indices of all other researchers in the graph, as the weights of the links will be taken as the average.

Period of the scientific activity

It would be unwise to compare weighting indices of two researchers in very different age or living in different times. The number of universities, scientists, journals, papers and citations keeps growing fast. Hence one should expect that a comparison of two scientists with equally valuable accomplishments, whose scientific contributions are already forgotten (and their papers are not quoted any more), would reveal that the scientist active more recently is characterized by a larger weighting factor.

Uniqueness of the leading eigenvector of the citation matrix

A matrix C is called reducible if it can be transformed by a permutation P into matrix with a zero block below the diagonal, $C^{\prime}=PCP^T= \left[ \begin{array}{cc} D_1 & Z \\ 0 & D_2 \end{array} \right],$ where D ₁ and D ₂ are square matrices. In the opposite case the matrix is called irreducible. The Frobenius-Perron theorem implies that for any irreducible non-negative matrix C its spectral gap is positive, γ: = z ₁ − |z ₂| > 0, so the real eigenvector ${\vec x}$ corresponding to the leading eigenvalue z ₁ is unique. The size of the spectral gap governs the speed of the convergence of any initial vector iterated several times by C to the invariant state ${\vec x}=C{\vec x}.$

The initial citation matrix C analyzed in this paper in principle could reducible, but due to numerous cross-citations between various researchers and subfields this possibility seems to be unlikely. Furthermore, the auxiliary (N + 1)-th node of the graph representing all scientists outside the ensemble under investigation introduces additional connectivity and hence increases (on average) the spectral gap.

The size of the spectral gap for the graph matrix describing entire scientific literature has to be determined in a numerical experiment. If the gap occurs to be too small to ensure convergence time realistic for practical implementations, one may always introduce a suitable modification of the citation matrix C. For instance, following the original idea of Page Rank (Brin et al. 1998), one could mix C with the flat matrix S such that S _ij  = 1/N (see also Langville et al. 2005, Bryan et al. 2006).