Project:Analytics/WCD: Difference between revisions
(New WCD) |
(+finding) |
||
| Line 21: | Line 21: | ||
* [https://citations.scatteranalytics.com/explorer/template/6385/report?parameter_key=isbn¶meter_value=978-0-226-77142-7 Same ISBN in "Cite book" template used across 22 articles with varying citations] | * [https://citations.scatteranalytics.com/explorer/template/6385/report?parameter_key=isbn¶meter_value=978-0-226-77142-7 Same ISBN in "Cite book" template used across 22 articles with varying citations] | ||
* [https://citations.scatteranalytics.com/explorer/citation/bdbb085b4fb9a609b67d8732f076eab3d8d6a536/other-articles?page_id=3410 Two citations to the same book, but without ISBN] | * [https://citations.scatteranalytics.com/explorer/citation/bdbb085b4fb9a609b67d8732f076eab3d8d6a536/other-articles?page_id=3410 Two citations to the same book, but without ISBN] | ||
* [https://citations.scatteranalytics.com/explorer/template/6385/report?parameter_key=isbn¶meter_value=%201-873410-89-1%0A%20 The same book cited on five articles with different citations] | |||
Potential bugs: | Potential bugs: | ||
Revision as of 19:52, 27 June 2026
Introduction
Wikipedia Citations Database is an effort supported by Internet Archive to build a comprehensive, historical database of each citation to appear on Wikipedia.
While methodologies of similar projects have focused on extracting standardized identifiers or easy-to-extract references, this project instead seeks to build a complete database, based on analyzing the structure.
Generations
- WCD Generation 1: Wikibase instance on Wikibase Cloud. Importing data was painfully slow, and it was decided Wikibase was not a good format for the data we were trying to store. The dataset produced is lost and was not very interesting or useful.
- WCD Generation 2: IARI; Postgres-based, but was extremely difficult to work with. Parts of it are used to support the Internet Archive Reference Explorer.
- WCD Generation 3: The latest attempt, using wiki-references-extractor and wiki-references-db (WRDB). WRDB is a core component of the broader WCD project. Version 2 adds RevisionChest as a pre-processing step.
- Version 1:
https://wikipediacitations.scatter.red- Partial build of English Wikipedia
- Has issues dealing with broken wikitext
- Version 2: https://citations.scatteranalytics.com
- Version 1:
- WCD Generation 4 (pending): To be built off of a combination of Generation 3 and UVAWikipediaCitationsDatabase which adds mapping between extracted citations and underlying sources
Interesting findings
All are English Wikipedia articles
- Non-templated citation used on three different articles
- Same ISBN in "Cite book" template used across 22 articles with varying citations
- Two citations to the same book, but without ISBN
- The same book cited on five articles with different citations
Potential bugs:
- Discrepancy between same-citation report and matching template parameter (if two citations are an exact match then they should appear in the same template parameter use report)
Longer term challenges
- Sometimes, Wikipedia includes statements from Wikidata, and those statements from Wikidata have citations, but they would not show up in revision text.
- Would this require cross-referencing with rendered HTML output?
- Or a "citations database plus" that includes the Wikidata item? (Wikipedia editors would probably like this for quickly comparing citations between Wikipedia article and Wikidata item.)