Project:Analytics/WCD: Difference between revisions
(Another report) |
(→Interesting findings: Resource Description Framework) |
||
| Line 30: | Line 30: | ||
*Only one page shows up on report when two should | *Only one page shows up on report when two should | ||
**[https://citations.scatteranalytics.com/explorer/citation/2ca3ae9d1d69ef6ba9336ce4df1581057a34ddd9/other-articles?page_id=30495845 This citation] appears on ''[https://citations.scatteranalytics.com/explorer/article?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FBoops_boops&follow_redirects=1 Boops boops]'' and ''[https://citations.scatteranalytics.com/explorer/article?url=https%3A//en.wikipedia.org/w/index.php%3Fcurid%3D60478829 Microcotyle isyebi].'' [https://citations.scatteranalytics.com/explorer/template/1374/report?parameter_key=doi¶meter_value=10.1007/s00436-019-06293-y Template parameter report] only shows one article even though the link on both resolve to the same URL (so not the issue of spaces at the parameter boundary) | **[https://citations.scatteranalytics.com/explorer/citation/2ca3ae9d1d69ef6ba9336ce4df1581057a34ddd9/other-articles?page_id=30495845 This citation] appears on ''[https://citations.scatteranalytics.com/explorer/article?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FBoops_boops&follow_redirects=1 Boops boops]'' and ''[https://citations.scatteranalytics.com/explorer/article?url=https%3A//en.wikipedia.org/w/index.php%3Fcurid%3D60478829 Microcotyle isyebi].'' [https://citations.scatteranalytics.com/explorer/template/1374/report?parameter_key=doi¶meter_value=10.1007/s00436-019-06293-y Template parameter report] only shows one article even though the link on both resolve to the same URL (so not the issue of spaces at the parameter boundary) | ||
---- | |||
* [https://citations.scatteranalytics.com/explorer/article?url=https%3A//en.wikipedia.org/w/index.php%3Fcurid%3D53847 Resource Description Framework] inspires all sorts of edge behavior | |||
== Longer term challenges == | == Longer term challenges == | ||
Latest revision as of 03:03, 14 August 2026
Introduction
Wikipedia Citations Database is an effort supported by Internet Archive to build a comprehensive, historical database of each citation to appear on Wikipedia.
While methodologies of similar projects have focused on extracting standardized identifiers or easy-to-extract references, this project instead seeks to build a complete database, based on analyzing the structure.
Generations
- WCD Generation 1: Wikibase instance on Wikibase Cloud. Importing data was painfully slow, and it was decided Wikibase was not a good format for the data we were trying to store. The dataset produced is lost and was not very interesting or useful.
- WCD Generation 2: IARI; Postgres-based, but was extremely difficult to work with. Parts of it are used to support the Internet Archive Reference Explorer.
- WCD Generation 3: The latest attempt, using wiki-references-extractor and wiki-references-db (WRDB). WRDB is a core component of the broader WCD project. Version 2 adds RevisionChest as a pre-processing step.
- Version 1:
https://wikipediacitations.scatter.red- Partial build of English Wikipedia
- Has issues dealing with broken wikitext
- Version 2: https://citations.scatteranalytics.com
- Version 1:
- WCD Generation 4 (pending): To be built off of a combination of Generation 3 and UVAWikipediaCitationsDatabase which adds mapping between extracted citations and underlying sources
Interesting findings
All are English Wikipedia articles
- Non-templated citation used on three different articles
- Same ISBN in "Cite book" template used across 22 articles with varying citations
- Two citations to the same book, but without ISBN
- The same book cited on five articles with different citations
Potential bugs:
- Discrepancy between same-citation report and matching template parameter (if two citations are an exact match then they should appear in the same template parameter use report)
- Only one page shows up on report when two should
- This citation appears on Boops boops and Microcotyle isyebi. Template parameter report only shows one article even though the link on both resolve to the same URL (so not the issue of spaces at the parameter boundary)
- Resource Description Framework inspires all sorts of edge behavior
Longer term challenges
- Sometimes, Wikipedia includes statements from Wikidata, and those statements from Wikidata have citations, but they would not show up in revision text.
- Would this require cross-referencing with rendered HTML output?
- Or a "citations database plus" that includes the Wikidata item? (Wikipedia editors would probably like this for quickly comparing citations between Wikipedia article and Wikidata item.)