Enabling open science articles

[ Home > Reproducibility ]

Assorted articles and presentations

• Bruce R. Childers et.al. "Active Curation of Artifacts and Experiments is Changing the Way Digital Libraries will Operate", WSSSPE4
• Simon Harper, "The Problem with Computer Science Research Prototypes"
• Grigori Fursin, et.al. Collective Knowledge: towards R&D sustainability, DATE 2016
• David H. Bailey, "Facilitating Reproducibility in Scientific Computing: Principles and Practice"
• G.Fursin and C.Dubach, "Community-driven reviewing and validation of publications", TRUST'14 @ PLDI'14
• Don Monroe, "When Data Is Not Enough", CACM
• David Rosenthal, "Emulation & Virtualization as Preservation Strategies", (PDF)
• Dirk Beyer et.al. "BenchExec: Reliable and Reproducible Measurement and Controlling of Computing Resources"
• Jure Triglav, "Thoughts on reproducibility of open scientific software"
• Amy E. Hodge, "SDR Deposit of the Week: Facilitating reproducibility"
• Ian P. Gent, "The Recomputation Manifesto"
• Konrad Hinsen, "ActivePapers: a platform for publishing and archiving computer-aided research"
• Daniel S. Katz, Arfon M. Smith, "Implementing Transitive Credit with JSON-LD"
• Ryan Chamberlain, Jennifer Schommer, "Using Docker to Support Reproducible Research"
• B. A. Nosek, "Promoting an open research culture"
• The Economist, "Let's just try that again"
• PL conference papers to get a journal - discussing ACM proposal to formally recognize conference publications as equal in quality as journal publications - controversial
• Our experience report with encountered problems in computer systems' research and new proposal for community-driven reviewing and validation of publications arXiv / ACM DL
• Collective Tuning Initiative: automating and accelerating development and optimization of computing systems
• Science's Reproducibility Problem, Bob Grant, The Scientist
• From Repeatability to Reproducibility and Corroboration, Dror G. Feitelson, SIGOPS Operating Syst. Rev. 49(1), pp. 3-11, Jan 2015
• Reviewing peer review, Jeannette M. Wing, ACM Blog
• How Should Peer Review Evolve?, Ed H. Chi, ACM Blog
• Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers, David H. Bailey, Supercomputing Review, June 11, 1991
• [Ten], Scott Pakin, HPCWire, December 13, 2011
• Cambridge University news about open and reproducible research
• Interesting Replicable "Badge" for journal articles, Daniel S. Katz
• What's in a Name? The Concepts and Language of Replication and Reproducibility
• Are Cycle Accurate Simulations a Waste of Time?, Vincent M. Weaver and Sally A. McKee
• Research Wranglers: Initiatives to Improve Reproducibility of Study Findings (Environmental Health Perspectives)
• Reproducible Research in Computational Science: Problems and Solutions For Data and Code Sharing, Victoria Stodden, ICML workshop 2010
• Predatory publishing
• Three myths about scientific peer review
• The repeatability experiment of SIGMOD 2008, SIGMOD Record, vol. 37, no. 1 (March 2008), 39-45.
• "Begin with an Author's response to Reviews" - proposal to submit past reviews along with articles
• Presentation by Professor Carole Goble
• Dennis McCafferty, "Should Code be Released?"
• Elsevier:Can data be peer-reviewed?
• Collective mind: Towards practical and collaborative auto-tuning
• Collective Mind, Part II: Towards Performance- and Cost-Aware Software Engineering as a Natural Science
• Elsevier:The Executable Paper Grand Challenge
• Special section on Data replication & reproducibility, Science magazine, 2 December 2011
• Chris Drummond, "Replicability is not Reproducibility: Nor is it Good Science"
• Science is in a reproducibility crisis - how do we resolve it?
• W3C Dataset Descriptions: HCLS Community Profile
• My blog article on "Automatic performance tuning and reproducibility as a side effect"  for the Software Sustainability Institute
• Trouble at the Lab, The Economist, 2013
• Joint Declaration of Data Citation Principles
• Puzzling Measurement of "Big G" Gravitational Constant Ignites Debate
• International vocabulary of metrology – Basic and general concepts and associated terms (VIM)
• White House takes notice of reproducibility in science, and wants your opinion
• Problems during performance benchmarking:
  • https://homes.cs.washington.edu/~bornholt/post/performance-evaluation.html
    We also experienced many similar issues during our work on auto-tuning and machine learning:
  • http://hal.inria.fr/hal-01054763
  • http://arxiv.org/abs/1406.4020
• ACM SIGOPS Operating Systems Review - Special Issue on Repeatability and Sharing of Experimental Artifacts
• Vinton G. Cerf. "Bit Rot: Long-Term Preservation of Digital Information" [Point]
• About citations: less related though interesting

Journals with artifact sharing and evaluation

• IPOL Journal: Image Processing On Line
  
• ACM TRANSACTIONS ON MATHEMATICAL SOFTWARE (TOMS)
• ACM SIGMOD 2015

National requirements

• UK EPSRC expectations on research data management
• University of Manchester, UK:
  • Policy
  • Data management planning
• Cornell University, USA: Best Practices

If you would like to discuss our open science initiative, do not hesitate to get in touch via CK mailing list!