Call for Participation: ACM SIGPLAN TRUST'14 @ PLDI'14 with a panel on conference artifact evaluation and reproducible research

===============================================================
                  CALL FOR PARTICIPATION

      ACM SIGPLAN TRUST 2014 co-located with PLDI 2014

     1st Workshop on Reproducible Research Methodologies
     and New Publication Models  in Computer Engineering

               June 12, 2014, Edinburgh, UK
   
            http://c-mind.org/events/trust2014
===============================================================
   Early PLDI and workshop registration deadline: 7 May 2014

           http://conferences.inf.ed.ac.uk/pldi2014

       Make sure to tick the box for the TRUST workshop 
===============================================================
Program:

* Panel on conference artifact evaluation experience

  Participants:

  - Jan Vitek (Purdue University, USA)
  - Shriram Krishnamurthi (Brown University, USA)
  - Christophe Dubach (University of Edinburgh, UK)
  - Grigori Fursin (INRIA, France)

* Talks:

  1) "Invited presentation"
     Christian Collberg, University of Arizona, USA

  2) "CARE, the Comprehensive Archiver for Reproducible Execution"
     Yves Janin, Cedric Vincent and Remi Duraffort
     STMicroelectronics, France

  3) "Academia 2.0: removing the publisher middle-man while retaining impact"
     Raphael Poss (A), Sebastian Altmeyer (A), Mark Thompson (B), Rob Jelier (C)
     (A) University of Amsterdam, Netherlands
     (B) L.U.M.C., Netherlands
     (C) KU Leuven, Belgium

  4) "Falsifiability of network security research:
     the Good, the Bad, and the Ugly"
     Dennis Gamayunov, Lomonosov Moscow State University, Russia

  5) "Software in reproducible research: advice and best practice
     collected from experiences at the Collaborations Workshop"
     Mario Antonioletti, Neil Chue Hong, Stephen Crouch, Alexander Hay,
     Simon Hettrick, Devasena Inupakutika, Mike Jackson, Aleksandra Pawlik,
     Giacomo Peru, John Robinson, Shoaib Sufi, Les Carr, David De Roure,
     Carole Goble, and Mark Parsons, UK

  6) "Introducing OCCAM project"
     Bruce Chillders, University of Pittsburgh, USA
===============================================================
Preliminary schedule and further information is available
at the TRUST workshop website:

http://c-mind.org/events/trust2014

Thematic session at HiPEAC CSW "Is Current Research on Heterogeneous HPC Platforms inline with Real-world Application needs?"

My colleagues Marisa Gill and Chris Fench are organizing an interesting
thematic session and a panel "Is Current Research on Heterogeneous HPC Platforms
inline with Real-world Application needs?".

It will be held on the 15th of May during HiPEAC Computing Systems Week
in Barcelona. Join us there:

http://www.hipeac.net/add/res/102/467
http://www.hipeac.net/csw/2014/barcelona

Weekend digest of tools and licenses related to reproducible R&D

Dear colleagues,

Here is a small summary of tools and licenses that may be useful
for reproducible research and experimentation that I assembled
after SEA'14 conference in Boulder:

* CDE tool: automatically create portable Linux applications
  with all dependencies:
  http://www.pgbovine.net/cde.html

* Docker tool: pack, ship and run applications as a lightweight container
  https://www.docker.io

* The IPython Notebook: a web-based interactive computational
  environment where you can combine code execution, text,
  mathematics, plots and rich media into a single document:
  http://ipython.org/notebook.html

* CRAPL license: promote academic code sharing even if it's ugly -
  it's an interesting concept, though not helping my research problems
  at this moment since I need (and promote) not just code and data
  sharing but also their APIs and meta-description to be reusable...
  http://matt.might.net/articles/crapl

Have a good weekend,
Grigori

Digest on reproducible research initiatives

Dear colleagues,

 

Just wanted to share a few links related to reproducible research:

 

* Many interesting software projects for research reproducibility

have been presented at the Collaborations Workshop 2014:
http://www.software.ac.uk/collaborations-workshop-2014-cw14-software-your-reproducible-research/cw14-agenda
All presentations should be available soon!

 

* Prof. Carole Globe presented a nice overview of software projects for reproducibility of research and experimentation:

https://www.dropbox.com/s/67n49i9k3cb28mo/GOBLE-CW2014.ppt

 

* Prof. Globe mentioned yet another interesting initiative on reproducible research:

http://reproduciblescience.org

 

* Interesting and a bit controversial article about scientific peer reviewing:

http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review

 

Hope of it will of any interest and take care,

Grigori

 

ADAPT'14 shared and validated research material

As a continuing effort for validation of experimental results by the community and for testing of Collective Mind repository, we validated two papers from ADAPT'14 workshop with the great help of volunters (Alberto Magni from the University of Edinburgh, UK and Sascha Hunold from Vienna University of Technology, Austria) and shared results here:

• Cedric Nugteren et.al. "Roofline-aware DVFS for GPUs" [ cM link ]
• Chris Schneider et.al. "Autonomous Fault Detection in Self-Healing Systems: Comparing Hidden Markov Models and Artificial Neural Networks" [ cM link ]

ADAPT'14 panel on reproducible research methodologies and new publication models

Finally, I found some time to add all slides from the ADAPT'14 panel on reproducible research methodologies and new publication models here: http://adapt-workshop.org/program.htm

 

I was very glad to see many participants and hot discussions on how to:

• improve publication reviewing process involving more reviewers, sharing of all related material for reproducibility, developing standards for fair and statistical evaluation , etc.
• involve ACM and encourage conferences/journals to promote sharing of related research material and validation of experimental results 
• validate and rank shared research artifacts
• encourage validation and public implementation of already published techniques
• develop common repository for shared material including benchmarks, data sets, tools, models, etc.
• encourage companies to share tools and data

Though there is now yet unified way on the above problems, I was very glad to see active participation and hot discussions. In order to continue and systematize these discussions, I strongly encourage to submit (short position) papers on these topics to our 1st ACM SIGPLAN TRUST'14 workshop that will be co-located with PLDI'14 (June 12, Edinburgh, UK).

This is first test message related to Collective Mind Methodology. I am looking for collaborators to gradually make computer engineering more systematic and reproducible through public repository of knowledge, auto-tuning and data mining framework and new publication model.

I understand that it is really not an easy task but our first results on cTuning.org are very promising so I hope that we will be able to build a strong community and make computer engineering a science!

This initiative is supported by [http://cTuning.org/lab cTuning community/virtual Lab] and HiPEAC network of excellence 2012-2016 (see our [http://www.hipeac.net/thematic-session/collective-characterization-optimization-and-design-computer-systems thematic session] and [http://exadapt.org/2011 EXADAPT 2011], [http://exadapt.org/2012 EXADAPT 2012] and [http://homepages.inf.ed.ac.uk/cdubach/adapt2013 ADAPT 2013] workshops).

You are very welcome to join or follow us!