When and what to share publically

In my last post in the series on analytical problems in dendroclimatology, the issue of sharing computer code was raised, which is part of the larger issue of being fully open about one’s claims, so that other people can check them.  I am pretty much 100% in the camp that everything in science needs to be defensible, and so all critical assumptions, inferential methods, data, computer code etc., does need to be made available to the world at large.  We all do definitely have to defend our work, and that’s how you defend it.  Saying “trust me” will not cut it.

But it’s not really as simple as just handing over everything you have done–there are questions of what exactly is owed, to whom, and in what form and when it is owed. There is a contingent of people that apparently thinks every scientist has some massive pile of federal grant money that funds their work and is therefore obligated to immediately release every piece of work they’ve ever done, upon request. That’s just not even remotely the case. In my case, a tremendous amount of completely unpaid work went into the coding and general discovery of the issues I’m addressing in these posts, while I have gotten utterly nothing at all in return for it.  If I were to go into detail on this, people would likely be surprised or shocked, judging from the looks on peoples’ faces when I tell them in person.  Not only did I have no grant money or salary, and receive no income, but in fact, so far the work has cost me a great deal of money, time, sleep and serenity.  I did it because I’m interested in the topic and I get pretty obsessive when I tear into something like this, usually being unable to drop it.

After that individual’s comment I later stated that I would put some code up demonstrating why what I have claimed in that post is true.  And indeed, because the original code is extremely long and complex and part of a much larger project that involves the development of new analytical methods, I can’t just release it and have it be useful in any meaningful way in concisely illustrating the concepts there–it’s too long and unwieldy and messy and full of comments.  It still needs cleaning up in various ways, even if the basic structure is sound.

This in turn means I would have to basically start from scratch and  write new code that concisely produces the results I showed in that post in order to adequately respond to a request/demand for code to be put up.  And so indeed that is exactly what I did the last three hours, and was just about to post up the code, when I got to thinking about just what is, and what is not, legitimate to demand from someone, especially anonymously on the internet, and especially when people accuse you of playing games and make statements that indicate that doing such work themselves is a burden.

It makes me wonder who the average person thinks we scientists are exactly.  Or perhaps my situation is just so out of the norm with respect to this work, that people naturally assume things that are far from true, I don’t know.

So I really don’t know if I’m going to post the code I just got done writing or not. I really don’t.

Advertisements

7 thoughts on “When and what to share publically

  1. Why do you do science? Of course, you need to get paid to live. But, over and above that, I think that we do science because it provides a contribution to the sum total of human knowledge – it assists humanity. And for that, I do not expect to get paid (or, indeed, receive much acknowledgement – even if I spend a lot of my spare time and my own money in the process). Your mileage (and that of medical researchers) may differ, of course.

    Getting plaudits from like-minded supporters is nice for the ego. But the most useful advances generally come from people who may not believe you and are prepared to spend a lot of their time looking for holes in your work. These people may be rude – they may have a grudge against you – they may not behave politely (and I have seen far worse than your example on the net). Nevertheless, these are the people who will test your hypothesis the best.

    If they try as hard as they can, and can only come up with minor amendments or made-up objections, then you know you have succeeded. This is the benefit you gain from releasing your work in sufficient detail to ‘someone who wants to find fault with it’.

    • I appreciate the thoughts Geezer. I do science because it’s interesting and useful. I would say the best reviewers are those who are simply the most objective, neither excessively critical nor lenient. If a reviewer has an attitude that leans either way, it’s most likely to come through in their decision. The problem is that during the peer review process, if you run into reviewers intent on keeping your paper from getting published, they will indeed make up the necessary objections to do so. This is exactly what happened in my case, but I’m still fully confident of what I’ve done nevertheless. And also, the other issue in play here is idea theft–people do that too.

    • And as I mentioned, its primarily the demanding attitude of “give me your code, now” that is aggravating, especially when the necessary information to do one’s own evaluations, has already been provided.

  2. Jim, it appears that the scientific world is seeing a shift. There are those that want to eschew the “old” peer review system. Throw your research onto the net and let everyone have a go. If you embrace this attitude, then post your code. However if you feel that the peer review script is solid and you prefer this route, then no code until published.

    IMO

    • DeNihilist, I’m solidly among those who believe that the peer review process has significant problems and can be greatly improved upon. This is a tough situation to be in and I’m considering it as carefully as I can.

  3. Hi Jim,
    Thank you so much for your dendro series. I understand enough to be fascinated but not enough to ask intelligent questions.

    As a software developer and former small business owner I am familiar with intellectual concerns. Your code is your property..

    The work you are doing to share your results to folk like me is wonderful. We partake as a privilege not a right. There is nothing wrong with having us wait patiently for you to get published and subsequently share what you want. After all, those of us who have been concerned about the “science of treenometers” for many years have gone without your systematic treatment. We can go without for a few more months or years.

    • Thank you for the kind words and the patience Robert, they are greatly appreciated. Scientific results/knowledge should be as fully open to the public as possible, and it’s up to all of us to try to make it so. And of course, a lot of people are trying their best to do just that, people that you never hear about, doing what they can every day to make that happen.

Have at it

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s