Monday, November 30, 2009

Jumping into the deep end

It make no sense to jump into the middle of a churning pool, especially when there is already blood in the water. As it happens, I don't have much in the way of good sense, but I do have some stories that I want to tell and that's going to mean getting wet.

And although I'm not terribly skilled at it, I think that the best thing that people can do in any community when we see a confrontation is to acknowledge it openly. The really hard thing is to do this without creating more discord. Paradoxically, I think that one way to do that is by offering our own experiences as directly and openly as possible but with the intention of being helpful. And this doesn't necessarily mean "seeing both sides". So I'll say from the very beginning that my personal and professional experience and interests make me more sympathetic to the foundation than to others. Ultimately there aren't any sides at all, only points of view that become more and more hardened the more we toss them back and forth at each other mindlessly. It's much more difficult to try and soften these points of view, because that involves exposing our own fears and vulnerabilities -- in other words our emotions. We have to actually be willing to say that our arguments aren't necessarily coming from lofty principles or having the right answer, but from wanting to express those feelings. When we get to the heart of the matter it isn't about the "issues" per se, but what kind of needs we think expressing our points of view about them are going to serve.

And actually I think I do have a few experiences that might be relevant and somewhat unique. I'm quite new to the Eclipse community itself. I have been contributing in some way or another for just a couple of years, and have been a committer and incubation project lead for less than a year. As I mentioned in an earlier post, I am not a part of a member company, and in fact there is no way that my tiny company can afford to pay membership fees, which means that although we are doing all of the work, I can really only mention my own commercial interests in passing. And on the other hand, the Eclipse foundation had no immediate hope of getting any positive return on my project. Actually there are great opportunities for the foundation and the community that Mike and others at the foundation have been extremely supportive of but at the time I put the project together there was no hint of that. Now notice something about what I just wrote -- with the exact same set of circumstances, I could make up a story that told about how the foundation unfairly favors large companies, how there is no way for me to build support for a small-scale commercial effort. Or I could tell people how the foundation has gone out of it's way to encourage independent efforts with no thought of immediate benefit.

I invested an enormous amount of my time navigating the internal processes including going through the laborious and sometimes intense IP process. My project had a number of very complex and political issues involved including existing source code that could not be made to be Eclipse compliant, and very challenging negotiations with another project to get code EPL'd. So my story could be that Eclipse invested an enormous amount of time and money in my little project, a project that could have turned out to contribute nothing to the overall eco-system. On the other hand, I could complain that it was just too much work, and who in their right mind would go through such an effort. How can we expect to have an open ecosystem, with such a high barrier to entry? On the other other hand, I could think of how much I would have had to pay for the specialized legal advice and due-diligence, custom IP scanning software and so on, and how Mike and the legal team were willing to go to bat for me when we were trying to resolve external difficulties. I think that people that don't work with large commercial and especially government institutions don't appreciate how incredibly valuable the Eclipse IP process actually is and how much work and thought have gone into it. Without very solid IP, many organizations won't even look at your software, let alone deploy it. Eclipse is really literally the gold standard of IP cleanliness in the Open Source world and having that mark on my little project's effort is enormously valuable. If I didn't find it valuable, I wouldn't be here, so how can I complain?

I am trying to build a software ecosystem, and I've been trying to convince people to get involved in a project while telling them that we have to follow the Eclipse project committer process guidelines and that I can't just give them committer access as I could for my sourceforge project. And my story there could be that the Eclipse process makes outside involvement nearly impossible except for the most committed individuals. Who would go through this unless someone else was paying them to do it? Wouldn't it be better to focus our resources on making the committer process as open as possible? Or I could instead tell the story about how badly I've seen this go in other projects -- where institutions have stuffed the project developer lists with their employees so that the efforts of the primary contributors would be less obvious, and where project leadership was based not on project contributions but on institutional management prerogatives and projects were designed and promoted to further the interests of one institution. And then I'd exclaim how important the meritocratic principles -- and most importantly practices -- are to making Eclipse what it is.

I have immersed myself in the culture and ethos of the Eclipse ecosystem. All of the efforts I've made over the past three or four years are tied up in the success of this ecosystem. If Eclipse manages to maintain and expand its reputation among developers and commercial, academic and government institutions then all of that time and effort will have been worth it. If it doesn't, then it will be largely wasted. So my story could be that there is something clearly wrong with the Eclipse foundation and we need to "fix it". The foundation needs to listen to all of the voices that have been calling for change. Look at the other successful Open Source projects out there. Things are working so well there -- why can't we be more like them? Or why don't we just leave Eclipse altogether? But why should we have to? Or my story could be that it is obvious to me that the foundation has been listening, and that they have made a really strong effort in responding to these concerns and that there are some really cool new efforts underway. There has been enormous progress in the last year alone on technical and process issues. But if you don't like the way things are going then you really should fork. I've done it myself and it worked out. But that was easy as I forked code that I had personally written. How many of the current Eclipse developers are going to come with you? Are you really appreciating just how much work and infrastructure is involved in Eclipse? Just getting a full build going would be a hugely significant effort.

OK, and now we come to the tough part. I've been watching the "Bjorn v. Foundation" drama for some time now, first completely ignorant of the bitter subtext of blog postings, then as a bemused observer, and finally now I guess as a walk-on. I could tell a story about how everything Bjorn has said seems perfectly reasonable. I mean, I don't know the guy -- I've never been to an Eclipse Con -- but he obviously has a lot of interesting insights. And at least one person who I've known for a long time and respect in the tech world has been strongly supportive of him. And where there is smoke there is fire. This wouldn't be the first time that an organization has been challenged by someone and turned on the messenger. And anyway, wow, isn't it a bit weird that the director of the foundation is calling him out like that in a public forum? What are people outside of the community going to think of this? Or my story could be that sure, there's smoke, and it's Bjorn who's holding the half-empty box of matches. I have met a bunch of folks from the foundation now and I can't say I've run into anyone who seems especially manipulative or who appears to be carrying a hidden agenda that goes against the broader development and user community. On the other hand, I have heard independently from a number of folks that in terms of day to day interactions Bjorn is a pain in the ass at best and a toxic influence at worse. This seems to be said with more of a sense of weariness and genuine hurt than with malice or hostility. To the extent that that is true, the fact that noone has said much publicly about Bjorn's difficulty in working with others -- to the contrary -- speaks of restraint, and it's actually really refreshing and honest for Mike to finally just come out directly and say what needed to be said. The worse thing that can happen is for the no longer subtle poisonous atmosphere to go unchallenged or unrecognized.

So it's obvious what storyline I'm aligned to. And I've really stuck my flag on one side of the line. I feel pretty uncomfortable about that. I have a lot of faults, but I'm a guy who can relate to and enjoy hanging out with all sorts of people. I really don't like the idea that I may have just hurt any chance of sharing a connection to someone in or out of the Eclipse community because of what I've said. But I guess that taking the risk that people won't be happy with what you have to say is part of involving oneself in an issue like this -- you can't have it both ways.

But my real message is that these are all stories. They aren't real, folks. There is no hidden truth to discover here. In fact I guarantee that the deeper one digs into this, the less clear anything will be. It is often convenient and sometimes seems almost necessary to align oneself with a particular story. But if we want to get beyond that and cultivate an atmosphere that combines sharpness and intelligence with some kind of warmth and appreciation for everyone's unique contributions then the first thing we can do is just see the story for what it is. And that we can change the story whenever we want to.

As I said up top, I think that involves being willing to discuss the feelings that we are carrying underneath the story. I'm completely serious about this and so I'll put my money where my mouth is and talk about my feelings here.. This isn't something that software engineers are very good at, and yeah, I know, ick, but if you don't like it, too fucking bad. My involvement with the Eclipse ecosystem is motivated in part by the fear that my own contributions won't be recognized, residual anger and resentment at being treated in the past as a kind of techno-serf, and that people won't think I'm competent at what I do. I have an embarrassing amount of my own sense self-worth wrapped up in all of this and it's surprising to me how difficult it is to admit that in public.

Other's have talked about how this has actually made them feel. The thing I appreciated about Mike's post is that he made clear that he was really angry and hurt and that that came from having his and other's motivations called into question repeatedly. I also admired what Doug Schaefer had to say in a recent post. He talks about being insulted and hurt by the perceptions of others about his role and motivation in his Open Source project. And it just struck me how both of those posts came from very different points of view, and a completely different story line, but expressed almost the same pain. We all (and I mean all) put a lot of effort and heart into what we are doing when we could be doing something else for probably more money and maybe even more recognition. We do Open Source because we love it, and when that is called into question it hurts.

Luckily, there is a light-weight practice we can use to work with this, and we are already really experienced with it. When someone has a legitimate or interesting point of view to contribute --even when that involves recognizing difficulty issues and relationships -- ACCEPT it. But when it is negativity about someone else's contributions, just mark it INVALID and hit the commit button.

8 comments:

  1. Excellent post Miles. Sometimes we can't see the forest through the trees.

    ReplyDelete
  2. "In fact I guarantee that the deeper one digs into this, the less clear anything will be." Indeed.

    Thank you very much for your post, Miles. I do hope you can make it to the next EclipseCon, would love to meet you in person!

    ReplyDelete
  3. Miles,

    We can use more of your wisdom in the Eclipse community. I hope you can make it to EclipseCon too.

    ReplyDelete
  4. This was a great read. Very interesting perspective. Thanks.

    ReplyDelete
  5. Miles,
    thanks for this great post.
    your experiences how to survive as a tiny company in eclipse ecosystem are very helpful for me (I'm also only representing my tiny company and spend much time for eclipse - on the other side I get so much value back)
    knowing you so many years from 'old' openarchitectureware forum I followed how you went the way to eclipse.
    hopefully we can meet at EclipseCon 2010.
    ...also your words about how to communicate and have different stories to tell could have come from my heart ;-)
    ekke

    ReplyDelete
  6. I've already suggested to Ian that we consider modeling the industry adoption of Eclipse with your agent framework. For version 0.2.0 of the model, perhaps we should add the 'churning pool' behavior (highlighted above) and create a few catalytic/disruptive agents to make the model more realistic?

    ;-)
    John

    ReplyDelete
  7. Already been done: http://www.brookings.edu/reports/2001/01technology_epstein.aspx . :) Well not exactly, but sometimes the Civil Violence model seems pretty close. ;)

    I do think that open source communities are a really interesting subject for study as we have really good behavioral and relationship data that can be inferred from project artifacts such as CVS logs, project meta-data, blog postings..

    and thanks to everyone for your kind words..

    ReplyDelete

Popular Posts

Recent Tweets

    follow me on Twitter