A few weeks ago, the Open Data Kit (ODK) team released an update to our mobile client, ODK Collect.  ODK is a suite of tools to help organizations collect, aggregate and visualize their data. The goals of ODK are to make open-source and standards-based tools which are easy to try, easy to use, easy to modify, and easy to scale.

Some of the new features in the most recent release include barcode scanning, image/audio/video capture and playback and editing of saved data.  We've also made location acquisition and form processing a lot faster, added a really cool way to review data entry and reworked the user interface to make training and use much easier.  Of course, we still support question grouping, repeats, constraints, complex logic and multiple languages -- functionality that we inherited from JavaRosa, another Open Mobile Consortium data collection project.  (For a video of some of the new features, see here.)

ODK and Java Rosa - The Obstacles to Working Together

The story of how the ODK and JavaRosa projects, two competitors in the mobile data collection space, came to work together is something we would like to share with the wider community.  Both of our respective projects faced a number of challenges that illustrate how hard it is to work together to benefit our users. For us, this story demonstrates the importance of the Open Mobile Consortium in creating an organizational and technical framework to minimize the obstacles faced in collaboration.

When our team started on the ODK project, we made what we thought was a simple choice: we would work with other organizations who had data collection systems to ensure compatibility with ODK. Our reasoning was simple: A truly "open" data kit would allow users to have a choice of platforms while maintaining data portability.

The first organization we interacted with was the OpenRosa Consortium.  OpenRosa is a community formed to create open source, standards-based tools for mobile data collection, aggregation, analysis, and reporting. OpenRosa's best-known project is JavaRosa, an OMC technology. We had worked with members of the OpenRosa community before and we decided to build ODK around their form description and data standard.

From our perspective, this would be a simple task of building our application to implement the standard so forms and data could flow from ODK to other OpenRosa-compliant projects like EpiSurveyor, OpenXData and OpenMRS. As we looked deeper into our task, we realized that because the standard was complex and still in flux, creating our own implementation would result in two versions of the standard -- effectively breaking compatibility and splitting the community.

Avoiding Fragmentation

The most obvious way to prevent this fragmentation would be to use and contribute to the existing JavaRosa codebase. Most developers see external dependencies of this sort as a liability, especially when differences in platforms exist. In our case, JavaRosa was designed for more common, but less powerful J2ME phones which are found in developing regions.

While ideal for price sensitive organizations, it did not enable the kind of data collection we wanted to provide -- primarily capturing of location, images, audio, video and barcodes.

Going Android

Additionally, as researchers, we wanted the processing power and freedom to transform the phone into everything from a portable ultrasound or even a cataract detector. The only available platform that could provide this functionality was Android -- Google's open source operating system. Unfortunately, even though Android and J2ME are based on Java, applications on those platforms are structurally quite different.

We approached the OpenRosa community with this problem and they were able to create a rough version of JavaRosa that would work on Android. Using this version of JavaRosa introduced a number of challenges that we had to manage. First, JavaRosa is complex and it took our team months to learn how to work with it. We would often make assumptions about its functionality only to find out weeks later that our approach was invalid. Although we had support from the JavaRosa core developers, this quickly became a frustrating experience for all involved.

Challenges

There were also organizational challenges. The core team that works on JavaRosa is from Dimagi, a Boston based consultancy and an OMC member. With our team in Seattle and both groups swamped with travel, we would often go for weeks before we could meet. This became more troublesome as we tried to add functionality to both ODK Collect and JavaRosa. These features required coordination with Dimagi and so our ideas would sit unimplemented for weeks.

We even had legal challenges. At the time, JavaRosa had not been properly licensed as open source. We discovered this a few days before our release of ODK Collect and it delayed the project for a month. We had to track down every JavaRosa contributor and have them sign a legal statement before we could release. While each of these challenges were frustrating for our team, they have translated into positive steps for both ODK and JavaRosa.

For example, because our team now has a thorough understanding of the JavaRosa code base, we have been able to make suggestions to improve its functionality and structure. It is also much easier to integrate JavaRosa on a number of mobile, desktop and server platforms -- improvements that ODK now relies on. In return, the JavaRosa community has helped us discover and fix a number of bugs in our code base.

Our struggles with the licensing have also paid off for the larger community. Organizations large and small can safely build their applications on JavaRosa and avoid legal ramifications of using unlicensed software.

How Working with JavaRosa Has Changed Our Work

The coordination challenges we faced with JavaRosa have changed the way we work on ODK. For example, we document our code much more extensively than we used to. We also make ourselves available via a mailing list and a chat room -- a technique we learned from the JavaRosa community.

More importantly, we try to ensure that developers don't need our help. That is, we would rather have something slow that is easy to understand and extend than something fast that only we could change. As one external developer working in the Central African Republic summarized, "The code is nicely commented so you can understand what's going on when you get under the hood. For example, I changed all the text strings to be in French. I [have] removed a couple buttons that the enumerators don't need to be messing with."

For our users, there have also been tangible benefits. When D-Tree (a NGO who participates in the OMC) decided to use ODK for a medical triage algorithm for pediatrics, they were able to take their existing forms and use them on ODK with only minor changes. Because of the work we did on JavaRosa, our users who work in disconnected environments can connect to an OpenXData local server for storage of their data. Dimagi's assistance has made them experts on ODK and they are now receiving contracts to implement projects using it.

Collaboration is Hard, But the Right Choice

Collaboration is a word that is bandied around in the development community as what we all should do. In our experience, real collaboration is much harder and more time consuming than most realize. There will always be moments of frustration and difficulty, but for us, the pros have far outweighed the cons. More importantly, we believe we made the right choice for our users.

As we continue our work building more ODK tools, we will continue to collaborate in code with our friends in the JavaRosa project. And as new OMC members, we are applying many of these lessons learned to our work to ensure that the OMC provides an organizational and technical framework that enables this kind of collaboration.