Recently a conflict emerged between two ARDC grant recipients – M17 Project and Multimode Digital Voice Modem (MMDVM). As the conflict has been unfolding, a variety of individuals have asked us to comment and provide our perspective. Out of respect for our grantees, our policy is to maintain confidentiality. Thus, we do not discuss specifics related to our grantees with anyone other than those directly involved in their project. Additionally, as a funder, it is inappropriate for us to mediate such conflicts. Still – in service of being as transparent as possible – it does feel appropriate to address some of the questions, comments, and concerns that have arisen related to how we select, evaluate, and exercise oversight of our funded projects.

We’re focusing on Research & Development (R&D) projects in this post, as they are quite different from other kinds of grants like educational programs or repeater installs. The nature of R&D is far more complex and often more expensive to execute. Even with the most talented teams, results can take a far different direction than initially conceived – at times to either the detriment or perhaps benefit of the project. There is a high level of financial risk, accompanied by potentially high rewards, but these rewards are not guaranteed. All told, these projects can be both difficult to pull off and equally difficult to evaluate.

This difficulty was demonstrated in our 2024 Grants Evaluation Team (GET) report, which showed that R&D projects had the greatest number of uncertain outcomes compared to other categories. This often had to do with:

• stark departures from a project’s proposal,
• a lack of information in final reports, and
• outcomes that can take many years to show whether they will have their intended impact.

Compared to Amateur Radio and Education projects, the GET found that the majority of R&D projects showed uncertain outcomes, and no score higher than Medium (below High and Very High). More details can be found in Appendix A of our 2024 GET report.

So, what can we do to improve these outcomes? As we’ve gotten more experience with R&D projects, and noting these challenges, we’ve looked for ways to improve the evaluation and oversight process on our side. First, in 2024, we initiated the GET, who reviewed every grant we’d made from 2019-2023 and provided insights and recommendations like the ones above. Now, they review both final and periodic reports closer to the time they are received, providing opportunities for quicker feedback to our grantees and Grants Advisory Committee (GAC). To support getting more useful information in our final reports, they now include questions that are specific to R&D. We are also researching other methods to ensure better ongoing oversight (including financial oversight) and opportunities to offer support for these projects, noting that there may not be a one-size-fits-all approach.

In addition to continually improving our evaluation process, we’ve also become clearer in our thinking about R&D projects, which affects what we look for in applications for new projects and funding extensions alike. Though we’ve added some language to our website and have had conversations with our R&D grantees, it’s worth clarifying here as well.

Notably, at a high level, ARDC does not aim to fund R&D projects indefinitely. We now aim to fund projects – or project components – that are clearly scoped and defined. For example, rather than a proposal for 8 different features, we prefer a proposal for 1-3 features that are going to add the most value to the project. Taking this approach enables us to better evaluate the project, which is especially important when considering requests for funding extensions. It also allows us to support a wide range of projects over time, particularly our priority projects defined earlier this year.

There are even more factors related to the project’s ongoing sustainability that we now know to look for. More than just a talented technical team with demonstrated development success, we now also seek the following skillsets on a project team:

• Project management (including funds management),
• Community management and engagement,
• Opportunities to test and incorporate feedback from stakeholders around usability, and
• Effective communication with relevant stakeholders (including us).

These skills are critical to a project’s success, both during and after the lifetime of the grant. They are particularly important for open source projects, because they demonstrate that a project will have sustainability beyond the end of our funding – which is what we want to see.

In support of manifesting ARDC’s vision, a goal in funding R&D work is to have the results spread broadly throughout the community and around the world. The kinds of issues listed below challenge that goal. Often, other proposals without these issues are more likely to succeed broadly, and are thus more attractive to us. With this in mind, ARDC often declines to support applications from projects that:

• have a fractured project team or contributor community,
• demonstrate an inability to work collaboratively with partners, and/or
• demonstrate an inability to communicate well with the stakeholders or community they hope to serve.

In addition, we have a fiduciary responsibility to ensure that any funds we provide are used solely for charitable or educational purposes within our mission. Thus, we have a strong tendency to decline subsequent grant applications from projects that:

• have strayed significantly from a previous project proposal without approval, explanation, or documentation, and/or
• show evidence of mismanagement of funds, or lack accurate accounting records.

Even if technical work produced in the past (with our funding or others’) is technically sound and useful, the above reasons will still affect our review of pending grant applications.

In the event that members of the public bring these concerns to our attention, we do look into them, particularly at the time of project evaluation. We make our conclusions based on our own evidence, not through hearsay, rumors, or similar.

As an organization, it is important to us that we, as well as our grantees, are always learning. We appreciate the opportunity to share about what we’ve learned with regards to approving and evaluating R&D projects in this post. We will certainly continue to find opportunities for improvement, and we are committed to acting on those opportunities when we find them. If you have any questions or ideas, please do not hesitate to reach out: contact@ardc.net.