For these reasons, the AHFMR recognized that a process that supported a more consistent application of criteria between reviewers and within competitions would be beneficial.
Finally, candidates had requested more feedback on a routine basis from the AHFMR about the review of the applications.
In the case of the AHFMR, a number of factors were identified that were thought to be predictive of the performance of trainees in the proposed research environment (see Table 1).
The reviewers' assessments are then electronically sent to the AHFMR and input into the ProGrid software for analysis.
Although the ProGrid-assisted review process is a recent introduction to the AHMFR, some progress made by the AHFMR in its adoption of the software-assisted approach can be clearly demonstrated.
Both of these limitations will be studied by the AHFMR, as will others as they arise.
While the new process has not been without its critics in the community, it has allowed the AHFMR to address specific process issues affecting the candidates, the reviewers, and the foundation itself, and has increased the organization's internal capacity for self-evaluation The pilot project described in this paper was applied first to the studentship application review process.
For example, the AHFMR has a Health Trainee Advisory Committee (concerned with applications in the fields of population health, behavioral research, and other non-biomedical health sciences), a Studentship Advisory Committee (concerned with applications in the fields of the biomedical sciences), and two Fellowship Advisory Committees.
While the original process required that a large percentage of time of in-person meetings be devoted to discussion of all of the applications, meetings can now be devoted to discussing those applications where there is a significant difference of opinion between committee members, with the remainder of the time spent discussing competition policy, as recognized through the pre-meeting assessments provided to the AHFMR.
In addition to these outcomes, the introduction of the ProGrid-facilitated process has other implications for program evaluation and review at the AHFMR.