Definitions of Criteria and Considerations for T32, T35 and K12 Critiques

Updated March 12, 2014

Overall Impact. Reviewers will provide an overall impact/priority score to reflect their assessment of the likelihood for the program to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following review criteria and additional review criteria (as applicable for the project proposed).

1. Training Program and Environment.

T32. Are the research facilities and research environment conducive to preparing trainees for successful careers as biomedical research scientists? Are the objectives, design and direction of the proposed research training program likely to ensure effective training? Do the courses, where relevant, and research experiences provide opportunities for trainees to acquire state-of-the-art scientific knowledge, methods, and tools that are relevant to the goals of the training program?  Does the program provide appropriate inter- or multidisciplinary research training opportunities? Is the proposed training program likely to ensure trainees will be prepared for research-intensive and research-related careers? Is the level of institutional commitment to the training program, including administrative and research training support, sufficient to ensure the success of the program? Is it clear how the proposed training program is distinguished from other externally funded training programs at the institution?

T35. Are the objectives, design, direction, and quality of the proposed short-term research training program appropriate and likely to ensure effective training? Does the proposed program provide suitable training for the levels of trainees being proposed and the area(s) of science to be supported by the program? Are the quality of proposed courses, if applicable, and training experiences appropriate for the trainees to be included in the program? If appropriate, does the program provide training in inter- or multi-disciplinary research and/or provide training in state-of-the-art or novel methodologies and techniques?  Is the proposed program of training likely to ensure that trainees will acquire knowledge and skills necessary to prepare them for a research career?  Does the program include features likely to encourage short-term trainees to consider careers in health-related research? Are the research facilities and research environment conducive to preparing trainees for successful careers as biomedical research scientists?  Is the level of institutional commitment to the training program, including administrative and research training support, sufficient to ensure the success of the program?

K12. Does the proposed program clearly outline a plan to recruit and develop well-qualified junior investigators for successful careers as biomedical or clinical researchers? Is there evidence of an adequate pool of potential scholars who could benefit from receiving career development support? Are the content and duration of any proposed didactic, training-related, and research-related activities of the program appropriate? Are appropriate timelines indicated for career progression and transition to independence? Does the institutional environment (e.g., research facilities and other relevant resources) in which the program will be conducted contribute to the probability of success? Does the proposed career development program benefit from unique features of the scientific environment, subject populations, or employ useful collaborative arrangements? Is the institutional commitment to the proposed program appropriate? If multiple sites are participating, is this adequately justified in terms of the career development and research experiences provided? Is there sufficient assurance that the required effort of the PD/PI, mentors and scholars will be devoted directly to the research training, career development, and related activities? When applicable, is there adequate documentation describing the responsibilities of the advisory committee with regard to the provision of input, guidance and oversight of the program?

2. Training Program Director(s)/Principal Investigator(s) (PDs/PIs).

T32 and T35.  Does the Training PD/PI have the scientific background, expertise, and administrative and training experience to provide strong leadership, direction, management, and administration of the proposed research training program? Does the Training PD/PI plan to commit sufficient effort to the program to ensure the program's success?

For applications designating multiple PDs/PIs: Is a strong justification provided that the multiple Training PD/PI leadership approach will benefit the training program and the trainees? Is a strong and compelling leadership approach evident, including the designated roles and responsibilities, governance, and organizational structure consistent with and justified by the aims of the training program and with the complementary expertise of each of the PDs/PIs?

K12. Do the PD/PI and Research Administrator (if applicable), have the experience to develop, direct and administer the proposed program? Does the leadership team bring complementary and integrated expertise to the program? Is there evidence that an appropriate level of effort will be devoted by the program leadership to ensure program objectives? Are the research qualifications, scientific stature, previous leadership and mentoring experience, and track record(s) appropriate for the proposed career development program? Are the PD(s)/PI(s) currently engaged in research relevant to the scientific area of the proposed program?

3. Preceptors/Mentors.

T32. Are sufficient numbers of experienced preceptors/mentors with appropriate expertise and funding available to support the number and level of trainees (including short-term trainees, if applicable) proposed in the application? Do the preceptors/mentors have strong records as researchers, including recent publications and successful competition for research support in areas directly related to the proposed research training program? Do the preceptors/mentors have strong records of training individuals at the level of trainees (including short-term trainees, if applicable) proposed in the program?  Are appropriate plans in place to ensure that preceptors lacking sufficient research training experience are likely to provide strong and successful mentoring?

T35.  Are sufficient numbers of experienced preceptors/mentors with appropriate expertise and funding available to support the number and level of trainees proposed in the application?  Do the preceptors/mentors have strong records as researchers, including recent publications and successful competition for research support in areas directly related to the proposed research training program?  Do the preceptors/mentors have strong records of training individuals at the level of trainees proposed in the program?  Are appropriate plans in place to ensure that preceptors lacking sufficient research training experience are likely to provide strong and successful mentoring?

K12. Do the mentors have expertise and experience, as well as track records of past mentoring and training? Are the quality and extent of the mentors' roles in providing guidance and scientific advice to the scholars acceptable? Are the mentors currently engaged in relevant research?

4. Trainees or Candidates/Scholars.

T32. Is a recruitment plan proposed with strategies to attract well-qualified candidates for the training program? Is there a competitive applicant pool of sufficient size and quality, at each of the proposed levels (predoctoral, postdoctoral and/or short-term), to ensure a successful training program? Are there well-defined and justified selection and re-appointment criteria as well as retention strategies?

T35. Is a recruitment plan proposed with strategies likely to attract high-quality candidates for the short-term training program? Does the program have access to and the ability to recruit high quality, short-term trainees from the applicant institution or another health-professional school? Are the size and quality of the applicant pool adequate to support the program? Are the recruiting procedures, and trainee selection criteria, appropriate and well defined?

K12. What is the quality of plans to identify, recruit, and select candidates, with a commitment to research relevant to the mission of the FOA, and the potential to develop as independent researchers? Is there an appropriate plan for the content, the phasing, and the proposed duration of the career development plan for achieving scientific independence for the prospective candidates? How useful is the research plan as a vehicle for ensuring research training for all scholars as described in the career development plan? What is the likelihood that the career development plan will contribute significantly to the scientific development of the scholars? Does the program include a plan to recruit, identify, and select scholars with a commitment to research relevant to the objectives of the FOA? Does the program description include a plan to recruit a diverse group of scholars locally and nationally? Does the plan for selection of the scholars include all of the eligibility criteria stated in the FOA?

5. Training Record.

T32. How successful are the trainees (or for new applications, other past students/fellows in similar training) in completing the program? Has the training program ensured that trainees are productive (or, for new applications, other past students/postdoctorates in similar training) in terms of research accomplishments, publication of research conducted during the training period, and subsequent training appointments and fellowship or career development awards?  How successful are the trainees (or for new applications other past students/postdoctorates in similar training) in achieving productive scientific careers as evidenced by successful competition for research science positions in industry, academia, government or other research venues; grants, receipt of honors, awards, or patents; high-impact publications; promotion to scientific leadership positions; and/or other such measures of success? To what extent do trainees' subsequent positions in industrial, academic, government, non-profit, or other sectors benefit from their NRSA-supported research training and directly benefit the broader biomedical research enterprise?  Does the program propose a rigorous evaluation plan to assess the quality and effectiveness of the training?  Are effective mechanisms in place for obtaining feedback from current and former trainees?

For programs that provide research training to health-professional doctorates: Is there a record of retaining health professionals in research training or other research activities for at least two years?

T35. How successful are the trainees (or for new applications, other past students/fellows in similar training) in completing the program? Do trainees subsequently enter careers in health-related research?  How productive are trainees (or for new applications, other past students/fellows in similar training) in terms of research accomplishments, publications, subsequent training appointments, fellowships, and individual research support? Does the program have a rigorous evaluation plan to assess the quality and effectiveness of the training? Are effective mechanisms in place for obtaining feedback from current and former trainees and monitoring trainees' subsequent career development? Are appropriate plans presented to follow the careers of short-term trainees and to assess the effect of the training program on subsequent career choices? Is the program successful in attracting trainees back for multiple appointments, when possible and appropriate?

K12. Is there evidence of a successful past training record of the PD/PI and mentors, including the success of former scholars in seeking independent support and establishing productive scientific careers? Does the program have a rigorous evaluation plan to assess the quality and effectiveness of the training?

 

Protections for Human Subjects. Generally not applicable. Reviewers should bring any concerns to the attention of the Scientific Review Officer. For additional information to assist you in making these determinations, please refer to the Guidelines for the Review of Human Subjects.

Inclusion of Women, Minorities, and Children. Generally not applicable. Reviewers should bring any concerns to the attention of the Scientific Review Officer. For additional information to assist you in making these determinations, please refer to the Guidelines for the Review of Inclusion in Clinical Research.

Vertebrate Animals. Generally not applicable. Reviewers should bring any concerns to the attention of the Scientific Review Officer. For additional information to assist you in making these determinations, please refer to the Worksheet for Review of the Vertebrate Animal section (VAS).

Biohazards. Generally not applicable. Reviewers should bring any concerns to the attention of the Scientific Review Officer.

Resubmission. For Resubmissions, the committee will evaluate the application as now presented, taking into consideration the responses to comments from the previous scientific review group and changes made to the project.

Renewal.

T32.  For Renewals, the committee will consider the progress made in the last funding period, including on the Recruitment and Retention Plan to Enhance Diversity. Does the application describe the program's accomplishments over the past funding period(s)? Is the program achieving its training objectives?  Has the program evaluated the quality and effectiveness of the training experience (and when applicable, short-term training experience), and is there evidence that the evaluation outcomes and feedback from trainees have been active upon?  Are changes proposed that are likely to improve or strengthen the research training experience during the next project period (may not be applicable to short-term training)?  Does the program continue to evolve and reflect changes in the research area in which the training occurs?

T35.  For Renewals, the committee will consider the progress made in the last funding period, including on the Recruitment and Retention Plan to Enhance Diversity.  Does the application describe the program's accomplishments over the past funding period(s)?  Is the program achieving its training objectives? Has the program evaluated the quality and effectiveness of the training experience, and is there evidence that the evaluation outcomes and feedback from trainees have been acted upon? Are changes proposed that would improve or strengthen the research training experience?  Has the program been successful in attracting trainees back for multiple appointments and to continue subsequently in health-related research positions?

Revision. For Revisions, the committee will consider the appropriateness of the proposed expansion of the scope of the project. If the Revision application relates to a specific line of investigation presented in the original application that was not recommended for approval by the committee, then the committee will consider whether the responses to comments from the previous scientific review group are adequate and whether substantial changes are clearly evident.

Recruitment and Retention Plan to Enhance Diversity.Peer reviewers will separately evaluate the recruitment and retention plan to enhance diversity after the overall score has been determined. Reviewers will examine the strategies to be used in the recruitment and retention of individuals from underrepresented groups. The review panel's evaluation will be included in an administrative note in the summary statement.

Training in the Responsible Conduct of Research.

T32 and T35. All applications for support under this FOA must include a plan to fulfill NIH requirements for instruction in the Responsible Conduct of Research (RCR).  Taking into account the specific characteristics of the training program, the level of trainee experience, and the particular circumstances of the trainees, the reviewers will evaluate the adequacy of the proposed RCR training in relation to the following five required components: 1) Format - Does the plan satisfactorily address the format of instruction, e.g. lectures, coursework and/or real-time discussion groups, including face-to-face interaction?  (A plan involving only on-line instruction is not acceptable.); 2) Subject Matter - Does the plan include a sufficiently broad selection of subject matter, such as conflict of interest, authorship, data management, human subjects and animal use, laboratory safety, research misconduct, research ethics? 3) Faculty Participation -Does the plan adequately describe how faculty will participate in the instruction?  For renewal applications, are all training faculty who served as course directors, speakers, lecturers, and/or discussion leaders during the past project period named in the application?  4) Duration of Instruction - Does the plan meet the minimum requirements for RCR, i.e., at least eight contact hours of instruction? 5) Frequency of Instruction " Does the plan meet the minimum requirements for RCR, i.e., at least once during each career stage (undergraduate, post-baccalaureate, predoctoral, postdoctoral, and faculty levels) and at a frequency of no less than once every four years? 

For renewal applications, does the progress report document acceptable RCR instruction in the five components described above? Does the plan describe how participation in RCR instruction is being monitored? Are appropriate changes in the plan for RCR instruction proposed in response to feedback and in response to evolving issues related to responsible conduct of research?

Plans and past record will be rated as ACCEPTABLE or UNACCEPTABLE, and the summary statement will provide the consensus of the review committee.

K12. Taking into account the specific characteristics of the career development program, level of scholar experience, and the particular circumstances of the scholars, the reviewers will address the following questions. Does the plan satisfactorily address the format of instruction, e.g. lectures, coursework and/or real-time discussion groups? Do plans include a sufficiently broad selection of subject matter, such as conflict of interest, authorship, data management, human subjects and animal use, laboratory safety? Do the plans adequately describe how faculty will participate in the instruction? Does the plan meet the minimum requirements for RCR, i.e., eight contact hours of instruction every four years? If this is a renewal, is there a report describing past instruction in the five components described above? Plans and past record will be rated as acceptable or unacceptable, and the summary statement will provide the consensus rating of the review committee. Applications rated unacceptable will not be funded until the applicant provides an acceptable, revised plan.

 

Select Agent Research. Generally not applicable.  Reviewers should bring any concerns to the attention of the Scientific Review Officer.  Budget and Period of Support. Reviewers will consider whether the budget and the requested number of training positions (predoctoral, postdoctoral, short-term), and the requested period of support are fully justified and reasonable in relation to the proposed research. For more details, please see Budget Information.

Additional Comments to the Applicant. Reviewers may provide guidance to the applicant or recommend against resubmission without fundamental revision.

[Return to Review Guidelines Page]