How the PECT Were Developed

The synopsis that follows describes the rigorous process that was used to carefully develop each of the PECT assessments.

1. Establish Advisory Committees

Content Advisory Committees were established to review testing materials using such criteria as accuracy, job-relatedness, and freedom from bias. An Equity Assurance Panel was formed to review testing materials for sensitivity, inclusion, and fairness. The members of these committees were chosen from applications received from well-qualified educators and teacher education faculty from throughout the state of Pennsylvania. In addition, Pennsylvania educators could nominate their colleagues to serve on a Content Advisory Committee or the Equity Assurance Panel. Nominations were sought from Pennsylvania LEAs and higher education program providers. Both the Content Advisory Committees and the Equity Assurance Panel comprised individuals who reflected the racial, gender, ethnic, and regional diversity of Pennsylvania.

2. Define and Review Test Content

For each test in the PECT program, a set of draft test objectives was developed, in collaboration with the Pennsylvania Department of Education (PDE), based on the Pennsylvania Program Framework Guidelines, the relevant Pennsylvania academic standards, and any other materials deemed appropriate by the PDE. The draft test objectives for each field were reviewed, revised as needed, and approved by the Equity Assurance Panel and the appropriate field-specific Content Advisory Committee.

3. Conduct the Content Validation Survey

The set of test objectives for each test, as approved by the Equity Assurance Panel and the field-specific Content Advisory Committee, was prepared for validation by means of a content validation survey. Pennsylvania public school teachers and teacher education faculty with expertise in the subject matter and relevant grade bands were randomly selected to participate in and complete the surveys. For the PreK–4, Special Education PreK–8, and Special Education 7–12 tests, Pennsylvania educators provided informed judgments and open-ended feedback about the importance of the knowledge and skills described by each objective, and the set of objectives as a whole, for the novice educator as an entry-level teacher in Pennsylvania public schools. For the PAPA, Pennsylvania educators provided judgments about the importance of the knowledge and skills described by each objective, and the set of objectives as a whole, as a requirement for admission to a state-approved teacher education program. Results from the content validation survey were analyzed and the objectives were revised, as needed, in consultation with the PDE, to ensure that all objectives represent knowledge and skills important to the job of a Pennsylvania educator, or, for the PAPA, all objectives represent basic skills in reading, writing, and mathematics required for admission to an approved Pennsylvania teacher education program. The results of these analyses were provided to the PDE, with copies of the revised objectives for the PDE's review and approval. The PDE-approved objectives were then posted on the PECT website.

4. Develop Draft Test Questions

For each test, questions were drafted based on the Pennsylvania-educator-reviewed and -approved test objectives. The questions were written to have content and other characteristics that relate to the purpose of the test (e.g., certification of entry-level teachers) and that support test results that identify candidates who have either met the "basic skills" requirement or who have proven subject matter mastery in PreK–4, Special Education PreK–8, or Special Education 7–12. Each draft test question was developed to correspond to a specific test objective. This correspondence established the alignment of the test questions—through the test objectives—to the Pennsylvania Program Framework Guidelines. Throughout the process, questions were reviewed for quality, alignment with the test objectives, psychometric characteristics, and freedom from bias.

5. Review of Draft Test Questions by Advisory Committees

The test questions for each PECT assessment were reviewed, revised as needed, and approved by both the Equity Assurance Panel and the appropriate field-specific Content Advisory Committee. The Equity Assurance Panel reviewed the test materials for sensitivity and fairness to all individuals regardless of their background, focusing its review on excluding from the test materials any language, content, or perspectives that might disadvantage or offend an examinee because of gender, race, ethnicity, sexual orientation, national origin, religion, age, disability, or cultural, economic, or geographic background. The Equity Assurance Panel also reviewed the materials for appropriate inclusion of content and perspectives that reflect the diversity of the Pennsylvania population. The Content Advisory Committees reviewed the questions for accuracy and freedom from bias, and they ensured that each question relates to the job of a Pennsylvania public school educator and is matched to one of the test objectives, which are aligned with Pennsylvania Program Framework Guidelines and relevant academic standards.

6. Conduct Field Testing

Test questions were field tested to determine whether they had acceptable statistical and qualitative characteristics to use on operational test forms. Students seeking admission to a Pennsylvania teacher education program and students enrolled in an educator preparation program (including graduate students), and beginning teachers with 1–3 years of teaching experience were eligible to participate in the field test. Notifications about field testing were sent to deans, department chairs, and faculty at approved Pennsylvania teacher education programs, who were asked to encourage their students to participate in the field tests. Notifications were also sent to Pennsylvania LEAs, who were asked to encourage newly certified teachers to participate. Members of the Content Advisory Committees and the Equity Assurance Panel were also asked to identify and notify eligible candidates. All participants were volunteers. At the conclusion of field testing, the performance of each question was analyzed; test items that did not perform acceptably were deleted.

7. Construct Preparation Materials

An online study guide for each test was prepared and published on the PECT website. Each study guide includes general test-taking strategies, the test objectives, a set of sample test questions, and a list of additional resources for further study. Additionally, full-length practice tests are available for each PECT assessment.

8. Determine Qualifying Scores

Prior to the first administration of the tests, panels of Pennsylvania educators were convened for a Standard Setting Conference. The panels participated in a structured process to make judgments that would be used by the PDE, based on feedback and recommendations from Evaluation Systems, to establish qualifying scores for each test module. Panelists for the PreK–4, Special Education PreK–8, and Special Education 7–12 tests provided informed judgments about the expected performance of effective novice teachers. Panelists for the PAPA provided informed judgments about the expected performance of candidates having the level of basic skills in reading, mathematics, and writing required by the PDE to be eligible for admission to an approved Pennsylvania teacher education program. Following the Standard Setting Conference, the panelists' judgments were compiled and presented to the PDE, who then set the qualifying scores for each test module based on recommendations from panelists and Evaluation Systems.

Learn about the test development process for the National Evaluation Series Dance test.