9 Training and Professional Development
The Dynamic Learning Maps® (DLM®) Alternate Assessment System provides comprehensive support and training to state education agency staff and local educators. The type of support provided for local educators is twofold: DLM Required Test Administrator Training, and optional professional development for instruction. DLM Required Test Administrator Training provides test administrators with both the context and practical knowledge of the assessment system design, administration, and security practices to administer the assessment with fidelity. All required training is therefore aligned with the Test Administration Manual (DLM Consortium, 2021). See Chapter 4 of this manual for a complete description of assessment administration practices. The purpose of the professional development component is to provide professional learning opportunities to support instructional practices for test administrators and educators of the target population of students who participate in DLM assessments.
This chapter describes the training that was offered in 2021–2022 for state and local education agency staff as well as the DLM Required Test Administrator Training, which was originally developed for the first operational administration of the DLM assessments in 2014–2015. Since the initial release of training materials, numerous updates have been made, which are reflected in the description of the training that was available in 2021–2022. Additionally, this chapter describes the participation rates and evaluation results for the optional instructional professional development modules.
9.1 Training for State Education Agency Staff
State education agency staff are integral to the implementation of the DLM assessment system. While there is no formal comprehensive training program for this audience, the staff have opportunities to participate in training designed for local education agency staff and test administrators. Throughout the year, state education agency staff also receive instruction during regularly scheduled meetings and through written documentation from DLM staff on state-level support topics, such as monitoring test administrators’ completion of required training, viewing and editing data in Educator Portal, and using data extracts from Educator Portal to monitor assessment administration.
9.2 Training for Local Education Staff
In 2021–2022, as in prior years, three local education staff roles supported implementation of the DLM System. These roles were typically held by one or more district-level staff members, but in some cases were fulfilled at the building level.
- The assessment coordinator oversees the assessment process, including managing staff roles and responsibilities, developing and implementing a comprehensive training plan, developing a schedule for test implementation, monitoring and supporting test preparations and administration, and developing a plan to facilitate communication with parents/guardians and staff.
- The data manager oversees educator, student, and roster data.
- Technology personnel verify that network and testing devices are prepared for test administration.
In August 2021, a prerecorded training video was posted to each state’s page on the DLM website. This training applied to assessment coordinators, data managers, and technology personnel. It discussed the checklist tasks from each role’s manual and explained how the roles work together to successfully implement the DLM assessment in their district. A training flyer was provided to state education agencies to distribute and promote this training opportunity. The flyer included information regarding two follow-up questions and answer opportunities, whereby DLM staff answered questions via live chat.
In January 2022, another prerecorded training video was posted to each state’s page on the DLM website. This video was provided for district superintendents and building principals. It provided an overview of the DLM assessment and explained the activities that should be ongoing in their buildings and districts to prepare for and monitor the spring assessment. Follow-up question and answer opportunities regarding this training were also offered via live chat.
9.3 Required Training for Test Administrators
Training is required annually for all educators who will administer the DLM alternate assessment to students. Test administrators are not allowed access to their student’s login credentials for the assessment until all required training components are completed.
The DLM Required Test Administrator Training is conducted in Moodle. First-time test administrators are enrolled in a training course for new test administrators that takes approximately three hours to complete. Test administrators who completed the training for the previous school year are enrolled in a course for returning test administrators that takes approximately an hour to complete. Each trainee has a secure login to the Moodle training site. Directions for accessing and completing the training are provided on each state’s page on the DLM website.
The training for new test administrators consists of four modules, each with a post-test. Trainees have to pass each post-test with a score of at least 80% to receive the certificate of completion and gain access to students’ login credentials for the assessment. The training for new test administrators is offered in two formats: self-directed, whereby the trainee navigates through the training course independently and completes the post-tests, or facilitated, whereby a trained facilitator presents or guides trainees through the training and then directs them to complete the post-tests on their own in Moodle.
For returning test administrators, the training consists of a single module that reviews the most salient aspects of the DLM assessment and test administration procedures. After the review, a four-part post-test is required. Each part corresponds to one of the four modules for new test administrators. Any returning test administrator who does not pass a part of their post-test with a score of at least 80% on the first attempt is redirected to the training for new test administrators for remediation before continuing the returning test administrator course. The training for returning test administrators is available in the self-directed format only.
Some states include one or more DLM professional development modules in their required test administrator training courses. When included, these additional modules are accessed in Moodle and are required to be completed to generate the certificate of completion, unless the state asked that the additional modules be included as optional modules. For a summary of participation in the professional development modules as part of the required training, see section 9.4.1.1 of this chapter. Some states also include one or two optional videos in their required test administrator training courses. These videos are posted on the DLM website but were accessed in Moodle if included in the required training. They do not have a post-test. Completing them is not required to generate the certificate of completion.
Figure 9.1 illustrates the differences between the two training formats for new test administrators.
Figure 9.1: Required Training Processes Flows for Facilitated and Self-Directed Training
9.3.1 Facilitated Training
The facilitated modules for new test administrators are intended to use with groups. This version of the modules is designed to meet the need for face-to-face training without requiring a train-the-trainers approach or requiring the facilitator to have deep expertise in the subject matter. Each state determines its own policy guidance regarding who serves as facilitators. Examples of individuals who serve as facilitators include district- and building-level test coordinators, district special education coordinators, instructional coaches, lead educators, state education agency staff, and trainers from regional education agencies that are responsible for professional development.
Facilitators are provided an agenda, a detailed guide, handouts, videos, and instructions to facilitate a meaningful face-to-face training. Facilitators play the DLM-produced videos and implement learning activities as described in their facilitator directions. Facilitators who wish to add to the training contents or deliver the content themselves rather than via video also have access to the PowerPoint slides and transcripts.
Facilitators for these sessions prepare for the training by reviewing all videos and all sections of the Test Administration Manual (DLM Consortium, 2021) addressed in the training. States also recommend (and some require) facilitators to complete the training requirements themselves; facilitators who are also test administrators are required to pass the post-tests. Facilitators are asked to ensure that participants have Educator Portal and Moodle accounts and access to them prior to the facilitated training session. Their responsibilities include setting up the training area with equipment, delivering the facilitated training modules, and directing users to return all equipment. Finally, facilitators direct test administrators to take each module post-test in Moodle with support from the Guide to DLM Required Test Administrator Training (Dynamic Learning Maps Consortium, 2021b) for details and access procedures. Facilitated training is flexibly structured so post-tests could be taken on-site during training sessions (e.g., in a computer lab) or independently after the training session was complete.
9.3.2 Self-Directed Training
The self-directed modules are designed to meet the needs of educators who are unable to attend facilitated sessions and those who otherwise need access to on-demand training. Some states and districts do not provide a facilitated training, so test administrators in those areas have to take the self-directed course. Self-directed modules combine videos, text, and online learning activities to engage educators with a range of content, strategies, and supports, as well as the opportunity to reflect upon and apply what they learned. The videos are identical to those used in facilitated training. The post-test for each module is the same as those for trainees who attend a facilitated training.
The self-directed training is completed entirely within Moodle with support from the Guide to DLM Required Test Administrator Training (Dynamic Learning Maps Consortium, 2021b) for detail and access procedures, including directions for completing all post-tests.
9.3.3 Training Content
Several pertinent resources provided on the DLM website are referenced throughout the training modules, including the Test Administration Manual, Accessibility Manual, Educator Portal User Guide, and the Guide to Practice Activities and Released Testlets. The individual modules that made up the training are described in more detail in the following sections.
9.3.3.1 Module 1: Overview of the Dynamic Learning Maps Alternate Assessment System
Module 1 of the test administrator training provides an overview of the DLM system components. The focus is on familiarizing trainees with DLM vocabulary, such as Essential Elements, linkage levels, assessment blueprints, claims and conceptual areas, and testlets. Accessibility supports are addressed in Module 1. The security standards are also addressed.
9.3.3.2 Module 2: Testlets
In Module 2, participants are shown examples of DLM testlet items. Testlet types (computer-delivered and educator-administered) are also explained.
9.3.3.3 Module 3: The Instructionally Embedded Model
Module 3 focuses on describing the Instructionally Embedded assessment model. The assessment blueprint requirements are explained, and the instructionally embedded process is described, including information about the Instruction and Assessment Planner in Educator Portal. The fall and spring windows are addressed, including how to access Testlet Information Pages. Finally, Module 3 provides an overview of the DLM scoring model.
9.3.3.4 Module 4: Becoming Familiar with DLM Resources
Module 4 concludes the training by explaining test administrator responsibilities. The module begins with a discussion about using the resources found on the DLM website. The module also describes considerations for utilizing the practice activities and released testlets, scheduling assessment sessions, and devising back-up plans to ensure all students complete the assessment. Test administration practices allowed and not allowed are addressed. Module 4 also includes information about the DLM professional development modules offered at https://dlmpd.com.
9.3.4 Completion of All Modules
Each of the four required training modules for new test administrators includes a post-test. Participants are required to complete each post-test with at least 80% accuracy, and they have to score at least an 80% to be able to move on to the next module. Participants are allowed to retake the post-test as many times as necessary to achieve a passing score. Feedback is provided in Moodle for any incorrect responses. Trainees always have access to the module’s contents, and the recommendation is that they review the module’s contents before attempting the post-test again. Whether participants complete the self-directed or facilitated version of the required training, post-tests are completed independently in Moodle. Details of participants’ progress are made available to state educational agencies through a data extract in Educator Portal.
9.4 Instructional Professional Development
The DLM professional development system is built to support educators in their efforts to teach English language arts and mathematics to their students who participate in the DLM System. In addition to instructional support, the instructional professional development modules also teach educators about the DLM System. While the modules were originally intended for educators who administer DLM assessments, demographic information suggests that preservice educators, related service providers, parents, and others also accessed and completed the modules.
The professional development system is built in WordPress, an open-source website content management system. The professional development modules and instructional support materials are available for anyone’s use at https://dlmpd.com or through a direct link from the DLM website. These DLM professional development modules address instruction in English language arts and mathematics. The modules also address processes educators can apply to create Individual Education Programs that are aligned with the DLM Essential Elements and supports they can provide to address the communication needs of the students they teach. Finally, the modules help them understand the components of the DLM assessment system more completely.
To support state and local education agencies in providing continuing education credits to educators who complete the modules, each module also includes a time-ordered agenda, learning objectives, and biographical information regarding the faculty who developed the training modules. There are a total of 51 modules, which are described in section 9.4.1.2 of this chapter.
The 51 modules are available in both self-directed and facilitated formats. The self-directed modules are available online, on-demand. The interactive modules include a combination of video-based content, embedded activities, and, for participants who would like to receive a certificate documenting their successful completion of the module, a five-item pre- and post-test. These certificates are sent directly to each participant’s email when they score 80% or higher on the post-test.
Modules in the facilitated format were created for groups and by individuals who prefer to learn by reading the contents rather than interacting with video and other package contents. The modules are organized on the DLM professional development website by claim and conceptual area and alphabetically. For more information on DLM claims and conceptual areas, see Chapter 2 of this manual. The modules have also been packaged into groups that are comprised of 5- and 10-hour focused packages for schools and school systems to use across the school year. These packages include the following:
Foundations of Instruction in DLM. This package provides educators with five hours of foundational content regarding the DLM System and the students who participate in DLM alternate assessments. Divided across five modules, the content can be split across five sessions or combined to meet local needs. The five modules are (1) Who are Students with Significant Cognitive Disabilities? (2) College and Career Readiness Standards: An Overview, (3) DLM Essential Elements Overview, (4) DLM Claims and Conceptual Areas, and (5) Individual Education Programs Linked to the DLM Essential Elements.
Getting Started with Students Working at an Initial Precursor Linkage Level. This package provides educators with 10 hours of important content and pedagogical knowledge pertaining to students with the most complex disabilities who are working at an Initial Precursor linkage level in the DLM alternate assessments in English language arts and mathematics. Divided across 10 modules, the content can be split across 10 sessions or combined to meet local needs. The 10 modules are (1) Beginning Communicators, (2) Symbols, (3) DLM Core Vocabulary and Communication, (4) Speaking and Listening, (5) Shared Reading, (6) Writing with Alternate Pencils, (7) Emergent Writing, (8) Predictable Chart Writing, (9) Unitizing, and (10) Forms of Number.
From Shared Reading to Reading Comprehension. This package provides educators with five hours of professional development focused on reading instruction, beginning with supporting engagement and interaction during shared reading and moving to reading and listening with comprehension. Divided across five modules, the content can be split across five sessions or combined to meet local needs. The five modules are (1) Shared Reading, (2) Teaching Text Comprehension: Anchor-Read-Apply, (3) Generating Purposes for Reading, (4) DR-TA and Other Text Comprehension Approaches, and (5) Supporting Participation in Discussion.
From Emergent Writing to Written Composition. This package provides educators with 10 hours of professional development focused on writing instruction, beginning with supporting access to writing tools and early engagement in writing, and moving through writing for various purposes and audiences. Divided across 10 modules, the content can be split across five sessions or combined to meet local needs. The 10 modules are (1) Universal Design for Learning, (2) Writing with Alternate Pencils, (3) Emergent Writing, (4) Writing: Text Types and Purposes, (5) Writing Information and Explanation Texts, (6) Writing: Production and Distribution, (7) Writing: Getting Started with Narrative Writing, (8) Writing: Getting Started in Writing Arguments, (9) Writing: Research and Range of Writing, and (10) Strategies and Formats for Presenting Ideas.
Moving Students from Initial to Distal Precursor Linkage Levels in Mathematics. This package provides educators with 10 hours of professional development focused on emergent mathematics understandings for students with complex disabilities participating in the DLM alternate assessment in mathematics at the Initial Precursor or Distal Precursor levels. Divided across 10 modules, the content can be split across five sessions or combined to meet local needs. The 10 modules are (1) Unitizing, (2) Forms of Number, (3) Composing and Decomposing Shapes and Area, (4) Composing, Decomposing, and Comparing Numbers, (5) Counting and Cardinality, (6) The Power of Ten-Frames, (7) Basic Geometric Shapes and Their Attributes, (8) Measuring and Comparing Lengths, (9) Time and Money, and (10) Patterns and Sequence.
In addition to the 51 modules, the instructional professional development site provides instructional resources for educators. These include DLM Essential Element unpacking documents; extended descriptions of the Initial and Distal Precursor linkage levels and how they relate to grade-level Essential Elements; links to dozens of texts that are at an appropriate level of complexity for students who take DLM assessments and are linked to the texts that are listed in Appendix B of the Common Core State Standards; vignettes that illustrate shared reading with students with the most complex needs across the grade levels; supports for augmentative and alternative communication for students who do not have a comprehensive, symbolic communication system; alternate pencils for educators to download and use with students who cannot use a standard pen, pencil, or computer keyboard; and links to Pinterest boards and other online supports.
9.4.1 Professional Development Participation and Evaluation
There are two ways in which test administrators and educators may complete professional development modules: required test administrator training or optional professional development. As described in section 9.3, some states require professional development modules to be completed as part of the required test administrator training. States can require certain modules be completed by new test administrators, returning test administrators, or both. Test administrators completing professional development modules as part of the required test administrator training access the modules through the Moodle training site where the rest of the required test administration training is located. The second way in which professional development modules are completed is through the DLM professional development website. The modules on the professional development website may be accessed by anyone and can be completed at any time. Additionally, participants completing modules on the professional development website are administered a short evaluation survey following the module. With the exception of the evaluation survey, the content of the modules is identical between the required test administrator training and the professional development website.
9.4.1.1 Required Professional Development Participation
A total of nine states required at least one professional development module as part of their required test administrator training. Table 9.1 shows the number of modules required, by state, for new and returning test administrators, as well as the total number of modules completed. In total, 21,888 professional development modules were completed by 4,431 new and 3,033 returning test administrators as part of the required training.
|
New test administrators
|
Returning test administrators
|
||||
|---|---|---|---|---|---|
| State | Required modules | Test Administrators | Required modules | Test Administrators | Total modules completed |
| Alaska | 1 | 84 | 1 | 130 | 214 |
| Delaware | 2 | 359 | — | — | 718 |
| Maryland | 4 | 1,466 | — | — | 5,864 |
| New Hampshire | 1 | 152 | — | — | 152 |
| New Mexico | 3 | 824 | — | — | 2,472 |
| Oklahoma | 5 | 649 | 5 | 1,305 | 9,770 |
| Rhode Island | 1 | 110 | 1 | 190 | 300 |
| West Virginia | 2 | 203 | 1 | 364 | 770 |
| Wisconsin | 1 | 584 | 1 | 1,044 | 1,628 |
Table 9.2 shows which modules were required for new and returning test administrators across all states choosing to include professional development modules in the required training. For example, the DLM Essential Elements module was required for new test administrators in four states and was required for returning test administrators in two states.
| Module | States requiring for new test administrators | States requiring for returning test administrators | Total modules completed |
|---|---|---|---|
| DLM Claims and Conceptual Areas | 1 | — | 1,466 |
| DLM Essential Elements | 4 | 2 | 3,993 |
| Effective Instruction in Mathematics | 3 | 2 | 3,987 |
| Individual Education Programs Linked to the DLM Essential Elements | 4 | 2 | 3,437 |
| Principles of Instruction in English Language Arts | 3 | 2 | 5,048 |
| Universal Design for Learning | 1 | — | 824 |
| Who are Students With Significant Cognitive Disabilities? | 4 | 1 | 3,133 |
9.4.1.2 Optional Professional Development Participation
Table 9.3 shows the number of individuals who completed optional professional development modules through the DLM professional development website, as well as the total number of test administrators from each state who had a student assigned to them for the DLM assessment. In total, 5,987 modules were completed in the self-directed format from August 1, 2021, to July 31, 2022. Since the first module was launched in the fall of 2012, a total of 77,061 modules have been completed on the professional development website.
| State | Participants | DLM test administrators | Total modules completed |
|---|---|---|---|
| Alaska | 20 | 180 | 42 |
| Arkansas | 24 | 785 | 62 |
| Colorado | 250 | 1,161 | 882 |
| Delaware | 83 | 259 | 201 |
| Illinois | 89 | 3,254 | 208 |
| Iowa | 78 | 939 | 358 |
| Kansas | 138 | 895 | 619 |
| Maryland | 125 | 1,405 | 448 |
| Missouri | 38 | 1,564 | 381 |
| New Hampshire | 9 | 328 | 28 |
| New Jersey | 79 | 2,928 | 456 |
| New Mexico | 15 | 643 | 100 |
| New York | 84 | 5,550 | 271 |
| North Dakota | 12 | 265 | 40 |
| Oklahoma | 16 | 1,484 | 121 |
| Pennsylvania | 53 | 4,380 | 525 |
| Rhode Island | 40 | 248 | 78 |
| Utah | 81 | 782 | 287 |
| West Virginia | 10 | 489 | 26 |
| Wisconsin | 178 | 1,543 | 846 |
| Non-DLM state and other locations | 4 | — | 8 |
| Note. Participant counts may include individuals who are not educators or test administrators (e.g., pre-service educators). |
To evaluate educator perceptions of the utility and applicability of the modules, DLM staff asked educators to respond to a series of evaluation questions upon completion of each self-directed module. Four questions asked about importance of content, whether new concepts were presented, the utility of the module, and whether educators planned to use what they learned. Educators responded using a four-point scale ranging from strongly disagree to strongly agree. During the 2021–2022 year, educators completed the evaluation questions 86% of the time. The responses were consistently positive, as illustrated in Table 9.4. Across all modules, 82% of respondents either agreed or strongly agreed with each statement.
To evaluate the consistency in the ratings for each module, we calculated Cronbach’s (1951) alpha from the four items for each module using all ratings from fall 2012 through the 2021–2022 year. Across all modules, alpha ranged from .85 to .96 with an average value of .92, suggesting high internal consistency in responses.
9.5 Conclusion
The DLM System makes training and instructional professional development modules available to test administrators and educators as well as state and local education agency staff. The Required Test Administrator Training is designed to ensure that test administrators understand the DLM System in order to administer the assessment with fidelity. Complementary to the required training are the professional development modules, which provide participants learning opportunities to support instructional practices aligned with content measured by DLM assessments. In 2021–2022, participants provided consistently positive feedback regarding the importance and relevance of the professional development modules.