Requirements to Becoming a Peer Online Course Reviewer
Experience
POCR reviewers must have successfully taken at least one of their fully online courses through POCR and received a “Quality Reviewed” badge from the California Virtual Campus (CVC).
Training & Certification
Reviewers must have successfully completed the following training courses. Contact Jennifer.Azzaro@deltacollege.edu for registration details.
♦ Canvas 2.0: Online Course Design (20 hours)
♦ Peer Online Course Review (45 hours)
About the Team
Delta’s dedicated team of POCR reviewers includes the Lead Instructional Designer, two Accessibility experts, and experienced online faculty from a variety of TrACs, the Academic Senate, CTA, and Equity Task Forces.
All reviewers are faculty.
All have completed an intensive training course and attend two statewide POCR meetings per year, to ensure a consistent review approach with peers across the CCC system.
The Lead Instructional Designer also serves as Delta’s POCR Lead and has regular interaction with the CVC-OEI to ensure process currency.
Many reviewers have been involved in POCR for 3+ years and have reviewed multiple courses.
You will be paired with an experienced reviewer to help guide you through your first review!
Click to Meet a Reviewer
What to Expect
Confidentiality
Peer Online Course Reviews will NOT be:
♦ Mandatory
♦ Shared outside the assigned review team
♦ Part of the faculty evaluation process
Per the SJDCTA 2021-24 contract, Article 16.1.2.7: The peer online course review process (POCR) evaluation shall not be used in a faculty member’s administrative evaluation and is for the purpose of identifying professional development opportunities.
Reviewer Pool and Course Assignments
Individuals who meet the above requirements will be added to the pool of available Reviewers, however this does not guarantee a course review. Review assignments will come from the Office of Regional and Distance Education and may be accepted or declined.
Staying Current with Norming Meetings
In order to remain in the pool of available reviewers, both active and inactive reviewers must participate in a minimum number of State and Local “Norming” meetings per year (usually 2 but subject to change). Regular norming helps to ensure currency in the CVC-OEI Rubric standards and understanding of what to look for when reviewing courses.
Course Teams, Timeline, and Workload
Each review team consists of 4 reviewers: the POCR Lead, Accessibility Reviewer, and two additional faculty. It takes roughly six months to take a single online class through POCR, using a 3-phase process:
Phase 1
(6 weeks): Kick-Off Meeting, Develop Initial Content, and Preliminary Review
♦ Course Authors work closely with the POCR Lead and Accessibility Reviewer, only.
Phase 2
(6 weeks): Develop the First Half of the Course and Informal Review
♦ Two additional faculty reviewers are assigned and required to attend ~2 check-in meetings with course author and review team. Informal review is done by the POCR Lead and Accessibility Reviewer.
Phase 3
(8-12 weeks): Develop the Second Half of the Course and Formal Review
♦ About 4 meetings, plus formal review (which may necessitate additional meetings). Formal review includes significant time looking over every part of the course (Modules, Assignments, Assessments, Discussions, Homepage, Syllabus, Announcements, Rubrics, Resources, and multimedia).
♦ Reviewers must fill out the CVC-OEI Rubric document and include substantive feedback that is both collegial and evidence-based. Reviewers confer with members of the team assigned to that course and ensure review is completed by the timeline set by the POCR Lead.
♦ Reviewers join the project with the understanding that each course may require several iterations of reviews and feedback.
For more detail, see our sample POCR Timeline.
Quantity and Quality of Feedback
Reviewers should expect to conduct a thorough, in-depth review of every aspect of the course. Review requires care. No course will align with every standard the first time through. This can be challenging for both course authors who take pride in their work, and reviewers who need to remain constructive, supportive, and encouraging. It’s not easy for authors to hear that multiple areas of a course are “Incomplete”. Even if a course is close to "Alignment," reviewers should still give suggestions to make it even better. For example: This element is in alignment. Well done! To move into the "exemplary" category, you might consider adding...
Feedback should always be:
♦ Substantive: Even if a rubric element is “Aligned,” the CVC requires more than simply re-phrasing the rubric language – and certainly more than, “Aligned: great work!” Feedback should have substance and speak to the uniqueness of the course.
♦ Collegial: Most feedback statements should attempt to start with something positive, even if the element is found to be “Incomplete” and in need of revision. Acknowledge the course author’s effort and intent.
♦ Evidence-based: Reviewers must cite the places they checked – and evidence they found – regardless of whether an area is Incomplete / Aligned / Exemplary.
♦ Supportive: Offer encouragement and suggestions for how the course author might bring an area into alignment.
♦ Confidential: Reviewer feedback should always be kept between the course author, review team, and POCR Lead.
See the next section for specific feedback examples.
Conferring With Teammates
Reviewers are expected to confer with their teammates and use various forms of State and Local guidance, paired with evidence from the course, to ensure accurate findings for each of the CVC-OEI Rubric standards. When in doubt, team members may seek further guidance from the POCR Lead.
Tips
♦ The review team is just that, a team! Be sure you agree in your shared feedback and use We statements throughout the review. “We looked in Modules 2, 4, 6, and 8…” “We were unable to find…”
♦ Critique the course, not the course author. Remember to ground your feedback in the rubric language and evidence.
Examples of Review Team Feedback
Incomplete
Issue: A9 (Instructions) marked “Incomplete” because the instructor has embedded several 10–20-minute YouTube videos on Canvas pages and given instructions to, “Watch the following video.”
Substantive review team comment:
A9: Instructions – Incomplete.
[Collegial Acknowledgement] Great job embedding videos on your course pages. This will help students stay focused on your content! [Evidence-based] We looked on the pages that include videos (below) as well as the Module Introduction pages, but could not find instructions for students as to what to do with the videos.
♦ Module 2: Intro page and Emotional Development page
♦ Module 3: Intro page and Behavioral Development page
♦ Module 4: Intro page and Social Development page
[Supportive Suggestions] Will students be assessed on the videos? What topics/ideas in each video should students focus on? We recommend giving instructions such as, “Please watch the following video, taking notes on the topics of .... as these will be covered in the unit discussion.” Adding context to the videos will help students get the most out of viewing them.
Aligned
Substantive review team comment:
A1: Placement of Objectives – Aligned.
[Evidence-based] We found Unit-level outcomes in each of the 5 course Modules, at the top of each page that specifically addresses that objective. [Collegial Acknowledgement] Students will find this direct mapping of content to objectives very helpful! [Supportive Suggestion] To bring this area to “Exemplary,” consider listing the unit-level objectives on the first page of each module, titled: "Unit X Introduction." Clear and consistent placement will help students anticipate what to expect as they begin each new Module.
Exemplary
Substantive review team comment:
A3: Alignment of Objectives – Exemplary.
[Evidence-based] We checked each module and found that all contain a consistent set of resources that support unit objectives (including video lectures, slides, notes, and learning resources). [Collegial Acknowledgement] Students will benefit from the explicitly clear connections on the Module Introduction pages, where unit objectives are mapped to both course objectives AND assessments within the unit. [Further Evidence] For example:
Unit 3, Objective #2: Calculate and interpret a Coefficient of Determination.
Mapping: Supports Course Objectives 2, 4, and 5. Assessed in the Unit 3 Worksheet and Exam.
Both the Unit 3 Worksheet and Exam require calculation of a Coefficient of Determination. Students will learn how to do this in the Unit 3 videos, slides, and notes.
Forms Used
Prior to formal review, the course author will provide the assigned review team with a completed Course Review Prep Form, citing examples of where the team can find alignment. This document serves as a starting point for reviewers to begin, but is not an exhaustive checklist.
The POCR Lead will provide each team member with a shared CVC-OEI Rubric document for the course, which can be edited and saved in a shared document repository such as Box.
Payment (if applicable)
A stipend of $1,500 per course will be approved upon completion of review, meaning all areas of the Rubric were found to be either “Aligned” or “Exemplary” and the assigned team has signed off on the course being ready to submit to the CVC for “Quality Reviewed” badging.
This stipend is effective through June 2025 (as per MOU).
Terminations
If a Reviewer cannot finish the review process due to illness or unexpected personal or professional concern, they can request, in writing, replacement by another individual from the pool of available reviewers, contingent upon the approval of the POCR Lead and Dean of Regional and Distance Education. Division of payment, if applicable, will be determined by the Office of Regional & Distance Education and CTA.
Other Important Notes
♦ Active Reviewers may simultaneously have their own course(s) undergo review.
Questions? Contact Us
Jennifer Azzaro
Instructional Designer and POCR Lead
Jennifer.Azzaro@deltacollege.edu
Vivie Sinou
Dean of Regional and Distance Education
Vivie.Sinou@deltacollege.edu
Psychology 2
Completing the POCR review process for my Psych 02 (Statistics for the Social Sciences) course was a challenging and rewarding experience. The process made me view my course through a new equity lens to make sure all students had the accessibility to be successful in the course. I learned a lot from the process and my skills on CANVAS were sharpened. I am able to use the information I learned from the POCR review and apply it to other courses that I teach.
Sociology 1A and 1B
I would highly recommend the POCR process for any instructors seeking to take their online course to the next level. Having gone through the process twice, I can say with confidence that my online courses are more organized and student-centered than ever. This process really allows you to dive deeply into online learning and provides you with the opportunity to develop a course that takes into consideration the varying learning styles of students.
English 1A
The POCR process helped me to fully align the course content with course learning objectives. Now each assignment has specific learning objectives that are measurable. Every learning task, whether it be a lecture, reading, written assignment, discussion, video, or image, is purposeful and contributes to the desired course outcomes. Students know the who, what, where, when, and why of every action in the course. The accessibility component was invaluable also.
Communication Studies 1A
POCR review has allowed me to offer a much better course, but more importantly, it allows students to learn more. It has transformed my online teaching and it’s the one thing I recommend everyone do if they want to teach online.
Music 4
I thought my classes were pretty well organized and dialed in until I went through the POCR process. I was able to raise my online course to a much higher standard. The process is very structured and clear and for me it revealed several areas that needed improvement. The support team were all amazing and very well organized and more importantly, they understand how challenging it is to design a class while teaching. I would encourage every instructor who teaches online to consider the POCR process.
English 1B
Moving my Eng 1B through POCR made me rethink every choice I make in my online teaching. Is this the most effective way to structure a lecture? Is this the most impactful and relatable assignment? Have I taken enough steps to ensure that no student gets lost? Looking at my class with fresh eyes has made my class more organized, accessible, and engaging for my students.
Education 10, 12, 13, 14 and 70
At a time when the Covid-19 pandemic required faculty to become proficient at executing virtual instruction, I found the POCR process to be invaluable to my goal of designing effective hybrid and asynchronous courses that supported my students’ success. I frequently receive positive feedback from students describing their appreciation for varied elements of our courses spanning content presentation, interaction, assessment, and accessibility which I attribute largely to the professional development that I have received through the POCR process.
Statistics 12
By going through the POCR review process, my class is more accessible, presentable, and organized. Students liked accessing this class which was organized and manageable.
Psychology 1, 2 and 5
Aligning my courses with POCR standards improved my awareness of online course best practices, including addressing diverse perspectives, overcoming accessibility barriers, enhancing content presentation, clarifying assignment instructions, boosting online student interaction, and optimizing the overall student experience, which has significantly improved my courses and student learning.
Canvas Lite & Users GroupAll Delta faculty are members of the Canvas Lite & Users Group.
Join us there to ask questions of experts and your colleagues! Let’s build a community of best practices.
|
||
