Online Course Design Quality Review
The Florida Board of Governor's identifies several strategic goals focused on quality practices. Starting in 2019, online courses within the State University System (SUS) that complete an online course design quality review will be tagged with a . The system goal is to increase the number of online courses in the SUS with a Quality designation. To support this statewide effort, online courses at 老澳门资料 follow the Quality Assurance measures outlined in 老澳门资料's DL policy. Specifically, within two years of initial delivery, online courses scheduled as distance learning must obtain the Florida High Quality (HQ) course design designation and renew that indicator every five years. Courses already tagged with Quality designation remain valid until expired.
To increase quality and accessibility and reduce barriers to student success, faculty are encouraged to submit their online course(s) for an internal QM review to receive the HQ designation. Any 老澳门资料 faculty member who completed the TOL Foundation Course and developed their course as part of TOL-DCD or Program Transition can receive a stipend upon completing the online course design review process. View 老澳门资料's QM-Reviewed Courses
Quality Matters
老澳门资料 implements the nationally recognized for the review of online course design and systematically builds and evaluates online courses based on QM's research-based standards. The QM Standards specifically focus on course design rather than course delivery or content and assure that the structure of online courses promotes learner engagement. A founding principle that drives QM is the need for standards to reflect current academic research on effective learning. The initial QM Standards and subsequent modifications are based on the insights of teams of experienced online instructors and instructional designers and the best practices and standards promulgated by accrediting bodies and national and international organizations. In addition, the QM Standards have been examined for consistency with the conclusions of the educational research literature regarding factors that increase learning and engagement and improve learner retention rates. Each specific QM Standard is centered on and supported by literature from the field. For more information, visit the FAQ section and .
QM-Certified Peer Reviewers
Critical to 老澳门资料’s implementation of Quality Matters is the assistance of QM-certified peer reviewers participating in the course design quality review process at 老澳门资料. To be eligible, full-time faculty members who have completed TOL and undergone a QM review of their own online course can register for online QM training courses, with the fees covered by CIRT. As an incentive, QM-certified peer reviewers receive a $500 stipend for every two internal reviews they complete at 老澳门资料.
Participating as a peer reviewer offers several benefits, including:
- Experience in a quality assurance process, enhancing your understanding of course design standards.
- Expansion of your professional community, connecting you with peers engaged in similar endeavors.
- Opportunity to review both internal and external courses, gaining valuable insights and ideas to improve your own courses.
- Enhancement of your professional development plan and portfolio, showcasing your engagement in quality assurance practices.
- Receipt of a $500 stipend for every two internal reviews completed at 老澳门资料, acknowledging your valuable contributions.
- Potential for additional payment from Quality Matters when serving as a member of national review teams, recognizing your expertise and involvement on a broader scale.
-
HOW TO BECOME A PEER REVIEWER +
There are two ways you can participate as a Quality Matters (QM) Certified Peer Reviewer.
- You can choose to be an internal reviewer, participating in course peer review teams only at 老澳门资料.
- You can aim to become a national reviewer, eligible to participate in both internal and official QM reviews.
To become a QM-certified peer reviewer, you must:
- Complete the TOL-DCD process (i.e., developed at least one course that passed a quality course review)
- Complete the two-week online training course facilitated by QM (CIRT will pay the course fee).
- Complete the three-week online training course facilitated by QM (CIRT will pay the course fee).
Note: Choose invoice as the form of payment at registration - they will bill us. If you already have a QM account, just login into the QM site and register for the training courses.
Getting Started
Before you register for the first online training course, please contact Kevin Hulen (kevin.hulen@unf.edu) to verify your eligibility and discuss the peer review process.
Internal Reviewers
Once you've completed the training (listed above), you will be listed on 老澳门资料's QM-Certified Peer Reviewer, and whenever we have a course ready for review, we'll reach out to check your availability. You will receive a $500 stipend for every two internal reviews completed at 老澳门资料. As an internal peer reviewer, you must:
- Review online courses using QM Standards and Rubric. The estimated time per review is 4-8 hours or less once you've conducted a few. You have 2-3 weeks to complete the review.
- Maintain strict confidentiality about any courses you access during the review process, including any student information contained in these courses.
- Not change, copy or modify any content within a course.
- Observe copyright laws and respect the intellectual property rights of faculty.
National Reviewers
If you want to get involved with conducting external (national) QM reviews with teams at other institutions, you must also:
- Have taught online within the last 18 months. If not, you must complete the Exemption section of the application documenting your qualifications for being a Peer Reviewer.
- Submit the , and resubmit annually (by updating information in .)
- Allow your name and institution affiliation to be listed on the national Quality Matters website.
Note: Quality Matters, not 老澳门资料, manages payment for participating in national reviews.
-
PEER REVIEWERS +
The following faculty members are certified to conduct internal QM Reviews at 老澳门资料, and some also participate in official QM-managed reviews (external). In addition, all instructional designers at 老澳门资料 are certified to conduct QM reviews and actively participate in each internal QM review.
Current QM-Certified Peer Reviewers Name College Department Brooks College of Health Health Administration (Master Reviewer) Brooks College of Health Nutrition and Dietetics Brooks College of Health Nutrition and Dietetics Brooks College of Health Nutrition and Dietetics Brooks College of Health Nutrition and Dietetics Coggin College of Business Economics College of Computing, Engineering and Construction School of Computing College of Computing, Engineering and Construction School of Computing College of Arts & Sciences Biology College of Arts & Sciences Communication College of Arts & Sciences Communication College of Arts & Sciences Communication College of Arts & Sciences Criminology and Criminal Justice College of Arts & Sciences Criminology and Criminal Justice College of Arts & Sciences History College of Arts & Sciences Languages, Literatures and Cultures College of Arts & Sciences Music College of Arts & Sciences Psychology College of Arts & Sciences Psychology College of Arts & Sciences Sociology, Anthropology and Social Work College of Education and Human Services Exceptional, Deaf, and Interpreter Education College of Education and Human Services Teaching, Learning & Curriculum (Master Reviewer) College of Education and Human Services Exceptional, Deaf, and Interpreter Education Library Thomas G. Carpenter Library Library Thomas G. Carpenter Library
Quality Designations
Online Course Design Quality reviews can result in a Quality (Q) or High-Quality (HQ) designation. Both quality designations rely on the , but the course review process is unique to Florida. The main differences between the two quality designations are in the number of course reviewers and the number of QM standards that must be met. Both are significant accomplishments that indicate a course has met rigorous, research-based course design standards. While instructors may opt to participate in either level of review, 老澳门资料 encourages instructors to pursue the HQ designation.
Quality
Courses designated as Quality (Q) will have been reviewed by the faculty course developer and one certified peer reviewer, met all essential QM Standards, and provide alternative means of access to course materials in formats that meet the needs of diverse learners.
High Quality
Courses designated as High Quality (HQ) will have been reviewed by the faculty course developer and two certified peer reviewers, met all essential QM standards, received at least 85% of the available points on the QM Rubric, and provide alternative means of access to course materials in formats that meet the needs of diverse learners.
Internal QM Review
Prior to initiating an internal QM review for your online course, it is essential to consult an Instructional Designer regarding the QM review criteria. Once your course is prepared for review, your designated instructional designer will initiate the internal QM review process.
The internal QM review process proceeds as follows:
See below for descriptions of each step in the Quality Review process.
Course Setup
After submitting your course for review, the QM coordinator will assign two reviewers: your instructional designer and a faculty QM-certified peer reviewer. The faculty reviewer will be enrolled in the Canvas Sandbox course and provided access to all course content, including external publisher materials if necessary. Reviewers have a 2-3 week timeframe to complete the review.
Review | Feedback | Revision
The reviewers assess the course, assigning a score based on QM standards and providing recommendations for measurable design revisions. If the course initially falls short of meeting the required QM standards, you will have multiple revision opportunities. Your instructional designer will share the review score and feedback with you, seeking your input on proceeding with the necessary revisions.
Course Meets Quality Expectations
To complete the review process and obtain the HQ designation, your course must meet all 3-point essential QM standards, achieve a total score of 86 or higher out of 101 points, and provide alternative means of accessing course materials in formats that cater to diverse learners, aligning with accessibility guidelines throughout the course. After meeting these criteria, you will receive a $2000 stipend.
Course Design Modifications
Online courses that complete an internal QM review are awarded the statewide High Quality (HQ) designation, valid for up to five years from the review completion date. During five years, most online courses typically undergo course design modifications to continuously improve subject matter content, technology integrations, and other aspects of online courses, which asserts that course design should and will change. To ensure online courses awarded the continue to meet QM standards, course instructors are encouraged to consider the following guidelines as they modify their online courses.
When there is a concern that significant course design modifications will impact the course, an instructional designer can ensure all critical course elements are aligned and meet QM standards. The following considerable course modifications could necessitate the course instructor to resubmit their online course for another QM review:
- Textbook adoptions that directly impact QM alignment standards. For example, a course adopts OER in place of a publisher’s textbook or course switches to using an entirely different online textbook publisher that necessitates significant changes to course design.
- Redesigning a course due to increasing or decreasing credit hours resulting in significant changes to course alignment might require another course review.
-
MINOR MODIFICATIONS +
Minor modifications typically do not impact course design quality and can be implemented by the course instructor as often as necessary. Examples include but not limited to:
- Changes to course policies
- Updates to instructor contact information or course introductions
- Addition of instructional materials
- Addition of graphics, multimedia, external links, and article
- Modifications to assignment instructions
- Updates that improve accessibility or usability
- Less than 20% of critical course design elements are modified (see Major Modifications list below)
-
MAJOR MODIFICATIONS +
Major modifications could negatively impact alignment between critical course elements and possibly invalidate the HQ designation of an online course. If more than 20% of ANY of the following QM alignment standards will be modified, these are considered major modifications, and you are encouraged to consult with an instructional designer:
- Course Level Objectives [QM Standard 2.1]
- Module Level Objectives [QM Standard 2.2]
- Assessments in the Course [QM Standard 3.1]
- Instructional Materials [QM Standard 4.1]
- Learning Activities (including Assignments) [QM Standard 5.1]
- Tools used in the Course [QM Standard 6.1]
-
COURSE ASSISTANCE +
The following support options are available to assist instructors with course modifications:
- CIRT Support for assistance with Canvas and accessibility
- Instructional Design Support for course design modifications and alignment with the QM Rubric and Standards.
Continuous Improvements
Course development is an ongoing process that evolves over time. Over a span of five years, online courses are likely to undergo multiple revisions, leading to variations in course design quality and consistency. These variations can potentially impact the delivery of the course and the success of the students. 老澳门资料's DL policy emphasizes the need for courses delivered online to maintain high-quality designs. According to this policy, such courses are required to undergo a thorough course design quality review every five years. This policy is founded on the understanding that technological advancements, updated instructional materials, and changes in pedagogical approaches can lead to significant alterations in online courses within a five-year timeframe. To ensure a continuous enhancement of online courses, instructors are strongly encouraged to submit their courses for an internal QM review every five years.
Courses listed under 老澳门资料's QM-Reviewed Courses are eligible for another internal QM review after five years or within a five-year period if certain course criteria are met, for example:
- Course modifications have been approved by the Academic Programs Committee (APC)
- The course has undergone major changes (i.e., more than 20% of the course as a whole has changed). View QM Course Design Modifications for details.
- The course is set to be delivered the following semester, and starting a few months early would benefit the instructor and students.
- The course will not be taught again for another year, putting the course at six years or more since the last QM review.
Quality Matters FAQ
-
WHAT IS IT? +
"Quality Matters (QM) is a national benchmark for online course design."
Quality Matters is a not-for-profit subscription service that provides a rubric and a process that academic institutions use for quality assurance and continuous improvement of online courses. The QM review process is a collegial, faculty-driven, research-based peer review process based on a consisting of 8 general standards and 44 specific standards nationally recognized as reflecting a shared understanding of quality in online and hybrid course design.
-
WHO'S DOING IT? +
Higher education institutions throughout the United States and internationally have subscribed to the Quality Matters rubric and services to assure quality and continuous improvement in their online education offerings. There are over 30 institutions and over participating in Florida.
-
HOW DOES IT WORK? +
An institution or a statewide consortium subscribes to QM for an annual fee depending on the institution's individual needs. This gives the subscribing institution's faculty and staff access to an automated, web-based QM rubric for reviewing online courses, the ability to conduct official course reviews and QM recognition (a seal) for those courses that meet standards, discounts on (or free seats in) QM training, and access to private web resources.
-
WHY IS IT SIGNIFICANT? +
The Quality Matters approach to quality assurance in online courses is unique. It emphasizes a collegial, peer-based review process in which the faculty course designer is a part of the review team. A course is never described as having "failed" a review; it is always "in the process of meeting standards" and is described as such until it meets the QM rubric standards at a rate of 85% or better. The QM rubric and process is also unique in that it is diagnostic (not evaluative or judgmental) and focuses solely on online course design and not on course delivery. According to the most current research literature, the QM standards are centered on the best practices in instructional design and online learning.
-
WHAT ARE THE DOWNSIDES? +
The cost to the institution of the privilege of using a rubric that may seem to be based on common sense and "what everyone already knows about best practices in instructional design and online learning" may seem excessive to some. However, the use of the rubric is a demonstration to stakeholders of the investment in and concern for quality in online education. Another possible problem that may occur with a prescribed set of standards is a potential for a sense of infringement on academic freedom, particularly in course design creativity.
-
WHERE IS IT GOING? +
The number of subscribers to Quality Matters continues to grow, and it appears that the QM seal will become more broadly recognized on online courses that have met this gold standard in distance education. Several new developments are emerging from the Quality Matters organization, which include the following:
- A project to implement QM certification of textbook publisher materials for online education
- A QM partnership with Sloan-C
- A growing interest in viewing the Quality Matters standards from a student perspective and making the standards speak directly to students regarding what is of value to them in online education
-
WHAT ARE THE IMPLICATIONS FOR TEACHING? +
The principles and standards embodied in the Quality Matters rubric and process are based on research that indicates that certain elements of course design if implemented correctly, will have the following effect on teaching and learning:
- Improvement in student learning outcomes and student retention in online courses
- Integration of new technologies using sound online instructional design principles and research-based best practices
- Efficiency in the use of instructional resources
- Continuous improvement in online and hybrid courses and ongoing faculty development
-
HOW DOES 老澳门资料 BENEFIT FROM QM? +
Quality Matters processes benefit both individual faculty members and their institutions in the following ways:
- Improved student engagement and learning outcomes
- Adoption of a systematic and comprehensive continuous quality assurance process that includes faculty training, course development, and course improvement processes that are aligned with accreditation standards
- Incorporation of new technologies and research findings
- Opportunity to engage in benchmarking activities with peer institutions
- Ongoing faculty professional development
- Opportunity for peer-to-peer collaboration and sharing across institutions
- Online courses that meet a consistent and widely-respected quality threshold