You're looking to improve your student's performance on written tests. It's time to talk about Excel. The Excel grid is the heart of PASS. It contains a Data Entry Form to input your results and a Results Analyzer to process them.
In order to help your student, you need the student's current performance. Traditionally, teachers use a student's test mark as the main indicator of their academic standing. But, as you know, the student’s mark on a test is only part of the story. The PASS Excel grid can process data and generate an individualized picture of the student's performance on a test, in the form of Student Performance Assessment Graphs (SPAG).
The PASS Excel grid is designed to:
- Use the categories and variables presented in Assessing Student Performance to generate a picture of the student’s performance on a written test in both table and graph form
- Compare the student's performance on the first and second halves of the test
- Compare the student's performance on two different tests
In this section we will show you how to help your students excel by:
- interpreting the data from the SPAG
- reviewing additional data from the student's completed test booklet
- summarizing the data
Interpreting The Data
From The Student Performance Assessment Graphs
Let us show you what the student graphs look like. We have found that students appreciate the impact of the graphs, as they clearly demonstrate their strengths and areas that need improvement. We'll start by looking at the categories presented in Assessing Student Performance.
Fields of Nursing
Here is a random example of a student’s SPAG generated from the PASS Excel grid. It is from a comprehensive test administered in the last semester of the Nursing program. Although not all Fields of Nursing are commonly used in one semester, it is appropriate to include all Fields of Nursing when administering a comprehensive test. One example would be when preparing nursing students for their OIIQ professional exam.
Test 1
Point of reference: the passing grade for this test was 60%.
Let us start by looking at the student’s strengths. The student achieved 73% in the Medicine & Surgery Field of Nursing. This is significant, because 50% of the questions on the test are from the Medicine & Surgery Field of Nursing. Therefore, the student passed the predominant portion of the test.
The student achieved 60% in both Geriatric Medicine and Pediatrics. We consider this a borderline performance.
The student had the most difficulty with Perinatal Care (40%) and Mental Health (40%). Therefore, there are several areas where the student needs to improve.
To help you better understand the student’s performance, we will talk briefly about blueprints. How do we, as teachers, use test blueprints to guide the student? The blueprint helps the student by informing them about the content that will be tested and how it will be weighted. For now, we will use the OIIQ Professional Examination blueprint as an example. It is our understanding that the weighting for each Field of Nursing is as follows (approximately):
- Medicine & Surgery 60%
- Geriatric Medicine 15%
- Perinatal Care 5%
- Pediatrics 5%
- Mental Health 15%
Based on the SPAG and the OIIQ weighting and the student’s performance assessment, how should the student organize their study time of the Fields of Nursing, according to their relative importance? Our answer is:
- Medicine & Surgery (this makes up the largest portion of the OIIQ exam and is always the most important)
- Mental Health (Mental Health and Geriatric Medicine are equally weighted but the student failed the Mental Health Field of Nursing)
- Geriatric Medicine (even though the student passed Geriatric Medicine, with a grade of 60%, this Field of Nursing makes up 15% of the exam, which is greater than the Perinatal Care and Pediatric Fields of Nursing)
- Perinatal Care (Perinatal Care and Pediatrics are equally weighted but the student failed the Perinatal Care Field of Nursing)
- Pediatrics (the student passed this Field of Nursing; in addition, Pediatrics makes up only 5% of the exam)
To present the remaining Categories (Class Content, Professional Component, Functional Component, and Bloom’s Taxonomy), we would like to introduce you to one of our former students. Using her SPAG as examples, we will demonstrate what we learned about her test performance when using the PASS Excel grid.
| Mina was born in Afghanistan and, due to war, had her schooling disrupted at age 8. She fled with her family to Russia. Mina immigrated to Canada with her family when she was 16. She was accepted into the Nursing program as a mature student at age 21. We met Mina in her second year of the Nursing program when she was 22 years old. Her name has been changed to protect her privacy. |
Class Content
The Class Content category is the most applicable to any test, as it provides information on how the student performed in each topic being tested.
Here is the SPAG from Mina's first test in Semester 3 of a six-semester program. As shown below, Mina received a grade of 54% on the test (passing grade: 60%). While this is a failing grade, remember that we want to begin by focusing on her strengths. As we explained in the Introduction, we have found that this helps to engage the student and start building their confidence. As we look closely at her SPAG, we see some areas where she is performing well.
Test 1
To interpret the data, let us separate the theory content from the lab content. While all topics were taught in the course, some were taught in the classroom and some in the laboratory. If you refer to the table, you will notice that there are more marks allocated to class content (40 marks) than lab content (10 marks). Mina should be congratulated for her strong performance on the lab portion of the test. Keep in mind, however, that the lab portion accounts for only 20% of the total test.
In looking at Mina’s performance on the theory content, we can see that she achieved 60% on the topic of Pneumonia. On the remaining topics, we see a range of 30% to 50%. In our experience, this performance suggests that the student is having difficulty understanding and applying pathophysiology content. This is a pattern we often see in students who try to memorize pathophysiology theory without understanding the disease process and the resulting nursing implications. This interpretation will require validation with Mina.
Professional Component
The Professional Component category provides important information on how the student performed in relation to different nursing activities. The Professional Component represents the nursing process as per the OIIQ. Since it mimics the steps in the nursing process, the Professional Component can be applied to all topics related to health problems. Therefore, of all the categories we are assessing, the Professional Component gives us the most information about the student’s ability to apply nursing knowledge.
We would like to remind you that we have added the Not Applicable variable to label test questions that do not relate directly to a nursing activity (assessment, intervention, and continuity of care).
Here is Mina’s SPAG for the Professional Component category:
Test 1
Recall that Mina achieved a grade of 54% on the test. Again, let us look at this from the perspective of her strengths and weaknesses in applying her nursing knowledge. She performed best on Not Applicable-type questions (79%). While this is a strong performance, this variable accounts for only 14% of the test. The Not Applicable variable is a catch-all grouping of questions that do not relate directly to a nursing activity; therefore, it does not help us to define patterns from which to build strategies.
Mina was more successful on Assessment-type questions (64%) than on Intervention-type questions (41%). We often see this pattern when students memorize signs and symptoms and diagnostic tests, rather than making links to the pathophysiology of the disease process. This also leads to difficulty in identifying the appropriate nursing interventions. We will validate our interpretation with Mina but, regardless, we need to work with her to improve her performance on both Assessment-type and Intervention-type questions.
Mina was least successful on Continuity of Care-type questions (33%). While this is also an area that needs improvement, it represents only 6% of the test and is therefore of less concern. In our experience, when students improve in Assessment-type and Intervention-type questions, improvement in Continuity of Care-type questions follows.
Functional Component
The Functional Component provides information about the student’s ability to use different types of knowledge, skills, and attitudes in clinical situations. While the OIIQ tests every Field of Knowledge, it is unlikely that a specific nursing course would have content on every Field of Knowledge in a single test. One exception might be a program comprehensive test.
Here is Mina’s SPAG for the Functional Component category:
Test 1
The Scientific Field of Knowledge is the foundation of nursing practice. Therefore, it is the predominant portion of most nursing tests. In this field, Mina achieved a grade of 52.5%. This is not a surprising result. A picture of her overall performance is starting to form: Mina is struggling with understanding and applying her knowledge of pathophysiology. We have concluded this because she is having difficulty with:
- Class Content: Topics related to disease processes
- Professional Component: Assessment-type and Intervention-type questions
- Functional Component: Scientific Field of Knowledge
Since the Scientific Field of Knowledge is the foundation for everything nurses do, it will be essential for us to address this with Mina.
On this test, the Communication Field of Knowledge was the second most tested field. Communication, an essential part of nursing, can be a difficult topic to test. In addition, factors related to language and culture often play an important role in the student’s ability to formulate answers to open-ended Communication questions. Mina scored 58% here, which demonstrates a foundation upon which to build. In order to help her to improve her performance, we need to examine where and why she lost marks, so as to better understand her areas of difficulty. This calls for a review of Mina’s test booklet.
Mina did not do well on the Legal Field of Knowledge (25%). Because it is difficult to discern patterns from just two questions, it is tempting to disregard this result. However, the Legal Field of Knowledge is important and, in order to help Mina, we need to look at where and why she lost marks when we review her test booklet.
Mina performed best on the Operational Field of Knowledge (100%). This makes sense given that Mina did well on the lab portion of the test. In addition, this was only a small part of the test, so it would not be an area of focus.
On this test, the Organizational and Moral and Professional Ethics Fields of Knowledge of the Functional Component were not tested.
Bloom’s Taxonomy
This category provides information about the student’s ability to answer different levels of questions as defined by Bloom’s Taxonomy.
Here is Mina’s SPAG for the Bloom’s Taxonomy category:
Test 1
Considering her mark of 54% on the test, Mina did relatively well on the Remember-level (69%) and Understand-level (67%) questions. These questions accounted for almost half of the test, so Mina certainly did something right in preparing for it. We need to explore with Mina what she thinks contributed to her success on the Remember- and Understand-level questions, so that we can build on her strengths.
What we are most concerned about is Mina’s poor performance on Apply-level questions (46%). We had hypothesized in the Professional Component that Mina was using memorization as a study strategy. This could explain her ability to answer the simpler-level questions correctly and her difficulty with answering Apply-level questions.
Let us look at the situation in more detail. In our experience, a student’s performance in Bloom’s Taxonomy Apply-level questions often mirrors their performance in Professional Component Intervention-type questions. This seems to be true for Mina. She scored 41% on Intervention-type questions, which corresponds to her 46% in Apply-level questions.
If we are correct in our hypothesis that Mina is using primarily memorization to learn, she will need help in learning how to apply knowledge. As she learns to apply her knowledge, Mina should improve in both Apply-level questions (Bloom’s Taxonomy) and Intervention-type questions (Professional Component).
Mina received a score of 17% on Analyze-level questions, but there were only three of these on the test. As noted above, Mina needs to learn how to answer Apply-level questions before she progresses to working on the skills required to answer Analyze-level questions. Therefore, this is not a concern right now.
Compare the student’s performance on the first half and the second half of the test
Sometimes students show a remarkable difference in their performance on the first and second halves of a test. This finding, on its own, does not provide you with much information. It is only when talking to the student that you can begin to understand the factors that may have caused the discrepancy.
Here is Mina’s SPAG for Comparison: First Half vs Second Half
The graph indicates a significant difference in Mina’s performance on the first half (60%) and the second half (48%) of the test. Without the data analysis from PASS, this observation would not have been evident. This means that Mina passed the first half of the test. This is a very positive result and should be pointed out to Mina, because it shows that if she can pass one half of the test she has the potential to pass both halves.
In our experience, poor performance on the second half of a test can indicate that, for whatever reason, the student did not finish the test. Again, this is a finding that needs to be explored with Mina.
Reviewing Additional Data
From Students' Completed Test Booklets
When interpreting your student's SPAG, you might identify areas that require further exploration. By reviewing your student's completed test booklet and/or computerized answer sheet, you may be able to gather additional data. As you review the student's test, the following questions may help you to identify patterns:
Overall:
- Did the student finish the test?
- Did the student leave many questions unanswered?
- Did the student erase and/or change many answers?
For each question that the student got wrong or lost marks on:
- Did the student understand what was being asked? (For example, the student might have answered an Assessment Question with an Intervention.)
- Did the student use relevant information in the case to answer the question?
- Was the answer clearly written?
- Was the answer vague or incomplete?
As you review the test, take note of the patterns that you are seeing. Although we cannot show you Mina’s completed test booklet, we learned that she:
- did not finish the test
- lost marks for answers that were not clearly written
- did not link all answers to the cases
Summarizing The Student Assessment Data
Summary following Test 1
Let us recap Mina's overall performance on Test 1. At first glance, this is a failing performance. Mina received 54% on the test, which had a passing grade of 60%. However, by analyzing and interpreting the SPAG and reviewing her test booklet, we were able to break down her performance. To help you organize the student data, we have compiled two worksheets: Summary of Student Assessment Data and Analysis of Student Data. We have drawn up a table to show you how to start organizing student data.
| Strengths | Areas needing improvement | Areas to explore with Mina |
|---|---|---|
|
Strong performance on lab portion of test (83%–100%) Passing grade on Assessment-type questions (64%) Beginning foundation in Communication Field of Knowledge (58%) Passing grade on simpler-level questions of Bloom’s Taxonomy (69% and 67%) Passing grade on first half of test (60%) |
Application of knowledge:
Assessment type questions (64%) Intervention questions (41%) Apply-level questions (46%) Poor performance on second half of test (48%) |
Test not completed Marks lost for answers not clearly written Not all answers linked to the case |
We will use this summary to build the Action Plan for Mina.
Now, you can take a look at another example, Student William, or move on to the Complementary Assessment Tools to gather additional information about your student.