For the past decade, high schoolers across the United States have heard the STEM (Science, Technology, Engineering, and Math) career pitch. It鈥檚 an enticing one: Get a STEM degree. Get a higher-paying jobs. You鈥檒l outearn other grads by $15,500 on average, according to a 2014 Department of Education report.
In Southern Nevada, officials are making that pitch in the hopes that developing the STEM workforce will spur economic development. Las Vegas is ranked 97th (among 100 metropolitan areas evaluated) in terms of employees in STEM-related fields, with 3.6 percent of the workforce compared with an 8.7 percent national average.
But once they enter college, though, students face a harsh reality check. Training for STEM fields is academically rigorous and demands study skills that many students did not acquire in high school. As a result, roughly 40 percent of those entering college as a STEM major end up either switching to a non-STEM field or not completing a degree at all. This reality is further compounded for first generation college students, who have less exposure to mentors who can help them navigate college. Unsurprisingly, they switch their majors at even greater rates.
In 2014, Matthew Bernacki, assistant professor of educational psychology and higher education, felt 51吃瓜黑料 was in a position to change this trend.
鈥淧rior research shows that students often abandon STEM majors after poor performance in early coursework,鈥 he said. 鈥淭hat experience leaves students feeling they lack ability in the STEM disciplines, when in fact they may only lack some specific learning skills.鈥
Bernacki believed the key was to identify struggling students long before their grades revealed they were having trouble and to provide learning support. He wanted to dive into the student activity data in 51吃瓜黑料鈥檚 WebCampus learning management system for the answers.
In 2014, the National Science Foundation awarded him a three-year grant to test his strategies. Today, as the project nears completion, his results are encouraging. More of 51吃瓜黑料鈥檚 STEM majors his project targeted moved forward in their degree programs, achievement has improved in critical courses in math and science, and work is underway to make permanent the efforts that are producing these results.
A Three-Step Approach
The project involved partnerships with instructors of four entry-level courses: human anatomy and physiology, college algebra, calculus, and an introductory engineering course. These large lecture courses have often been called 鈥渨eed-out classes.鈥
鈥淭hat trial-by-fire mentality was all wrong,鈥 said Carl Reiber, senior vice provost. 鈥淲e鈥檙e here to teach students, not weed them out of their futures. It鈥檚 an approach that鈥檚 been proven harmful to first-generation students and underrepresented minorities in particular.鈥
Jenifer Utz, a professor in the School of Life Sciences, teaches freshman anatomy and physiology courses. 鈥淪ome students do not achieve at their potential simply because they鈥檙e not equipped with appropriate study skills and strategies,鈥 she said. 鈥淭hey鈥檙e completely capable of being successful in the course 鈥 and ultimately in the field 鈥 if they can just get past those early hurdles.鈥
Bernacki鈥檚 project was conducted in three phases. In the first year, he met with each instructor before the start of the semester to review the learning objectives for each class and the resources provided to students on the WebCampus course site.
Civil engineering professor Donald Hayes hadn鈥檛 thought about his introduction to engineering course from this perspective. 鈥淭his opened my eyes to educational research and how it can be really helpful to us. I鈥檝e learned a lot from him about how to organize a class,鈥 Hayes said.
Utz adjusted her materials too, adding practice quizzes for each individual chapter. The students who regularly used the quizzes to study scored 12 percent higher on the final exam than non-users. Differences were even larger among students who entered the course with minimal prior biology knowledge. 鈥淭o simply say 鈥楬ere are some practice tests to see if you鈥檙e on track鈥 and then see someone鈥檚 grade suddenly jump 10 percent was impressive,鈥 she said.
Bernacki also used that first year to observe students鈥 behavior in the WebCampus environments for each course. He collected data about which resources students accessed 鈥 or ignored 鈥 and when. Then he observed how students鈥 behaviors correlated with their grades. This would be critical information for the third year of the study.
Learning to Learn
In the second year, Bernacki studied whether a program called 鈥淭he Science of Learning to Learn鈥 could be delivered in WebCampus to improve performance. It first acknowledges the challenges students face when transitioning to college coursework. It then introduces important learning principles 鈥 like 鈥渞etrieval practice鈥 to improve factual information and 鈥渟elf-explanation strategies鈥 to break down complex concepts 鈥 and helps students select the most appropriate strategy for their course. It trains them on behavioral strategies for managing a college lifestyle.
鈥淚t鈥檚 one thing to study, but it鈥檚 another to be aware of the correct objectives to study and to keep track that the knowledge is being learned and retained,鈥 said one engineering student who provided anonymous feedback. 鈥淪ounds simple, but it鈥檚 something I never thought about.鈥
In the initial study with anatomy and physiology students, those who completed 鈥淟earning to Learn鈥 modules after their first unit exam outperformed a control group on the next two exams. The pattern of results was replicated the next semester when students completed the modules in the first weeks of the course. A third run of the study showed similar results in math courses. College algebra students who completed the training outperformed peers (who spent equivalent time solving algebra problems) on the next two course exams.
Power of Prediction
After gathering data about students鈥 online behavior for the first two years, Bernacki created an algorithm to predict performance. 鈥淲e鈥檙e at a point where after four weeks into the biology, three weeks into the calculus, or five weeks into the engineering course, we can identify which students are going to earn that poor outcome about 80 percent of the time,鈥 he said.
Students typically need to pass with a B or a C in these initial courses to advance in their STEM coursework. Without prediction modeling, students may not know if they are on track to make that grade until after their first exam.
鈥淭hat鈥檚 a problem because when a poor first exam grade arrives 鈥 often as late as mid-semester 鈥 it means the time available to adjust learning methods is short,鈥 Bernacki said. 鈥淲hat鈥檚 worse, the first chance to perform well in the course has now been missed, which raises the stakes of the remaining exams.鈥
Utz added, 鈥淚t can become mathematically impossible to recover a passing grade.鈥
Bernacki and his team created an early alert system so students know they need to change study habits. A week before their first exam, a message from their instructor reminded students about the upcoming test and proposed they use some powerful learning methods 鈥 things that had worked for students in past semesters.
They were directed to an advice page authored by real 51吃瓜黑料 students and faculty and hosted on WebCampus. Students also were encouraged to use the 鈥淟earning to Learn鈥 modules.
In spring 2016, more than 300 anatomy and physiology students identified as likely to struggle received the message; more than one-third beat projections and earned A鈥檚 or B鈥檚 in their course. Follow-up studies showed similar improvements in the biology course. And when applied to calculus, messaged students outperformed others predicted to struggle by nine to 15 points on all five exams.
鈥淪o far, those who get the message and take us up on the offer [of learning support] ultimately outperform those who don鈥檛 get a message,鈥 Bernacki added. 鈥淚 was pleasantly surprised that when students did what we鈥檇 hoped they would do they were as successful as we thought they could be.鈥
The Clicks of Academic Success
Cam Johnson can get roped into a lot of interesting projects, but the operations manager in 51吃瓜黑料鈥檚 office of information technology, didn鈥檛 see this one coming. In 2014, backed by a National Science Foundation (NSF) grant, Matthew Bernacki approached Johnson. The education professor wanted to collect student data from 51吃瓜黑料鈥檚 WebCampus course management system to create an early warning system for student success.
鈥淲e do work with faculty on occasion, maybe just to mentor a group or speak to a class, but we鈥檝e never done something like this with hands-on research,鈥 Johnson said.
His team had been working with Splunk, a system used by Fortune 100 companies to collect data and effectively organize it for searches.
鈥淚t allows us to collect data from disparate sources and make it searchable,鈥 Johnson said 鈥 a powerful tool for organizations where systems tend to be developed over time for specific functions. These systems don鈥檛 always 鈥渢alk鈥 to each other. 鈥淔irst we had to answer the questions, 鈥楧o we have the data he needs?鈥 and 鈥楥an we curate it in a way to support his research?鈥欌 he said.
The project鈥檚 novel use of the data 鈥 it was the first time Splunk was applied to faculty research 鈥 garnered a 2016 Splunk Public Service Innovation Award. Johnson鈥檚 team developed a data modeling solution that recorded clicks within the WebCampus system. Their solution allowed Bernacki to see exactly what students are 鈥 or are not 鈥 clicking on for a course and what kinds of learning resources they accessed. They then created a model that aggregates and mines these data in order to predict each student鈥檚 success.
鈥淥ne of the things we鈥檙e trying to deal with now is how to scale this out, help students graduate, and do things we know are predictive of them doing well at the university,鈥 Johnson said. Bernacki is working to apply what he has learned to other STEM classes on campus and he hopes other institutions will learn from 51吃瓜黑料鈥檚 success.
鈥淸Bernacki] saw data and found a way to interact with it,鈥 Johnson noted. 鈥淲e built a solution I wouldn鈥檛 describe as an enterprise solution, but it was a challenge and it was fascinating.鈥
Most important: It worked.