The goal of creating these chapters was to maximize their usefulness to the Tackling a Wicked Problem (TWP) course taught in the first year of the Plymouth State University (PSU) undergraduate curriculum. I have emphasized the specific Habits of Mind that are central to the PSU curriculum and the kind of research that is most likely to arise in a project based learning environment. With the understanding that the projects produced by the students in this class will not necessarily (or even likely) be the standard academic papers, these chapters
My own experience working as a support librarian for TWP has suggested to me that:
- students struggle to come up with topic
- students are drawn to Google and see little reason to try other search tools
- no one has told them what the different kinds of sources are that they are likely to encounter
These chapters can be used in the order in which they appear, and it is true that many chapters refer to content in other chapters. But I have tried to make each chapter work as a stand alone entity. When other chapters are referenced, they are linked, so they can be consulted on an as needed basis. I hope that you will take whatever pieces fit well with and reinforce the work your section is doing.
What follows are notes on the content and reflection and discussion questions, as well as some activity suggestions that didn’t make it into the chapters.
Right now I have only my own experience to draw on but I hope to expand this section to include notes, tips and feedback from TWP instructors. If you use these materials, please let me know how it went, what worked for you, and any suggested changes or additions. I’d love to hear from you at chwixson (at) plymouth (dot) edu or fill out as much of [this form] as you’d like.
Throughout the chapters, I tried to generate Reflection & Discussion Questions that could be used either as in class (whole group or think/pair/share) discussion prompts or as written reflections assigned out of class. If your students generate any written answers to any of the Reflection & Discussion Questions in this chapter, I would be very interested to see them.
Our Mental Shortcuts
If you’d like to reinforce Kahneman’s ideas about System 1 and System 2 thinking the video below (12 minutes) is very good, (thanks to Mike Davidson for this suggestion.)
This is a revised version of an exercise from in the original TWP OER chapters. I modified it to be more focused on students critically assessing their existing knowledge than on identifying gaps. This version is very similar to a strategies for promoting metacognitive reflection used in motivational interviewing, deep canvassing and street epistemology.
Regarding the previous exercise: asking students to look at what they already know as a way to determine what is missing can’t help them discover information they don’t even know that they don’t know. The next exercise on mindmapping is aimed at helping students notice gaps in their knowledge.
I suggest this exercise be used in class rather than assigned outside of class. In my experience coming up with search terms is challenging for students, full stop. (I think this is related to the problem of not knowing what you don’t know.) Coming up specifically with slanted or neutral search terms is even harder. Students may not be able to come up with a variety of search terms on their own, but collectively as a class they may be able to generate a variety of examples of both kinds of search terms.
Another approach if students struggle with this, would be to revisit these questions as you work through the preliminary research suggestions in the chapter Identifying a Topic. Then instead of generating search terms on their own, they can try to recognize them in the sources they are reading.
The amount of data Google, Facebook, Amazon, and Apple have on us is astounding, so it’s relatively easy for them to target information to our preferences either to sell us products or to keep our attention on their site. Shoshana Zuboff coined the term surveillance capitalism to describe this situation, and it works in exactly the same direction as our cognitive biases. Our choices often amount to accepting the collection of our data or not using their services. Obfuscation is almost the only weapon we have against online tracking. I almost guarantee students will already have practices in this vein, maintaining multiple accounts or online personas, using privacy-oriented services like DuckDuckGo or the Firefox browser, using the Tor broswer, etc.
If this is a topic that catches your or your students’ interest, here are further readings and discussion questions to expand on these ideas:
The Age of Surveillance Capitalism (2019) ~Shoshana Zuboff at The Institute of Art and Ideas (video – about 30 minutes)
Brunton, F., & Nissenbaum, H. (2019). The Fantasy of opting out. The MIT Press Reader.
Shaffer, K. (2019). Swimming upstream: how content recommendation engines impact information and manipulate attention. In Data versus Democracy: How Big Data Algorithms Shape Opinions and Alter the Course of History (pp. 31-44). Apress.
- How does the data gathered about you affect what you see online?
- Do you think that personality-based and emotion-based targeted advertisements are effective?
- What agency do you have around the data that is gathered about you? How much agency do we have within the system of surveillance capitalism?
Identifying a Topic
I have usually mentioned “brainstorming” as a path towards coming up with a topic. But then I realized I was telling students to try a practice that isn’t remotely close to how I actually work. I don’t just sit down and try to come up with a topic out of thin air. Why would I tell students to do something that I don’t find works for me? I read things, find what piques my interest, and write about how it applies to my context. (But maybe you do something different? Let me know.)
There is a brainstorming exercise in the previous chapter that is about examining what you know and where it came from. That can be a starting point for asking questions too, but I would recommend against brainstorming as the only strategy towards topic and question identification since it does not enable students to get to topics they didn’t know existed.
I struggle with getting students to actually read the sources we find together in our research consultations. They seem to want to do all the searching first and all the reading later. No matter how I tell them it’s iterative and you need to go back and forth between reading and searching many many times, the messages wasn’t landing. This chapter is my next iteration in how to talk about the research process, but I really don’t now what the secret recipe is yet. Let me know if you think this one lands.
Types of Sources
I am a big fan of Mike Caulfield’s information literacy work (see the next chapter, SIFTing Information.) Sometimes I have found my attempts to use his strategies in the classroom were hard for students. For example, when I’ve tried the exercise about the American Academy of Pediatrics and the American College of Pediatricians (Reflection & Discussion Question 1) without first talking about professional organizations, students rarely got how they were different, and it did not build their confidence.
It’s hard to identify a legitimate professional association if you’ve never heard of the concept of professional associations. This chapter may be long, but I felt it was important to enumerate at least some of the dimensions of the sources they may find, so that when we get to Caulfield’s SIFT method they are set up for success.
After students do this exercise, direct them to this video explainer of the answer that also includes discussion of the best strategy to use to find the answer.
You may also be interested in discussing the original study by the Standford Education Group that used a version of this exercise to examine and compare the fact checking skills of students, history professors, and fact checkers:
This exercise is a preview of the kind of work students will do in the SIFTing Information chapter. If you think this kind of source research will be challenging for students, you may want to consider switching the order of the chapters and covering SIFT first.
Other advice that might smooth the way for this exercise is to remind students right before they start that we aren’t interested in what these organizations’ websites say about themselves, but what they can learn about them from the rest of the internet. Encourage use of Wikipedia for this type of source research. Encourage them to slow down and to practice “click restraint” once they have Googled one of these orgs. What can they learn from looking at just the search results page, without clicking through to anything? What is the overall impression from a variety of results?
- Center for Consumer Freedom: Many of the Google search results (with or without including the search term funding) indicate this is astroturing. A look at the Wikipedia page tells us that this org was started by a pretty well known PR guy and the sidebar lists their focus as “represents the interests of restaurant and food companies” and their method as “lobbying.”
- National Consumers League: Students may note that it has been around since 1899, has no critical results on the first page of Google results, and even has an entry in the Encyclopedia Britannica.
- One Fair Wage: a legitimately grass-roots effort to raise the minimum wage for restaurant workers.
- Save Our Tips: This is one case where adding the word funding to the search helps a bit. If we do that we find sources indicating that this group is funded in part by the National Restaurant Association and a conservative strategy and consulting group. Not what you would expect for a grassroots effort lead by waitstaff.
I find that students really like this and other Google tricks. For other advanced Google techniques check out Google’s page on how to refine web searches.
Another useful one is to use the minus sign (-) in front of a term in the Google search box to exclude sites that use that term. For example patriots -football -nfl
I often use the site limiter and the minus sign together to find what others are saying about an organization, instead of what they say about themselves, as in:
Center for Consumer Freedom -site:consumerfreedom.com
Possible directions this discussion might go in:
- Editors: Usually editors are chosen based on their expertise, and putting an expert in charge provides some protection against blatantly incorrect information getting through. But this gatekeeping process concentrates a lot of power into the hands of one person, and it is possible for the cognitive biases of any individual to color their decisions, even when they’re acting in good faith.
- Credentials: Academic credentials tend to represent a significant commitment of time towards gaining mastery of a subject, and therefore requiring a particular degree may increase the likelihood of accurate information. However, not all groups are equally represented in higher education. Degree completion is uneven across race and income factors (among others), making academia not demographically representative of our society as a whole. Some perspectives are therefore systematically underrepresented in groups with advanced degrees.
- Peer Review: Peer review sometimes only results in collaborative improvements to a work. It can also prevent the publication of very obviously flawed or poorly executed or analyzed research. Very new or radical ideas may be initially rejected because they are such a departure from existing dogma. Peer review is largely a practice of academia, therefore has the same exclusionary problems mentioned in the credentials section. It is possible for individual reviewers to act in a biased or unethical way to prevent the publication of some works.
- Fact Checking: Not a lot of downside here. Let me know if your students come up with anything good.
- Domains: For some top level domains (mostly just .gov and .edu) looking at the domain provides some assurance that the web content there is an official communication of a particular institution. There really isn’t any problem with domains excluding voices, since there are so many unregulated domains that can be used instead of the regulated ones.
Access & Searching
Open access is an area I’m always happy to talk about. If you are interested in having this discussion with your students, (and I recognize it might not be important to everyone or relevant to every TWP section,) I’d be happy to join in and be part of the discussion.
If you’d like to talk more about open access on your own, may I suggest the following resources:
- Open Access Explained is an 8 minute animated video. It’s from 2012, but I find it holds up really well.
- Paywall: The Business of Scholarship, a 1 hour documentary from 2016, could be good for a longer, out of class assigned viewing. Also it’s openly licensed so we could also arrange a campus showing if there’s interest.
I have observed students really struggle with this part, (a significant percent of my reference interactions start with “there’s nothing on my topic.”) I find they don’t readily believe search terms matter or that spending time thinking of alternative terms is worth the effort. I suggest modeling the process before asking them to try, (or ask your librarian to do this if you have one come in to your class.) Also, if you figure out a way of talking about this that seems to resonate with your students, please tell me about it!
Another variation on this exercise I’ve used is to break the class into groups and give each group a database to explore. When I have done this I have offered the prompts like these. I invite each group to make a slide about their database in a shared Google slide deck, then each group gets to tell the class about their database (example 1, example 2) In class this can be a tight squeeze for time and it might work better if the exploration were longer, more guided, and/or assigned outside of class. Let me know if you’d like to work together on something like this.
I acknowledge that it’s pretty provocative to ask students “Why do you think we even talk about these databases if you might not have lifelong access to them? Do you think we are right to teach you about them?” If this discussion doesn’t get past the indignation stage, prompt students to think about it in light of the first Reflection & Discussion Question (about access to COVID research), and what role the databases might have in getting them acquainted with their disciplines.
This chapter is derived from that material in the Check, Please! Starter Course, including all of the exercises. Each exercise has an explainer page that I will link to below. Depending on how important you consider these skills to your particular section, you may want to consider directing your students to the Check, Please! course instead of using this chapter. The whole course has 5 modules each taking about 30 minutes, and Caulfield indicates it is an appropriate amount of homework to give over a week or two. If you or your students are interested in “fake news” and determining the credibility of sources found online, I cannot recommend this course highly enough.
For those who do use this chapter, the explainer pages below are part of the Check, Please! course, and having students work through them on their own could save a lot of in-class time.
Check, Please! is an actively maintained site, and examples are updated periodically. If you find that an explainer page is now missing, please let me know.
This discussion page also ties this exercise into questions of authority and local news sources.
This page has some really good discussion prompts about websites that claim to be satire but fool more people than they entertain.
This discussion references the practice of mis-captioning images and videos and walks through another example of a mis-captioned image.
The sunscreen exercise is not the only one discussed on this page, it starts about half way down the page.
Evaluating News Sources
The content of this chapter is derived from Web-literacy for Fact Checkers and the Check, Please! Starter Course, both my Mike Caulfield.
- Washington Post is regularly cited as doing a really good job marking their opinion pieces as opinion. In this case article 2 is an opinion piece and you can tell that because the first word of the title is Opinion.
- NBC News is much less clear. Their opinion pieces are in a section labeled “Think” with “Opinion, Analysis, Essays” written in smaller font underneath. The font of the regular news story, labeled “US News” is also pretty small. The background colors of the pages are very different – if you spent a lot of time on this one site, you might learn cues like that.
- Arizona Republic tends to label and organize their articles by topic, but recently they added a bar across the top of opinion pieces.
Audience, Presentation & Citation
I suspect that some of you have your own assignments and exercises to facilitate student peer review and peer feedback. I’d love to hear about them or suggestions you have for edits to this prompt.
- Edward Bernays
- Wikipedia. Public Relations
- Pinterest. Retrieved June 10, 2021.
- Bernays, Edward. Crystalizing Public Opinion.
- Encyclopedia of Propaganda
Possible directions for the discussion:
- What the sources suggest about the level of research. Do sources like Wikipedia and Pinterest indicate a deep engagement with the topic? What about the Encyclopedia of Propaganda? Call back to the chapter, Identifying a Topic, encyclopedias are good preliminary sources, but if research stops with an overview source, how valuable is it?
- Ways in which the citations are ambiguous. Is enough information provided that readers can find the original information? Is number 1 about that person or written by that person? Is number 4 a book or an article? It has implications for how we would look for it. For number 5, there is more than one book with the title Encyclopedia of Propaganda, and also it’s unlikely they meant to refer to the whole encyclopedia.
- The difference between discovering a source on a social media platform and citing the content. Is enough information given to find the Pinterest source? Revisit the creator concept from the chapter, Types of Sources. Social media companies distribute but do not create content, so they are not the ones that should be cited. Opportunity to talk about specific sources students have found on social media and how to use the moves from the chapter, SIFTing Information to trace those posts to the original information or a better source.