A summary of presentations from the weekly Summit partner webinars
June 15, 2023 – The latest Summit Summary
- Recruitment Help for a New Tool to Address Misinformation – Angela K. Shen (CHOP VEC)
- Text Reminders to Increase Vaccine Uptake Research Update – Professor Katy Milkman (Wharton School of the University of Pennsylvania, Behavior Change for Good Initiative)
- Announcements
Recruitment Help for a New Tool to Address Misinformation – Angela K. Shen (CHOP VEC)
Angela K. Shen, ScD, MPH, Principal Project Investigator, Children’s Hospital of Philadelphia, Vaccine Education Center gave a presentation on the ARTT tool guide.
Analysis and Response Tool for Trust (ARTT) is a web-based software that’s currently in development. The toolkit provides insights into points of analysis and response during complicated conversations requiring trust, and is focused on helping people engage in trust-building ways to discuss vaccine efficacy and other topics. ARTT shows different possibilities for managing these conversations.
Recruitment enrollment begins July 10, 2023, lead by CHOP
- Vaccines will be a test case for the toolkit
- Angela Shen will be the PI for this project
- Contact email: shenak@chop.edu
Survey Design
- Pre- and post- surveys
- Between the two surveys there will be a 4-week period of using the ARTT guide with text reminders
- If participants wish, they will be asked to join in small focus groups at the end of survey two
- Target audience for recruitment:
- People in immunization and public health
- Community members—those embedded in the community in their work
- People who are technology savvy
- Incentives are offered for completing participation in each component
Online Communication
- Can be emotionally exhausting for individuals working against misinformation online
- ARTT will help craft responses and figure out what to say to combat misinformation
- Useful for communicators in online conversations
Features of ARTT
- With the ARTT guide, the user can craft responses with the help of research-backed tips using a drafting space and list of tips
- Nine different strategies will be shown to users depending on the goal of their conversation under categories of:
- Understand
- Participate
- Inform
- All of the strategies are organized as tips, templates, and samples
- Tip: The theory being the response strategy that’s described in laypeople terms
- Template: How a user could apply that strategy
- Sample: A sample response using that strategy
- Users will be able to analyze messages for:
- Source quality: will offer best source articles for content
- Emotional sentiment of the article
- Wikipedia articles: Can show the health of the article
- Journalistic quality
- Misinformation harm
- Twitter bot detection
- Users will be able to find support and community in others doing the same work
- Currently this is a separate platform, but ideally will be something that is integrated into the tool
- Nine different strategies will be shown to users depending on the goal of their conversation under categories of:
Contact Angela Shen for more information at: shenak@chop.edu
Text Reminders to Increase Vaccine Uptake Research Update – Professor Katy Milkman (Wharton School of the University of Pennsylvania, Behavior Change for Good Initiative)
Katy Milkman, PhD, James G. Dinan Professor of Operations, Information and Management at the Wharton School of the University of Pennsylvania, Co-Director of the Behavior Change for Good Initiative presented on the use of text reminders to increase vaccine uptake.
Intention doesn’t always equal action
- The purpose of this work is not to change minds or inform, but to prompt people to take action
- Many people who intend to get a vaccine do not follow through
- Data on flu vaccination shows 22% of those saying they will absolutely get a flu shot do not get one
Three mega studies tested messages to promote vaccination
- Mega study with University of Pennsylvania and Geisinger (2020)
- Tested 19 different text-messaging strategies
- Including 47,306 patients with healthy check-ups
- All assigned a random text message in the three days leading up to their visit
- Layering a flu vaccine message on top of the provider’s reminder
- Control group did not get the reminders
- Flu vaccination rates
- Control group accepted their vaccines at a rate of about 42%
- On average, all of the flu-focused messages increased flu uptake rate by at least 2 percentage points
- About 1/3 of the messages significantly outperformed usual care
- While all text reminders improved uptake by 5 percentage points, best performing increased by 11%. These were():
- 24 hours before the appointment
- “A flu vaccine has been reserved for you.”
- Of the 19 messages tested, the top-performing used the “reserve” language
- Takeaways
- Text reminders increased flu vaccinations by 5% on average
- Top-performing reminder increasing vaccinations by 11%
- Two factors drove reminder success:
- “Reserved for you” language
- Messages congruent with the sort of communications patients expected to receive from their healthcare provider
- Mega study with Walmart Pharmacies (2020)
- Tested 22 different text-messaging strategies
- Including 689,693 Walmart Pharmacy patients who had received a flu vaccine in the previous year
- Vaccination rates
- Control group doesn’t get reminders but gets vaccinated at rate of 29.4%
- Every text message treatment tested significantly improved vaccination rates over and above usual care
- Takeaways
- Text reminders increased flu vaccinations by 7% on average
- Top-performing reminder increasing vaccinations by 10%
- Two factors drove reminder success:
- “Waiting for you” language
- Sending repeated reminders
- In 2022,an intervention that used 2 text messages instead of one was found to be more effective
- Mega study with a Pharmacy Chaim to Promote COVID Boosters (ongoing)
- Tested 8 different text messaging strategies along with the incentive of free Lyft ride to the pharmacy
- Includes 3.66 million patients
- Looked at 30-day booster rates in the control group. Only 5.09% got vaccinated for COVID_19 in the 30 days following
- Takeaways
- Text reminders increased COVID-19 vaccination by 20% on average
- Top-performing reminder increasing vaccinations by 23%
- Free rides to pharmacies for vaccines did not add value over and above simple reminders to get a booster
- Top-preforming reminders:
- Suggested a date/time/location matching person’s last vaccination
- Communicated that infection rates were currently high in the patient’s county
- Were sent on behalf of patient’s local pharmacy team
Questions
Q: Do you feel that the “reserved for you” language, even though targeted towards flu vaccine, might have increased patient receptivity due to temporal trends of COVID-19 vaccine coming out and being limited initially to certain age groups for persons with chronic conditions? For example, do you think results might differ now in the current context?
Katy Milkman: I’m pretty confident we can rule that out and the reason for that is that we ran a study in September of 2020 when there were no COVID-19 vaccines available, along with the flu shot. We also ran another version of the study over the course of the entire fall of 2020 when there were still no COVID-19 vaccines, but we looked at September–December patients who came in and we still see the effect. Then we ran it again this past fall with COVID-19 boosters and we saw the same benefits plus there were two replications with COVID-19 vaccines across all those different samples. We saw the same effects, which suggest this is not all that variable in its impact. There is a study that used this language in May of 2022 in Rhode Island with people who had not yet received a vaccine (when everyone’s eligible) at the point where we hit that cliff [with acceptance of COIVD vaccines] and everybody stopped coming in for COVID-19 vaccines. In that study there was no effect. My interpretation of that is that there was no effect so we were not at the point where text messages with slightly different wording was the persuasion people needed. I do not think this is the thing to do. I think maybe the conversation tools that were discussed in the previous presentation could be more useful. I think of this as closing the intention action gap as opposed to changing minds.
Q: To follow up on that, you said the second sample was in 2022, so the season that we just came out of. Is that right?
Katy: I presented three different groups. We had flu in 2020 at Penn medicine. Then we had flu in 2020 again with Walmart, but those messages went out September, October, November, and December of 2020 instead of just September. The third one from this fall of 2022 with COVID-19 boosters was exactly the last vaccine season we went through so those are the three different studies.
Q: Did you look at flu at all for this last season or was it just COVID-19 boosters?
Katy: We did look at whether or not these messages increased flu vaccination as a secondary outcome even though they were communicating about COVID-19. We saw about a 7% increase so it’s a one percentage point lift. On average they basically all increase at about the same amount. There’s no differentiation. The simple explanation I have for that is that at this pharmacy, when you go to the scheduler to schedule a COVID-19 vaccine, they also asked if they want to get a flu vaccine at the same time. Just by lifting the number of people coming in for a vaccine we simultaneously pulled in some extra flu vaccines.
Q: Is it possible to get links to these studies?
Katy: The first two that I presented are already published. You can find them on my website katiemilkman.com. If you Google Katie Milkman in Google Scholar you will find them. The latest one on COVID boosters is under review and it’s not publicly posted now. You can e-mail me at Kmilkman@Wharton.upenn.edu for more.
Q: In the second study with Walmart did you look at any other incentives? Sometimes Walmart does something like $5 off if you get your flu shot, as well.
Katy: We proposed it and it was not something that Walmart was comfortable doing at that time so none of our messages were accompanied by any incentive. There’s some great research on vaccine incentives from Sweden that was led by Devin Pope and Stephen Meyer showing benefits of small incentives for COVID-19 vaccines in that context. But everything I’ve seen in the U.S. has been a null result. I co-authored a piece that reviewed all of the statewide incentives for COVID-19 vaccines, looing at pre- and post-vaccine roll out. We saw precisely a zero effect. There’s another study by Tom Chang of USC that looked at COVID-19 vaccine incentives and they saw some negative effects on people who are particularly hesitant, suggesting that the incentives might have led people to feel it was a riskier behavior than they thought.
All of those incentives were quite small, and I think it’s unclear if really large incentives are very different. You can think of a mandate as an incentive when we say you can’t come out to restaurants or museums or go to work if you haven’t been vaccinated. That moved behavior. So, it’s really the small amounts of cash that in the U.S. research has suggested are not useful. I don’t think that means we know all incentives would be useless or even what would happen with boosters because that’s a whole new ball game.
Q: Regarding the messages with the pre-signed date and time for the vaccine, I would think it might be more inconvenient if you texted me a date and time I didn’t want and I had to go reschedule. Was there feedback on that?
Katy: There’s some terrific work that’s been done and it was published in JAMA in 2011 by Gretchen Chapman and collaborators showing that if you pre-schedule people for flu vaccines, even in an inconvenient time, saying that you can reschedule but we’ve got an appointment for you already, significantly increases turn out. It’s moving the needle from 34% to 42% getting a flu vaccine at an on-site clinic, so that is very effective despite the inconvenience. There’s a lot more no shows when you pre-schedule appointments but there’s also a lot more show ups and a lot of people rescheduling to arrange an appointment. The time we pulled was specifically chosen to match the day of the week, the time of day, and the location of the pharmacy where they had last received a vaccine. We’re trying to personalize the suggestion, but they still must make an appointment no matter what. It did seem to be effective. We are going to run a follow-up experiment as soon as possible. We are very interested to know whether it’s about the personalized suggestion date that matches when you got it or is it just having a suggestion anytime. Gretchen Chapman’s prior work suggested that any arbitrary time works because now you feel like you have an appointment you need to change. We don’t yet know from this experiment whether it’s about matching the time to your past preferences. We’re just providing a default time that does the heavy lifting. We’ll hopefully know in six to nine months.
Announcements
- You can find the approved presentations and bios for the speakers from the in person Summit meeting on the website at: izsummitpartners.org/2023-naiis.
- During the summer our weekly meetings become monthly meetings. Beginning in September we resume our weekly schedule.
- If you are registered for the Summit not getting the emails from Mailchimp, please add “NAIIS” at info@izsummitpartners.org to your contact list.
- If you have any agenda items that you are interested in sharing with the Summit, please let us know and we can add you to an upcoming call as a speaker or panelist. Contact information: info@izsummitpartners.org.
