Edited by: Luciana Leite, Instituto Butantan, Brazil
Reviewed by: Pier Luigi Lopalco, Università degli Studi di Pisa, Italy; Ana Wheelock, Imperial College London, United Kingdom
Specialty section: This article was submitted to Vaccines and Molecular Therapeutics, a section of the journal Frontiers in Immunology
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
The 1998
香京julia种子在线播放
Acceptance and uptake of vaccination is important for reaching public health targets. The information available, either from books, television news, newspaper articles, or online sources, has a major impact on how the public perceives vaccines. In this respect, the most impactful information was the publication by Andrew Wakefield in the medical journal The Lancet in 1998, supporting a link between the mumps, measles, and rubella (MMR) vaccine and autism (
It has often been pointed out that antivaccine information available on the Internet has a high prevalence and could impact negatively vaccination decisions (
Furthermore, the information on the prevalence of antivaccine websites is not consistent. A study in the USA analyzing 89 websites on human papilloma virus (HPV) returned by Google, Yahoo, and Bing reported less than 10% of websites with negative tone about vaccines (
The purpose of this study is to analyze the information available to the public, 20 years on from the publication of the above mentioned Lancet paper, on the link between vaccines and autism. The study does not analyze the impact of online information of vaccination rates or on public health views on vaccines but provides an approach to monitor vaccine-related information on the web. Using a methodology used previously for similar studies, we obtained a sample of the existing information using Google as the search engine (
The analysis was performed in different countries on localized versions of the search engine in different languages (
We also investigated the visibility, in terms of ranking, given by the search engine to webpages with a negative tone on vaccines. This has been overlooked by most studies, and it is known that users typically spend a short time on each website (
The results indicate differences in the composition of the antivaccine websites across the world and the footprint left by Wakefield’s Lancet paper. They also show differences in the ranking of antivaccine websites in the different localized versions of Google.
We searched the two keywords “vaccines” and “autism” in Google between June and September 2017. It was decided to use only those keywords because we wanted to obtain a sample of the websites returned independently of the expression used. For this reason, we decided not to use questions such as “do vaccines cause autism?” because the results would be different depending on how the question was formulated and we needed to be consistent across the different languages. Although “vaccines” could be synonymous to “immunization,” particularly in the scientific literature, we decided to use the search term “vaccines” as this best represents what the lay public would search on the Internet.
Before performing the search, the investigators deleted cookies and browsing history from their browsers to avoid the results of the search being influenced by previous searches done on the same computer (
The first 200 websites returned in each SERP were transferred to a spreadsheet and then the websites visited individually. When searching
The total number of webpages considered for the analysis were as follows: English (
For each website, we recorded the typology of the website using the classification previously described (
Definitions and examples of typology of websites.
Typology | Description | Examples |
---|---|---|
Government (G) | Website of a governmental body | |
Health Portal (HP) | Website that contains information on a variety of health topics | |
News (N) | A website from newspapers, magazines, or TV | |
Non-Profit (NP) | Website from a no-profit organization |
|
Professional (P) | Websites created by a health professional organization (medical school, clinic/hospitals, medical board) | |
Commercial (C) | Selling of producing drugs, supplements, or other | |
Scientific journal | Academic journals |
To assess the JAMA score, we searched the webpage for the presence of the following information: author, date, references, owner of website (
We also annotated webpages according to the following features:
(1) The name of the vaccine mentioned; (2) the overall stance on vaccines (positive, negative, or neutral); (3) the chemicals or adjuvants mentioned; (4) whether the page mentioned complementary and alternative medicine (CAM) and its stance toward it (positive, neutral, or negative); (5) whether religion was mentioned; (6) whether the page contained a testimonial (e.g., a personal story); (7) whether a celebrity was mentioned. For websites associated with the typology “News,” we recorded the most mentioned stories in each SERP.
When indicated, statistical analysis was performed using GraphPad Prism version 7 for Windows (GraphPad Software Inc., La Jolla, CA, USA).
A two-tailed Fishers Exact test was used when comparing frequencies; when comparing multiple groups, the Bonferroni correction for multiplicity was applied.
When comparing JAMA scores across more than two groups, ANOVA was performed followed by Kruskal–Wallis test corrected for multiplicity by controlling the false discovery rate using the method of Benjamini and Hochberg.
A Pearson correlation coefficient test was used to assess the correlation between two variables, following D’Agostino and Pearson normality test (when the number of samples was too small, a Kolmogorov–Smirnov test was used to determine normality, a pre-requisite for the Pearson’s test). For non-normally distributed samples, correlation was assessed using a Spearman Rank test. An alpha value of 0.05 was used for all statistical tests unless otherwise specified.
The statistical test used is described in the text or in the legends to figures and tables.
Word count to detect the number of occurrences of the names of celebrities was performed using natural language processing. Briefly, text corpora were extracted using WebBootCaT, an online tool for bootstrapping text corpora from Internet. Then word counts were obtained using the corpus analysis software Sketch Engine by Lexical Computing, Brno-Královo Pole, Czechia (
The raw data containing the list of websites analyzed and how they were annotated in provided in Data Sheet S1 in Supplementary Material.
Because we only used the word “vaccine” without specifying further, we first analyzed the vaccines mentioned in the webpages returned. As shown in Table
Vaccines discussed by webpages in the different search engine result page (SERPs).
Com | UK | AUS | FR | IT | Man | Port | ARA | Total | |
---|---|---|---|---|---|---|---|---|---|
Mumps, measles, and rubella | 123 | 133 | 112 | 96 | 93 | 116 | 88 | 71 | 832 |
Influenza | 23 | 20 | 3 | 11 | 6 | 21 | 9 | 4 | 100 |
Hep | 16 | 10 | 10 | 13 | 2 | 34 | 13 | 0 | 98 |
Diphtheria–tetanus–pertussis | 10 | 9 | 4 | 4 | 0 | 37 | 8 | 0 | 72 |
Polio | 5 | 10 | 5 | 6 | 18 | 25 | 0 | 0 | 69 |
Hib/Men | 8 | 4 | 3 | 2 | 4 | 29 | 0 | 0 | 50 |
Human papilloma virus | 6 | 6 | 0 | 1 | 2 | 8 | 0 | 0 | 23 |
Chickenpox | 4 | 3 | 2 | 5 | 2 | 9 | 0 | 0 | 25 |
Pertussis | 10 | 5 | 2 | 7 | 0 | 0 | 0 | 0 | 24 |
Rotavirus | 3 | 2 | 0 | 1 | 2 | 10 | 0 | 0 | 18 |
Pneumococcal | 3 | 3 | 0 | 5 | 0 | 10 | 0 | 0 | 21 |
Smallpox | 4 | 2 | 0 | 20 | 4 | 4 | 0 | 0 | 34 |
BCG | 0 | 0 | 0 | 0 | 0 | 10 | 0 | 0 | 10 |
Yellow fever | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 0 | 3 |
Measles | 0 | 0 | 0 | 3 | 0 | 0 | 3 | 0 | 6 |
Table
Composition of the search engine result page (SERP) by typology of webpages.
Typology | UK | AUS | FR | IT | Man | Port | ARA | |
---|---|---|---|---|---|---|---|---|
Comm | 4.0 | 5.3 | 3.1 | 6.3 | 0.0 | 2.2 | 4.5 | 0.0 |
Gov | 1.7 | 6.4 | 6.7 | 1.3 | 3.1 | 5.0 | 3.0 | 3.4 |
HP | 3.4 | 3.7 | 6.2 | 1.9 | 4.2 | 10.1 | 10.6 | 11.6 |
News | 41.7 | 30.3 | 26.3 | 31.6 | 49.7 | 36.9 | 31.1 | 34.9 |
NP | 11.4 | 13.8 | 10.8 | 7.0 | 6.3 | 7.3 | 2.3 | 2.1 |
Other | 26.9 | 26.6 | 29.9 | 41.1 | 32.5 | 31.8 | 29.5 | 39.0 |
Prof | 7.4 | 6.9 | 10.3 | 8.9 | 4.2 | 5.6 | 17.4 | 8.2 |
ScJ | 3.4 | 6.9 | 6.7 | 1.9 | 0.0 | 1.1 | 1.5 | 0.7 |
Total | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 |
Non-profit organizations, health portals and professional websites followed in various proportion. Commercial websites had a presence (except in Italian) between 2 and 6%. SJs online were present in a significant percentage (3–7%) only in the three SERPs in English, which is not surprising if we consider that scientific literature is mostly in English.
On the other hand, the pattern in the top 10 websites is completely different (Table
Composition of the top 10 webpages by typology.
Typology | UK | AUS | FR | IT | Man | Port | ARA | |
---|---|---|---|---|---|---|---|---|
Comm | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Gov | 1 | 2 | 3 | 0 | 0 | 1 | 0 | 1 |
HP | 1 | 0 | 1 | 0 | 1 | 2 | 1 | 1 |
News | 1 | 1 | 0 | 6 | 3 | 2 | 4 | 2 |
NP | 3 | 4 | 1 | 1 | 1 | 0 | 1 | 1 |
Other | 1 | 1 | 1 | 3 | 3 | 4 | 2 | 5 |
Prof | 2 | 0 | 3 | 0 | 2 | 0 | 2 | 0 |
ScJ | 1 | 2 | 1 | 0 | 0 | 1 | 0 | 0 |
We investigated whether websites contained a testimonial (personal story), mentioned a celebrity, or mentioned CAM.
As shown in Figure
Percentage of webpages containing testimonials
Celebrities (Figure
Celebrities most mentioned in the search engine result pages.
Name | Context |
---|---|
Donald Trump | US president, suggest vaccine cause autism on Twitter |
Robert F. Kennedy Jr. | US Environmental attorney, claim links between vaccines and autism, rumored to be appointed by Donald Trump to lead a committee on vaccine safety |
Jenny McCarthy | US actress and Playboy model, blames vaccination for his son’s autism |
Robert De Niro | US actor, founder of Tribeca festival. He has a son with autism and was linked to belief of the link between vaccines and autism and critical of the Center for Disease Control. He reversed his initial decision to include the film “Vaxxed” from the festival |
Jim Carrey | US actor with autistic son (from Jenny McCarthy), led a “green our vaccines” march in Washington, DC and is critical of the Center for Disease Control |
Chuck Norris | US actor, accused government to hide data on links between vaccines and autism |
Dan Burton | US representative, grandfather of a child with autism, believer that thimerosal causes autism. Previously expressed support of laetrile, a complementary therapy for cancer |
Luc Montagnier | French scientist, Nobel prize for the discovery of HIV. Attended vaccine skeptical conferences and highlighted an association between vaccine and autism (however, he warned that this may not mean causation). Previously linked to condescendence toward homeopathy |
Martine Ferguson-André | French politician. Suspects vaccines caused his son’s autism |
Agnès Buzyn | French health minister, introduced 11 vaccines compulsory |
Beatrice Lorenzin | Italian health minister, passed a law making 10 vaccines compulsory |
Few websites mentioned CAM, and their frequency was higher in Mandarin and Portuguese websites (4–5%), while in other SERPs, they accounted for no more than 2% of the websites (Figure
The most important aspect of the content analysis was to assess the stance of websites toward vaccines, whether pro-vaccine, vaccine-negative, or neutral. A pro-vaccine stance would be that of websites promoting vaccination or denying the causal link with autism. A vaccine-negative stance would be that of supporting a link with autism or discouraging vaccinations, like the so-called “anti-vaxxers.” An example of neutral stance would be that of a news website reporting the existence of this controversy or a scientific paper reporting findings from an epidemiological study.
Figure
Webpages with different stance on vaccines in the entire search engine result page
Figure
Visualization of the ranking of webpages with a negative stance on vaccines in the first 100 websites in each search engine result page (SERP). Webpages are listed in the same order they are ranked in the SERP. Yellow, vaccine-negative websites; blue vaccine-positive or -neutral. The black bar on the right indicate the top 10 webpages.
The observed frequency of vaccine-negative webpages across the different typologies of websites is reported in Table
Frequency of vaccine-negative webpages in each typology.
UK | AUS | FR | IT | Man | Port | Ara | ||
---|---|---|---|---|---|---|---|---|
Comm | 71.4 | 60.0 | 16.7 | 77.8 | 0.0 | 25.0 | 16.7 | 0.0 |
Gov | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
HP | 0.0 | 0.0 | 16.7 | 0.0 | 12.5 | 27.8 | 7.1 | 0.0 |
News | 6.8 | 7.0 | 2.0 | 0.0 | 2.1 | 12.1 | 7.3 | 3.9 |
NP | 10.0 | 11.5 | 4.8 | 30.0 | 0.0 | 15.4 | 33.3 | 0.0 |
Other | 34.0 | 40.0 | 39.7 | 30.8 | 30.6 | 29.8 | 56.4 | 15.8 |
Prof | 7.7 | 7.7 | 5.0 | 7.1 | 0.0 | 10.0 | 17.4 | 0.0 |
ScJ | 0.0 | 23.1 | 7.7 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
16.6 | 19.7 | 15.5 | 19.6 | 11.5 | 19.0 | 24.2 | 7.5 |
We also analyzed whether the mention of testimonials, celebrities, CAM, or religion was associated with a particular stance on vaccines. Figure
Vaccine stance in webpages from all search engine result page (SERPs) mentioning testimonials, celebrities, religion, or complementary and alternative medicine. Blue, vaccine-positive, gray, neutral, red, negative. * Denotes a higher frequency of vaccine-negative webpages compared to the total SERP (
There is a diffuse concern that the chemicals, including adjuvants and preservatives, added to vaccines to act as adjuvants may be a cause of autism. We, therefore, took note of when a webpage mentioned the presence of it in the text. The chemical name occurring with the highest frequency was thimerosal (441 webpages, 60% of total), followed by the partially synonym mercury (184 webpages, 25% of total), aluminum (101, 14%), and formaldehyde (15, 2%). These adjuvants and preservatives were mentioned in a large proportion of the websites: 56% in
A sub-analysis of the adjuvant mentioned by websites and the stance of the website on vaccines showed that vaccine-negative websites mentioned aluminum with a frequency that was nine-times higher than pro-vaccine, and four-times higher than neutral, websites (Table
Main chemicals mentioned in webpages with different stance.
Positive | Neutral | Negative | |
---|---|---|---|
Thimerosal | 274 (37%) | 86 (22%) |
81 (36%) |
Mercury | 92 (12%) | 32 (8%) | 60 (27%) |
Aluminum | 21 (3%) |
23 (6%) | 57 (25%) |
Formaldehyde | 7 (1%) | 2 (1%) | 6 (3%) |
Total | 744 (100%) | 384 (100%) | 225 (100%) |
Because of the high frequency of news websites, accounting for about one-third of all SERPs, we have summarized in Table
Main topics in news webpages.
Search engine result page | Topic | Examples |
---|---|---|
Tribeca film festival and the anti-vaccine film “Vaxxed” | ||
Donald Trump and political debate on vaccinations | ||
Theory that the Center for Disease Prevention and Control (CDC) have withheld evidence that that African-American boys are at an increased risk of developing autism | ||
Portuguese | Report the story of the origin of the myth of the link autism-mumps, measles, and rubella (MMR) Wakefield paper | |
French | Report on a new law to make 11 vaccines compulsory in France, and of the opposition by Martine Ferguson-André, member of Europe Ecologie-les Verts | |
Italian | Reports of courts cases and final sentences of the Supreme Court in June 2016 and July 2017, which denied the causal link between vaccines and autism. Most news take the stance that connection between vaccines and autism is a hoax (“bufala”) except one vaccine-negative article in “Corriere Quotidiano” | |
Report on the law, approved by the Italian Parliament in July 2017, making ten vaccinations compulsory for all children aged 10–16 | ||
Mandarin | China Shandong Illegal Vaccine Scandal on vaccines purchased from illegal sources and not stored properly | |
Donald Trump’s stance on vaccines | ||
Andrew Wakefield. Talks about the revocation of his medical license and his fraudulent research paper published in The Lancet linking MMR vaccines to autism, which has since been withdrawn |
As mentioned above, vaccine-negative news webpages were less frequent than expected in the whole SERPs. Vaccine-negative news articles were highest in Mandarin, Portuguese, UK, and
The median JAMA score for all SERPs is shown in Figure
JAMA score of webpages in the different search engine result pages (SERPs).
We also analyzed, for each SERP, the JAMA score of vaccine-positive, -neutral, or -negative and could not find any difference in the JAMA score of websites with different stance on vaccines (data not shown). Furthermore, for any SERP, we could not find any significant difference in the JAMA score of the top 10 websites compared to the rest of the SERP.
The varied composition of the SERP returned by Google, with only 30% being non-affiliated websites or blogs, and the rest representing a wide range of news outlets, professional or government organizations, and scientific journals, represents a good sample of the information on the topic of vaccines and autism that the public is exposed to.
Because we analyzed the first 200 websites returned by Google, the list is not just a sample of all that is available in what has been called the infosphere (
Despite retraction of his paper in 2010, Dr. Wakefield is still highly mentioned (a word count found his name recurring 462 times in the
It is important to be aware that the autism-MMR scare was not borne out of an obscure sect but from scientific papers published in respectable and authoritative journals, leading to a widespread concern even among health professionals.
This seems to be true today when articles published in academic journals of varied respectability can have a significant impact as they may be perceived as providing a scientific basis for antivaccine, or just vaccine-skeptical, positions. A study has shown that, in the US, a drop in the MMR vaccination rate was observed soon after the publication of original scientific reports, even before this was the subject of media coverage (
It may be surprising that in the UK and Australian websites, but not in
In particular, the 2002 paper published by Oxford University Press was ranked second in the UK SERP. Repeating the “autism vaccines” search on
In the Australian search, two websites were collections of scientific papers supporting a causal link between vaccines and autism, a third the Wakefield letter mentioned above, and a fourth a paper by the organization “Informed Consent Action Network” that, even if not published in a journal, and it might be questionable whether it could be legitimately defined a scientific paper as it is unclear whether it was peer reviewed, has all the features of a scientific review. Classifying these papers as vaccine-negative was a shared but subjective decision of the authors who reviewed those websites, and we provide the references in Data Sheet S1 in Supplementary Material in case the reader wishes to reassess our coding from a different perspective.
As noted in a Nature editorial by Leask (
Despite the science behind it being discredited, there are several reasons as to why the association between the MMR vaccine and autism is still present amongst the lay public. Flaherty pointed out that this is partly due to autism being a complex condition without a single, established causal mechanism (
The strong association between vaccine-negative stance and CAM, as well as commercial websites often selling “natural products,” confirms that cultural factors may reinforce an antivaccine stance by the association of vaccines with capitalism, big pharma, and profit.
Another finding of the present study is that government organizations accounted for only 1.3–6.7% of websites (Table
The fact that Trump is the most frequently mentioned celebrity reminds us of the difference between countries, where in some countries antivaccine sentiment is prevalent among alternative, left-wing groups, and right-wing, individualist, groups in others. We could not find a significant association between mention of religious issues and sentiment about vaccines. In fact, religious beliefs may be important in the confidence in vaccines (
The fact that news outlets represent 30–50% of the websites indicates that the link between vaccine and autism is a topical and newsworthy topic. From this point of view, it is reassuring that news websites returned by Google have a low frequency of vaccine-negative articles. This is not to say that there are no antivaccine news articles (many vaccine-negative articles have been published by top tabloid newspapers in the UK) but rather that these are not given visibility by the algorithm used by Google.
However, the information quality criteria used by Google do not always penalize vaccine-negative websites. This study shows that, while in
Interestingly, this is similar to what we observed in a previous research where we analyzed the information returned by Google on influenza vaccine or influenza prevention in English and Italian. While in
Of course, here we only use Google as a mesh to collect a sample of the web and the websites returned in the SERP might just reflect “what is out there.” However, it is important to note that the overall frequency of vaccine-negative webpages was not so different in the different SERPs, and we have no explanation for this observation. One wonders whether the vaccine-negative study published in a SJ was ranked high in the UK SERP because the publisher is Oxford University Press, or whether the one in the top 10 in the Australian SERP was ranked high because the .org domain was taken as a proxy of authority and quality. It is also possible that the higher ranking of vaccine-negative webpages in some SERPs is due to the fact that they receive a high number of clicks in that country or language.
Another interesting finding of this study is the difference in the JAMA score of different SERPs. Websites in Arabic showed the lowest JAMA score than all other languages. Websites from
One obvious question is how much the antivaccine information impacts on the uptake of vaccines. Data from the Organization for Economic Co-operation and Development (OECD) show that, in 2015, Italy had the lowest vaccination rate for measles (85%), People’s Republic of China the highest (99%), Australia and France 91%, the USA 92%, the UK 95%, Belgium and Brazil 96%, Portugal and Saudi Arabia, 98% (
We assessed whether there was a correlation between the percentage of vaccine-negative webpages from Figure
It should also be noted that the search in Mandarin was performed using the localized version of Google in Singapore; because the Google search engine is not available in the People’s Republic of China, our results cannot be extrapolated to the information available in that country. We should also bear in mind that most of the languages investigated are not specific to a single country. Hence, making correlation between webpages in one language and vaccination rate or sentiment in one country, is not immediate.
This lack of correlation might support the view that the impact of online information on vaccination acceptance may be exaggerated. For instance, a study among French mothers reported that the main source of information on vaccination is the family physician or pediatrician (84–90%) and the Internet accounts for only 10–12% (
A major limitation of this study is that we only looked at webpages and did not investigate social networks. Studies have previously explored this area of the Internet and have analyzed their features in English and French (
In summary, the main findings of this study are the marked differences in the visibility of websites with a negative stance on vaccines given by the ranking by Google across not only different languages but also in different localized searches in English. Public health authorities, particularly those acting internationally, will need to take these differences into account when designing websites aiming at promoting vaccinations. They will also need to consider the relevance that issues like the adjuvants included in vaccine preparation have in the information available and clarify these issues to correct misinformation. Counteracting disinformation about vaccines by health authorities is part of the solution, but the loss of confidence in vaccines goes far beyond misinformation. Communities, social environment, educational level, are few examples of factors affecting the vaccine confidence. Education, as well as transparency, would be an important aspect to keep in mind when trying to increase vaccine confidence.
NA, PG, and MN designed the study. NA, MA-J, IB, GP, KC, MM, MN, and PG performed research. All authors analyzed data, all authors wrote the paper.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Supplementary Material for this article can be found online at