Links or other measures of popularity are not reliable criteria for reliability !! I don't think there is an automatic way to know if some story is reliable. Sometimes you can't trust even a widely reputable newspaper. That said, although there are some exceptions, the most reliable sources of online news are still the good old news organizations. They still are who provide most of the original news material on the web. So if you find a story on a website and want to know if it is reliable, the best way to check it is to see if that information is also present in at least one reputable media: the main newspapers (the broadsheet ones) or news agencies. This is a basic criterion in journalism: cross-check information using reliable sources !!!

You can analyze hyperlinks on Google. Type in link:http://nytimes.com in search and look at the number of results. NYTimes for example has more links than http://cnn.com.  I think you can find studies in bibliometrics about this. 

While it might not give you a precise answer to whether or not a newspaper is reputable, you can use it as a proxy for data that is unavailable.

You can also compare metrics on Twitter, Facebook, and other social media.

There are also services that measure traffic to sites: eg. http://www.alexa.com/siteinfo/cnn.com

According to this site, CNN is #84 in the world, NY times is 108. 

Links or other measures of popularity are not reliable criteria for reliability !! I don't think there is an automatic way to know if some story is reliable. Sometimes you can't trust even a widely reputable newspaper. That said, although there are some exceptions, the most reliable sources of online news are still the good old news organizations. They still are who provide most of the original news material on the web. So if you find a story on a website and want to know if it is reliable, the best way to check it is to see if that information is also present in at least one reputable media: the main newspapers (the broadsheet ones) or news agencies. This is a basic criterion in journalism: cross-check information using reliable sources !!!

From the business point of view: look who owns the news source media company ...

I think that now it is really difficult to know, because everybody in social media could publish everywhere and everything. It is really important the sources, the media and the journalist. Although there are some times, the most reputation media fail (Jayson Blair and NYT). And you can read something interested in this article

http://en.wikipedia.org/wiki/Jayson_Blair

http://now.snopes.com/2015/05/12/five-fake-news-sites/

I agree with what's been said. If it's a national or international story it may be carried by multiple outlets. Their sources can be scanned to see if there are independent sources. Also, Fact Check can serve as a resource.

There are some very specific but important situations that my previous answer has overlooked .It has been said that the Internet has a democratizing potential due to the presence of alternative voices, other than those offered by traditional media. Where there is a free press, usually the facts themselves are not hidden. Alternative voices are views or interpretations different from those prevailing in mainstream media. But in countries where the press is controlled, there may be organizations or individuals attempting to use the Internet to reveal to the world the facts that the regime hides. It may happen that the foreign media did not echo those revelations because they are not of much importance to the public. In these cases, the reputation of the organizations or individuals who report the facts or, if they are not known, knowledge of the political context --which can be obtained at institutions such as Freedom House, Amnesty International, Human Rights Watch and the like-- may indicate the plausibility of the information to decide to undertake an investigation or not.

If it goes against what I would normally expect, I would look for corroboration from a second source.

I can speak from experience in covering crime and courts. A great deal of my best sources were police officers. The best recipe for success I found was to develop strong personalised relationships with them. I treated them as people first and police officers second and found that when major stories were breaking, they would tip me off or give me information they would not divulge to other media outlets. So relationship building is a significant dynamic of the journalist-source relationship.

You may want to see the book "Blur: How to Know What's True in the Age of Information Overload."   Other books by the same authors, Bill Kovach and Tom Rosenstiel, distinguish the "journalism of verification" from less trustworthy "news" that aims only to affirm audience prejudices or to promote a point of view.

Here are some of my reactions to the notes you've left so far:

Carol, thanks so much, just borrowed the book and see its direct relevance!

Olga and José Eduardo, thank you for the popularity metric discussion and links (and your enthusiasm on the topic, Olga! Good to hear from you!) And, yes, José Eduardo, I agree with the point on leaks/ revelations from behind the curtains of tight regimes, be they political or economic monopolies. Third party critics play a huge role in such cases, your point well taken!

José María Herranz, the links to the fake sites are useful. I'm always on the look out for the known fake, fraud stories, especially if they are picked up by the mainstream media or go viral on social media. Do you (or anyone else) knows of any other ones?

John Budd, good to hear from you! Yes, we like Fact Check as well, though it's not completely unbiased, is it? Neither are several other sites that show some favoritism to a particular side in the conflict.

Arvind, some form of corroboration used to be paramount in journalistic practices. Isn't it ironic that the onus is now on the reader rather than the news producer (or often re-sender)?

Lucian, of course. Do you, by the way, have any particular examples that you consider cases of economic interests biasing the whole news source? Or do you mean more of a political affiliation bias?

Mark, thanks for your great insight on relationship building with law enforcement officers! I'm curious about your level of trust for other sources. Is that a continuum of some sort (from official to non-official sources)? And what you'd use to corroborate, fact-check or verify sources' information in different circumstances?

With many thanks to all who read and/or participated in the discussion up till now. I look forward to more insights and reactions.

Great, Victoria! I’m not familiarized with the topic, so I didn’t suggest any answer, but your summary of the various contributions is a nice example of how to build a sense of sharing and belonging in a debate. Lovely!

Off the topic, but close enough, I hope, you might like to have a look at a recent blog post on the relativity of traditional research paradigms before the contemporary challenges of big data and crowdsourcing.

Along that line, I remember that a couple of years ago I came across a paper on the use of automatic statistical data collection and processing by data journalists. Although I couldn't find that paper now (yet), I could swear it mentioned the automatic collection and processing of news data to increase reliability.

http://ieet.org/index.php/IEET/more/swan20150524

Antonio, thanks for your compliments and complements.

As for news text processing, certain features wouldn't qualify news as big data. The 3V principles include Volume, Velocity and Variety + the 4th proposed V - veracity. News don't just come is as fast, steady and furious as purchasing transactions or imagery from cross-road intersections, but perhaps news form part of the big data variety (still laboriously manually produced) yet circulated in volumes via digital re-distribution channels, twitted, re-twitted, re-posted, basically re-and-regurgitated.

[As for your tangential comment on the very-large scale science, my take on it is that big data methods have their own merits, esp. with the powers of observing patterns, habits, etc., but I still wouldn't discount statistical predictions on smaller quality datasets. The article you pointed out seems to be written by "a philosopher, science and technology futurist" and not a data-mining geek, from what I can tell. So I'm not 100% sure her grasp is that well-rounded, esp. with admittedly sensationalized claim of the need to "retire the scientific method" as a whole. I assume she means the positivist paradigm (i.e., experimentation, etc.).]

It would be nice to see that article on auto-processing of news, esp. if it claims increase in reliability of the result. Was it in any of the data-mining conferences?

We have to assume that most news, while not being phoney is biased to an extent that it is not fully accurate.  News tends to be presented, no matter how 'unbiased' the source claims to be from a perspective of the culture in which the news provider is based.  

This can be excellently observed in watching the reports on the multiple wars in the Middle East.  The BBC portrays it from the perspective of the modern democracy (not necessarily as benign as westerners often like to think).  RT presents it from a purely chauvinistic Russian perspective and even Al Jazeera, purporting to be the Middle eastern view adopts a capitalist, corporate view of a minority in a geopolitical zone with a vast range of cultures, political and religious belief systems.

News is biased because people are biased.  British and American newspapers are deliberately biased in order to pander to their readership.  Many readers are less concerned with the accuracy and veracity of news stories than the pandering to their prejudices.  

Victoria, Many thanks for your answer!

As I said in my previous comment, I'm not an expert on the topic. In fact, I took an interest in journalism because one of my PhD students was a journalist. So, in the five years she took to finish her thesis, I made an effort to learn from our common intellectual struggles. And I found the topic very attractive!

I could not locate the paper I mentioned to you (I'll keep an eye on it, and I'll tell you if I do), but I found literature that belonged to the bunch consulted in the same period. In particular, I recalled the description of the Guardian Databolog's coverage of the UK riots of 2011, in "The Data Journalism Handbook" (edited by J. Gray, L. Chambers, and L. Bounegru, O'Reilly, 2012). The Guardian team developed the "Reading the Riots" project together with the London School of Economics and the University of Manchester, and worked on 2.6 million riot tweets to understand and explain what was actually happening. In the process, it also analyzed how the mainstream media fared in debunking, or, on the contrary, confirming, as news, the multitude of rumors that had been put in circulation. I don't know if this qualifies as the kind of example you were looking for! ...

As to the provocative article by the "philosopher, science and technology futurist", I can only agree with you! :) I was not trying to find any truth in it, but rather serendipitous provocation. I have a pet interest in accidental discovery, I wrote about it in the past, and I value much its role in contributing to the progress of knowledge :)

Hello, thanks for posing this question. It has been very educational to read through the answers and identify some quantitative pattern based methods to judge credibility. I would like to suggest a different approach as well - I have taught a course on news and journalism from the perspective of anthropology and media/cultural studies and it brings a more qualitative and critical perspective to the discussion. The question "How do we evaluate credibility?" was one of the guiding themes of my seminar. I had my students read a variety of texts that focused on language use in news production, the political economy of news organizations, and the social impact of news from the perspectives of readership, professionalization, and the affects of news on local situations - ranging from local politics to collective violence. I am sharing some insights and texts from that class here.

I found that getting students to focus on what kinds of words are used to tell stories help them to unpack biases and prejudices that we might otherwise be blind to - so John Hartley's Understand News is a great text for learning how to be critical of news discourse. Credibility and telling the 'whole story' relies a great deal on the creation of 'us' and 'them' binaries - which shift depending on whose perspective you are analyzing the situation from. For this, Amahl Bishara's book on Palestinian stringers who work anonymously and without credit for major US and international news agencies like the NYTimes is a fascinating read. Zeynep Gursel's work on the images in news is also great for this line of thinking. Pierre Bourdieu's essay on the notion of the field and specifically on the journalistic field and whether or not it is, or can ever be, an independent and autonomous entity free from politics or social bias is also a useful way to introduce students to the idea of how vested interests affect what goes into our news and how we read, watch, or listen to it. An oldie but a goodie is Gaye Tuchman's work on Making News - which is a sociological study of the news room and how 'facts' are created - not out of thin air, but how they are established as facts. Philip Schlesinger is another person whose work on the BBC, also from the 1970s if I am recalling correctly, takes on this line of investigation.

For my own reading, I tend to find more credible or convincing those pieces that try to cover multiple perspectives in a story, leaving us with more questions rather than a sense of closure. Stories that align themselves with a subaltern position, or are more likely to tell the less told side of the story are more likely to catch my attention, but even there, I pay attention to the words and imagery that a story conjures up, and for it to be credible it would have to be as fair as possible to multiple perspective, which is not the same as appealing to objectivity (Michael Schudson on the cult of objectivity and how it developed in US journalism is a great book), which often disguises deep-seated injustices and inequalities. This, of course, displays my political commitments, which is to be constantly critical of mainstream media, no matter what the credibility of particular organizations. 

Your question has really got me thinking more about how such a simple question can open up so many pedagogic and intellectual possibilities - so thank you again! I hope some of my thoughts on the question are helpful.

Determining if a news source is reputable or a news story is reliable can be done based on some known criteria. There are two key sides of a news source or a news story. To clarify, a news story depends on a news sources. There are two main common types of news sources. Besides, there are many sources such as local, national, regional, and international agencies.

     The first kind refers to individuals, families, groups, business, communities, organisations, or institutions. They provide information, news, or adverts to the media (clients). And the second categories are media institutions or media houses (both print and electronic media).

     A news source may be influenced by the media ownership, editorial policy, accident or chance, media trust, outreaches, corporate social responsibility, circulation, and public image. So, in order to determine if a news source is reputable or story is reliable check its media house or clients to whom the reported news story has been attributed (as the source) for the:

Possibilities of verification or sending a feedback on the subject, editorial contacts of editors and reporters, circulation figures or area coverage, clients, advertisements, and quality of journalism (the news story); because news story logically answers these questions: what? Who? When? Where? Why? and how?  If it is a local story, confirm with multiple sources.

     A story also indicates a by-line or full name of the reporter, dateline (place and date of filing the story), title or rank of the reporter(s), history of legal cases or public complaints, staffing levels, annual reports, polls ranking, and compare with other media reports / sources.

     I addition, we need to be aware of some special circumstances as well. For example, April 1 fools day media reports, some advertorials, special supplements, opinions, and propaganda. Most success news stories which are attributed to public or private corporations, institutions, organisations, businesses, associations, or individuals are public relations stories.

     If we are to be successful in assessing or evaluating the credibility of most news stories or sources, then we need to work on ourselves first. For example, we should build our own capacity, competitiveness, or competence to do so. Media literacy is therefore vital for us all.

     To conclude, good journalism is based on social responsibility of the press, verification, accuracy, truthfulness, fairness, balance, and the right to a reply by the affected audience.

by the credible authentic evidences and data they come out with

My first rule from distinguishing actual news from advocacy written to look like news is whether there are value judgements (opinions) stated in the article without attributing them to anyone.

Journalists to not incorporate their own opinion into stories. Opinions and value judgments must be attributed.

Then as a next step, one can evaluate whether both/all sides are treated fairly, who owns the publication or has editorial control, etc.

Thank you for your response to this question posited about 4 years ago, some time in 2014, way before the start of the 'fake news' hype. The issue remains unresolved, Dr. Marek, doesn't it.

I think you mention valuable points on lack of attribution and fairness of multiple perspectives. On the other hand, in a journalist is tasked with writing an op-ed (i.e., an opinion editorial by the definition of the sub-genre), does it change your perspective on value judgements? Or would you never take any op-eds into account, even as professionally written as say, the Undark, from MIT?

As a journalist, editor and former broadcaster in Canada for 30 years, I can tell you how stories are researched first before it ever hits the page. Now, this does not dictate that all news agencies follow this process, as tabloid, cable TV outlets, online news and podcasts etc., may decide to do things much differently.

Part of the issue is that we assume that those who are on these stations/channels, are actually trained journalists. Many are not, but heck, they're on television or the internet, they look slick and talk smart, but that doesn't mean they received the same training that requires them to follow the rules set out specifically for the fourth estate in that particular jurisdiction.

Remember too that standard/traditional journalists and broadcast reporters are required to follow specific laws and ethics according to that particular country, where the agency, its editors, and the individual are bound to adhere. Tabloid and cable news don't always follow this, as per the direction of their producers and executive producers and legal counsel. The goal is always to get readership, viewership and make money, but trust and professionalism factor into the mix more than we realize, sometimes unconsciously so.

When handed (or stumbling upon) a story, reporters will have their stock of specialists they turn to for a balanced opinion, or they seek out new, fresh and credible feedback. These are always vetted before the interview, checking other coverage, education, community impact, reputation online, and asking other reporters/editors. They are also required to seek the insight of the parties involved, (perpetrators, companies, victims, etc.) if they choose to do so or are allowed to do so.

Most answers/statements that come from corporations/groups in a story are often spun and controlled by public relations/communications specialists or lawyers, so we have to present that to the public (as to what the company's line is), because that's all we get at the time. Editorial never really embraces public relations as a credible source as a result (we don't like advertisers either), because it's not coming raw from 'the street, the moment'--it's manufactured. But we rely on them to feed us information on who is doing what, but most of the time, we prefer to chase our own quotes, rather than borrow one from a press release.

The harder, more front-facing and credible perspective comes from people on the street, who saw, heard, felt, etc., and give their own experience. But it's not the only piece of the 'truth'. It's only one perspective. The goal is to gain several in interviews, and then piece together the five W's and the H for the reader/viewer.

Any source in an interview may often just represent the personal/professional view of that individual or 'company', but are not always the only opinion. Ask 50 people what the weather is like today, and you'll possibly get 20 different answers, depending on their perspective.

A story typically requires three sources, or more, and all of them need to be accountable, identifiable (very rarely are pseudonyms or anonymous sources used in Canadian journalism, unless it is to protect the identities of vulnerable individuals who are in fear of retribution, such as women, children, or occasionally employees who blow the whistle.). This is not encouraged, as it lack the accountability and attribution on behalf of the person giving the quote and information and the reporter/agency that delivers it.

'According to'...attribution, is the key to credibility, but that all has to be checked and double checked, especially if there is a claim to be made. That's why statistics from a reliable source are key to backing up a source's claim.

My journalism professor, Geoff Lane, who worked for national British and Canadian newspapers, told us on the first day of class that "Your information is only as good as your source." Ergo, if sources are dependable, salient, logical, experienced, and can offer a balanced view (which means you ask balanced questions, pro and con to the situation in the interviews), and determine the strongest and most unique and relative quotes to the story's focus, you can guide the reader through the various perspectives you collected, allowing them to trust the writer, the sources, the agency and ultimately, themselves, to make up their own mind on how they feel about the situation.

To determine if a story is credible, it would be judged on the source/agency, checking out the credibility (if one takes the time), of the individuals who are quoted, and then viewing other stories that cover the same subject.

Most rank and file will not take the time to do this, as a digital information age is highly competitive in terms of attention, focus and discernment.

There's my perspective. I hope it helps widen the understanding. Maybe we can talk about the perception of 'fake news' sometime!

Best,

Donna

@DonnaGray, thank you for sharing your professional editor's/broadcaster's perspective. You've added depth to this somewhat forgotten thread of conversation.The valuable groundwork with credible sources seems to be given less priority and time, and some is often outsourced to post-print fact-checking. I came across Bill Kovach and Tom Rosenstiel's (2009) book "Blur: How to Know What's True in the Age of Information Overload" that expressed similar concerns with obtaining credible evidence from sources at length. Your points of view seem to converge, as Kovach and Rosenstiel seem to lament the lost (or at least obscured) art of the journalism of verification. I also wonder how often and how consistently the procedures you describe have been properly followed under the current ad-revenue model in digital news (or other content dissemination).

Hi Victoria,

I used to live in Sarnia, and miss Grand Bend and Lake Huron and Dr. Disc! I appreciate your response. I would guess that in theory, the people hired to chase stories (or borrow, copy edit, rewrite/pare down, similar to the traditional Broadcast News feed) may not be pushed to follow all the steps I listed. It almost seems like digital news is perceived to less accountable even to editors, but ironically, more impactful, since a copy of the New York Times is only to subscribers, rather than general news being picked up by everyone. There is no 'overlord' for the news, unfortunately. The only guidepost is good taste and professionalism. Sadly, that doesn't seem to stick in tabloid or digital formats.

What do you think?

La pregunta es muy interesante y la respuesta es compleja.

Desde mi punto de vista se requieren varios pasos que ayudan a incrementar la credibilidad de la información.

En primer lugar el medio que publica la noticia: su prestigio, la existencia de un código de ética interno, la presencia de un defensor de la audiencia.

En segundo lugar, el género del que se trata: la noticia excluye la opinión de quien informa, ya sea explícita o implícitamente. Lo segundo suele caracterizarse por el uso de adjetivos para calificar el hecho y los actores del mismo, así como las reiteraciones. Esto demanda un análisis de contenido.

De esto último tengo muy presente el caso de Nicaragua durante la era Somoza y la de los Sandinistas. Los opositores a Somoza eran presentados como "insurgentes", en tanto que quienes combatían al sandinismo eran "contras". Del mismo modo, a fin de magnificar los hechos y las batallas, había quienes al referirse a un combate, sumaban el número de muertos de combates anteriores, a fin de "agregar valor" a la información. Todo ello evidenciaba sesgo en la noticia.

Otro elemento es la comparación con otros medios, cuando es posible, a fin de poder evaluar coincidencias y diferentes.

Todo ello en un contexto de que la objetividad pura no existe. En todo caso, yo prefiero la referencia a la veracidad. El información selecciona, jerarquiza y sintetiza el hecho desde una valoración en la que intervienen su concepción del mundo, sus valores y, finalmente, su competencia técnica para comprender el tema sobre el que informa. Finalmente, interviene el factor humano, en unos casos de forma deliberada y en otros sin intención, pero a fin de cuentas presentando el hecho de una manera limitada que, en ocasiones, nos obliga a ir y contrastar la información en las fuentes directas (un diario de debates del Congreso, por ejemplo), cuando esto es posible y, después, por analogía, aplicar un juicio general acerca del comportamiento del reportero y del medio.

¡Qué informativo, José de Jesús Castellanos! Me gusta vuestro ejemplo. La forma de nombrar "combatientes rebeldes" en lugar de "separatistas" aparece en conflictos fronterizos todo el tiempo y puede fácilmente revelar la percepción de la verdad.