Methodology question: I am trying to figure out how to measure the degree to which a risk-related problem has become a public issue. The metric needs to be easily analyzed, accessible, quantitative, and stable over at least five years (from 2015 to 2020) and have high face validity. The measurement should be robust – it does not have to be particularly refined or capable of resolving small differences.
The application for this metric is that I am developing a project that has to do with factors (such as Peter Sandeman’s “outrage”) that determine which risk-related (environmental, health, sustainability) issues become public issues over time. I need a way to measure outcomes and compare them against predictions.
Social media is the most obvious approach. One metric would be Google hits, which is convenient, free, and cumulative, and which almost certainly will be around in five years in roughly the present form without too much bias from algorithm changes introduced over the period. On the other hand, I am concerned about Twitter because I’m not sure it will be as stable a platform over the time period and I’m not sure how much people tweet about issues as opposed to people and events. Newspaper inches in a journal of record (such as the New York Times), which used to be an old standby, might be completely obsolete by 2020.
I would be grateful for practical ideas.
I agree that Google hits would be a good starting point. I think that Twitter is still worth considering as an avenue - there are a lot of issue-based tweets, and the events that raise the issues in the first place are often tweeted. Shark attacks for example, raise the issues of bather/surfer safety and shark culling - all heavily tweeted. I also agree that newspaper inches might be an obsolete idea by 2020. However, depending on how techie you are, you could easily create a 'web-metres' metric. By correcting for font size etc across articles, you could just update the newspaper inches idea for 21st century use.
Interesting metric idea. I don't know how to do this but maybe can get some help. The context of this is that next winter/spring I'll be a Fulbright Visiting Chair at McMaster. My priorities will be to teach my new book on health and sustainability (not out yet) and to initiate this particular study. Hoping to get some tech-enthusiast students to help me figure out a robust protocol than even I can follow through with in 5 years time.
I am surprised! I thought there would be lots more suggestions! Maybe the problem is not so simple.
You could also try using metrics for the relevant groups on the LinkedIn network.
Yes, if there is a relevant group. The basic idea is to identify emerging topics that have not surfaced yet and rate them by an instrument, then to track them over five years and see which broke through to media attention and public debate and whether the instrument predicted it. By definition, the topics will not be visible enough to have a LinkedIn Group at the beginning, but if there is one at the end of the period, that would certainly indicate that the issue had broken through. Good idea!
I think your metric ideas all sound good. I think Twitter will still be relevant in 5 years and not too different - its a pretty simple concept. Also how about Facebook? Maybe you could do an index using all of the above?
I am returning to this thread because the project I outlined in the initial question will be implemented shortly, when I take up residence in Ottawa (not McMaster - I got reassigned). Does anyone have other ideas for metrics in social media to determine public interest/concern in a particular issue related to risk?
So far, we have:
1. Google hits on topic
2. Twitter tweets
3. LinkedIn - coverage by Groups
4. Web-area density metric
Other ideas welcome. I will be putting out a call shortly for nominations for candidate issues. These will be plausible issues concerning health risk (environmental, consumer product, sustainability-related) that appears to be at the beginning of its cycle, limited to concerns at the community level or talked about by a small group, not yet experiencing media coverage. The idea is to identify factors that predict which risk issues become public issues and which stay in the background.
We all know which issues went from relative obscurity to media attention over the years. Our inboxes are full of them. A good example of one that did not is microcystins. This is an important, serious water contamination issue, but until Toledo, there were only about 130k hits on Google for "microcystins in water", compared to >2.5M hits for "TCE in water". Could it be predicted that microcystins, as an issue, would fail to break through to general public awareness?
Thanks for any ideas you can share.
We once contemplated a study of the salience of public health issues (in our case stress related) that were loaded up and viewed on u-tube. This can be monitored if you have a good tech person. Another metric to think about Tee.
- Badges
- Science topic