Coarse Coding and Coarse Behavior

I confess that I am fascinated with the ongoing saga of Toronto Mayor Rob Ford’s scandalous behavior, his bid for reelection, and the coverage of it all by the Toronto, Canadian, and international media.  I am particularly intrigued when the same event is described completely differently by different news outlets (with competing political aims in their editorial policies).  At the Leafs home game on April 5, was the key story about a happy, lucid, sober Mr. Ford mobbed by a frenzy of adoring supporters?  Or was the news instead about a belligerent, irate, possibly drinking Mayor being warned about his behavior by security?

Regarding this most recent incident, it is of particular interest that social media provided the most detailed account of the Mayor’s evening out.  Posts on Twitter, several including links to video and photographs, tracked his movements from the start of the hockey game to his confrontation with security, alerted followers to his late night visit to City Hall (What was that mysterious burning rubber smell detected by security there? Why is that not mentioned in the traditional media?), and established his early morning presence at the Muzik nightclub.  In this case, it seemed that Twitter was breaking the news, and that the traditional media were playing catchup.  As reports appeared in the media the next day some posts on Twitter accused traditional outlets of not telling the complete story!

All of this made me think of Twitter as representing reality in what connectionist cognitive scientists call a coarse code.  Many artificial neural networks generate highly accurate responses by pooling the signals of individual elements, where each individual element has noisy, sketchy, or inaccurate information about what is going on.  The ‘coarseness’ of this type of representation is reflected in the fact that every processor inside the network is an inaccurate detector.  The surprising power of this representation comes from the fact that if you combine all of these poor measures together, a highly accurate measure is generated.

For coarse coding to work, the individual (inaccurate) measures generally require two different properties.  First, different measuring elements must have overlapping sensitivity: many of them will be measuring similar things.  Second, different measuring elements must also have to have different perspectives on what is being detected.  In short, their sensitivities overlap, but are not identical.  When these two properties are true high accuracy can be produced by combining measures.  This is because if each detector has a different perspective, it will be providing different ‘noise’ than is provided another.  Combining the different noise from different detectors will tend to cancel it all out.  What remains is the amplified ‘signal’ – the ‘truth’ – that is also being sensed to a limited extent by the various processors in the network (due to their overlapping sensitivities).

Each individual tweet on Twitter can be viewed as some information being provided by an inaccurate detector.  If the sources of a large number of these tweets have slightly different perspectives, or provide different kinds of information (statements vs pictures vs videos), then their combined effect provides information that has a strong sense of accuracy.

Not surprisingly, researchers interested in Big Data are actively exploring this characteristic of social media.  For instance, some researchers are using the content of tweets to predict the results of elections, although the accuracy of this approach is subject to a healthy debate.  Importantly, the accuracy of such predictions requires that the two key properties of coarse coding (presenting information that is similar, but different) be true.  When these properties are not true – for instance, when many people retweet the same information, so that different perspectives are not provided – social media can misinform, as shown by Twitter being a continual source of celebrity death hoaxes.

To me, the parallel between tweets and successful coarse coding in artificial neural networks clearly indicates that Twitter can be a source of a great deal of accurate information, and makes me reflect on how neural network paradigms might be tweaked to explore tweet contents.

The parallel also makes me think that if I was a politician seeking reelection – particularly one who is such a notorious celebrity that my frequent encounters with the public immediately appear on social media – I would strive to be on my best behavior.  The image of me emerging from all of those Tweets might be more accurate and telling than the one that the traditional news media feels safe to publish!

 

A Deep Breath, Democracy, and Twitter Blocks

Thomas Lukaszuk, Deputy Premier  of Alberta and Minister of Enterprise and Advanced Education, has a very active Twitter account (@LukaszukMLA) – 8923 tweets and counting as of today.  Apparently this account has been in existence for less than two years.  It is commendable that he is taking advantage of social media, though I must admit that I am often bemused by the disconnect between the flow of information that he provides (via the #abpse hashtag) and that provided by others participating in the same dialogue.

Recently a number of Twitter users have reported that their accounts have been blocked by the Minister’s.  There is obvious anger about the Minister’s blocking, frequently expressed in conjunction with alarm about the state of democracy in Alberta.  The image below gives some sense of the situation.

My opinion? Everyone should take a deep breath, exhale slowly, step back, and consider things a bit more objectively and calmly.

First, I find it a bit of a stretch to consider that direct access to a politician via an unblocked Twitter account is a cornerstone of democracy.  Hopefully political dialogue requires more depth than is available in missives that can be no longer than 140 characters, and on average contain only 11 words!  And presumably democracy has been around a lot longer than Twitter, which was created in March of 2006.

Second, an objective examination of the tone of #abpse tweets over the past few months indicates a growing (and understandable) rancor.  There have been many negative tweets directed towards the Minister that have been personal in nature, and typically include his account name to appear directly in his Twitter feed.  I, for one, do not feel that this is particularly helpful or constructive, even though it conveys the emotions of the many affected by #abpse budget cuts.  If I was on the receiving end of this, then I would deal with it with whatever tools are available.  And, of course, account blocking is a perfectly reasonable tool to use in this case.  Sometime I’m surprised that the Minister’s account hasn’t disappeared completely!

Third, and related to the point above, if an open line of Twitter communication to the Minister is important, then perhaps we should consider the tone of our tweets before we share them.  I have no idea how many accounts he has blocked, I do not know who he has blocked, and I have no knowledge about the tweet content that results in being blocked.  If innocuous tweets lead to an account being blocked, then that would be unfortunate (but see below).  However, I am sure that the Minister’s account has seen a flood of tweets that were more than a little charged.  

Fourth, account blocking doesn’t prevent Twitter posts from being processed.  A blocked account cannot communicate with @LukaszukMLA directly, but can easily search for his tweets.  Similarly, I would be surprised if the Minister himself doesn’t search for #abpse tweets, seeing as he often uses that hashtag.  He can easily keep up with what he might be missing due to his blocking of some accounts.

Of course, keeping up with the public opinion expressed in Twitter is something that may be very important for the Minister to do.

In a recent column in The Guardian, John Naughton argues that Twitter provides a key technology for societies to discuss political issues, and suggests that recent debates by governments on either side of the Atlantic have been affected by electronically expressed public opinion.  Naughton’s column in The Guardian provides a link to an interesting paper from Carnegie Mellon University that shows that Twitter data is not only highly consistent with ‘gold standard’ public opinion surveys, but actually provides a leading indicator of survey results.  That is, Twitter analysis can ‘scoop’ the pollsters.  Apparently Twitter predicted not only the overall result of the latest provincial election in British Columbia, but also made accurate predictions about the seat distribution across all parties.  (It is not perfect, though.  Twitter analyses did not predict the results of the recent provincial election in Nova Scotia.)

In short, there are good reasons to keep #abpse tweets flowing, and they may provide important and accurate information to the Minister and his government regarding public opinion.  It is unfortunate that he has blocked some accounts.  But don’t worry about democracy: we still have it, and can exercise it at the ballot box during the next provincial election.

On An Unnamed Institute to Commercialize Research

One of the themes concerning Alberta’s reduced postsecondary funding is the need to reorganize provincial universities and colleges so that they can contribute more to the provincial economy.  The sense is that currently our institutions are too concerned with pure research and are not conducting enough applied research.  The government is using reduced funding to, in part, encourage postsecondary schools to change their priorities.

The most recent example of the government’s intent comes from Thomas Lukaszuk, minister of Enterprise and Advanced Education.  The Edmonton Journal reported his announcement that a new institute that will help colleges and universities commercialize their research in partnership with private companies and other agencies is underway.   According to the Journal, Lukaszuk said that “the institute, as yet unnamed, will be open to researchers and students from any campus in Alberta, and serve as a major vehicle for diversifying the economy. It should eventually generate a stream of royalties for campuses and businesses.”

Proper evaluation of this announcement requires more details.  In addition to having no name, there is no indication about funding for this institute, whether it will employ its own researchers, or whether it will offer degrees.  The whole idea could be good, but it could also be very, very bad.  Though lacking crucial information, I feel obliged to make a couple of pre-emptive points.  A previous post of mine, “Enterprise and Advanced Education”, considered some topics related to research commercialization that could stand some repetition.  I will just give some highlights here; for more details, please see that earlier blog entry.

First, there is a core incompatibility between the mission of publicly funded research and the mission of private, commercialized research.  The former is governed by the rule “publish or perish”; its mandate is to make its results publicly available as soon as possible.  The latter follows the rule “silence is golden”, because publishing research results poses problems for patenting discoveries.  The creators of this institute must chart a very difficult course between these opposing mandates, addressing such thorny ideas as ownership of ideas, publication of discoveries, and the like if they wish to recruit current university scientists in Alberta.

Second, there already exist channels for commercializing university research.  For example, the University of Alberta was involved in creating TEC Alberta in 2006; last year it advised hundreds of clients who generated millions of dollars in revenue.  The creators of this institute must defend the need for yet another channel for research commercialization, particularly if the government funds it handsomely.  One of the goals of the March budget’s large cut to postsecondary funding was to inspire the discovery of administrative efficiencies and the removal of redundancies.  Presumably, strong arguments that the new institute is not a move in a redundant direction are on the horizon.

 
Third, Lukaszuks’ vision of the new institute is ambitious. According to the Edmonton Journal, the minister hopes that it will evolve into something similar to the world-renowned Massachusetts Institute of Technology.”  Given this vision, one would expect that the new institute – like MIT – will grant degrees.  This raises the redundancy issue again.  If our current postsecondary model is redundant enough to warrant huge cuts in operating funds, then why is a new degree granting institute required? If this is not so, then a different problem emerges. There are prototypical examples of non-degree granting organizations that use researchers to develop new products, like Apple and Google. We do not call them institutes – we call them companies.

Enterprise And Advanced Education

The Alberta government’s ministry responsible for postsecondary education is ‘Enterprise and Advanced Education’.  Over the past few years, there has been a tremendous push to align the two concepts of ‘enterprise’ and ‘advanced education’, both provincially and federally, and both from outside postsecondary institutions and from within.  In principle, such alignment is possibly a very good idea.  In practice, however, it raises some serious concerns.  For a university researcher like me, it produces a conflict between one maxim for applied research (“silence is golden”) and another for public research (“publish or perish”).  Let me elaborate. 

The provincial government is signalling postsecondary institutions to align their research and teaching outcomes with the provincial economy.  For instance, in its 2012 Alberta Research & Innovation Plan, the ministry of Enterprise and Advanced Education notes “The alignment of Alberta Innovates corporations and Campus Alberta brings together learning, research and application, management and technical skills training, and new knowledge and commercialization, into a system that will create a culture of inquiry and entrepreneurialism. A culture that fosters the acquisition of commercial skills, financial abilities and business acumen will enable Alberta to move from a solid economy to a resilient, diversified knowledge-driven economy for future generations” (p. 6).  It is possible that other messages lay not far below quotes like this one.  There is concern that universities are under pressure to favor applied research over pure research, or to favor programs that emphasize particular career paths over others.  One case in point is a recent column in the National Post by Matt Gurney that argues that governments should only provide financial aid for students taking useful programs.

The push to align enterprise with advanced education has sharpened with recent budget cuts to postsecondary institutions and accompanying “Letters of Expectation” from the ministry.  For example, the draft letter of expectation for the University of Alberta asks that it allocate its resources “in ways that best achieve the following desired outcomes:

• Albertans are engaged in lifelong learning;
• Alberta’s workforce is skilled and productive;
• Alberta demonstrates excellence in research, innovation, and commercialization; and,
• Alberta’s economy is competitive and sustainable.”

A recent Globe & Mail column by University of Alberta Emerson Csorba argues eloquently that the last three of these outcomes appear to be strongly related to short-term economic goals.

Nonetheless, at a very general level all four of these outcomes do not appear to conflict with the university’s role of discovering new information, and disseminating its discoveries (and the methodologies used to make them) to the public and to its students.  In her “State of the University” address on February 28, 2013, Dr. Indira Samarasekera, President of the University of Alberta, pointed out that “our goal is, in fact, to do more than prepare students for particular jobs. Our goal is to give our students the knowledge and skills they need to think for themselves, to be creative and entrepreneurial, to seek solutions that others have missed … to undertake today’s work in a way that addresses tomorrow’s challenges. They must develop the very knowledge and skills that can help them diversify and be resilient—to bring all that they have to bear on whatever task they take up.”  I think that when I try to integrate my research with my teaching I am playing a part in achieving this goal.

Of course, this is not the only approach undertaken by the University of Alberta that harmonizes with the government’s desired outcomes.  In 2006, the University of Alberta and the Edmonton Economic Development Corporation launched a joint, not-for-profit, venture called TEC Edmonton.  Its general goal is to transition “science solutions into business opportunities”; TEC stands for “Technology, Entrepreneur & Commercialization”.  It is the exclusive technology transfer agent for the University of Alberta, and provides aid for U. of A. researchers and students to explore the possibility of commercializing their research.  According to TEC Edmonton’s annual report, in 2011-12 they advised 529 entrepreneurs and researchers, 70 of their clients combined to generate $73.4 million in revenue, and 28% of their clients were University of Alberta spinoffs.  Clearly, a significant amount of research commercialization goes on at the University of Alberta.

However, a university researcher faces a serious conflict because the goals of discovering and disseminating knowledge to our colleagues, our students, and the public does not align very well with the goal of transitioning science into business.

The cliché “publish or perish” governs knowledge discovery and dissemination.  An academic career faces constant evaluation, and a large part of this evaluation concerns research productivity.  To receive annual merit pay, to achieve tenure or promotion, to obtain grant funds to support future research, university researchers must publish their results.  A competitive CV requires both quantity and quality.  That is, to succeed in this business you have to publish many papers, and they have to appear in respected outlets.  I constantly remind graduate students that they are only as good as their next publication.  I do this because ‘publish or perish’ is the critical lesson to pass on to apprentice academics.

Long ago, though, I found that “publish or perish” is very bad when it comes to commercializing research.  A lot of my own research involves artificial neural networks, and these networks offer many possibilities for solving various applied classification problems.  Because of this, one of my more entrepreneurially minded colleagues set up a meeting between the two of us and a representative with the Industry Liaison Office, which was an ancestor of TEC Edmonton.  To my surprise, the ILO representative was more than a little concerned about me publishing my research in the public domain.  Indeed, I left with the strong impression that the ILO view of my job (or perhaps more correctly, the view of how my job should be conducted if I aimed to commercialize my discoveries) was completely opposite to my own view (expressed in the previous paragraph).

The reason for this was straightforward: patent law.  In Canada, a researcher has only a year to patent an invention or discovery after its public disclosure.  If more than a year passes, it is not patentable.  This means that if a researcher like me wants to commercialize research, “silence is golden”.  Published research is in the public domain, so if I publish my work – as is expected by my employers, not to mention my government granting agencies – this places severe time constraints on research commercialization.  The ILO’s solution seemed to require me to be silent about my research, and as a result I never pursued its commercialization.

 
In short, my view is that the combination of enterprise and advanced education leads to some perplexing problems for university researchers.  When I accept my salary, or when I accept government grants – public money – to conduct research, I believe that “publish or perish” is an acceptable position to be in, both in terms of lab research and course instruction.  “Silence is golden” strikes me as being contrary to the goals of a public institution.  It will be interesting to see how the combination of enterprise and advanced education proceeds in Alberta in the coming months.

The Extended Mind: Alberta Primetime Edition

Last Wednesday I, along with Ray Bilodeau of the J.R. Shaw School of Business at NAIT, was in a television discussion about the effects of smart technology on human intelligence.   The discussion was spawned when the team that puts the Alberta Primetime program together wondered  whether offloading memory and knowledge to smart phones and the like had detrimental effects on the human abilities of memorizing and thinking.  I think both panelists agreed that smart technology does not make us stupid, but you can be the judge: a link to that portion of the Alberta Primetime telecast is below.

There were a number of delightful ironies related to that panel.  How did I get the opportunity to be part of it?  My name came up in a web search by a producer, they sent me an e-mail, I pointed them to my February 18 blog post on the extended mind (“Where is the mind, and why does its location matter?”), and that was that.  In short, smart technology was responsible for me talking about its effects!

After agreeing to appear on the panel, I decided that I needed to prepare, and performed some additional web browsing and reading.  This immediately led me to several interesting sources, including Nicholas Carr’s famous article in The Atlantic that asked if Google was making us stupid.  The producer helpfully e-mailed links to some additional material, which also included Carr’s piece.

I do not agree with Carr’s argument (it is too anecdotal), but it led me to a wonderful example of how this debate about the effects of technology on intelligence is effectively 2400 years old.  Carr cites Plato’s dialogue between Socrates and Phaedrus as providing an argument about the negative effects of the written word.  I hunted down Plato’s Complete Works (1997, edited by John M. Cooper, Hackett publishing) from my son’s extensive philosophy library.  There I found the claim that writing “will introduce forgetfulness into the soil of those who learned: they will not practice using their memory because they will put their trust in writing, which is external and depends on signs that belong to others, instead of trying to remember from the inside, completely on their own. You have not discovered a potion for remembering, but for reminding; you provide your students with the appearance of wisdom, not with its reality” (pp. 551-552).  Of course, we only know of Socrates’ teachings because Plato wrote.  Carr cites Plato, who wrote about Socrates 2400 years ago, and I extend my knowledge a little further by doing some reading!

Of course, the World Wide Web and smart phones are only the latest tools for extending the mind.  To the probable dismay of Socrates, the written word is a far more pervasive technology for extending human cognition.  Not long after my television appearance, I finished reading a book on the history of computer programming languages (Go To by Steve Lohr, Basic Books, 2001).  In a chapter on the history of the GUI and the Mackintosh, Lohr cites some classic work – completely unknown by me – by J.C.R. Licklider, who was a Harvard-trained psychologist.  While working for Bolt, Beranek and Newman, Licklider published a paper that presciently argued that the goal of computing was to augment intelligence, and not to substitute for it.

I was intrigued by Lohr’s mention of this work, partly because Licklider was a psychologist, and partly because I had been thinking more about the extended mind after appearing on the Alberta Primetime panel.  Naturally, I used a new technology (the World Wide Web) to access an old tool (the written word) when I retrieved Licklider’s 1960 paper “Man-machine symbiosis”.  This paper is full of key ideas, and is particularly interesting to consider in terms of developments in computer technology (and its use) 50 years after it first appeared.  To me, two key themes of embodied cognitive science run throughout Licklider’s piece.  First, his notion of a symbiotic relationship between humans and computers is essentially one of emergence: the extended mind created by this symbiosis is capable of solving different problems than either component is on its own.  Second, Licklider in essence argued that the creation of this symbiosis required advances in affordances: that is, computer technology had to develop in a fashion that led to a seamless man-machine interface.  “It seems likely that the contributions of human operators and equipment will blend together so completely in many operations that it will be difficult to separate them neatly in analysis” (Licklider, 1960, p. 6).

One theme that runs through Licklider’s article is that the human contribution to man-computer symbiosis is to help guide the problem-solving process, and to evaluate its results.  To me, this suggests that by offloading some cognitive tasks to modern technology, we may be freeing our minds for making alternative contributions to extended thought.  Smart technology is not making us stupid; it is making us use our minds in different ways to take advantage of new affordances in the world.

Links:

• Alberta Primetime panel discussion of smart technology, televised March 20, 2013
•  Nicholas Carr. Is Google making us stupid?  (Original appearance in The Atlantic, July 1 2008)
• Licklider, J. C. R. (1960). Man-computer symbiosis. IRE Transactions on Human Factors in Electronics, HFE-1(1), 4-11.