SolrSherlock: towards Watson-like agent capabilities

February 12, 2013

SolrSherlock is the name I have given to a project which aims to take advantage of one of the gifts of IBM’s Smarter Planet initiative, the gift of UIMA to the Apache Foundation. The project itself lives, at this time, at DebateGraph, where a structured conversation and topic map are growing around  design ideas, with plenty of room and encouragement for other design ideas to emerge. The DebateGraph site is aimed at the creation of more than a single Watson-like agent.  Eventually, soon, in fact, there will be some components of the codebase at Github.

The underlying thesis of SolrSherlock rests on the marriage of the Apache Solr project with UIMA, as is well documented at the Apache site and around the web, together with a topic map.  That marriage came from a history of writing programs in Forth, where the lessons learned included “let the compiler do the work for you”. What is the mapping from such lessons to this project? A topic map is a kind of compiled information bank, well structured and navigable. The compiler, in this case, is the natural language machine reading to topic map structure harvesting platform. That’s one among many tasks for UIMA and its many kinds of annotators.

The earliest codes to hit Github will constitute these components:

  • A Solr Update Request interceptor which lives in the chain of processes described in a Solr configuration declaration; its task is to send the new document just indexed by Solr out to a society of agents for further processing
  • An agent coordinator system, this one based on the tuple space concept, is accessible over a network; this system accepts documents from the Solr interceptors
  • An agent framework, which provides an API for plug-in agents, some based on UIMA, some completely different. In each case, the agent has access to the coordinator to fetch or return resources (documents), and access through the internet to the Solr platform as needed. One particular plug in agent is that of the topic map’s merge platform. A topic map maintains the one location per topic promise by merging new resources into those topics in which topic identity is identified to be the same. For complex topics such as those created during conversations, merging requires the services of machine reading to map sentences to structures which support comparison.
  • Solr configurations and schema definitions which permit a single Solr installation, or a SolrCloud to behave as a topic map in this environment

The project grew legs when it was suggested that the book Taming Text  already had Apache-licensed code at Github which showed how to use Solr together with another Apache project, OpenNLP, to build a question answering system with Solr. The book is rich in ideas which parallel many of the concepts documented in the rich literature being published by Watson’s creators. That, by no means, signals that the project is a slam dunk. It is not. But, it is a worthy mountain to climb.

Consistent with the SolrSherlock project, others are invited, as explained here, to participate, either in this or in similar or related projects. There will be many tasks associated with the SolrSherlock project; the society of agents is a plug in framework, which means that many different experiments can bloom, have their day in the sun, and maybe grow roots and stay around.

There is much more to say about the project; I hope to do so after setting up the Github repo for this project.

Thoughts after playing Foresight Engine games

January 31, 2013

Background

I started talking about online games that matter  followed by a report on the BreakthroughToCures game, which was followed by some observations on that game; when the game results and report appeared, I reported on that here.

Earlier, two other games were played; I did no reporting on either. The first was CatalystsForChange, and the second was the U.S. Navy’s MMOWGLI. MMOWGLI was played on a platform that was licensed from IFTF, and created at the Naval Postgraduate School; that platform is to be made available as an open source project. In each of those games, I achieved highest score on the leaderboard. I remain skeptical that being at the top of the leaderboard conveys much information beyond levels of persistence, and perhaps innate skills at gaming the scoring rubric. But, that score can bear some meaning in a ValueMatrix sense when combined with other metrics such as the count of “super interesting” marks achieved by that player. An overall observation from having played 4 such games starting in early 2010 is this: the quality of players and their game moves is improving; that’s an important trend.

The next section of this report relates to experience gained while playing a another game; this one sought ideas about next-generation hospitals. All of these games are based on the Institute for the Future’s Foresight Engine. To anticipate, game play in a Foresight Engine game, a kind of card game, involves choosing a particular card type (question, answer, etc), and making a statement or question. You never actually make a de novo game move; you are always responding to another card.

Comments on Gameplay

These comments related to the hospital game. It’s extremely draining when you play to win, where winning is defined as sitting at the top of the leaderboard. I won;  in fact, I helped #2 to get there from being #4 on the leaderboard just by game play directly with him in a fashion not unlike ping pong. Let me explain.

Ping pong, as you know, is a game in which, to win points, two people swat at a flying ball, trying to get the other to miss. David Bohm  spoke in terms of ping pong as not the way to conduct conversations.

But, I just said ping pong with another player moved him from #4 to #2 on the leaderboard. Sound like the opposite effect? Sure. In this case, we kept “hitting the ball” by responding to each other’s game moves, and each got lots of points for each swat of that ball (read: game play move). Sounds like win-win, and it is. Why? The scoring rubrics.

In the past, I argued that the particular scoring mechanism in a Foresight Engine game encouraged the wrong kind of game play; today, I will argue otherwise. Here is what I mean.

Let me explain my use of the term ValueMatrix.  Ordinarily, we think of things as valued in some way. But, sometimes the way in which we value something might not be appropriate to some context. Value ought to be a matrix; it is a high-dimensional concept.

Let me sketch the Foresight Engine. I view it through two lenses. First, the user experience is that of a card game. Then, the actual conversation is that of a tree, a particular kind of tree, one which structures the conversation along lines suggested by Issue-based Information Systems (IBIS), which came into play in the search for finding resolutions to wicked problems; visit Cognexus, DebateGraph, and Compendium to learn more (those are by no means the only resources–this is a rich and active field).

The scoring rubric says this (my interpretation, which could be weak or wrong):

  1. you get points for making a game move, but only when someone responds to your move with one of their own. In some sense, that’s a ValueMatrix-like approach: if nobody reads and responds to your card (game move), it isn’t worth anything, and that is one dimension in a ValueMatrix.
  2. when people respond to your card in droves, building a “tall tree structure” (called a “Build” in the game), the deeper they go into that tree (longer the tree branch) the more valuable your own move(s) (really, all moves in that tree). At this time, it is not clear whether points are allocated up the tree to all the cards in that branch, but they could be.
  3. if a “game runner” — expert (wizard as in oz) reading cards behind a curtain, likes a card, that card gets marked as “super interesting” and that game move is now worth way more than otherwise, meaning game play off that card (see 2) is even more valuable.

My view is that (2) encourages ping pong. I tend to stick with that interpretation (eyes wide open to other ideas). Build tall trees to win. That’s what we did. As #4 rose to to #2, our later moves appeared to be worth many hundreds of points each.  It did help that at least one card in that tree had a “super interesting” notation (3).

Now, my original notion was that ping pong encourages chatter, mindless game moves just to get points. Indeed, I would argue that some game play is like that, and that was my complaint before now. Early in the Hospital game, I began to realize that mindless chatter is necessary. It is the noise in a Boltzmann machine  that jiggles things around, letting them have opportunities to re-settle into different patterns. Let me unbundle that statement.

The setting is a fitness landscape.  Think of a hill as a representation of a big problem you need to solve. You start climbing that hill. At some point, you find a plateau (it’s foggy) which you think to be the top of the hill, so you stop climbing, sit down, have lunch, and tweet your victory. In some cases, you are at the top; in others, you are not there yet and you need something to “kick you in the butt” and start you climbing again. Kennan Salinero  reminded me of  energy minima landscapes which allow movement across activation energies into new local minima, as in thermal movement.

Here’s the deal as I see it. Some fitness landscapes (hills) are gentle, others are rugged, steep, jagged. A Boltzmann machine is a learning machine that needs randomness to stir it up so it can re-settle to new patterns. That metaphor suggests that if you have a lot of noise on a gentle landscape, not much will happen. But, given a rugged fitness landscape, noise might actually knock you off the plateau and force you to start climbing again.

Map that back to game play. If you read lots of (nearly all) cards as I do, then you get ideas; the more the better. Mindless ping ponging might just offer the noise necessary to get me thinking outside whatever box is driving me at the moment.

But, this is also a picture of a double-edged sword. Here’s what I mean.

The context for this thought is that of a WorldOfWarcraft-like framework in which an imagined Hospital game posed a Quest, to be played by Guilds. If the Quest’s final game tree included mountains of mindless chit-chat, it’s hard to imagine very many “people who matter” reading that game tree. So, Guilds serve to generate game moves. It’s the task of Guild members to apply whatever mindless chatter it takes to evolve solid, concise, novel, profound game moves, and to keep that mindless chatter out of public view; only the final game moves are published to the Quest’s game tree. The thought here hints at the notion that we could conduct games in such a way that noise occurs where noise adds value, leading to crisp, clean, concise, and comprehensive game moves which reach those who need to see them.

Let me now return to the scoring rubric one more time. Consider (3), marking a game move as “super interesting”.  Let me suggest that to be a kind of missed opportunity. Let me explain.

Some masked person marks my card as super interesting. On the surface, now I get more points, bragging rights, maybe a better place in line at the supermarket. I have no clue what it is about my game move that made it interesting. Don’t get me wrong: “more points” is good! But, that’s not the point of game play for me; in my view, the point of game play is to maximize our understanding of some situation (Quest) and maybe make possibly profound discoveries along the way. For that, we need all the information we can get. The label “super interesting”, while nice, is useful since it serves as an attractor basin (ants on honey) to get others to play that hand;  it would be more valuable if it included statements which justify the award.

I can anticipate a valid counter argument to what I just said: learning what a domain expert thinks of one’s move, in particular, their justifications might bias further game play. So, we are sitting in the middle of a mildly wicked problem, in which potential gains to the game players in their understandings (knowledge) might actually bias their game play such that we miss something they might otherwise have “thunk up”. It’s a tough call.

In my view, a key point is this: (3) is the only mechanism by which some game move is rated in terms of possible semantics in a direct way.

If we revisit ants on honey (attractor basins), game moves should be interesting, and in particular, they should gain ValueMatrix scores based precisely on why they are interesting. It is that why which counts. Because someone thought so is not quite as useful as why that person thought so.  Just consider the echo chambers which serve political dialogue to think of why this is important. We can build into our game engines all sorts of software agents that roam about a knowledgebase and form relations between a game move and other topics. Consider this: one of my “super interesting” cards (one of three I got) is this:

hjp
With that card, I referenced the topics “other cards” (this game), “PTSD patients”, “benefits”, “game play”, “avatars”, “storytelling”, and “guild activity”. You can analyze that claim (the card’s statement) in as many different was as there are stars in the universe, but we don’t need to do that. We can link those topics out to a trusted knowledgebase and appeal to those topics for ValueMatrix merits. For instance, consider PTSD. How does it stand in this precise context?

Firstly, the overriding context is next generation hospitals, and PTSD patients sometimes engage hospital ecosystems in one way or another. There, you get some points. Nextly, PTSD, as a medical issue, is a hot topic. More points.  After you do that analysis for all of those topics, then you start looking at the coherence of the claim itself; all the mentioned topics seem to fit together as a coherent claim. More points.

Maybe that analysis is precisely what the game runner did; we shall never know. But we do know that the analysis is useful, so we strive to build such analytical capabilities into our platform. IBM’s Watson does something like this.

Final Thoughts

Overall, I believe that the road ahead for, let us call them IBIS Games, lies along three parallel but necessarily converging paths:

  1.  User Experience
  2.  Scoring Metrics
  3. Collaborative Games (Quests, Guilds, Avatars).

I would like to think that there are much larger uses of games, particularly in the fields of education and sensemaking (think: politics and health, research). Let me close with these questions: What would a manifesto on sensemaking/learning games look like?; What roles might sensemaking games play when combined with MOOCs?

IBIS meets MediaWiki

May 5, 2011

Some slides are now online at slideshare which are drawn from training materials for the Bloomer project which is a component in the collective intelligence platform being installed in some Millennium Project nodes. The IBIS MediaWiki extension can be added to any MediaWiki installation (though it’s not tested on the latest MediaWiki build); it should be possible for a good PHP developer to adapt its code to other platforms such as Drupal.

The extension presently is configured to maintain an index of conversations. Each conversation starts as a Wiki topic, and each question, answer, or argument (see below) is also an individual Wiki topic.

IBIS stands for Issue-based Information Systems, and was a target in my thesis research. IBIS conversations are structured, meaning each question, answer, or argument occupies its own node which is linked through a coherence-relation to another node. Some references are found at the Compendium website.

A lone question or idea can start a conversation; answers or questions respond to questions. Answers respond to other answers to expand on them. Pro or con arguments follow answers. As a conversational tool, online structured conversation platforms are part of the argument web. They are also highly appropriate to #CCK11 connectivist thought.

Examples of structured conversation platforms include Compendium, Cohere, DebategraphTruthMapping, Climate Collaboratorium, and Argument Mapping and an emerging list of others. It should be noted that Jane McGonigal has introduced IBIS as playing cards in her online games, including the MRF Game I mentioned earlier.

<copied here from previous blog>

MRF Game Results Posted

May 5, 2011

The Myelin Repair Foundation game on which I reported previously is now discussed at the Robert Wood Johnson website. The 30 page pdf is found here. The report opens with this:

On October 7–8, and November 9–10, 2010, Institute for the Future (IFTF), in cooperation with the Myelin Repair Foundation and the Robert Wood Johnson  Foundation, hosted a Foresight Engine thought experiment called Breakthroughs to Cures.  Designed as an open, non-partisan environment where models for innovation in medical research can be freely explored and developed, the purpose was to generate  “outlier” ideas and strategies that could lead to more effective and efficient ways to fund  and conduct medical research with the goal of speeding up the development of patient  treatments and cures.

Played as a “card game” where each card resembles a node in an Issue-based information systems (IBIS) conversation as seen in, for example, Compendium which I illustrated from my own MRF game moves here, or at Debategraph, the game provided wide opportunity for journalistic discovery and reporting. The report says this:

In sum, what game play pointed to was a variety of opportunities—particularly in terms of technological infrastructure and in terms of the types of relationships that could be built to bring new ideas to basic science research and to make better use of current resources. Many of these ideas point toward long-term opportunities to facilitate connection and accelerate, and in this sense, provide the outlines for actions to take over time to accelerate medical research.

I believe that an important contribution provided by the MRF game report as produced by IFTF members is its illustration of how a crowd-sourced research project could produce results that journalists could then synthesize into a report worthy of any sensemaking project which leads to decision making.

Where could the MRF games go from here?  I believe the answer to that question lies in the hands of those who created, conducted, and funded that project. What value can those of us who research and practice the art and science of sensemaking through hypermedia discourse gain from the MRF game? The answer to that lies precisely in what we do with not only the report linked above, but also what we do as we study the game boards ourselves seeking to better understand the craft exhibited.

<copied here from previous blog>

A first look at an MRF game move

May 5, 2011

In my previous post

on online games that matter, I described the Myelin Repair Foundation’s research game. That game was mounted in concert with Justine Lam and the Institute for the Future, and was funded by the Pioneer Fund of the Robert Wood Johnson Foundation.  I look forward to continued explorations of online games that matter. Meanwhile,  as part of my thesis research, I began to analyze game moves, starting with one of my own.

In that game, players create game moves by filling in cards, each of a specific type: a question, or several types of answers. The result is a tree structure, not unlike those created with Compendium.  I therefore lifted one of my game moves together with the entire subtree it anchors and copied that into Compendium. The tree’s image is online here (click on it to expand its size), and the report is here.  It’s worth noting that the tree I crafted represents my interpretation of the game moves; it is entirely reasonable to expect there to be other interpretations, as well as errors in my own.

One goal of this analysis is to begin the process of discovering and evolving a set of best practices associated with structured conversation, be it in games or otherwise.  From the nature of each contributed card (node), I look for evidence of issues related the contribution, and seek ways to improve the process.

The simplest observation is that multiple topics made in any given card make it difficult to establish a coherent subtree of responses to that node. Here is a trivial example, not taken from the game:

  • Q: What are the causes of climate change?
    • A: Upper atmosphere carbon dioxide and refrigerator magnets

One should not dwell on the apparent humour in that answer, since there are skilled people who could turn that into a really thoughtful conversational arc around the energetics of making refrigerator magnets and entailed effects on climate. Our interest lies in a suspicion that the subtree that grows from that answer will use a lot of coherence factors to separate out the two topics, then deal with each, separately.

A preference, cast in the light of conversation federation, is to seek simple answers, and use lots of tree (child) nodes to expand on those answers such that each expansion, itself, is an addressable assertion that can, where appropriate, serve as a root for a new subtree.

I think there is room for a large (global) conversation that orbits a well-posed core question that seeks best practices in hypermedia discourse.

<copied here from previous blog>

Online games that matter

May 5, 2011

The Institute for the Future (IFTF) teamed with the Myelin Repair Foundation, funded by the Robert Wood Johnson Foundation to host a game with this title:

How would you advise the President to reinvent the process of medical discovery?

The game was played here, ending today. Following the game, as part of my thesis project, I wrote a quick summary report, found here.

My overall impression from playing the game is that online games that matter, with the Foresight Engine being a shiny new example, will play an increasingly important role in social sensemaking and learning.

<copied here from previous blog>

Hello world!

April 28, 2011

I am moving here from my earlier blog at Open University, UK.  It’s going to take a while to get this blog set up. Busy trying to import the entire blog. Having some import difficulties. Stay tuned.


Follow

Get every new post delivered to your Inbox.