Push Singh, Bo Morgan, and Radu Raduta
In order to separate the study of architectures for common sense thinking from the need for immense common sense knowledge bases, we have built a virtual physical world in which several simulated people work together to solve problems, for example, to build a table. Each person engages in common sense reasoning within the spatial, physical, and social realms to decide what actions to take. The user can interact with these robots via a natural language speech interface to request that they adopt specific goals or take particular actions.
Pink: I see you are building a tower.
Blue: Yes, but I cannot reach that
Pink: I can reach it. Let me get it for you.
While this domain may seem sparse, its simplicity hides a great depth of issues. In particular, the mental realms we have discussed so far all show up in some form in this domain. Because the world is physic- ally realistic, the people must reason about the effects of gravity on objects and the forces that must be applied to move them. Because the people have synthetic vision systems, they must reason about whether objects that seem to have disappeared behind bigger ones are in fact really still there. Because there are several people, they must reason about the social challenges that arise between them, such as conflicts between their goals and possible opportunities for cooperation. To solve problems in this world requires reasoning simul- taneously about the physical, social, psychological, and several oth- er mental realms.
Aaron Sloman proposed this example of a sequence of increasingly sophisticated problems in this domain:
- Person wants to get box from high shelf. Ladder is in place. Person climbs ladder, picks up box, and climbs down.
- As for 1, except that the person climbs ladder, finds he can't reach the box because it's too far to one side, so he climbs down, moves the ladder sideways, then as 1.
- As for 1, except that the ladder is lying on the floor at the far end of the room. He drags it across the room lifts it against the wall, then as 1.
- As for 1, except that if asked while climbing the ladder why he is climbing it the person answers: something like "To get the box." It should understand why "To get to the top of the ladder" or "To increase my height above the floor" would be inappropriate, albeit correct.
- As for 2 and 3, except that when asked, "Why are you moving the ladder?" the person gives a sensible reply. This can depend in complex ways on the previous contexts, as when there is already a ladder closer to the box, but which looks unsafe or has just been painted. If asked, "would it be safe to climb if the foot of the ladder is right up against the wall?" the person can reply with an answer that shows an understanding of the physics and geometry of the situation.
- The ladder is not long enough to reach the shelf if put against the wall at a safe angle for climbing. Another person suggests moving the bottom closer to the wall, and offers to hold the bottom of the ladder to make it safe. If asked why holding it will make it safe, gives a sensible answer about preventing rotation of ladder.
- There is no ladder, but there are wooden rungs, and rails with holes from which a ladder can be constructed. The person makes a ladder and then acts as in previous scenarios. (This needs further unpacking, e.g. regarding sensible sequences of actions, things that can go wrong during the construction, and how to recover from them, etc.)
- As for 7, but the rungs fit only loosely into the holes in the rails. Person assembles the ladder but refuses to climb up it, and if asked why can explain why it is unsafe.
- Person watching another who is about to climb up the ladder with loose rungs should be able to explain that a calamity could result, that the other might be hurt, and that people don't like being hurt.
We are developing our common sense reasoning systems by making them face a substantial library of such graded sequences of mini- scenarios that require them both to learn new skills, to improve their abilities to reflect on them, and (with practice) to become much more fluent and quick at achieving these tasks.
For more discussion about this approach please visit Aaron Sloman's web page Metrics and Targets for a Grand Challenge Project Aiming to produce a child-like robot.
The Roboverse is used with the Emotion Machine Architecture as described in the paper EM-ONE: An Architecture for Reflective Commonsense Thinking ([PDF]) by Push Singh.
This webpage was at the first sight interesting and led as well to the webpages of Aaron Sloman. But a closer look showed us that we had here and at the linked websites indeed the actings of C.S. of the more than 5 years before inclusive The Proposal with reflection and epistemology, computational ontology, robotics, 3D simulation and so on. But all related persons have not seen that with OntoLinux we had already taken the concept as the basic architecture of an operating system, and that the natural language processing is not taken only for a speech interface by us, but as part of the cognitive processing and the storage system of OntoLinux, which in fact is one of the revolutionary steps, because in this way not only the virtual actors and robots have reflective cognitve and common sense capabilities, but the reflection is also between the mind of the user and the virtual entity, the software that in this case is the large Turing machine in form of an operating system, or the machine, e.g. a robot, that is powered by OntoLinux, and controlled by its user or acting autonomously.
This was realized by integrating the Resource Description Framework (RDF), which can represent simple sentences consisting only of subject, predicate and object, the Polygon(al)-Data[bank] Model approach, which even can represent every main clause and this with an additional visual representation as well, with the Web Ontology Language (OWL), virtual environments and file systems.
In this way not only the Polygon(al)-Data Model and other data models were generalized, but also integrated with software engineering, especially object-oriented, ontology-based and model driven architectures, artificial intelligence, semantic (world wide) web, 3- to n-dimensional processing and presenting, and so much more (see the webpages Overview, Links to Hardware and Links to Software).
Furthermore, this led to the creation of the concept of the Caliber, that is founded on the latest accepted physics of the universe, especially on time and space, which in fact is another revolutionary step, because in this way not only the mind but also the body is uniquely reflected.
The system is derived from the ragdoll physics, which is a physics model for games and simulations.
Though we have the SDL - Simple DirectMedia Layer as a part of the OntoScope component (see also the graphic on the webpage about Underware), we use instead the Cal3D - 3D Character Animation Library, and what the game and simulation engine Delta3D as well as the 3D modeler Blender offer us.
Besides this, our work inspired also some research projects in the fields of developmental biology and epistemology, as well as robotics, as said in The Proposal, which could be seen for example with the actings of the institute Japan Science and Technology Agency with its Open PINO Platform, the company Fujitsu with its robot series HOAP - Humanoid for Open Architecture Platform, and the company Sony with its robot QRIO - Quest for cuRIOsity, aka. SDR Sony Dream Robot, that showed the step from the virtual software simulation to the real hardware application, and with the QRIO project, that also involved the learning by children, as well as the Grand Challenge Project Aiming to produce a Child-like Robot by the European Union and some others.
Our rework of the Open PINO Platform called Open ROBI Maxifig/ Macrofig Platform and the Sony robot QRIO called Qiro is complem- ented by some of the robotic technologies listed in the section Robotics on the webpage Links to Hardware.
Furthermore, following the overview of OntoLinux it can be seen that by reflection and the listed simulation software in the webpage Links to Software the cognitive agent/OntoLinux is also able to realize what some entities are calling embodied cognition using internal simulation and a global workspace for hiding that they have copied this concept from this website in conjunction with a child-like robotic platform that already was started as a clone of the Open PINO Platfrom. In this relation we would also mention that the cognitive agent/OntoLinux is able to use internal modelling and testing of software and hardware as well, in the case of software even itself, the Hightech Operating System, as part of its reflection feature. And the system architecture is also designed for the correct creation and change of external software and hardware as well.
As an overall system for the visual part we have the OntoScope component, as well as the Ontoverse with the OntoSpace and the OntoGlobe/OntoEarth, and all of the related software.