To help people remember key facts, IBM unveiled a software technology
created in its Research Labs that uses the images, sounds, and text
recorded on everyday mobile devices to help people recall names, faces,
conversations and other important information.
The technology, nicknamed "PENSIEVE" by the IBM team, uses associative
recall to make connections between pieces of related data acquired by a
person. The advantage of the new technology is its ability to
understand the context in which data is captured, then connect various
data, and then use this knowledge to help bring the correct information
to a person when it is needed.
"This is like having a personal assistant for your memory," said Dr.
Yaakov Navon, the lead researcher and image processing expert from
IBM's Haifa Research Lab. "Our daily routines are overflowing with
situations where we gain new information through meetings,
advertisements, conferences, events, surfing the web, or even window
shopping. Instead of going home and using a general web search to find
that information, PENSIEVE helps the brain recall those everyday things
you might normally forget."
Today's mobile devices have an endless number of functions that can
record data in real time. IBM's new software blends techniques from
image processing, GPS information, smart clustering, optical character
recognition, speech recognition, and information retrieval to index and
tag the information.
Researchers at IBM's Haifa Research Lab in Israel are pairing advanced
mobile technologies with memory cues to develop a system that can
analyze acquired data, create hooks to related experiences, and use
them to populate a person's information management applications. Once
the address books and calendars are updated, the technology enables
memory recall triggered by time, location or the introduction of
For example, if you meet someone at a conference and use your phone to
take a picture of him or her and another picture of that person's
business card, the new technology will associate the two pieces of data
because they were taken at the same time and location. It then creates
a virtual briefcase of data that includes the person's image, the name
of the conference where you met, the date and time, and any other
The knowledge base is unique in its ability to integrate contextual
information with image, time, and location data. Prior to a future
meeting with the same person, you can be prompted by your calendar to
review your notes about that person, along with other facts you may
have recorded or items the system associated for you.
This year, for the first time ever, more people in the world will have
a mobile phone than a land-line. Mobile phones already double as music
players, video players, game devices, and calendars. And while using
mobile phones to take photographs and record sound or video bites is
common practice, much of this data tends to get stored away on the
device with little information attached to it, rendering it of little
value for future use. To address this, IBM recently launched a new
initiative to bring even more features and functions to the mobile
Many people encounter situations where you have been introduced to
someone but you can't quite recall how you know them. By simply typing
the person's name into PENSIEVE, you can recall when and where you met
them, and any related information garnered at that time. You could even
browse forwards or backwards in time to find out what events transpired
before or after the initial meeting.
Another use of this technology is in reconstructing and sharing an
experience or memory. If enough media-rich data was collected about a
particular event, it can be used to build a more complex visual
associative representation of the experience.
"This is where the real power of collaboration kicks in," said Eran
Belinsky, research team leader and a specialist in collaboration. "You
can recall the name of the person you met right before you entered a
meeting by traversing a timeline of your experiences, or share a
business trip with colleagues by creating a mashup that shows a map
with an animation of your trail and the pictures you took in every
This technology builds on IBM Research's expertise in image processing,
speech recognition, information discovery and retrieval, collaboration,
and much more.