Augmentology" a concise manual of reality for our digital age."

Mark Hancock,_Augmentology: Interfaccia Tra Due Mondi_

[Sponsored by The Ars Virtua Foundation/CADRE Laboratory for New Media]

This TED talk by Peter Molyneux:

…demos Milo, a hotly anticipated video game for Microsoft’s Kinect controller. Perceptive and impressionable like a real 11-year-old, the virtual boy watches, listens and learns — recognizing and responding to you.

The demonstration begins with an explanation of how Milo is constructed. A combination of the following three elements allow Milo to exist:

  1. A Kinect Camera
  2. Artificial Intelligence developed by Microsoft
  3. Emotional Artificial Intelligence built by Lionhead Studios.

Milo moves through a synthetic environment predicated on User-directed biofeedback/body gestures: no mechanical controllers are necessary. Unfortunately, Milo’s introductory learning curve [which is integral to the "game" leveling system] involves inherent gender bias: if you’re a girl, your initial game variable is a Butterfly whereas if your a boy, you’ll be presented with a Snail.

The demonstration goes on to illustrate how Milo’s face is comprehensively AI driven. His facial movements include blush response, nostril “flare” size [indicating stress], “body matching” [causing neuro-linguistically driven facial alterations] and responses to verbal cues. Peter then describes how Milo’s personality development is predicated on a Cause-and-Effect dynamic. This causality is showcased via 3 examples:

  1. The User can choose to direct Milo to squash a snail: if the User does it will effect “…how Milo develops”.  The specifics of the verbal stimulus employed including how the User vocalises [specific phrases and intonations] all contribute to a database that informs and effects future interactions.
  2. The User teaches Milo to skim stones over the surface of a river [skewed gender stereotyping is again evident here].
  3. The User choosing to clean Milo’s room: Milo’s recognition of the User’s beneficial intervention and verbal engagement promotes sustained developmental interaction based on [what Peter terms] “deep psychology”.

This “deep psychology” [or what is described in synthaptic terms as "augmentology"] encourages a User’s empathy loadings. This in turn allows such games to shift towards complex experientially-defined engagement. These games surpass the hollow reinforcement of contemporary Social Games such as Farmville: instead, the User “levels up” by knitting fictionalised engagement with personality/identity construction and personalised growth variables. The element of cloud-directed learning [coaching synthetic humans whose social and chronological development depends on "crowdsourced" input] creates enormous opportunities for instruction and feedback via these types of  “Reality Gaming” systems [highlighted here by Seth Priebatsch]:


Sidequest! Screenshot

Don Woods famously added the more “Tolkienised” aspects of the Colossal Cave Adventure game after rewriting it in 1977. Woods’ rewriting/recoding/versioning of the game is another [TGOTT] inspirational treasure: Woods and others who have remixed and rewritten Adventure have opened this game to a constant flow of imaginative reinterpretations.

When immersed in Sidequest!, players may sometimes find themselves following an Ancient Path to a city inside the Hollow earth called Agartha. We members of [TGOTT] found an entrance at Mammoth Cave in Kentucky. As we crawled through the portal we passed through – and became – Will Crowther, crafting his computer-mediated memory of the present moment to share with his daughters. With Admiral Richard E. Byrd flying overhead (in a story told to us by Dr. Raymond Bernard in 1969) and 2,300 miles beyond the South Pole through a hole in Antarctica, we/he began watching the sky mirror reflect the sky below as he went inside the Hollow Earth. UFOs and Governmental Secrecy? December, 1929? February, 1947? November, 1955? January, 1956? February, 2009? Many unanswered player questions can be found frozen in the Arctic ice.

There are more Sidequest! entries buried underground than the Library of the Mystic Arts. Since 1969, the ARPANET has run on dedicated cables which were themselves buried underground. This internet backbone stretched across the United States and now has (obviously) expanded to an international and ubiquitous scale. ARPANET, that early military-industrial-academic-complex of only a few nodes, literally lay the underground network for us to crawl through. We (as William Crowther) were instrumental in the original ARPANET development team when we worked at Bolt Beranek and Newman building core technologies. In Sidequest! there is an option to begin at this starting point and crawl into lower levels of the military-industrial-academic-operating-system. This point resides closer to the kernel beside a datastream tumbling along a Classical Von Neumann machine.

Sometimes we drift out of this datastream, sailing upstream against assumptions to the The Stanford Artificial Intelligence Lab: another Sidequest! story starting place. We surface in the warm Californian sun at the end of Arastradero Road. This road sits before the D.C. Power Building in the foothills of the Santa Cruz Mountains. Californian foothills roll across the view with scattered trees underneath. Graduate Computer Science students run through these trees in infinite loops while attempting to decode the possibilities of playing chess with a majikal machine. A small stream flows out of the building and down a gully and you, William Crowther, can remember when it was made of pure data and flowing along underneath the cave floor at 56K.

Sidequest! is written in Python with entries authored by jonCates and programmed by Tamas Kemenczy and Jake Elliott: metaphors are mixed and transcendentally transposed over unstable timespaces. As we crawl closer to the center of the game, the connections are more random and fleeting, and flirt with more self-reflective recursions. This is what we mean when we say Sidequest! is “cyberpsychedelic” through combining the effects of mixing Cybernetics and Psychedelics as cultural influences, technologies and aesthetic principles.


The Guardians of the Tradition are an Art Games guild that recently created Sidequest!: A Classic Cyberpsychedelic Text Adventure. In Sidequest! you – as the game character William Crowther – crawl through a generative, cut-up and recombined twisty little maze of passages through timespace that criss-crosses multiple networks. Examples of these networks include: Mammoth Cave, ARPANET, The Stanford Artificial Intelligence Lab (SAIL) and The Hollow Earth. This Art Game is playable online as well as being Free & Open Source Software Art.

The Guardians of the Tradition (or simply [TGOTT]) formed in 2009 after an invitation to participate in the “Play Up!” exhibition at the Jack Olson Gallery, Northern Illinois University School of Art. “Play Up!”, curated by Mike Salmond, featured work by Eddo Stern, Ben Chang and The Guardians of the Tradition. The Games Guild is made up of jonCates, Jake Elliott and Tamas Kemenczy.

In 2006, [TGOTT] member Tamas Kemenczy created a text adventure engine/editor called Forklore, a collaborative platform for writing forked narratives via telephones. For our text adventures in Sidequest! Tamas employed a more generative approach so as to keep the game narratives coherent while also creating a significantly randomized, responsive and psychotropic narrative environment. This environment can collapse and regenerate depending on the gameplay. We define parameters (in order to keep the actions/subjects coherent) by keeping track of the avatar, chosen actions and the subjects acted upon. We use that ‘game data’ to generate text adventures (having the generated text adventures self-reflexively/recursively reflect the underlying state of the machine).

Sidequest! finds you, William Crowther, crawling through a generative twisty maze of passages, all different. The maze of passages is sometimes:

  • the caves William Crowther explored with his daughters prior to his divorce
  • the physical network of computers that constituted ARPANET
  • the original location of The Stanford Artificial Intelligence Lab (SAIL)
  • the mythical/mystical Hollow Earth
  • a cyberpsychedelic cutup of all these places/networks.

Will Crowther wrote Colossal Cave Adventure in 1975/1976. Colossal Cave Adventure is recognized as the classic/original text adventure that established this genre of gaming/Interactive Fiction. Patricia and Will Crowther:

  • traversed a tight passage between two networks of caves previously thought to have been unconnected in 1972
  • helped map Mammoth Caves in Kentucky
  • contributed to the field of spelunking
  • were in love
  • married
  • had two daughters (Sandy, born in 1967, and Laura, born in 1970)
  • by the mid-1970′s were divorced when Will Crowther wrote “Colossal Cave Adventure”.

Apparently Will wrote Colossal Cave Adventure to recreate his shared experience of crawling through/exploring cave systems (based on Mammoth Caves). His construction of the game was to allow himself and his girls a asynchronous existence in a virtual-memory-space after his divorce from Patricia. [TGOTT] are excited/inspired by the conceptual/emotional basis of this kind of game development and gameplay.

Post Reenactment – “Don’t Kill Mr. Gandhi!”:

The 240 mile march across Second Life with my treadmill powered Gandhi ended on April 6th, 2008. At the conclusion of the walk, I had a great sense of accomplishment – feeling content and quite frankly looking forward to a much needed rest! I had completed a rather fascinating journey – both in the physical and conceptual sense. In the days that followed the end of the reenactment, this feeling of euphoria was slowly overtaken by a sense of melancholy – I found myself both saddened and conflicted for the march to be concluded.

One of the avatars who walked with me the most was named Gyovanna. She told me she was from Brazil. She had the endearing habit of referring to me as “Mr. Gandhi”. Several days after the conclusion of the walk, I received a message from Gyovanna pleading with me “Don’t Kill Mr. Gandhi!”. In the days after the reenactment, such sentiments were expressed to me by many others. I logged in once or twice as Gandhi after the march but it was just not the same. The layered factors necessary to conduct the reenactment were crucial to creating a symbiosis between the treadmill, me, and my avatar – these could not be replaced or duplicated by merely visiting SL in the traditional keyboard-based manner.

Over the course of the march I had been thinking about how best to document and, in a way, process the experience of the march into a series of artefacts and objects. As a way towards further exploring my connection with my SL avatar, MGandhi, I worked to extract the 3-D model from Second Life and process this information into several sculptural renditions – the first being an 8” 3-D print made using the rapid prototyping equipment at Eyebeam. Working with my intern, Lenny Correa, we extracted Gandhi using an open source application called OGLE developed, oddly enough, at the Eyebeam Open Labs.

17’ tall Gandhi Reproduction

As an Eyebeam resident artist, part of the award was to have an exhibition onsite at the end of the residency period. At some point during thinking about the exhibition and processing the 8” figure, I envisioned the creation of a monumental sized Gandhi figure. Most statues of Gandhi, including the one in NYC at Union Square, are roughly human scale. The thought of creating a larger than life representation of my Gandhi avatar seemed quite audacious and oddly appropriate. I settled upon using another open source application, Pepakura Designer, which is a popular papercraft program used by enthusiasts to create some rather amazing – albeit generally tabletop scale – reproductions of everything from anime figures to airplanes. I adapted this program to create a 17’ tall reproduction of my Gandhi avatar out of cardboard and hot glue. The entire process has been described and documented, in detail, on – my first instructable which I see as an unorthodox but highly effective way to further disseminate the project. The resulting 17’ figure is a monumental physical representation of Gandi created from very simple materials. The figure was made to be the same height as Michelangelo’s David – a fitting conceptual connection to this iconic work of art history depicting the Biblical figure of David just before slaying Goliath.

Most recently, I have completed a second 3D print of the Gandhi figure that stands 12” tall. For this piece, I have treated the figure in genuine gold leaf – creating a rather contradictory sculptural representation of my avatar as a shining, fetishized object.

3D Print of the Gandhi Figure in Gold

I’ve also created a 6000+ frame stop action video created from screenshots recorded automatically every 60 seconds documenting the entirety of the reenactment in SL. The 9 minute long piece is compiled at 12 frames per second. I showed this piece at Eyebeam in an installation featuring the original treadmill; it includes an LCD panel laid upon the treadmill showing a looping segment shot from above, and in front of, my legs walking during the reenactment. For the exhibition, the re-installation of the treadmill, the 17’ Gandhi and the 8” Gandhi were complimented by six screen shots shown as large formatted prints. Also included was a mural size print of my recreation of the famous image of Gandhi making the salt at the beach at Dandhi.

The entirety of the Gandhi work is to be featured as part of the Guangzhou Triennial 2008, entitled Farewell to Post-Colonialism in China this fall. For this exhibition, the museum will be working with me to build a second 17’ Gandhi – a way of having a local connection to the creation of the works for the show. The first Gandhi was, btw, designed to allow for disassembly into parts for shipping – he is now on his way home to me in Reno in two very large cardboard containers, big and bulky but very light ;-) .

What pleases me most about the entirety of the reenactment experience, and ongoing exploration or processing of this Mixed Reality performance, is that through this work I have, for the first time, unified what have previously have been two different threads of my artistic practice. For years, my work in physical installation, sculpture and kinetic art has been separate from my work performing in computer games and online communities. The synthesis of the real and the virtual during the reenactment, and the resulting physical explorations post-reenactment have proven to be a revelation to my creative process that will surely inform my activities for years to come.

On the March:

The reenactment of Gandhi’s Salt March in Second Life began on the South Western edge of the largest mainland on April 12, 2008. Gandhi’s basic navigational strategy in SL was to walk towards other avatars (represented by green dots on the map), greet them, describe the project, offer to be “friends”, share a copy of his walking stick, and to invite others to join him on the march. I set very specific rules for my travels online, including a strict avoidance of the typical SL transportation methods of flying and teleporting from one location to another.

I set out on this “journey” for a variety of reasons – to pay tribute to the vision and creativity of Mahatma Gandhi, continue to explore the nature of protest, investigate notions of physical and virtual embodiment, challenge the expectations of avatar representation, incorporate bodily exertion and durational constraints to an online performance – and to simply do something that has never been attempted previously in a combined online and real world context. What I discovered was that, while the reenactment certainly touched on the aforementioned ideas, I found myself, unexpectedly, to be profoundly transformed and engaged by the experience.

Gandi + Other Avies Interacting

For the month of the performance my life revolved entirely around conducting this reenactment. The engagement in this performance involved what became a daily ritual of walking on the treadmill to guide my Gandhi avatar in Second Life, commencing at 12 noon each day for 26 days, walking six days a week, six hours on average each day. My daily regimen was essentially totally dedicated towards walking on the treadmill and interacting with the world of Second Life. My immersion into Second Life combined with the physical exertion of my body necessary to engage in the work created an interesting symbiosis of the virtual and real.

By the end of the reenactment I had become so engaged with and attached to my Gandhi avatar that my ability to clearly delineate between the online and the real world had become temporarily muddled. On more than one occasion I found himself wanting to “click” on people and experiencing brief instances of déjà vu during off hours in NYC where my mind’s eye was briefly confused as to my physical location. The action of walking on the treadmill further reinforced this confusion of physical and virtual space – it became a daily occurrence to find myself nearly falling off the treadmill during the walk as Gandhi stumbled either due to connection lag or when he would invariably stumble off a mountain or otherwise take a virtual misstep.

TreadMill/Laptop Running

At the start of the journey, I was not sure if I would be bored or find myself disinterested by spending such extended periods within the confines of Second Life – this online environment had failed to secure my interest in the past. Contrary to my expectations, the walk across SL became a daily fascination – the sense of discovery and wonder was very intense. As I was only able to walk, and only able to move my avatar by physically walking on the treadmill, an odd synthesis of physical labor and exploration ensued that was quite rewarding. Gandhi believed wealth without work to be meaningless – I found the walk to be fulfilling in part due to the combination of my physical activity and the need to find my way on foot – I began to appreciate the fact that I was earning this particular online experience through intense physical exertion.

The interactions with other avatars in SL and with people in RL stopping by to watch me at Eyebeam were as well significant to the overall experience. The typically private act of engaging in online activities from a home computer was transformed into a public, physical spectacle. I stopped along the way to chat with hundreds of avatars, informing each one as to the nature of my “walk across Second Life” telling them that “my human is on a treadmill making me go”. Many stood next to me or sat behind me in easy chairs at Eyebeam, watching transfixed and talking with me as I navigated through SL – one spectator who watched me for over an hour said it was curious, as Gandhi was my avatar she was starting to think of me on the treadmill as being her avatar.

Gandhi Meets Nixon

It is perhaps more difficult for me to write specifically regarding the “why” of being Gandhi than to describe to effect of the overall experience – I could go on about this for quite some time – the excitement of others in greeting my Gandhi avatar, how easily he disarmed strangers in the online world, how shy I found myself the day after the conclusion of the reenactment upon joining an actual Gandhi walk for peace in NYC.

In my third and final posting next week I will detail my ongoing efforts to process and further explore this performance project – including the creation of a 17’ cardboard replica of my Gandhi.