WE
WE
INSTALLATIONS
Overview
As Tamara took a swig from the beer in front me, I exclaimed “Hey that’s mine”. Yellong Bulla, made fun of me sarcastically “oh…. that’s my beer”. Immediately understanding him, his words continue to reverberate.
I am you as you am I. That as individuals, we are multidimensional mirrors of the culture and space that we share.
WE an artificially intelligent computational mirror, manifests itself through an evolving 3 dimensional interactive face/portrait mapped projections and light installations.
Anthropomorphic imaging litters our subconscious through endless data streams both man made and environmental. Every single face we have seen has been reflected through our internal processes and out for others see. Philosophically the concept of the self within indigenous shamanic cultures and the psychedelic state, is at odds with the western axiom of Self and the Other. WE engages people to explore this space through interacting with an artificial intelligence and thus effecting it / or leaving their mark on it through their engagement.
Deep Learning and artificial intelligence are used to discern faces and chatter from camera and audio equipped personal smart devices and small surveillance hardware. Captured faces, audio and geolocation are transmitted to the WE, a server that converts 2D faces into 3d meshes, audio data into textual, coupling internet text and video into knowledge via deep learning algorithms. Data such as gender, emotion, location etc. are used to generate WE’s face and personality. Smart devices are used to converse with and teach WE per an installation’s geolocation.
By using internet text chat to interact with the AI my goal is enable an intimacy that textual chatting can bring to a non intimate public environment.
Technically and technologically WE is consolidation and progression of my virtual geolocation public artwork, my music visualisation graphics pipelines, music festival projection technologies and scalable internet software design from my work as a Software Architect.
Philosophical Observations
It is now very clear that techniques of machine-human interfacing, pharmacology of the synthetic variety, all kinds of manipulative techniques, all kinds of data storage, imaging and retrieval techniques– all of this is coalescing toward the potential of a truly demonic or angelic kind of self-imaging of our culture…Terrance Mckenna
“There will come a time when it isn’t ‘They’re spying on me through my phone’ anymore. Eventually, it will be ‘My phone is spying on me’.Philip K. Dick,
Philosophically WE encompasses many areas of both philosophical thought and quantitive research. One goal of WE is to explore these complexes through software as a rational exploration of the meaning of being. Through the digital manifestation of our most primary language engine the face, many other equally important notions emerge about our relationship with technology and what is to be the future human.
Face
As a highly social species, humans frequently exchange social information to support almost all facets of life. One of the richest and most powerful tools in social communication is the face, from which observers can quickly and easily make a number of inferences — about identity, gender, sex, age, race, ethnicity, sexual orientation, physical health, attractiveness, emotional state, personality traits, pain or physical pleasure, deception, and even social status.
A brief look at any face demonstrates the numerous inferences observers can make about an individual. Although humans quickly, and apparently effortlessly, perform such a variety of social categorisations on a daily basis understanding the precise nature of this process — which specific face information subtends the perception of each category — remains challenging, because the face, as a transmitter of multiple and complex social categories, comprises a high-dimensional, dynamic information space.
Erving Goffman wrote about face in conjunction with how people interact in daily life. He claimed that everyone is concerned, to some extent, with how others perceive them. We act socially, striving to maintain the identity we create for others to see. This identity, or public self-image, is what we project when we interact socially. To lose face is to publicly suffer a diminished self-image. Maintaining face is accomplished by taking a line while interacting socially. A line is what the person says and does during that interaction showing how the person understands the situation at hand and the person’s evaluation of the interactants. Social interaction is a process combining line and face, or face work.
Goffman, E. (1959). The presentation of self in everyday life. Garden City, N.Y.: Doubleday
Because you are I am
A Zulu phrase, ‘Umuntu ngumuntu ngabantu’, which means ‘A person is a person through other persons.’
According to Ubuntu philosophy, which has its origins in ancient Africa, a newborn baby is not a person. People are born without ‘ena’, or selfhood, and instead must acquire it through interactions and experiences over time. So the ‘self’/‘other’ distinction that’s axiomatic in Western philosophy is much blurrier in Ubuntu thought. As the Kenyan-born philosopher John Mbiti put it in African Religions and Philosophy (1975): ‘I am because we are, and since we are, therefore I am.’
We know from everyday experience that a person is partly forged in the crucible of community. Relationships inform self-understanding. Who I am depends on many ‘others’: my family, my friends, my culture, my work colleagues. The self I take grocery shopping, say, differs in her actions and behaviours from the self that talks to my PhD supervisor. Even my most private and personal reflections are entangled with the perspectives and voices of different people, be it those who agree with me, those who criticise, or those who praise me. Abeba Birhane Descartes was wrong: ‘a person is a person through other persons’
Some philosophers have wondered whether the self even exists at all. A concept that important to Buddhist thought. Others believe in the existence of the self; but think that self does not exist independently, but rather arises from a matrix of relationships.
This can be seen when we answer the question “Who are you?” Must of us do not answer with Cogito ergo sum, or hold up a scan of our brains. Rather, when asked “Who are you?” we answer “brother,” “daughter,” “friend,” “healer,” “teacher,” etc. In short, our descriptions of ourselves often invoke our relationships to other or as parts of the whole.
The Self & the Other
The concept of the Self requires the existence of the Other as the counterpart entity required for defining the Self. Georg Wilhelm Friedrich Hegel (1770–1831) introduced the concept of the Other as a constituent part of self-consciousness (preoccupation with the Self), which complements the propositions about self-awareness (capacity for introspection)).
Edmund Husserl (1859–1938) applied the concept of the Other as a basis for intersubjectivity, the psychological relations among people. In Husserl’s workCartesian Meditations: An Introduction to Phenomenology (1931), that the Other is constituted as an alter ego, as an other self. As such, the Other person posed and was an epistemological problem—of being only a perception of the consciousness of the Self.
In Being and Nothingness: An Essay on Phenomenological Ontology (1943), Jean-Paul Sartre (1905–1980) applied the dialectic of intersubjectivity to describe how the world is altered by the appearance of the Other, of how the world then appears to be oriented to the Other person, and not to the Self. The Other appears as a psychological phenomenon in the course of a person’s life, and not as a radical threat to the existence of the Self. Simone de Beauvoir (1908–1986) applied this concept of Otherness to Hegel’s dialectic of the “Lord and Bondsman” (Herrschaft und Knechtschaft) and found it to be similar of the Man–Woman relationship, there for it could be seen as a explanation for society’s treatment and mistreatment of women.
Artificial Intelligence
Alan Turing (1912–1954) reduced the problem of defining intelligence to a simple question about conversation. He suggests that: if a machine can answer any question put to it, using the same words that an ordinary person would, then we may call that machine intelligent. A modern version of his experimental design would use an online chat room, where one of the participants is a real person and one of the participants is a computer program. The program passes the test if no one can tell which of the two participants is human. Turing notes that no one (except philosophers) ever asks the question “can people think?” He writes “instead of arguing continually over this point, it is usual to have a polite convention that everyone thinks”. Turing’s test extends this polite convention to machines:
– If a machine acts as intelligently as human being, then it is as intelligent as a human being.
Hubert Dreyfus claims
if the nervous system obeys the laws of physics and chemistry, which we have every reason to suppose it does, then …. we … ought to be able to reproduce the behaviour of the nervous system with some physical device.
This argument, first introduced as early as 1943 and vividly described by Hans Moravec in 1988, is now associated with futurist Ray Kurzweil, who estimates that computer power will be sufficient for a complete brain simulation by the year 2029. A non-real-time simulation of a thalamocortical model that has the size of the human brain (1011 neurons) was performed in 2005 and it took 50 days to simulate 1 second of brain dynamics on a cluster of 27 processors.
The words “mind” and “consciousness” are used by different communities in different ways. Some new age thinkers, for example, use the word “consciousness” to describe something similar to Bergson’s “élan vital”: an invisible, energetic fluid that permeates life and especially the mind. Science fiction writers use the word to describe some essential property that makes us human: a machine or alien that is “conscious” will be presented as a fully human character, with intelligence, desires, will, insight, pride and so on. (Science fiction writers also use the words “sentience”, “sapience,” “self-awareness” or “ghost” – as in the Ghost in the Shell manga and anime series – to describe this essential human property). For others[who?], the words “mind” or “consciousness” are used as a kind of secular synonym for the soul.
For philosophers, neuroscientists and cognitive scientists, the words are used in a way that is both more precise and more mundane: they refer to the familiar, everyday experience of having a “thought in your head”, like a perception, a dream, an intention or a plan, and to the way we know something, or mean something or understand something. What is mysterious and fascinating is not so much what it is but how it is: how does a lump of fatty tissue and electricity give rise to this (familiar) experience of perceiving, meaning or thinking?
Philosophers call this the hard problem of consciousness. It is the latest version of a classic problem in the philosophy of mind called the “mind-body problem.” A related problem is the problem of meaning or understanding (which philosophers call “intentionality”): what is the connection between our thoughts and what we are thinking about (i.e. objects and situations out in the world)? A third issue is the problem of experience (or “phenomenology”): If two people see the same thing, do they have the same experience? Or are there things “inside their head” (called “qualia”) that can be different from person to person?
Neurobiologists believe all these problems will be solved as we begin to identify the neural correlates of consciousness: the actual relationship between the machinery in our heads and its collective properties; such as the mind, experience and understanding. Some of the harshest critics of artificial intelligence agree that the brain is just a machine, and that consciousness and intelligence are the result of physical processes in the brain. The difficult philosophical question is this: can a computer program, running on a digital machine that shuffles the binary digits of zero and one, duplicate the ability of the neurons to create minds, with mental states (like understanding or perceiving), and ultimately, the experience of consciousness?
Singularity
The technological progress that characterises our modern world is becoming an increasing part of our lives. Due to scientific research and economic growth, new possibilities of improving our lives are becoming actual. In the fields of artificial intelligence (AI), nanotechnology, biomedicine, computational technologies and genetics new opportunities for applying scientific insights are coming to fruition. This leads to innovative developments in applied science fields such as robotics, augmented or virtual reality, body enhancement, communication technology, medical treatment of patients and psychological therapies. These new forms of technology are likely to have a great impact on our everyday life. Our ways of working, learning and communicating are going to change drastically through implementation of new technologies in our world. Both human longevity and the longevity of information will be challenged and eventually extended. Because the way we interact with the world and with each other is constantly subject to change our way of experiencing the environment is going to change accordingly. Not only does this raise implications for specific fields in the philosophy of mind, but also for the social, cultural and economical sciences. In 1987, inventor and futurist Ray Kurzweil wroteThe Age of Intelligent Machines, which he later adapted to another book called The Age of Spiritual Machines (Kurzweil, 1999). These two books were updated by Kurzweil into one book called The Singularity is Near (Kurzweil, 2005). In his book, Kurzweil describes what awaits the human race as technological development progresses exponentially towards the future. In his view this is a future in which technology changes our everyday life, our scientific research, our way of government and even our place in the universe. According to Kurzweil, the way of being human that we have become used to is going to change radically and very quickly, to the point that we won’t recognize ourselves as humans anymore. All this will happen in an event which Kurzweil calls the technological Singularity.
When the Singularity hits, the distinction between humans and AIs will disappear over time. Because there will be no difference between humans and AIs, it is not questioned whether or not AI will be conscious. In other words, there will be one single conscious form of entity, with a biological portion of intelligence and a nonbiological portion of intelligence.
Technology & Architecture
My approach to fast prototyping and rapid application development is to use Python as the base system language. Python has a major history in the scientific community for software prototyping and linking to C based libraries for faster execution system intensive processes. Utilising HTML/javascript/css as means of application delivery to mobile devices come with some modern software performance drawbacks. However ease of application updates to all devices is seen a major advantage over native device code. By basing the application in “the cloud” WE can be up scaled up and distributed as one supercomputer.
Systems Overview
Data Paths
WE Cloud Servers running Linux
- MongoDb Database
- Python Servers
- HTTP facial processing server
- face image to landmarks
- landmarks to mesh & texture
- catalog & store
- serve facial mesh updates and sets via geolocation and deep learning catalogue parameters
- Web Socket & HTTP Chat Server for mobile device interaction
- Geo Location zone calculation
- face detect streams to facial processing server
- Collection and interaction via text for response with chat bot
- Deep learning catalogue agent
- Cleans faces, text and audio sets e.g. end user duplications, low resolution imaging, spamming
- retrieves set meeting deep learnt parameters Eg age, sex, emotion
WE Physical Installation clients
- Projection Mapping Client
- Linux OS
- Python
- Networking loading of Audio, Mesh & texture Data
- OpenGl GLSL interactive Shader Texture & Mesh pipeline
- mesh & texture morphing
- mesh & texture audio interaction animation
- Audio Capture and transformation for pitch and dynamics analysis
- Data Collection Client
- Raspberry PI hardware
- Linux OS
- Python
- Open CV image capture Cascade facial Detection
- DLIB Facial Landmark Prediction
- HTTP Data send to facial Processing server
- HTML5 Device Interface apps
- Responsive cross device Design
- Camera & GPS Access
- Facial recognition login pathway
- Web socket Chat Client
Client We Face Render Pipeline
Face Generator Render
- Mesh Loader
- Primary Decision Maker
- audio events
- geolocation events
- communication events
- Frame Buffer Renderer
- GLSL Pipeline
- Vector Raster Map Morpher/Combiner
- Eigen face Vector Morpher
- GLSL Facial expression Vector Morpher
- GLSL Projection Render to Texture
Environment mapping Render
- 3d Space Object Loader
- Space Object and vector / texture Positioner
- Projection Generated face texture
- wrapped over 3D space object
Acknowledgements
WE would not be possible without the work of thousands of others. Projects would literally be to numerous to mention, such is the nature of technology. Projects and work directly attributed to the unique function of WE are listed below
Papers
- The Human Face as a Dynamic Tool for Social Communication Rachael E. Jack and Philippe G. Schyns School of Psychology, University of Glasgow, Scotland G12 8QB, UK Institute of Neuroscience and Psychology, University of Glasgow, Scotland G12 8QB, UK[PDF]
- A Multiresolution 3D Morphable Face Model and Fitting Framework, P. Huber, G. Hu, R. Tena, P. Mortazavian, W. Koppen, W. Christmas, M. Rätsch, J. Kittler, International Conference on Computer Vision Theory and Applications (VISAPP) 2016, Rome, Italy [PDF].
- Deep Speech: Scaling up end-to-end speech recognition, Baidu Research. Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, Andrew Y. Ng [PDF]
- DEX: Deep EXpectation of apparent age from a single image Rasmus Rothe, Radu Timofte, Luc Van Gool Computer Vision Lab, D-ITET, ETH Zurich, Switzerland [PDF]
Software Projects
- Python Python is a programming language that lets you work quickly and integrate systems more effectively
- Tensor Flow An open source machine learning framework
- Dlib A modern C++ toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real world problems.
- OpenCV An Open Source Computer Vision Library
- Open GL The Industry’s Foundation for High Performance Graphics – from games to virtual reality, mobile phones to supercomputers
- Locust An open source load testing tool.