Questions about work in progress

ChatBot using speech recognition  p5jS > create a River script for the interaction between audience and computer , (D.Shiffman Tutorial)

The bot is triggered by proximity sensor monitored via Arduino. Arduino send via Serial to Computer , 1 if there is someone in front of the sculpture. If there is no one, bot is only displaying DataText.

Speaker sculpture is made of a sound sensor connected to an Arduino and a speaker plugged to computer. The audio speaker sculpture is acting as an actuator for the first servo which body is spinning and touching the other sculptures.

Still have to find the link between the proximity sensor with the bot and the speaker sculpture

DataText is produced by training Texts on masks and object in theatre and Text about components specifications. Training model is Markov Chain.

River script is based on waiting for keywords , and ability to display long sentences. This is a limitation to the conversation.

Need to set up a private router , to have the installation free from overcrowded network

Questions about training Neural Networks from the maker

 

The maker : For who’s pleasure do we train the models?

Who is the curator/ the audience?

What is the purpose?

“If we employ machine intelligence to augment our writing activities, it’s worth asking how such technology would affect how we think about writing as well as how we think in the general sense.” (Ross Godwin )

How to start

https://towardsdatascience.com/how-to-build-a-neural-network-with-keras-e8faa33d0ae4

Questions about the first actuator

  • recommendation from R. Fiebrink about sound and triggering the installation

Use sound recognition to find if there is someone talking to one of the artefact, will solve the problem of how many people are interacting effectively with the installation, play with the sensitivity level , to get the data sound we need

  • Build up one “ear” node, with microphone , check this  and that to understand wich type of microphone we need. > lavalier microphone
  • Build up a “mouth” to display the response of the chatbot installation, play with idea that a microphone is a reverse speaker?
    • version 1 simple version via p5js with the need to have access to cloud to do speech recognition ,
    • version 2 for speech recognition with BitVoicer API , speech recognition, check Arduino Tuto : speech recognition and synthesis,
    • would need to use a private router to secure access to the cloud for the text to speech > check authorisation for plugging the private router to internet, 

To generate responses from the ChatBot : build up different training set with different methods : Markov Chain , char RNN review and  basic char RNN training with github source,   , Tensor Flow‘s sequence to sequence library feed ,

write different scripts via RiverScript to compute the input text  to ouput text, use array in Riverscript? generate different characters , who you are talking to, use a list of trigger words in an array to output a certain sentence

or use the mic from MAC

  • after tutorial with H. Pritchard

produce a mix reality with the electronic node devices and human interpretation by the nodes Node A activate node B/ one sensor, one motor, one movement, / small (or big)size nodes

create the narrative: when (no one/ or not enough people) is (looking or talking) , the nodes are interacting  with each other- when there is enough people or one person( looking or talking), the nodes stop moving, and talk back

Using text to speech and maybe speech recognition

 

 

Question about Network characterization

Reviewing different ways for the characterization of the Network and first tests on how about telling a story with sensors and motors?

About Sensing and controlling the physical world with computers. O’Sullivan, D., & Igoe, T. (2004). Boston, MA: Thomson. interesting quotes about “lost” characters in ASCII table “Another point of confusion stems from the fact that the first 32 entries in the ASII table are control characters, like the “carriage return”( ASCII 13) and “line feed”( ASCII 10), are familiar to you from word processing. Some of them, like “bell”(ASII 7), are left over from the old teletype days. Sending these numbers can cause confusion in environments that can only interpret text characters becaue either nothing shos up, or you get a bunch of garbage characters( square, smiley faces, and other dingbat characters).p142

It is noticeable that the machines talk to each other in a foreign language, we gave and forgot about it , they still talk this language. How about checking the ASCII control characters table from old teletype days>?

Possibility to define the Network of artefacts controlled via OSC touch?  Network would be the Mask, its artifacts,  the nodes.

network projects its desire onto us- through its material affordances, AfNet

Artists references and approaches to the concept of Network and cybernetic behaviour :

Things about Textility of Code

FoAM . Collective looking into Weaving and Coding

A catalogue of errors, by David Griffiths and Alex McLean : This article presents a series of informal experiments in software and weaving, most of which were conducted as part of the Weaving Codes, Coding Weaves project.

FoAM github   Weaving experiment

 

Questions about installation – K- cells-Knitted cells

the cells, could functions as kinetic drawing bots:

01/10/2018

 In relation to my essay proposal about the  ”NetWork : the Ultimate Mask “
Find the Poetic Space of the Network , I have the intention is to build up some Knitted cells units.  I would like to be able to stitch them together to make either a costume , a quilt , a mask,  a 3D sculpture  , a sensitive environment, or reactive bodies for puppets…

“Rules” :

  • The cells will have different functions , and different structures . They could store visual memories . They could produce or record sounds, be thermo sensitives.  They could produce lights etc…
  • Combined and stiched together they form a body. Each of this unit has its own functions and life, stich together they form a collective and have a collective behaviour. Individually they have a autonomous behaviour.
  • Like puppets can do one thing , dancing versus walking, or a mask is expressing one quality ( neutral, clown, buffon ….)  each type of K cells have limited functions , either data feed, ligth or sound. their design should reflect this particular feature.
  • Decide whether together they are stronger or weaker ? amplification process? they can also relate to each other remotely ie not stitch together.

Strategy

  • >Build K cells in electronics and transpose into soft e textile
  • > Build two families KL, knitted light fiber cells .and KA knitted audio fiber cells

    • Wearable Antennae Speaker FM Transmitter (KA)
  •  Puppetry and mask making process of animating objects
  • then WA weaved audio cells
  • combine the two families in a collective structure
    yep
    Birdy, found objects puppet by theatredu1k
Mona Luison
credit : Mona Louison

“Variables and functions”:

To achieve the design of each family of K cells,  I will use ( list to be completed)

  • mapping technology ( Max/ MSP, Mad Mapper)
  • generative code to find interesting patterns for each type of cells
generative sin pattern in Processing by SRQ
 

Luce - Lucette & Lulu

Questions about questions

02/14/2018

about Practice Methodology

My approach is from the material not from preconceived ideas. Thus playing with codes and trust the focus from the right brain who will lead the code play towards the direction

01/30/2018

Pattern for a PCB? : Icosahedral-stereographic-projection-in-the-direction-of-a-2-fold-axis 

01/29/2018

Looking for patterns , how the nodes move

, do they have to move?

 

01/09/2018

update with a more detailed proposal for the practical side : K- cells-Knitted cells – MFA year 2 practical project

This project is about exploring the poetic space lying between nodes and paranodes of the Network using mask improvisation and shadow theatre technics combined with coding and mapping technology

Annexe of the Essay with a list of ideas including even what is left as we try to keep everything even if not used yet to share with other artists willing to use it
what is not used by me could be used by other not only by big companies
not erasing , but put aside, to be revisited
Theatre piece : Head mask , shadow silhouette defined by the edge, producing different scenarios to illustrate the actualization of the Network in human bodies.
Mask improvisation workshops to understand how we actualise in our bodies , the network;
Mask improvisations, craft approach ,
creating the space of mask improvisation and mask making ( mapping + artefact)
looking for symbols produced by the network , logos? how do we choose define the mask
Mask improvisation workshops to understand how we actualise in our bodies , the network;
Human actors. Use the human imagination of the network , how do we perceive the network?
Human actors and machines , create a mask for human actors
Machines, create a humanoid mask for the machine
Exploring the mask and methods of training actors with masks as a way to understand and play the networks .I suggest that the social network and its interfaces (screens, avatars, social media selves…) could be understood as masks. Like masks they have a specific language and specific qualities. They frame, mirror, mediate, catalyse, transform, show emotions, classify or hide.

propose a scenario for a mask to explore the relations between humans and machines in the network.
you are the machine, the hardware , the software, the code, the portal, the algorithm, the coder, the screen, the human, a data, ….list all the possible characters and try to mach a mask

one way to plan this mask workshop could be to rely on tested scenarios experimented without mask like in Ihde, D. 2001, Bodies in technology, University of Minnesota or like in out of body experience by Ehrsson
Body silhouette as a shadow behind a screen made of sensible material,
look through a microscope

Artefact evolving in time and space with dynamic rules
you can inherit the artefact
Artificial Plant, only if you look at her ( bitcoin Plantoid), or with data garbage
network as monstrous chimera knitted , mixing humans and machines with the data garbage , revisit your garbage ( Thomas Mann, the field, Sabrina ’s poem )
animated Rhizome evolving in time with dynamic rules
Japanese tree with prayer papers , printed out
London Bridge , train station and railways from above miniature

experiment the Network as Anthromorphic frame, The Goddess as Void

Sound Installation with knitted sculptures mixing crochet with conductive thread and rope. (inspired by the work by Judy Tadmans or Ruth Asawa) .
Beating the Bounds experiment the limit of the Network , boundaries , sounds of the boundaries

Staging observers and users
listen to data, with wearables antennas to plastron , headband,
producing Interferences to others like it was with Talkie Walkie,
with an umbrella , when under the umbrella you could experience other data
cushion . when you hug it , memories appears (thermo ink)
Petri dishes , biologic bacterias will show the contact
generative algorithm human and machine collaboration for a music quilt continuous renegotiation , control/ freedom ratio

Remote sensors Explore ways to “tell” something happening in other space
Mask is in the exhibition area, and “tell “ (Max MSP?) what is happening elsewhere, outside where the data is collected. Data storage and Data source remote.
Wind automata in display in exhibition place linked to weather data sources

Wearable memory
knitted cuffin , plastron, headband, scarve….recording and play sound.
the one you want to share / the one you want not to erase but that you can inherit from someone else , transferable ,
for index visualisation artefact, disconnection between index in virtual and analog reality they are supposed to tell,
information embodied

Autonomous artefact collecting informations and selection algorithm for read and erase?
Caterpillar , snail
Memory storage as a food chain , eating and digestion remixing the left over ? food artist
navigate through a Mosaic, Puzzle

Mask machine teaching us,
producing a list of lost/ forgotten/ erased behaviours that other machines will teach : good manners, body interactions not reduced to sex ,

use your body perception and affect (excluding eye) for something else than food processing, sex relations
input by the mask, output by the body, input by the body, output by the mask

revisit your garbage ( Thomas Mann, the field, Sabrina ’s poem )
producing a comic with images in memory

Is there a Sustainable program, software , show the evidence of sustainability?
list of program language and number of updates javascript , C++….
list of devices / software by number of updates
quilt
hieroglyph

I will explore knitting , embroidery, weaving, ( flexible) wire, thermo painting, and fibre optic as a light and also data transmitter in order to make every other week sculpture. The mask/object will reflect the actualisation of the entanglement between human and machines. The objects will have a practical use or not, will move or not, will have sensors or not etc… and will be displayed , like a dada surrealist “cabinet de curiosite” .

Ultimately , for the exhibition , I am willing to explore also how to create an embroidered tapestry quilt mixing network and human with live input. The tapestry will be a tangible memory of the meeting between machines and humans. I will either hack a sewing machine to build very simple patterns with lines, triangle or rectangles and maybe circle , then combine with embroidery made by hands during exhibition workshop or use an embroidery machine. In order to achieve this goal, I will explore different embroidery and knitting hacking technics and look how to alter their looping process just enough to reveal the encounter , like knitted glitch

Things about Essay planning

Notes from Helen Pritchard 15th december

  • –  Invasion of the body by the network
  • –  Exploitation of the body of the network
  • –  performativity of the network
  • –  Relations of power
  • –  Which side of the string are you – is there a side in entanglements?
  • –  Automation
  • –  Entanglement – Karen Barad
  • –  Body and Society Journal
  • –  Technology, Culture and Society Journal
  • –  Off the Network – https://www.upress.umn.edu/book-division/books/off-the-network

 

–       Anna Munster’s work
–       Erin Manning’s work on relationscapes

Things about Bibliography

01/09/2018

BiblioGraphy

Albu, C. 2016, Chapter 4, Mirror Portal. Mirror Affect, Seeing Self, Observing Others in Contemporary Art, University of Minnesota
Assaraf, A, 2017, « Toutes les émotions en deux forces : Damasio et le “système JP” », PSN, volume 15, 1993, Quand dire, c’est lier, Nouveaux Actes Sémiotiques, Université de Limoges, PULIM, no 28.
Austin , J. L., 1976, How to do things with words, London : Oxford University Press
Barad, K. (2007). Meeting the universe halfway : Quantum physics and the entanglement of matter and meaning. Durham: Duke University Press.
Butler, J. 1993. Bodies that Matter. On the Discursive Limits of Sex. London and New York: Routledge. Butler, Judith. « Performativity’s Social Magic » chapter 7 in Bourdieu: A Critical Reader. Richard Shusterman (Editor). Critical Readers. Blackwell, 1998.
Chun, W. 2011, Programmed vision, MIT Press.
Chun,W. 2016, Updating to remain the same,MIT Press.
Hills, K. 2015, Networked affect, MIT Press
Hills, K. 1999, Digital sensations, University of Minnesota
Ihde, D. 2001, Bodies in technology, University of Minnesota
Landy, R.J. (1986). Drama therapy concepts and practices. Springfield, IL: Charles C. Thomas.
Manovitch, Lev 2002, The Language of New Media, Cambridge, Mass.: MIT Press.
Monteiro , S. 2017. The fabric of interface mobile, MIT Press
Munster, A, 2013, An Aesthesia of Networks: Conjunctive Experience in Art and Technology, MIT Press
Meijas , Ulises Ali, 2013, Off the Network, Electronic Mediations , University of Minnesota Press
Shaviro, S ,2003 , Connected, or, What it means to live in the network society, Minneapolis : University of Minnesota Press
Simanowski, R. 2011, Digital Art and MeaningReading Kinetic Poetry, Text Machines, Mapping Art, and Interactive Installations, University of Minnesota
Simondon,G. 1965-1966, Imagination and Invention, Puf 2008

Artworks, Artist, Mask

Best, S. 2013, Visualizing Feeling affect and the feminine avant-garde, I.B.Tauris; Reprint edition3
Ekman,U Ubiquitous Computing, Complexity and Culture, 2016 Edited by Ulrik Ekman, Jay David Bolter, Lily Díaz, Morten Søndergaard, Maria Engberg – Routledge
Sears A.Eldredge Mask Improvisation for Actor Training and Performance, the Compelling image, 1996. Northwestern University Press Evanston, Illinois
Ishii Hiroshi, Tangible Media Group.
Jeremijenko,N, 2007 the “Dangling String”
Jonze, S. 2013, Her. Film synopsis available at <https://en.wikipedia.org/wiki/Her_(film)>
Mondloch, K . 2010, Screens, Viewing Media Installation Art, University of Minnesota
Napier, A. (1996). Foreign bodies : Performance, art, and symbolic anthropology. Berkeley: University of California Press.
Oursler, T . Available at < http://tonyoursler.com/>
Perona , Francesca
Posch, Irene , the knitted Radio http://www.stitchingworlds.net/speculation/the-knitted-radio/
Rokeby, D, Very nervous System
Simanowski, R. 2011, Digital Art and Meaning Reading Kinetic Poetry, Text Machines, Mapping Art, and Interactive Installations
Thomas, R, Vision , 1970, comic character ,android, synthetic human from the Avengers, informations available at <http://marvelcinematicuniverse.wikia.com/wiki/Vision>

Technology
how can technology evolves to enrich the relation between human and machines, Hiroshi Ishii, about tangible interfaces available at <https://www.media.mit.edu/people/ISHII><https://www.fastcodesign.com/video/mit-legend-hiroshi-ishii-urges-tech-to-think-beyond-the-iphone/hPj1HLTA>
Embroidery and Related Manufacturing Techniques for Wearable Antennas: Challenges and Opportunities available at <http://www.mdpi.com/2079-9292/3/2/314/htm>
Williams, C. 2014,Knitted Antenna research project, available at <https://xxxclairewilliamsxxx.wordpress.com/electronic-textiles/>

Articles

Marc Jeannerod, M , 2003, De la vision à l’imagination available at <https://www.scienceshumaines.com/de-la-vision-a-l-imagination_fr_13502.html>
Lise Amy Hansen* and Andrew Morrison , Materializing Movement—Designing for Movement-based Digital Interaction Institute of Design, The Oslo School of Architecture and Design, Norway available at <http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/1245/614>
Justin Marshall, Jon Rogers and Jayne Wallace, Crafting the digital <https://github.com/DigiTransGlasgow/crafting_our_digital_futures/blob/master/_contributions/27.md>
Ford Morie , J, 2007. Performing in (virtual) spaces: Embodiment and being in virtual environments, International Journal of Performance Arts and Digital Media, Volume 3 Number 2 & 3. University of Southern California, available at <http://ict.usc.edu/pubs/Performing%20in%20(virtual)%20spaces%20-%20embodiment%20and%20being%20in%20virtual%20environments.pdf>
Heather C. Trepal-Wollenzier MEd & Kelly L. Wester MA (2002) The Use of Masks in Counseling, Journal of Clinical Activities, Assignments & Handouts in Psychotherapy Practice, 2:2, 123-130, DOI: 10.1300/J182v02n02_13 , available at <https://doi.org/10.1300/J182v02n02_13>

Things about Crafting the Digital

Make the invisible visible-crafting the digital

While researching about network embodiment , I came across an article written by  Justin Marshall, Jon Rogers and Jayne Wallace  . The authors  investigate how craft sensibility” can extend beyond the scope of material interaction, to engagement with people and situations as a broadly empathetic way of exploring the world. ” Their proposition is to “create a crafted digital future” .

In that matter, Irene Posch , artist and researcher, explore the integration of technological development into the fields of art and craft with projects like the knitted radio  and crafting logic (crafting computer)