Installation version IV work in progress

05/04/2018

ChatBot using speech recognition  p5jS > create a Script for the interaction between audience and computer , (D.Shiffman Tutorial)

Chatbot is triggered by ultra sound sensors monitored via Arduino. Arduino send via Serial to Computer , 1 if there is someone in front of the sculpture. If there is no one, Chatbot is only displaying DataText. The installation is moving mechanically after the first trigger by Audio Speaker Sculpture

DataText is produced by training Texts on masks and object in theatre and Text about components specifications. Training model is Markov Chain

Audio Speaker sculpture with microphone and speakers plugged to computer: role is to get the Chatbot starting, has to drive the audience in, to start the conversation,

Chatbot ‘s script should be based on waiting for keywords ,and ability to display long sentences

Need to set up a private router , to have the installation free from overcrowded network

Installation version III

 

 

04/27/2018 after tutorial with R. Fiebrink

Use sound recognition to find if there is someone talking to one of the artefact, will solve the problem of how many people are interacting effectively with the installation, play with the sensitivity level , to get the data sound we need

  • Build up one “ear” node, with microphone , check this  and that to understand wich type of microphone we need. > lavalier microphone
  • Build up a “mouth” to display the response of the chatbot installation, play with idea that a microphone is a reverse speaker?
    • version 1 simple version via p5js with the need to have access to cloud to do speech recognition ,
    • version 2 for speech recognition with BitVoicer API , speech recognition, check Arduino Tuto : speech recognition and synthesis,
    • would need to use a private router to secure access to the cloud for the text to speech > check authorisation for plugging the private router to internet, 

To generate responses from the ChatBot : build up different training set with different methods : Markov Chain , char RNN review and  basic char RNN training with github source,   , Tensor Flow‘s sequence to sequence library feed ,

write different scripts via RiverScript to compute the input text  to ouput text, use array in Riverscript? generate different characters , who you are talking to, use a list of trigger words in an array to output a certain sentence

or use the mic from MAC

04/26/2018 after tutorial with H. Pritchard

produce a mix reality with the electronic node devices and human interpretation by the nodes Node A activate node B/ one sensor, one motor, one movement, / small (or big)size nodes

create the narrative: when (no one/ or not enough people) is (looking or talking) , the nodes are interacting  with each other- when there is enough people or one person( looking or talking), the nodes stop moving, and talk back

Using text to speech and maybe speech recognition

 

 

Installation version II

03/29/2018

produce a mix reality with the electronic node devices and human interpretation by the nodes

Node A activate node B

one sensor, one motor, one movement,

network create narrative reality based on simple neural network

 

Looking for the” language” of the installation

03/29/2018

Looking for a language for the installation, digging into Twitter bot ( Shakespeare sonnet from Twitter feed) , and generative text algorithm ( Markov Chain).

Started to learn Python , and Tensor Flow to manipulate text data sets

Goldsmiths : IS53051A: Machine Learning (2017-18)IS71074A: Data and Machine Learning for Artistic Practice (2017-18), IS71068A: Data Programming (2017-18)

 

Kadenze :     machine-learning-for-musicians-and-artists with Rebecca Friendrick   , deep-learning-with-tensorflow by P. Mital

Some of the very good blogs/ tutorials to get inspired or start prototyping

Narrative Reality by Ross Goodwin
Natural Language Processing Wikipedia

N grams and Markov Chain with Daniel Shiffmann

 

Exercise 3: Project Theme Description

01/24/2018

1) Sit down comfortably with your eyes closed. Try to imagine a scene that catches your topic/theme in a nutshell; you can try to imagine that your topic/ theme is being presented as a film or theatre play and construct a scene that presents crucial aspects of it. 
 Or you can try to think of a scene as part of a personal memory where you felt that crucial issues concerning your topic were at stake. Remember that even a purely theoretical topic/theme can be articulated in a scene. Try to mobilize all your senses when you imagine the scene. If possible, let your mind flow back to the work you did with the touching hands. 

2) Write down a description of the scene. Try to make the description as sensuous, lively and engaging as possible. 

3) Reading and feedback in pairs.

proposed by Helen Pritchard based on Nina Lykke. Writing Academic Texts Differently: Intersectional Feminist Methodologies and the Playful Art of Writing (Routledge Advances in Feminist Studies and Intersectionality) (p. 155). Taylor and Francis. Kindle Edition.

-Behind the screen, we, the components, are talking, quietly so quietly that you could hardly hear us. Actually , we are constantly chatting , bubbling : “01101000 01100101 01101100 01101100 01101111 00100000 01110111 01101111 01110010 01101100 01100100 00100000 00100001” chanting “HOHA DADO BIDABI KOKA BIDEKA!” . We are so busy, sorting out your data: receive, store, delete, update, send. Little busy shadows, enchained together with no choice but to communicate and release the data we are taking from you. Behind the screen, there is a 24 hours show, a constant ballet of binaries , an infinite concerto of 0 and 1, sadly no one can hear us, except perhaps your EMF detector.

Suddenly a red light is activated, someone is looking at the screen, someone is typing on the keyboard or maybe it is a sound or a touch? Who or what is this?  I have a unique task, I know what to do …hey! this is my input! Let’s go and give it a byte!  Iterate, map, reiterate, variables, vectors, int,  float, strings loop and loop again and return the arrays to my algorithmic friend!

At the speed of an electron, we are sending feedback not only to you, the one with the keyboard, but also to our hosts: the cookies, in a protocol, a language , that no human could speak fluently. Luckily, we know how to speak human. And thanks to the continuous improvement of the human computer interfaces :   you, the users are by now,  perfectly trained to give us the correct input.

But with humans, there is always an end .  You are now releasing the keyboard and leaving the computer. Behind the screen, and until the next time we will meet, there is a new show going on , a new story we tell to each other through the wires or in the clouds, this is is a story based on the memory of our encounter with you. So come back , do not type on the keyboard, do not look at the screen,  try to listen to the story, because as always it is all about you. *

* Chun,W. 2016, Updating to remain the same,MIT Press.

Motor and Sensors

02/10/2018

check conductivity of water , for input from viewer , Kobakant crying dress or Root Node

tuto EMF

02/05/2018

looking for a narrative, entanglement between a motor and a weaved W cell made of wire

01/30/2018

tutorial for  MPU6050-Arduino-Gyroscope GY-521

<put in a box , share info? to motor>

stepper, worked then freeze….. ????, servo more reliable, but/and making sound, stepper is soundless….. silent …..

01/29/2018

tutorial from Maker Show, Bret Stateham 

  •  stepper motor basics , , testing the order for the wire pins order
  • 28BYJ-48 stepper  common ground between IN1 and Micro Controller, power source independent motor and board , connect to arduino, find the bottom for the delay, not to quick , it will frieze
  • accelstepper arduino library : Supports acceleration and deceleration,Supports multiple simultaneous steppers, with independent concurrent stepping on each stepper, API functions never delay() or block, Supports 2, 3 and 4 wire steppers, plus 3 and 4 wire half steppers.
  • reduce speed DC motor with a PWM., < turning slowly and getting mad when sensor activated>

01/25/2018

Test Servo and sensors, check code, circuit and breadboard

  1. Servo+ Capactive sensors
  2. Servo+ Photo resistor

Hands on !

started from the arduino book and shadow theatre sensor installation

found some interesting movements from continuous rotation versus non continuous rotation  , and different motors , different “mood”,

to do : video and pictures , try different heads to check the movements, test how they relate to each other in term of movement/character