OK, ROBOT
REBOOT (VOICE)
WEEK 7
A worm that uses peristaltic wave motion
and recoils when it is in proximity to humans
Develop a Semi-Autonomous Robot
and recoils when it is in proximity to humans
Develop a Semi-Autonomous Robot
This robot is semi-autonomous and nature-like. :) My goal with this project was to make a semi-autonomous robot that communicates consent through its movement. Its default state is to move linearly (peristaltic wave motion), but when it senses something is close, it will sometimes recoil by contracting to half its size. Other times, it will express comfort by engaging in a slightly more subtle wave before ultimately deciding to continue moving along. I wanted to create a robot that can reject human interaction. It’s more reflective of nature for a non-human species to be afraid of us (we are also afraid of us).
How it works:
• The mesh has 6 segments made of vertex joints with union 4mm push connectors, 1/8” poly tubing, and 1 motor per segment
• Circumferential cable -> wraps around the inside of the mesh → when the motor is spinning, it pulls the cord in, and the diameter of the worm decreases, and the segment will lengthen
• Longitudinal cable -> forms a zigzag line through the mesh → when the motor is spinning, the diameter increases, and the segment shortens
• Double spool that also serves as a mount for the motors -> moves the cables in opposite directions
• Peristaltic wave -> the 6 segments move sequentially, the contraction moves the robot forward
- ToF and IMU: measure how close a person is, and that triggers a recoil response
My own research on the subject has opened up a can of worms ;’) but reading about the advancements made in soft robotics has made me feel a little bit optimistic about our role in the world. It feels like we’re building on things for the greater good, instead of just for the sake of building BIGGER. A few inventions that come to mind are soft biomedical devices for supportive prosthetics and less invasive surgical tools. The research papers I used for this project (on the peristaltic wave motion of worms) demonstrate the worm’s ability to be malleable and adaptable enough to move through confined spaces. WALL-E could NEVER. There are also examples of other worm inventions that can navigate rougher terrain (think of a city after a hurricane has torn through it). Worm motion, coupled with a set of wheels, could allow a robot to roll through debris, fallen trees, and even climb at a steep incline. It could serve as a vital tool in search and rescue missions, and to carry supplies where cars and drones can't go. As an act of contrition, we have an opportunity to engage with the living world in a more responsible way. I like the idea that we can imagine our role in the world as one that moves with it and doesn't behave destructively for the sake of innovation.
Issues I encountered: The vertex pieces and motor mount from the research paper are verrryyyy specific (a lot of geometry that my brain couldn’t manage in the time I had). I ended up improvising by taking two 4x4mm push-connectors, HAND-modifying them, and using M3 screws to connect them. The idea is to make the vertex hinge a 30-40 degree difference. Also, the carrier board for the motors would intermittently not communicate with my Mac. I’m going to custom-design the vertex pieces and motor mounts this week and use a different carrier board.
References:
For this project, I’m using the mesh and peristaltic wave design by PhD students and postdoc/research engineers at Case Western Reserve University (Department of Mechanical and Aerospace Engineering, Biologically Inspired Robotics Lab). The details can be found in these papers:
WEEK 6
Two examples of Robotic Art
One direct and one indirect
One direct and one indirect
Can’t Help Myself, 2016 (Sun Yuan and Peng Yu)
Can't Help Myself is a giant robotic arm that’s programmed to sweep up dark oil that spills from its own body. Over the course of three years, we can watch footage of its slow decline and eventual “death” in 2019, where it finally succumbs to its own mechanical exhaustion.
If I had to guess, the robot is programmed to keep rotating so that it can keep visualizing the oil. I'm not sure if there is an "else if" programmed in it that might indicate there is no more oil to clean up, but I assume there is because, unbeknownst to the robot, the artists have designed it so that the oil never stops spilling. It just keeps cleaning, and it never gets the chance to reach the “else if” because that condition never arrives. And there’s no human perspective that comes in to reassess and decide if it should keep going.
What makes this feel like art is that we can immediately identify with the robot. It’s doing exactly what a robot would do, input -> output, but something is unsettling about the robot’s unawareness of the built-in, never-ending oil sabotage. Meaning, we know the mission is futile, but the robot doesn’t, and that is just sad.
It is such a robot thing to keep chipping away at the task, and a very human thing to feel the burden of blind belief, where we have to invoke our own resilience before eventually succumbing to our limitations. Also, at a base level, it feels like you are witnessing a living thing slowly die. Maybe when we look at the robot, it brings up our collective feelings of existential dread, where we project our own experiences in blind faith as well as our relentless devotion to survival.
If I had to guess, the robot is programmed to keep rotating so that it can keep visualizing the oil. I'm not sure if there is an "else if" programmed in it that might indicate there is no more oil to clean up, but I assume there is because, unbeknownst to the robot, the artists have designed it so that the oil never stops spilling. It just keeps cleaning, and it never gets the chance to reach the “else if” because that condition never arrives. And there’s no human perspective that comes in to reassess and decide if it should keep going.
What makes this feel like art is that we can immediately identify with the robot. It’s doing exactly what a robot would do, input -> output, but something is unsettling about the robot’s unawareness of the built-in, never-ending oil sabotage. Meaning, we know the mission is futile, but the robot doesn’t, and that is just sad.
It is such a robot thing to keep chipping away at the task, and a very human thing to feel the burden of blind belief, where we have to invoke our own resilience before eventually succumbing to our limitations. Also, at a base level, it feels like you are witnessing a living thing slowly die. Maybe when we look at the robot, it brings up our collective feelings of existential dread, where we project our own experiences in blind faith as well as our relentless devotion to survival.
Rhythm 0, 1974 (Marina Abramovic)
In Rhythm 0, the artist stands completely still in a room with various objects ranging from a rose to a loaded gun, inviting the audience to do whatever they want to her.
Rhythm 0, to me, is an example of indirect robotic art because in it she resigns herself to self-objectification. Aside from mirroring the participants, she sets out to act as somewhat of a catatonic puppet. For example, if they looked at her, she would then look back, and if they decided to move her arms, she’d slowly move theirs. And this is all with the understanding that there would be no intervention from her or the gallery.
As you can imagine, the participants let their curiosity guide them, as some of the acts involved the participants offering her a rose, and others involved committing sexual acts. Some cut off her clothes with razors, and eventually cut her skin. This would escalate to one participant picking up the loaded gun and raising it to her head. And because the audience had formed a protective barrier over the course of that evening, the group of protectors would eventually fight the gun from their hands.
This form of self-objectification opened a world of discussion on human predictability and the presence of "no consequences," where, much like in the Stanford prison experiment, they found that if you give humans unlimited control without consequences, there is a whole world of possibilities that you are exposing yourself to. We tend to build robots and objects for our own convenience and design them with the understanding that at some point, they'll be handled not with care or gentleness, but with brutality.
As you can imagine, the participants let their curiosity guide them, as some of the acts involved the participants offering her a rose, and others involved committing sexual acts. Some cut off her clothes with razors, and eventually cut her skin. This would escalate to one participant picking up the loaded gun and raising it to her head. And because the audience had formed a protective barrier over the course of that evening, the group of protectors would eventually fight the gun from their hands.
This form of self-objectification opened a world of discussion on human predictability and the presence of "no consequences," where, much like in the Stanford prison experiment, they found that if you give humans unlimited control without consequences, there is a whole world of possibilities that you are exposing yourself to. We tend to build robots and objects for our own convenience and design them with the understanding that at some point, they'll be handled not with care or gentleness, but with brutality.
WEEK 5
Radio(head) Robot
The Future-BOT: Choose an existing inanimate object and turn it into a robot.
radio.html
This week, I used stepper motors to create a voice-activated robot that plays 2 different Radiohead songs. The logic is: Once you open the .html file in Chrome, it prompts to connect to the Arduino Uno, then once the page loads, you’re able to click “Start Radio”. After this, it listens for “Hey Radio”. Once you say “Hey Radio”, it asks, “Do you want to hear a sad or anxiety-inducing song? It listens for “sad” or “anxious” and if you say “sad”, then the stepper motors play No Surprises by Radiohead, and if you say “anxious”, Everything in its Right Place plays.
After that, it listens for either “Hey Radio” or “Stop”, which stops the music. I also have a switch that acts as a master enabler, where if it’s in the off setting, nothing plays, but if it’s on, the radio robot is able to initiate the stepper motors.
TLDR: You speak → Web Speech API → JavaScript understands it ->
Web Serial API → Arduino Serial → Motors play music
In the .html,
I’m using things that don’t require any installs (all built-in browser tools):
-
the “Agnes” voice
- Speech Synthesis API
- Web Serial APIs (to communicate with the Arduino)
-
Web Speech APIs
- Vanilla JavaScript
- plain HTML/CSS
- the help of Claude. Claude helped in putting together the HTML portion of the project, and helped to solve the puzzle of figuring out how to not have the hardware switch override the Robot.
For the stepper motors: WHAT IS HAPPENING HERE??
(in the code and overall):
THERE ARE 6 FILES IN THE ARDUINO SKETCH.
I mainly did this to keep everything clean and compartmentalized. That way, if I need to update each song's details, I can just pull it up and not worry about breaking anything.
1) okrobot.ino:
- Declares the pin arrays for STEP and DIR (X,Y,Z).
- Chooses the play/stop input pin and the LED pin.
- Calls the library setup, then keeps the engine running in loop().
2) okrobot.h: - A list of functions that the main Arduino sketch calls to control the stepper music: like setup, start/stop, run loop, optional LED attach.Keeps things clear so the .cpp can be edited safely.
3) okrobot.cpp:
- Implements the playback stuff.
- Handles all the playback/timing for the motors.
- Each motor stores its own info: pins, current note, and timing.
- Uses two clocks: microseconds for fast steps and milliseconds for note lengths.
- Switch uses INPUT_PULLUP: ON = enables voice control, OFF = killswitch (but ON doesn't automatically start music)
- The switch is wired to Abort (A0) to stop playback (the robot starts it).
- The EN pin is handled by the code (it turns the stepper drivers on while playing and off when stopped).
- Clear variable names (STEP_PINS, DIR_PINS, TRACKS) make editing easier for me (lol).
- LED blinks SOS with a small timing pattern that runs in the background.
4) 5) 6) everything.h, nosurprises.h, okrobot_tonesglue.h
- The song files are where I keep all the musical info (frequencies, length) for each motor.
- The tonesglue file: this needs to come before the individual song note files. They basically say “Here are all the musical notes that will be played,” so the files themselves know what to play.
radio.html
This week, I used stepper motors to create a voice-activated robot that plays 2 different Radiohead songs. The logic is: Once you open the .html file in Chrome, it prompts to connect to the Arduino Uno, then once the page loads, you’re able to click “Start Radio”. After this, it listens for “Hey Radio”. Once you say “Hey Radio”, it asks, “Do you want to hear a sad or anxiety-inducing song? It listens for “sad” or “anxious” and if you say “sad”, then the stepper motors play No Surprises by Radiohead, and if you say “anxious”, Everything in its Right Place plays.
After that, it listens for either “Hey Radio” or “Stop”, which stops the music. I also have a switch that acts as a master enabler, where if it’s in the off setting, nothing plays, but if it’s on, the radio robot is able to initiate the stepper motors.
TLDR: You speak → Web Speech API → JavaScript understands it -> Web Serial API → Arduino Serial → Motors play music
In the .html, I’m using things that don’t require any installs (all built-in browser tools):
- the “Agnes” voice
- Speech Synthesis API
- Web Serial APIs (to communicate with the Arduino)
- Web Speech APIs
- Vanilla JavaScript
- plain HTML/CSS
- the help of Claude. Claude helped in putting together the HTML portion of the project, and helped to solve the puzzle of figuring out how to not have the hardware switch override the Robot.
For the stepper motors: WHAT IS HAPPENING HERE??
(in the code and overall):
THERE ARE 6 FILES IN THE ARDUINO SKETCH. (in the code and overall):
I mainly did this to keep everything clean and compartmentalized. That way, if I need to update each song's details, I can just pull it up and not worry about breaking anything.
1) okrobot.ino:
- Declares the pin arrays for STEP and DIR (X,Y,Z).
- Chooses the play/stop input pin and the LED pin.
- Calls the library setup, then keeps the engine running in loop().
- A list of functions that the main Arduino sketch calls to control the stepper music: like setup, start/stop, run loop, optional LED attach.Keeps things clear so the .cpp can be edited safely.
- Implements the playback stuff.
- Handles all the playback/timing for the motors.
- Each motor stores its own info: pins, current note, and timing.
- Uses two clocks: microseconds for fast steps and milliseconds for note lengths.
- Switch uses INPUT_PULLUP: ON = enables voice control, OFF = killswitch (but ON doesn't automatically start music)
- The switch is wired to Abort (A0) to stop playback (the robot starts it).
- The EN pin is handled by the code (it turns the stepper drivers on while playing and off when stopped).
- Clear variable names (STEP_PINS, DIR_PINS, TRACKS) make editing easier for me (lol).
- LED blinks SOS with a small timing pattern that runs in the background.
- The song files are where I keep all the musical info (frequencies, length) for each motor.
- The tonesglue file: this needs to come before the individual song note files. They basically say “Here are all the musical notes that will be played,” so the files themselves know what to play.
WEEK 3
Passive-aggressive IVR Experience
Imperfect robot interaction that uses speech-to-text (STT)
thebank.html
This week, May and I built an IVR system called “The Bank.”
It uses speech-to-text (STT) as the input and browser-based speech synthesis as output/voice to simulate a very imperfect customer service exchange (the one you’d normally have before speaking with a human). This “agent” is a gatekeeper in many ways! The thing that makes this system imperfect is that the IVR agent never gets your name right, but rather they can only understand your name to be “Maxine”, and they don’t allow you to correct them. They do eventually pass you on to a representative, but you’ll most likely need to restate all of your information. Also, it’s a funny idea that a robot would impersonate someone by calling. I guess crazier things have happened.
Constraints:
- Uses existing web technologies: browser SpeechRecognition API (voice input/output)
- Runs entirely in the browser
- Voice is the primary input method
- Has multiple exchanges (name, number, riddle)
What is the user expectation for this experience?
How does the BOT's personality influence the interaction?
How does the implemented technology enhance or constrain the experience?
The caller expects efficiency and speed, but the limitations we set (speech recognition errors, forced flow) make the system inefficient and a waste of time. I think this reflects a lot of our experiences with IVR technology; it’s meant to save time for both the caller and the company, but it often has the opposite effect. I find Alex’s voice to be cloyingly neutral and yet also passive aggressive; it’s somehow managing to hit all of the markers of a “helper” by repeating what they incorrectly heard, but never asking, “Did I get that right?”
How does the implemented technology enhance or constrain the experience?
The caller expects efficiency and speed, but the limitations we set (speech recognition errors, forced flow) make the system inefficient and a waste of time. I think this reflects a lot of our experiences with IVR technology; it’s meant to save time for both the caller and the company, but it often has the opposite effect. I find Alex’s voice to be cloyingly neutral and yet also passive aggressive; it’s somehow managing to hit all of the markers of a “helper” by repeating what they incorrectly heard, but never asking, “Did I get that right?”
WEEK 2
A possessive health check-in experience.
Command-line interaction that uses text-to-speech
ritual.mjs
I used Eleven Labs to clone my voice and then customized a ritual.mjs file to engage in a possessive “health check-in experience”.
Constraints:
I used Eleven Labs to clone my voice and then customized a ritual.mjs file to engage in a possessive “health check-in experience”.
Constraints:
- Takes text inputs and delivers speech outputs.
- Triggered on the terminal with a single command and runs without a GUI.
VOICE TRAINING:
- In Eleven Labs, I used around 4 voice memos. In two of them, I just rambled, and in the other two, I read a few excerpts from this Reddit post, where folks offer scripts that help to optimize the cloning process. Let me tell you, this clone sounds exactly like me, and it’s extremely bizarre listening to it.
- I grabbed an API key as well as a Voice ID key, both of which I used in the ritual.mjs file.
- CODE: A combination of this Medium article by Ivan Koop, assistance from Claude, as well as node.js stuff.
Does this interaction serve any purpose?
Can you make this interaction more playful?
How does the technology enhance or constrain the experience?
This interaction is meant to read as a practical health check-in assistant bot. While previously the chatbot may have engaged in an impersonal way, this time it's starting to show possessiveness and judgment. I think I meant it to be funny due to the uncanniness and how it weaves in and out of practicality and insubordination. After hearing my own voice back, I ended up feeling weirdly comforted by the chat's possessiveness. It made me think to myself, “wow, this voice is obsessively caring for me... in the way I should be”, which was an unexpected feeling.