Process

Ideation & sketches

Two Modes of Interaction​

*Exhibition Mode — The Bracelet​

In galleries, participants wear a discreet bracelet that captures biometric signals.​ As they stand in front of a screen or projection, their internal state generates a unique visual composition in real time.​

*Personal Mode — The Glove​

For intimate or customized use, the glove expands the system.

Bracalet

Glove

Design & Fabrication

Heartbit sensor

To connect the MAX30102 sensor involves an I2C interface, which allows the board to read oxygen saturation and heart rate data. The sensor is primarily designed to be used on parts of the body with thin skin and high blood flow, allowing the red and infrared light to effectively penetrate the tissue and reflect back to the photodetector, such as fingertip, wrist or earlob. It is required to use the SparkFun_MAX3010x_Sensor_Library in the Arduino IDE.

The circuit will be attached to a wristband to sense the heartbit of the user.

Pinout connection

MAX30102	Description	Xiao ESP32C3 pinv
GND	Ground	GND
VIN	Power	3.3v
SCL	I2C Clock	D5
SDA	I2C Data	D4

Code to read the heartbit signals

#include <Wire.h>
#include "MAX30105.h"
#include "heartRate.h"

MAX30105 particleSensor;

const byte RATE_SIZE = 4; // Increase for more averaging
byte rates[RATE_SIZE]; // Heart rates
byte rateSpot = 0;
long lastBeat = 0; // Time at which last beat occurred
float beatsPerMinute;
int beatAvg;

void setup() {
  Serial.begin(115200);
  if (!particleSensor.begin(Wire, I2C_SPEED_FAST)) {
    Serial.println("MAX30102 was not found.");
    while (1);
  }
  // Setup sensor to use red light for heart rate
  particleSensor.setup(); 
  particleSensor.setPulseAmplitudeRed(0x0A); // Turn red LED to low
}

void loop() {
  long irValue = particleSensor.getIR(); // Read IR value
  if (checkForBeat(irValue) == true) {
    // A beat is detected
    long delta = millis() - lastBeat;
    lastBeat = millis();
    beatsPerMinute = 60 / (delta / 1000.0);

    if (beatsPerMinute < 255 && beatsPerMinute > 20) {
      rates[rateSpot++] = (byte)beatsPerMinute;
      rateSpot %= RATE_SIZE;

      // Calculate average
      beatAvg = 0;
      for (byte x = 0 ; x < RATE_SIZE ; x++)
        beatAvg += rates[x];
      beatAvg /= RATE_SIZE;
    }
  }
  Serial.print("IR=");
  Serial.print(irValue);
  Serial.print(", BPM=");
  Serial.print(beatsPerMinute);
  Serial.print(", Avg BPM=");
  Serial.println(beatAvg);
}

The circuit will be attached to a wristband to sense the heartbit of the user.

Galvanic skin response sensor

Connecting a Galvanic Skin Response (GSR) sensor to the Seeed Studio XIAO ESP32S3 Sense involves connecting the sensor's analog output to an available ADC pin on the XIAO board and providing power. Since the XIAO ESP32S3 Sense operates at 3.3V, it is important to power the sensor with 3.3V and read from the appropriate analog pin.

The electrodes will be connected to the index and middle fingers.

Pinout connection

GSR sensor	Description	Xiao ESP32C3 pin
VCC	Power	3.3v
GND	Ground	GND
SIG Analog	Analog Signal	A0 or any A0-A10

Code to read the changes in skin resistance

#define GSR_PIN A0 // Pin connected to SIG on the GSR sensor

const int GSR_PIN = A0; // Connect Grove-GSR to A0
int sensorValue = 0;

void setup() {
  Serial.begin(115200); // Start serial communication
}

void loop() {
  sensorValue = analogRead(GSR_PIN); // Read analog value (0-4095)
  Serial.print("GSR Value: ");
  Serial.println(sensorValue);
  delay(500); // Read every 0.5 seconds
}

Galvanic skin response and Heartbeat sensors

I connected both the GSR and heartbeat sensors together to have more signals and data to work with in a more accurate way.

The electrodes will be connected to the index and middle fingers and the heartbeat sensor on the back of wrist.

Code to read the changes in both the skin resistance and heartbeat

#include <Wire.h>
#include "MAX30105.h"
#include "heartRate.h"

MAX30105 particleSensor;

// ================= HEART RATE =================
const byte RATE_SIZE = 8;
byte rates[RATE_SIZE];
byte rateSpot = 0;
long lastBeat = 0;

float beatsPerMinute;
int beatAvg = 0;

// ================= GSR =================
const int GSR_PIN = A0;
const int CALIBRATION_TIME = 15000;   // 15 seconds

int gsrMin = 4095;
int gsrMax = 0;
long gsrSum = 0;
long gsrSamples = 0;

int gsrBaseline = 0;
int gsrRange = 1;

bool calibrating = true;
unsigned long calibrationStart;

// ================= EMOTION =================
String currentEmotion = "";
String lastEmotion = "";

void setup()
{
  Serial.begin(115200);
  delay(1000);

  Serial.println("Initializing sensors...");

  if (!particleSensor.begin(Wire, I2C_SPEED_STANDARD))
  {
    Serial.println("MAX30102 not found. Check wiring.");
    while (1);
  }

  particleSensor.setup();
  particleSensor.setPulseAmplitudeRed(0x0A);
  particleSensor.setPulseAmplitudeIR(0x0A);

  pinMode(GSR_PIN, INPUT);

  calibrationStart = millis();
  Serial.println("Place fingers... Calibrating GSR for 15 seconds...");
}

void loop()
{
  int gsrValue = analogRead(GSR_PIN);

  // ================= CALIBRATION PHASE =================
  if (calibrating)
  {
    gsrSum += gsrValue;
    gsrSamples++;

    if (gsrValue < gsrMin) gsrMin = gsrValue;
    if (gsrValue > gsrMax) gsrMax = gsrValue;

    if (millis() - calibrationStart > CALIBRATION_TIME)
    {
      gsrBaseline = gsrSum / gsrSamples;
      gsrRange = gsrMax - gsrMin;

      if (gsrRange < 50) gsrRange = 50; // avoid too small range

      calibrating = false;

      Serial.println("Calibration complete.");
      Serial.print("Baseline: ");
      Serial.println(gsrBaseline);
      Serial.print("Range: ");
      Serial.println(gsrRange);
    }

    delay(20);
    return;
  }

  // ================= NORMALIZED GSR =================
  float gsrNormalized = (float)(gsrValue - gsrBaseline) / gsrRange;
  gsrNormalized = constrain(gsrNormalized, 0.0, 1.0);

  // ================= HEART RATE =================
  long irValue = particleSensor.getIR();

  if (checkForBeat(irValue))
  {
    long delta = millis() - lastBeat;
    lastBeat = millis();

    beatsPerMinute = 60 / (delta / 1000.0);

    if (beatsPerMinute > 40 && beatsPerMinute < 180)
    {
      rates[rateSpot++] = (byte)beatsPerMinute;
      rateSpot %= RATE_SIZE;

      beatAvg = 0;
      for (byte x = 0 ; x < RATE_SIZE ; x++)
        beatAvg += rates[x];

      beatAvg /= RATE_SIZE;

      currentEmotion = classifyEmotion(beatAvg, gsrNormalized);

      if (currentEmotion != lastEmotion)
      {
        Serial.print("EMOTION:");
        Serial.println(currentEmotion);
        lastEmotion = currentEmotion;
      }

      // Send structured data to Processing
      Serial.print("BPM:");
      Serial.print(beatAvg);
      Serial.print(",GSR_RAW:");
      Serial.print(gsrValue);
      Serial.print(",GSR_NORM:");
      Serial.print(gsrNormalized, 3);
      Serial.print(",STATE:");
      Serial.println(currentEmotion);
    }
  }

  delay(20);
}

// ================= EMOTION CLASSIFICATION =================
String classifyEmotion(int bpm, float gsrNorm)
{
  // LOW AROUSAL
  if (bpm < 65 && gsrNorm < 0.3)
    return "CALM";

  // RELAXED
  if (bpm < 80 && gsrNorm < 0.5)
    return "STABLE";

  // FOCUSED
  if (bpm < 95 && gsrNorm < 0.7)
    return "FOCUSED";

  // HIGH AROUSAL
  if (gsrNorm > 0.7 || bpm > 105)
    return "INTENSE";

  return "ELEVATED";
}

Challenges

Errors encountered

Before using the ESP32C3, I tested with the ESP32. When installing the ESP32 library I received the error:

4 DEADLINE_EXCEEDED error when installing the esp32:esp32:3.3.5

I solved the issue adding the next line to the file

C:\Users\.arduinoIDE\arduino-cli.yaml

network: connection_timeout: 1200s

After that the library was installed succesfully, however I found another error when I tried to upload the program to the board.

Failed uploading: no upload port provided

The port option was disable which is why I could not select any.

In the device manager I found CP2102 USB to UART bridge controller unavailable.

I downloaded and installed the driver from the following link

https://www.silabs.com/software-and-tools/usb-to-uart-bridge-vcp-drivers?tab=downloads

The issue was solved and I could select the port

Then, the following error appeared:

A fatal error occurred: Failed to connect to ESP32: Wrong boot mode detected (0x13)! The chip needs to be in download mode. For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html

Failed uploading: uploading error: exit status 2

References

Introducción a la Serie Seeed Studio XIAO ESP32S3

Getting started with XIAO-ESP32-S3-Sense

Getting Started With ESP32-C3 XIAO

How to Build a DIY WiFi Smart Oximeter Using MAX30102 and Arduino ESP32

Technical Specification datasheets

Seeed Studio XIAO ESP32C3 datasheet

Max30102 datasheet

Prototypes

Glove

GSR sensor output

Videos

Heartbit sensor test

GSR sensor test

Mood Paint

Mood Paint

## Mentoring notes ## Midterm Presentation Feedback

Anastasia the system could combine the heartbeat and galvanic with the persons hand movement, for example different colors are the heartbit, the galvanic value is the type of brush and the movement of the hand is the tracing on the lines (drawing) iti is important to explore the drawing part. your hardware can be simple and functional and the drawing can be developed further to be more attractive.

Claudia I recommend you checking out touchdesigner where you can learn how to connect the inputs of electronics through wifi in real time to custom visuals that collectively appear and interact. You can establish your own rules and guidelines within the software. See also related projects by artists that make use of those technologies in their practices. * https://www.youtube.com/shorts/Txzaid2XVow * https://www.youtube.com/watch?v=cI6uLbyJeUY * https://www.youtube.com/watch?v=FAtgupffVbA

Nuria Dear Emma, your project is evolving beautifully, and the concept is very strong, you’re translating internal states into visual expression in a way that feels poetic and accessible. As you move forward, I think it would really help to improve the photos and videos of your prototypes so the signals, reactions, and materials can be clearly appreciated. This will make your process and results much more understandable for others. I’m happy to help you work on capturing better visuals, so you can communicate the project with the clarity it deserves.You’re doing excellent work, keep going!

Lousi Emma, thank you for your presentation! I was wondering how are you going to relate the heartbeat and skin conductance with personal emotions? Maybe you have a chart? Will you map your paintings’ aesthetics depending on emotional states? It would be nice to see painting references for each emotional states.

Carolina Hi Emma, it's interesting that you created a relation with painting/screen. The glove could be adjustable so you can fit more people, but I agree that the bracelet is better for an exhibition. I´m curiouse with the final gadjet-art they can take home. Good work!

Maddie Olsen Some people have a hard time identifying and describing feelings they have, or their mind-body connection is not strong. While you admit it’s not informed by any medical/clinical data, it may be therapeutic and educational for those who struggle to interpret the way their body feels to the moods they have.

## Half-fabrication files [^1]: Test file: 3d modelling test