Tangible Interaction Workshop – Lighting Controller

IMG_1635

This is a wireless controller for the type of stage lights found in many performance venues.  The controller selects colors, changes the brightness, and turns the light on and off. The enclosure itself has a strong visual design that doubles as a control panel, suggesting to users how to interact with a circular soft potentiometer, a sensor not familiar to most people.

Tools: Soft potentiometer and rotary encoder, Arduino MKR1000 microcontroller, Tom Igoe’s sACNSource.h library, and vinyl adhesive sheets.

Demonstrating the controller’s color selection feature.

How It Works

The user’s interactions with the tangible sensors causes analog and digital signals to be sent to the Arduino microcontroller. Code saved on the microcontroller runs logic to determine what to do based on these signals, and then stores answers that are sent via wifi to lights using the DMX protocol (the standard language used to communicate with these standard lights found in performance and retail spaces).

IMG_1624

The circuitry inside.

Creative Process

Prototype 1 – Test Connectivity

  • Ran generic tests to guarantee connectivity to existing wifi and lights systems. Used simple code to connect microcontroller to private WiFi network and send DMX commands to stage lights.

Prototype 2 – Draft Pseudo Code & Test Features

  • Drafted code’s logic and structure to realize new designs for lighting controller.
  • Got help from Tom Igoe doing office hours to clarify understanding of rotary encoder’s behavior and how data is compiled to be sent as packets.
  • Obtained clean readings from sensors & tested new features by implementing one block of code at a time to isolate bugs and ensure smooth progress.

Final Prototype – Write Final Code & Fabricate

  • Controlled lights and fabricated enclosure.

 

Challenges & Next Time

I’d like to understand why my all of my controller’s features work when powered by my laptop, but only half work when powered by a LiPo battery (I used a 3.7v 2000mAh battery, which should be enough but I will have to find out). I also have some flow issues in the structure of my code, so that while a command exists to entirely shut off the lights, it’s not being activated. I hope to completely finish this controller and the MIDI music controller this spring.

Code

Capacitive Touch Sensor Report: Breakout Boards

This report is part of a Sensor Library developed by Tom Igoe’s Tangible Interactions Workshop course at NYU’s ITP program.

Below you can read my report on capacitive touch and a few breakout boards.

You can view mine and my classmates’ reports on the course’s syllabus site over here.

Contents

This sensor report covers capacitive touch & compares a few breakout boards. I created a chart and went into more detail regarding a specific breakout board that is simple & reliable.

  • What is Capacitive Touch?
  • Pros – What is it good for?
  • Cons – Why buy a breakout board if you can DIY?
  • Comparison Chart
  • In-depth look at Sparkfun’s Breakout Board for Capacitive Touch using Microchip’s AT42QT101X (1010 or 1011)
  • Sample Code
  • References

animated (7)

One example of capacitive touch in action. This simple breakout board is covered below.

animated (2)

Behind a possible enclosure of acrylic. You can disconnect the LED if you like.

What is Capacitive Touch?
Capacitive Touch measures anything conductive or insulating, most commonly human touch. Capacitive Touch is great for both creative and useful applications because it works with many different materials, either in the form of the capacitive touch sensors themselves or paired with unique enclosures. This is because as mentioned, capacitive touch can measure anything that is either conductive or more insulating than air, even through an enclosure.

If your project has a lot of inputs, using a breakout board can help a lot. Here are a couple recent ITP projects that use cap touch. …Imagine if you wanted to have 20 cap touch buttons, moss controllers, or 20 sound lamps. A breakout board or two would come in handy.

Examples

Uses cap touch!

This interactive is by Brett Peterson, ITP alum, as part of the Senses exhibit at the American Museum of Natural History. Visitors can arrange puzzle pieces to teach a computer how to recognize imagery.

Capacitive touch sensors are used underneath the projected controls and table top surface, providing a very streamlined appearance.  As you can see, the sensors work most of the time, but not as regularly as arcade buttons might. 

American Museum of Natural History's interactive for machine learning.

Projector and web cam as part of interactive.

ITP Project page for Chelsea Pfohl’s Moss Touch. This uses capacitive sensors to control color and sound in a screen-based but 3D world.

 

ITP Project page for Yining Shi’s Atmospheric Sound Lamp. This uses cap touch to control light and sound of physical light sources.

 

How does it work?
Capacitive Touch works because the body’s water content is enough for a single finger or hand to change the electrical field surrounding two plates inside a capacitive sensor. The two plates can be thought of as antennae that senses changing electrodes.  The change in electrical field caused by a finger results in a reading that you can use in your code to trigger whatever event you’d like.

To clarify, the user can be near or far away from actual capacitive sensor itself. You can move the electrode, the pad that receives the user’s input, as far away as you like. It all depends on your materials and wiring.

Pros – What is it good for?
As for benefits, there are a few. Capacitive touch:

  • Allows your user to give inputs with something other than a typical button or slider — but still physically touch something (which is more intuitive than gesturing in the air).
  • Replaces mechanical components such as buttons or knobs which can wear and tear over time.
  • Can provide a more streamlined or unique appearance to your project’s enclosures & user inputs.

Cons – Why buy a breakout board if you can DIY?

Sure, for a one week assignment, you can DIY your electrode with copper, foil, velostat and conductive fabric, etc. And yes, you can use the Arduino Captouch library and put off buying a breakout board.

But for a larger project, you’ll want to spend a little money because your time is better spent figuring out the higher-level or trickier parts of your project, instead of debugging what you did by hand.  While yes, you can make your own quick electrodes (foil, copper, fruit),  by using a manufactured electrode and/or breakout board you can trust it will all work reliably. And using the Arduino Captouch library is good, but plan to only use an Arduino Uno, as the library is only written for that microcontroller (until it’s updated).

 

Comparison Chart of a Few Options

Name Product Image Price Inputs Chip & Maker Set Up Why You Might Want To Use It
Sparkfun’s Breakout Board AT42Qt101X $5.95 1 digital input AT42Qt101X by Microchip Microcontroller with GPIO (just input and output pins) Peace of mind. The board will reliably get cap touch readings.Built-in Button. It’s ready to go with no extra fabricating by you.

Aesthetic reasons.You may want a flat, streamlined look for your control panel or enclosure.

Adafruit’sCAP1188 – 8-Key Capacitive Touch Sensor Breakout – I2C or SPI

(Similar to the Adafruit 12-Key Capacitive Touch Sensor Breakout – MPR121)

 CAP1188 - 8-Key Capacitive Touch Sensor Breakout $7.95 8 or over 40 digital inputs if you create separate addresses CAP1188 by Microchip Microcontroller with I2C or SPI Peace of mind. The board will reliably get cap touch readings.Provides a lot of inputs.

Visual feedback.Built-in LEDs give you feedback while debugging lots of inputs.

Bare Conductive’s Pi Cap $39.95 12 digital inputs MPR121 by ROHRS Raspberry Pi with Bare Conductive’s Libraries If you want to use a Raspberry Pi. Let users use capacitive touch to interact with the internet, video file, interlocking programs, or calculations, thanks to the Raspberry Pi.
Captouch Library for Arduino Also see this page. 13 digital inputs Only works with the Arduino Uno, until the library is updated. No waiting.If you have an Arduino Uno, you can start right away without a special breakout board.

In-depth Report:

Sparkfun’s Breakout Board for Capacitive Touch using Microchip’s AT42QT101X (1010 or 1011)

animated (7)

The board in action. No mechanical parts due to the magic of capacitive touch.

What is it?  What does it do well?

Sparkfun’s breakout board is a cap touch button in and of itself. You don’t have to build your own electrode/button, unless you want to. It works well as a replacement for a key press.

What is it bad at?
Sparkfun and the chip maker say it’s built for proximity sensor readings. However, this barely worked for me, even on its most sensitive mode. To a regular user, it’s actually harder to hover 1 cm above the pad (proximity) then actually press it (touch).

animated (5)

Not great at proximity readings.

But on a related note, the board’s readings can be switched from Fast Mode to Slow Mode. Although a pain, you can move the solder on the back of the chip from Fast Mode to Slow Mode. Slow Mode is helpful if you want to use less power by sending fewer frequencies to the electrode’s pad, but be sure to test that it’s still sensitive enough to get the readings you want.

What does it NOT do?
Measure how hard the user is pressing!

Tips to Avoid Debugging

  • If you wire several of them together, be sure to figure out how to sync their clocks together.
  • If you wire one or more into a larger circuits, don’t share power. This might give you strange readings. Add a voltage regulator so that this board is protected from changes in power.

But how does it REALLY work? 
The AT42QT101X chip reads the conductive area in order to detect “touches” by a user. It does this by sending high frequencies to the pad. This technology is called “digital burst mode charge transfer” or “spread spectrum charge transfer”. If a finger is pressed, the pad changes the shape of the pulses. When the chip detects these slight changes it raises the OUT line to HIGH indicating that a touch is present. The amount of time you can say it’s “on” depends on whether you’re using the 1010 or 1011 chips (with the 1010 turns off after 60 seconds of being “on”; the 1011 stays on as long as the user presses it).

What would you use it for? You might use it in things like:

  • Control panels
  • Lighting controls
  • Anywhere a mechanical switch or button may be found
  • Large installations
  • The makers say it’s also good for toys.

You might also find it solves your design needs, such as:

  • For aesthetic reasons, you might want to create a streamlined look for your user input button behavior by hiding these underneath material.
  • For physically large projects, you can have your user be far away from your microcontroller when giving inputs.
  • For peace of mind, you can isolate your inputs into this specialized board so they’re more likely to work consistently (over creating your own).

Enclosure Ideas – What works or not?
You probably want to cover it up as part of an ITP project. The makers say it should work with a piece of acrylic (⅛” or thinner), cardboard, stone, ceramic, and most kinds of wood. I tested as many of these as I could. Some worked, others didn’t.

 

animated (2) animated (8) animated (3)
Works – Acrylic. Works – Mat board Works – Metal by itself
animated (6) animated (1)
: ( Doesn’t work – Thin wood. : ( Doesn’t work – Cardboard, surprisingly.

 

What code can I use for the AT42QT101X?

Sparkfun’s Shawn Hymel provides sample code you can start with, see below. You can also find it as part of the entire hook-up guide here.

/*
12-23-2013
SparkFun Electronics 2013
Shawn Hymel

This code is public domain but you buy me a beer if you use this
and we meet someday (Beerware license).

Description:

This sketch shows how to use the SparkFun AT42QT101X Breakout
Board. If you touch the Capacitive Touch area on the breakout
board, the LED attached to the Arduino will light up (in addition
to the LED on the AT42QT101X breakout board).

Simply connect power and ground to the breakout board,
and the AT42QT101X handles all the capacitive touch functions.
By default, the board will light up the green LED when the pad
is touched. A wire may also be connected from OUT on the
breakout board to a digital input pin on an Arduino. This signal
is normally LOW but goes HIGH on a touch detection.

The “LED Enable” solder jumper may be de-soldered in order to
control the LED directly from the LED pin. This is useful if you
want to light up a button that the user needs to push.

Hardware connections:

Uno Pin AT42QT101X Board Function

+5V VDD Power supply
GND GND Ground
2 OUT Capacitive touch state output
*/

// Constants
const int TOUCH_BUTTON_PIN = 2; // Input pin for touch state
const int LED_PIN = 13; // Pin number for LED

// Global Variables
int buttonState = 0; // Variable for reading button

void setup() {

// Configure button pin as input
pinMode(TOUCH_BUTTON_PIN, INPUT);

// Configure LED pin as output
pinMode(LED_PIN, OUTPUT);

}

void loop() {

// Read the state of the capacitive touch board
buttonState = digitalRead(TOUCH_BUTTON_PIN);

// If a touch is detected, turn on the LED
if (buttonState == HIGH) {
digitalWrite(LED_PIN, HIGH);
} else {
digitalWrite(LED_PIN, LOW);
}
}

Here’s the link to the AT42QT101X hook-up guide from Sparkfun:

https://learn.sparkfun.com/tutorials/at42qt101x-capacitive-touch-breakout-hookup-guide

References

Allaboutcircuit’s overview of capacitive touch:

https://www.allaboutcircuits.com/technical-articles/introduction-to-capacitive-touch-sensing/

Texas Instrument’s overview of capacitive touch:

http://www.ti.com/lit/an/snoa927/snoa927.pdf

EE Times’ overview of capacity touch:

https://www.eetimes.com/document.asp?doc_id=1279563

Sparkfun’s hook-up guide for the AT42QT101X:

https://learn.sparkfun.com/tutorials/at42qt101x-capacitive-touch-breakout-hookup-guide

Adafruit’s tutorial for their CAP1188 Breakout board:

https://learn.adafruit.com/adafruit-cap1188-breakout/downloads

Bare Conductive’s information and directions for their Pi Cap:

https://www.bareconductive.com/shop/pi-cap/

Paul Stoffregen and Paul Badger’s CapacitiveSensor library:

https://github.com/PaulStoffregen/CapacitiveSensor

https://playground.arduino.cc/Main/CapacitiveSensor?from=Main.CapSense

Tangible Interaction Workshop – MIDI Music Instrument

For this assignment, we created musical instruments that use the MIDI protocol. Mine combines melody and percussion, and frees up the musician to use their voice if they wish. I’m imagining that lead vocalists in bands may want an instrument like this. If this prototype were to become wireless and portable, they would be free to leave their set-up on stage to move around. It could also work just as well for other musicians whether accompanying themselves, playing in an ensemble, or backing up another musician.

IMG_1246.jpg

The soft pot is for volume, and the buttons on the right are for choosing pitches. The orange shape signifies where to strike with your drum stick. 

IMG_1260

Inspiration

I was inspired by two things: One, having seen many musician friends and colleagues perform over the years while I managed and produced live music programs and concerts. I admire anyone on stage, but especially those who are up front most engaging with the audience while also playing instruments. I thought it might be fun to create something they could use that amplified how they communicate energy to the audience, all while singing and moving around the stage. And two, I’ve always loved how much sound even a small hand held instrument adds to a band. Not only that, but they usually call for such a specific gesture that the audience can really see and enjoy.

I was also inspired by these handheld instruments from across the African Diaspora and Latin America. They each add their own unique sound and specific gesture to what’s happening on stage. I admired these and thought I would make something that builds on their shape and purpose.

claves.jpeg

Claves, Afro-Cuban

 

guiro

Guiro, Puerto Rican, Cuban and other forms of Latin American music

 

agbe

Agbe, African

 

Assignment from Class

Make a device to control playback of synthesized or pre-recorded music using MIDI. You do not have to build the playback engine, just the physical control device. Your device will send MIDI messages to another device which will play the music.

Your device should support the following features:

    • start and stop the playing of a note
    • play multiple notes simultaneously
    • sustain a note
    • pitch-bend a note

Make a housing for your controller. Document it according to the project documentation guidelines at the end of this page.

Design Process

First I thought of a few ideas. At first I played it safe and was going to make a guitar with buttons. But I decided to challenge myself to make an instrument that doesn’t quite exist yet.

I bought my materials, including some cool buttons, wire connectors (these are amazing – no soldering to your buttons!), and containers from where else, but the Container Store. Thanks to Nick, my classmate, for keeping it real and suggesting I just get an off the shelf container rather than make my own at this stage.

Here’s a diagram of the enclosure.

[uploading]

 

Circuit Process

As usual, I started small so that I could get simple sensor readings from my five buttons, shake sensor, and soft pot.

IMG_1216.JPG

Testing MIDI Code & MIDI Hardware Set Up

After that, I started working on code to connect those readings to MIDI messages. As an easy start, I tested my circuit with code provided by Tom Igoe to see if I was getting the expected serial monitor messages. Then I set up MIDI hardware with the example code, to see if I had connected the equipment properly. This took a few tries.

Creating My Own Code

Once I had example code and a hardware set up working, I started to try and write my own code that would allow the musician to use the instrument as I had in mind. My takeaway is that I learned a lot deciding for myself how to organize and structure my code. But despite getting lots of help, I hit a wall and will need some more help finishing it up.

I did accomplish a lot, though. For example, I wanted the built-in notes to be a normal C major scale, not half tones as in the Github example. I also wanted the musician to be able to control the volume by pressing up or down the soft pot, which meant passing that value into the MIDI message function, and then remembering it using state change detection. That way the musician could press once and let go, and not have to continually hold the volume soft pot sensor while pressing other notes.

Essentially I created a template function to run on every sensor input, which requires the three values of the static pin, static MIDI pitch value, and changing volume value. I wrote separate logic to capture the soft pot’s value, map it to MIDI’s volume range of 0 – 127, and use state change detection to save the latest reading. I funneled the three values into the master MIDI function which actually sends the data using Serial1.write.

At the moment, my on and off commands are mixing up, and I don’t have enough state change detection written to change the state of those on/off commands.

You can see my code at the bottom.

Fabrication Process

Fabricating was done mostly by hand, as the circular enclosure meant I couldn’t use the laser cutter. First I taped on paper and used a pen to mark the outside locations of my panel controls, and the inside locations of the breadboards and battery on the inside. I used a drill to make holes. And I found that colored adhesive vinyl sheets get across a lot of personality and even can serve as control panel instructions.

IMG_1224

Proof It Worked

I had a mostly working circuit just before enclosing it  & trying to add more state detection change. In addition to needing to work on the code, I need to make sure the shake sensor data really works with a drumstick.  But proof it worked!

Final Code Documentation

Here’s my final code, as of now. I have a few quirks I look forward to figuring out, maybe as my last project in the class.

 

Tangible Interaction Workshop – Lunar Landing Game Controller

For this first assignment, we were asked to create a game controller for Atari’s 1979 arcade game Lunar Lander. You can play the game yourself online. My controller is below.

IMG_0947

My game controller plays with the idea of taking on a character before starting the game. Will you choose to be an astronaut or alien? Perhaps this will change your decisions, or add more of a story to the interaction. This is represented in the visual language of the panel, and a small switch that the user can toggle between either character. In a further prototype, I would change the code a bit to have the alien behave differently than the astronaut.

In an even more ideal scenario, I’d like users to pick between two head pieces, which are the enclosures for the game controllers. The choices are either an astronaut’s helmet or an alien’s head.  Both have controllers on the sides of the head for up, down, thrust, and start.  Maybe the two players battle against each other.

 

Assignment: Game controller with HID output

  • Send up, down, left, right keypresses, or W, A, S, D keypresses as a USB keyboard
  • Send mouse clicks
  • Move mouse in X and Y direction to desired location to click
  • Your controls should be arranged so that the player can watch the screen, not concentrate on the controls
  • Make a housing for your controller.

 

Background

In this assignment we take advantage of standard technical protocols written for HIDs (Human Interface Devices) such as keyboards and mice. But we get to replace the keyboard and mouse with a reimagined interface using tangible sensors of our choosing. This way, the user can press a more interesting type of sensor but still be able to interact with content on a computer, such as the web browser.

Shout out to the Arduino libraries written for this keyboard and mouse functionality. The libraries have a lot of power. For example, you don’t want to take over the mouse or keyboard and accidentally write random code into your  program! That’s what happened right before this.

But here’s the button behaving as a mouse as I actually designed it.

 

Interaction

Me playing a good old game of Lunar Lander. It’s hard to play while filming yourself.

 

Diagram

In addition to this diagram, I created little icons to put inside the windows of the arcade buttons.

GameControllerPanelDiagram.png

 

I made more finished cut outs, but you can see a draft below.

Sound on. Very satisfying.

 

Schematic

Note: I used a MKR1000 microcontroller, not an Uno, to be able to access the keyboard and mouse using HID protocol.

 

TIWGameController_schem.jpg

 

Prototyping & Fabrication 

First I created a simple circuit and slowly built my code’s functions one button at a time.

IMG_0917

I used a bamboo box found at the Container Store that comes without a lid. To create a lid, I first prototyped with paper templates.

IMG_0918

Then moved on to cardboard.

IMG_0952

Then, thanks to Keerthana, I was able to add this fancy black acrylic panel. Here you can see that I had to raise the panel higher than the box. I did this by layering pieces of cardboard. This was necessary because the arcade buttons were a quarter inch taller than the box itself… I had to raise up the panel to accommodate them. I also didn’t have a fourth white button, so had to use a blue button for the start. With limited materials, I made the best of it.

IMG_0932

 

As a note to myself, these are the settings I used on the 75 watt laser cutter.

Acrylic: 1/4 inch think acrylic.  Etch: 600 dpi, 85 speed, 60 power. I went 5 digits slower in speed than the recommended 90, this must be why my engravings are deeper. Cut: 100 frequency, 3 speed, 100 power.

Cardboard: Regular amazon cardboard. Etch: 90 speed, 10 power. Cut: 30 power, 60 speed, 10 frequency.

Bonus

It would be cool to light the entire panel from below. I’m halfway there with these light-up buttons.

IMG_0945

 

Code

 

References – Tech and Fabrication

https://www.arduino.cc/en/Reference/KeyboardModifiers

https://www.arduino.cc/en/Tutorial/KeyboardAndMouseControl

Tom Igoe’s code shared in class:

#include “Keyboard.h”

void setup() {
Keyboard.begin();
pinMode(5, INPUT_PULLUP);
pinMode(4, OUTPUT);

}

void loop() {
if (digitalRead(5) == LOW) {
digitalWrite(4, HIGH);
Keyboard.write(“u”);
//Keyboard.write(KEY_UP_ARROW);
} else {
digitalWrite(4, LOW);
}
}
References – Cultural Background

Here’s a version of the original game, reviewed on youtube. Lunar Lander games are a genre of first person games in which the player controls the landing of a spaceship. As a fun fact, the first version was written by a high school student at the time.

 

Check out the original panel! The control for thrust looks like the most realistic control one might find in a space ship. The rest are a little harder to interpret. It looks like there are two settings for left movement and two for right movement, which is something the mouse and arrow keys do not afford.

lunarlanderpanel

 

The original enclosure – an arcade unit. Great graphics.

lunarlanderunit