Using a physical object to control my web browser
In this lab for Intro to Physical Computing, I’m using a physical object to control what’s happening in my web browser. To do this, I’m applying what I learned about in the last lab – asynchronous serial communication – to send a flex sensor’s data through my microcontroller, serial port, Arduino code, and finally to my p5.js sketch.
It’s not very common to to control a web browser with external hardware via a laptop’s serial port. Personally, I don’t think I’ve come across this in my daily life. I’m curious as to why this isn’t always possible? I know “historical reasons” were mentioned in one of the ITP videos online. There also might not be enough daily applications to be worth building it into general consumer computers. And maybe it opens the door to nefarious activity?
But with this additional capability, I can add physical inputs from the world me into my visual coded projects in P5.js, Processing, Max/MSP, and/or OpenFrameworks.
Part I: Reading smaller sensor values that fit into 1 byte, with raw binary numbers
First, add code to your Arduino IDE to read your microcontroller
This is some simple code to send the value of a flex sensor to your serial monitor, using serial communication with the command Serial.write().
Second, prepare the P5.serialcontrol app and P5.js serialport library
Third, set up your P5.js sketch to connect with your microcontroller
Next I’ll add some code to my p5.js sketch so that it’s connected to my USB serial port and microcontroller.
But Dan Schiffman had sent our class some simpler code, which worked well:
Then I write this code below to ask for a list of available serial ports on my laptop. To do this, I first create an instance of the serialport library and set up a call back function to list my available ports.
Next, Use Events and Call Backs to Create Behavior
I lost all my text in this section of the blog! DARN!
Basically, I talked about how to set up my p5.js code to expect events from my serial port, and define call back functions to perform if those events happen. For example in this lab, if data comes in through the serial port, then perform a new behavior, such as display the incoming values on the screen. A simpler example in p5.js might be changing a ball from red to blue if I’ve clicked my mouse, because I’ve written a “call back” function that requires going to find additional code I’ve written to perform that new behavior.
What’s Happening Here?
I’m imagining this all as a relay race with a special baton with written words on it, which is really data. Each runner waits for the last runner to pass it the baton. But the runner needs to change the language of the baton’s words each time they receive it, so they can understand what it says!
In other words, the microcontroller sends bytes via serial communication using Serial.write(), which when the computer receives that byte, understands it a ‘data event’. This triggers the serialEvent() command in p5.js to be called, which stores the bytes into a variable called inData, at the same time turning it into a number. From there, the draw() function takes that number, and displays it on the web page.
Draw a Graph With the Sensor Values
I also lost this text. : (
Here, the sensor’s value is being mapped to the x position of the graph lines being drawn.
Part II: Reading larger sensor values that fit into more than 1 byte, with ASC-II encoded values
Aka reading serial data as a string
Because I’m using Serial.println(), extra bytes will be used to communicate a carriage return and line break in between each sensor value.
On the p5.js sketch side, I add the serial.readLine() command as the method of interpreting the serial data. This command is unique in that reads the incoming serial data as a string of bytes (not just one byte as with the serial.read() command that we used before). And, when that string happens to be all-numeric, it converts it to a number, which is useful for us as we want numbers to be able to display to the canvas.
However, at first this leads to an issue because the p5.js sketch will get confused when it reads the carriage returns and line breaks, which are sent in the ASCII-encoded language as either \r (for carriage return), or \n (for new line). When it reads those respective bytes, it displays nothing on the screen, which look like gaps in the graph or flickers in the text display sketches.
To circumvent this, you need to be very explicit with the p5.js program, and tell it to only display bytes coming in through the serial port that are actual ASCII-encoded numbers, and not characters. To do so, you add a function to the serial.Event call back function. Here’s the complete code.
I’m beginning to see how much effort has been put into creating commands that allow someone to switch between reading raw binary and ASC-II encoded values. For now, I’m guessing that I’d personally switch between the two when testing new sensors. I’m sure there are other typical applications? For example, I’d use Serial.write() when testing a new sensor’s range with a simple mapped range that fits into one byte. And then switch over to analogRead() to test applications of that new sensor. This is because I can now see how analogRead() in Arduino IDE and serial.readLine() in p5.js can quickly lead to needing more code to navigate interpretations of ASC-II encoded values.