Daniel Dimoski
Colin Foe-Parker
Ramya Iyer
The semester is coming to an end. Snow is in the forecast and it is crunch time. We have been tasked to develop a "haptic instrument." A device that not only makes sound, but also imparts "feel" to the user. As Prof. Gillespie keeps mentioning, electronics are taking over. However, people are loosing the ability to connect with objects. This project is an attempt to remedy this trend. We want to combine the relatively boundless opportunities that electronics and software present with the feel, touch, and react experiences that are dominant in the physical world. By combining these two domains, new and better devices will undoubtedly develop.
Team Five, (Dan Colin and Ramya), has decided to do something novel. While we are pushing the bounds of the electric / physical interface we might as well push the boundaries farther and create something completely new. (Rather than imitating an instrument already in existance) So, we have been throwing ideas around. A lot of our inspirations so far have come from stringed instruments from multiple continents and cultures.
Musical System:
(A little farther in our design process....)
For our instrument, we decided on combining many of the individual elements that we learned in class. Specifically how to utilize, fader motors, H-bridges, microprocessors, Pure Data, Mosfets, encoders, motor models, multifaceted tinkering, and countless other skills. As stated above, our plan is to create something completely new that plays to the strengths of electronics; flexibility. How, do we provide flexibility, you ask? We make a hardware system that is easily adjusted to the sounds, feels, strengths and desires of the performer. Our hardware system consists of three motorized fader sliders as well as a haptic wheel/stick. The relation of these four inputs can provide the response necessary for a satisfactory experience as well as the flexibility of software.
Required Parts:
Our Project used some pretty sophisticated hardware. We used three fader motors. For those of you who don't know, Fader motors use a linear potentiometer to discern location of a tab along the track. By hooking the potentiometer in series with a 10K resistor a voltage divider is created. The location of the slider can be found by measuring the voltage at the middle node of the divider with an Analog to Digital Converter. These devices also have a small rotary motor/belt drive. This provides actuation to the device. Combine this with a control algorithm and the fader is set for haptic feedback.
We used a hefty dc motor as the primary haptic medium. Using a Pololu supplied current sensing chip, we were able to invoke a high gain proportional feedback loop to control torque. We coupled the current information with the bi-directional flexibility provided by a H-bridge. This gave us the ability to control the "feel" and "direction" of the experience. As our "metric," we measured the stall torque of the motor to be .74676 N-m. This torque proved to be sufficient to generate an adequate feedback to user input. While also not proving dangerous for the user if misused. The motor, however, was not ideal for this application. The motor had a relatively high moment of inertia as well as coarse gradation on the commutator. As a result, precise control of the device's torque was difficult.
To determine the rotor's position, we used an 8 bit (512 position) encoder attached to the rear of the motor. This provided .70 degree resolution. The granularity of the encoder was not ideal. It was a primary reason that instability developed when attempting to create a virtual wall.
The final component of our device was an ATmega 1280 microprocessor hidden in the form of an Arduino Mega. The Arduino was the brains of the operation. It ran off on a 10 millisecond interrupt driven loop. The Arduino was in charge of determining the state of all the sensors and implementing the correct feedback of the actuators. Additionally, the Arduino sent MIDI signals out to the a computer running PureData.
Case Fabrication:
We used the laser cutter in the AutoLab grad shop to cut the intricate shapes required to make these wooden cases for our project. An Auto CAD file of the required parts was drafted and than saved as a dxf file. A really neat color coding method was used to differentiate the correct cutting sequence and laser intensity. See pictures below. Some common designs for the larger electric motor stand were superimposed upon each other, in different colors, and then during the cutting process some color coded parts were called upon while others were left dormant.
The other wooden components of the cases were cut using traditional methods; then fastened together with nails, screws, and glue. Each case was designed with adequate space to allow for the concealment of the circuit boards and the Arduino Mega.
Software:
The arduino environment makes software development on a micro controller extremely easy. It does this by providing simple api's to control hardware functions. The downside is that it is often difficult to discern exactly what low level changes can be made without detrimentally affecting the operation of the device. And for our project... we needed to make some of those afore mentioned changes. We implemented a round robin software architecture running every 10 ms off an internal interrupt. The interrupt was fired every time Timer 0 (an 8 bit timer) overflowed. We had to do some low level register manipulation to make this happen as well as searching through the Arduino development files to see the side effects of our changes. The two most apparent changes were to the millis() and delay() functions. (An example of the unfortunate side effects mentioned earlier)
We also implemented two other hardware driven interrupts: one for both channels of the encoder. Upon seeing a rising edge on either of the encoder channels, the arduino would pause and increment/decrement the encoder count. The ability to discern discrete time increments as well as position measurements allowed us to determine the velocity of the haptic wheel. With the combination of position, velocity, and torque measurements, we have created a device that can easily implement a multitude of virtual environments.
During our development we implemented a couple test virtual environments. Particularly we demonstrated, mass springs, mass dampers, virtual walls, and pure springs. These environments were implemented in an attempt to discern exactly what feelings were reproducible using our hardware/software skills. Although we could "identify" the specific feelings of all of the aforementioned environments, we found that a couple hardware limitations resulted in non-ideal behaviours. Those limitations were the 16 Mhz clock speed of the ATmega 1280, the motor inertia, and the encoder resolution. These three elements made the most difference when trying to implement an "analog" feel using digital circuitry.
Sound Production with PureData (PD):
PureData is a powerful software which can be easily interfaced with other platforms like the Arduino or Labview. We used a Midi box to hook up the Arduino to PD. The circuit diagram and basic program to send signals from Arduino to PD can be found at http://itp.nyu.edu/physcomp/Labs/MIDIOutput.
In its simplest form, PD generates a sine wave to produce a note of a desired pitch and volume from computer/attached speakers. Manipulating the waveform in different ways changes the way the sound is perceived. People can distinguish between two instruments playing the same note based on the timbre of the note. For example, one can add "partials" to the fundamental frequency by multiplying it with some integer and adding the overtone to the output. The presence or absence of overtones or subharmonics is one of the characteristics of the sound produced by any instrument. Another important characteristic would be the attack, decay, sustain and release times (ADSR) of the note. PD can thus be used to create a variety of sounds by manipulating these parameters.
The arduino code as well as the Pure Data spreadsheet are included at the bottom of the post.
The Instrument:
We programmed our extremely flexible instrument to behave as either a gong or a virtual musical spring for demo day.
The gong feature was developed by implementing a horizontal virtual wall with the large motor. When the lever is rotated as to to hit the virtual wall/gong a signal is sent from the arduino to PD to initiate the sound. The wall behaves just as a drum head would. If the mallet is forced deep into the wall it will be repelled with a larger torque. Thus bouncing it back away from the object. An additional feature that we took advantage of was the instability that arises at the wall. Due to our coarse sensor resolution, we were unable to model an analog world using a digital one. So, as the user applies continuous force into the wall the system explodes; resulting in reverberations at the wall that are very similar to drum rolls. So, the system models discrete hits as well as continuous drum actions.
To imitate the sound of a gong, we used a reverberator available as one of the examples in PD.
We used the fader motors to control the volume and pitch of the device. The volume is varied by moving fader motor 2. The pitch was modulated by moving fader motor 3, which has a virtual spring feeling bringing it back to an equilibrium position at the bottom of the potentiometer. The feedback on fader motor 3 allows the musician to feel the slider's position by allow the cognitive understanding of the position.
An important design aspect for this device was the effect of the haptic feedback. We wanted to develop a system where physical feedback wasn't merely extraneous information for the human to sift through. Instead, it would be necessary in order to really master the device. We succeeded. It is almost impossible to to play this instrument without the virtual wall. Believe me, we tried. Additionally, during our demo day we asked some people to try the device w/ and w/o the feedback. The response was unanimous that the wall was "required" in order to make the device feasible to learn, play, or master.
As stated before, our other incentive was to develop a device that was flexible. So, as an afterthought we put in a "second instrument." By moving fader 1 we are able to switch between the haptic instruments.
The virtual spring is played using all the same hardware. However, the feel of the device changed drastically. The large motor no longer represented a drum head. Instead, by rotating the lever past the virtual equilibrium point it acted like a spring. In a very appropriate manner (for a spring that is), we accompanied the feel with an eerie wawa tone. The wawa frequency is varied by rotating the mallet further past the equilibrium point. So, more generated spring force accompanied a slower wawa frequency. This sound was generated by using frequency modulation. Rotating the lever would change the modulation index. Fader motors 2 and 3 accompanied the instrument and fulfilled the same function as in the gong.
At the Expo:
Future Work:
This device could be improved. The biggest shortcoming was the limited sounds that we could generate with PureData. (The limited sounds are due to our inexperience with the software, not the limitations of the software itself.) So, developing a larger ensemble of virtual instruments that can be played along with the gong and virtual spring would be fun.
/*
Colin Foe Parker
Daniel Dimoski
Ramya Iyer
This Code Controls the big haptic motor....
*/
#define encoder0PinA 2
#define encoder0PinB 3
#define DirectionPin 52
#define CurrentPin 6
#define Motor1PWMPin 13
#define ledPinGreen 24
#define ledPinYellow 22
#define FaderPin1 7
#define FaderPin2 2
#define FaderPin3 3
#define FaderMosPin1 8
#define FaderMosPin2 9
#define FaderMosPin3 10
/////////////////////////////////////////////////////////////
//Variable Declarations
volatile uint16_t gTickCount = 0; //For Interrupt Timing
volatile uint8_t prevCount = 0; //For Interrupt Timing
volatile uint16_t timer=0;
volatile uint16_t encoder0Pos = 32768;
volatile int16_t Torque = 0;
unsigned int Direction=0;
uint16_t count=0;
uint8_t count2=30;
unsigned int tmp_Pos = 1;
unsigned int valx;
unsigned int valy;
unsigned int valz;
unsigned int analogValue;
int Print;
int16_t Lastencoder;
boolean A_set,enable;
boolean B_set;
int16_t speed1;
int16_t Fader1=0;
int16_t Fader2=0;
int16_t Fader3=0;
uint8_t modulation;
uint16_t note,gongvol;
////////////////////////////////////////////////////////////
void setup() {
Serial.begin(115200);
Serial2.begin(31250);
pinMode(encoder0PinA, INPUT);
pinMode(encoder0PinB, INPUT);
pinMode(ledPinGreen,OUTPUT);
pinMode(ledPinYellow,OUTPUT);
// encoder pin on interrupt 0 (pin 2)
attachInterrupt(0, doEncoderA, CHANGE);
// encoder pin on interrupt 1 (pin 3)
attachInterrupt(1, doEncoderB, CHANGE);
pinMode(DirectionPin,OUTPUT);
}
////////////////////////////////////////////////////////////
//This loop will run ever 10 ms off a Timer0 overflow interrupt
void loop() {
Fader1=analogRead(FaderPin1);
note=(uint16_t)((Fader3-353)*.12+40);
speed1= (int16_t)(encoder0Pos-Lastencoder)*12.27;
Lastencoder=encoder0Pos;
analogValue = analogRead(CurrentPin) - 99;
Fader2=analogRead(FaderPin2);
Fader2=(uint16_t)((Fader2-353)*.4);
Fader3=analogRead(FaderPin3);
modulation=(uint8_t)(encoder0Pos-32768);
if (modulation > 200) modulation = 200;
if (Fader1 > 505) {
if ((count%40)==0){
digitalWrite(ledPinGreen,HIGH);
digitalWrite(ledPinYellow,LOW);
}
if ((count%80)==0){
digitalWrite(ledPinGreen,LOW);
digitalWrite(ledPinYellow,HIGH);
}
SpringDamper();
noteOn(0x91,note,Fader2);
noteOn(0x92,modulation,0);
}else{
note=(uint16_t)((Fader3-353)*.12+50);
virtualspring(FaderMosPin3,353,Fader3);
Print=VirtualWallBig(0);
//The flashing lights
if ((count%20)==0){
digitalWrite(ledPinGreen,HIGH);
digitalWrite(ledPinYellow,LOW);
}
if ((count%40)==0){
digitalWrite(ledPinGreen,LOW);
digitalWrite(ledPinYellow,HIGH);
}
if(enable ==1){
noteOn(0x90,note,Fader2);
noteOn(0x91,0,0);
enable =0;
}
gongvol=0;
}
//The Serial Stuff
if ((count%20)==0){
Serial.print(" Encoder Pos");
Serial.print(encoder0Pos);
Serial.print(" Note ");
Serial.print(note );
Serial.print(" GongVol ");
Serial.print(gongvol);
Serial.print(" Count1 ");
Serial.print(count);
Serial.print(" Fader 1 ");
Serial.print(Fader1);
Serial.print(" Fader 2 ");
Serial.print(Fader2);
Serial.print(" Fader 3 ");
Serial.print(Fader3);
Serial.print(" Speed ");
Serial.println(speed1);
}
count ++;
if (count ==10001){count = -1;}
WaitForTimer0Rollover();
}
/////////////////////////////////////////////////////////
//Functions
//Gong Function
uint8_t Gong(void){
uint8_t gongvalue;
gongvalue = (uint8_t)((speed1/150)*127);
return(gongvalue);
}
//Waits for the timer to roll over (10 ms)
void WaitForTimer0Rollover( void )
{
prevCount = gTickCount;
while ( gTickCount == prevCount )
{
;
}
} // WaitForTimer0Rollover
//To send notes to the midi
void noteOn(byte cmd, byte data1, byte data2) {
Serial2.print(cmd, BYTE);
Serial2.print(data1, BYTE);
Serial2.print(data2, BYTE);
}
//Virtual Springs for the Fader Motors
void virtualspring(int Channel, int ZeroPoint, int x) {
float TorqueFader;
float k = 3;
TorqueFader = (uint8_t) (x-ZeroPoint)*k;
if ((x-353)<20){ torquefader =" 0;"> 32768){
Temp_Power= (int16_t)(.5*(encoder0Pos-32768));
Direction = 0;
}else {
Temp_Power= (int16_t)(.5*(32768-encoder0Pos));
Direction = 1;
}
digitalWrite(DirectionPin,Direction);
TorqueControl(analogValue, Temp_Power);
}
// Code to translate the voltage control to torque control (Max Current ~250)
void TorqueControl(uint16_t Current, uint16_t DesiredCurrent){
float Pgain = 40;
if (DesiredCurrent > 350){
DesiredCurrent = 350;
}
Torque += Pgain*(DesiredCurrent-Current);
if (Torque < torque =" 0;"> 255){
Torque = 255;
}
analogWrite(Motor1PWMPin,Torque);
}
//Code to do a Virual Wall for the Big Motor
int VirtualWallBig(uint16_t Setpoint2){
int16_t Temp_Power;
float Pgain = 20;
float Dgain = .01;
Setpoint2 += 32768;
Temp_Power =(int16_t) ( Pgain*(encoder0Pos-Setpoint2))-Dgain*speed1;
if (encoder0Pos > Setpoint2) {
digitalWrite(DirectionPin,0);
}else{
Temp_Power = 0;
}
TorqueControl(analogValue, Temp_Power);
return(Temp_Power);
}
//Code to do a Virtual Spring Mass for the Big Motor
int16_t VirtualSpringMassBig(int Setpoint,float frequency){
static unsigned int counter=0;
int Temp_Power;
Temp_Power=(int16_t) (encoder0Pos-32768)*cos((counter-Setpoint)*frequency/(2*PI));
if (Temp_Power > 0){
Direction = 0;
}else {
Direction = 1;
Temp_Power = -Temp_Power;
}
counter ++;
digitalWrite(DirectionPin,Direction);
TorqueControl(analogValue,Temp_Power);
return(Temp_Power);
}
/*Quad Code taken from http://www.arduino.cc/playground/Main/RotaryEncoders
Interrupt on A changing state*/
void doEncoderA(){
// Low to High transition?
if (digitalRead(encoder0PinA) == HIGH) {
A_set = true;
if (!B_set) {
encoder0Pos = encoder0Pos + 1;
if (encoder0Pos >32760 && encoder0Pos < gongvol="abs(speed1);" enable="1;" a_set =" false;" b_set =" true;" encoder0pos =" encoder0Pos">32760 && encoder0Pos < gongvol="abs(speed1);" enable ="1;" b_set =" false;">