A brief history


2015 David Antón Sánchez built (under supervision of René Bohne) OpenVNAVI: A Vibrotactile Navigation Aid for the Visually Impaired, similar as the one Sean Benson built for Hackaday, but with a much higher resolution.



While it turns out that the vest works surprisingly well, we had some reliability issues when the vibration motor got stuck in their 3d-printed housing.

Furthermore, soldering all the pcbs and especially the wiring is far to complex this way.

Personal Photonics: Haptivest

Beginning with June 2016 we have now funding from the BMBF as part of their Open Photonics call to improve the vest and develop a haptic toolkit for easier prototyping of different kind of vibration feedback systems.

As a first step, we reduced wiring of Davids vest by exchanging the central I2C expander hub on the backside with 8 Adafruit 16*Servo boards on the front, making it easier to maintain the vest and reducing weight - on the cost of a less balanced version (the front part is now far heavier than the back side).



This updated vest - along with future versions - will be displayed on different locations:

This allows is to get us feedback and testing the reliability of the hardware setup.



Control a bunch of vibration motors:



The original vest used pcbs with a amplifier circuit (left picture) in a 3d-printed housing for each vibration motor, which was mounted on the pcb within the housing. This allowed to use cheap surface mounted vibration motors, but needed both power lines for voltage supply and a data line for each motor.
The standard 3pol pin header allows an easy exchange of these motor units (along with the velcro attachment of the housing and vest). Each half row is connected with the two supply lines in parallel, which are bundled in the middle in a central node as one power supply line, while the data lines goes in between the housings upwards to the controller unit.

This at the beginning a central unit (left picture) in a box at the back, where each of eight 16* I2C PWM-driver on a central PCB controls one row of vibration motors. The layouts and schematics for both boards can be found here, as well as the files for the 3D-printed housings (and of course a more detailed description in his thesis).

For reducing wires and simplify exchanging the driver units this board was later replaced with the standard Adafruit 16* Servo boards, which also allows the decrease significant the size of the back box, which now only contains the raspberry pi as brain and a Jtron buck converter for the power supply, which could be either a battery pack or cable connection.



Personal Photonics: Haptic Toolkit

For future versions, we developed until now different methods to use huge numbers of vibration motors (pcb layouts and circuit diagrams can be found here, with the goal of developing an easy-to-use toolkit to develop different kind of systems with huge numbers of vibration motors with acceptable effort.

  • I2C PWM driver:
Can be used int he same way as the Adafruit 16*Servo driver, because it uses the same controller, but has an integrated voltage regulator and driver for the vibration motors, such that either encapsulated or pancake vibration motors can be directly connected with the board. Six address pins (solder on resistor) allows to use up to 62 of these boards at one I2C bus. For wiring use a flexible wire, and stabilize the solder joints either with shrink tube (vibration motor side) or hot glue (on the board itself). These boards are then interconnected with each other (and e.g. with the raspberry) with the two voltage and the two I2C lines. Since a voltage converter is on each board, we don't need the bulky central unit anymore and can supply a higher voltage to each board, therefore cables with smaller diameter can be used. This is at the moment our best way to control the vibration motor arrays, because of the small amounts of components to solder and the simple wiring.



  • Serial Connection
Here we use the WS2811 driver, which is normally used for RGB led stripes with a serial one wire data bus. Instead of a led, we control the vibration motor with this ic. While it is possible to control up to three motors with one ic (one for each color), we decided to control just one and add two (single-color) control leds instead. Therefore, we can daisy chain this board just like the led boards, and use the same libraries and programs like the Adafruit Neopixel library to control them, up to 1024 at one data pin of a micro controller. Main disadvantage is due to the serial connection if one board breaks, the rest behind will be also dead. Therefore we would only recommend it for non-wearable solution, where the wiring won't be stressed to much or use cases were the motors are spread over a big area. A one wire data lines and the two voltage lines are connecting each board in series.



  • Parallel Connection
A dedicated micro controller on each board controls up to four vibration motor. Since the PWM is now done with a general-purpose controller, we can control even linear resonant actuators, which is not be possible for the above mentioned systems. A micro plug system for flat ribbon cable allows an easy setup with crimping 4 pol connectors in parallel on a cable, for the voltage, the serial data line and for programming the micro controller. Current on a standard ribbon cable is limited, therefore after a certain number of vibration motor boards a voltage controller board is plugged in between (again with the same micro plug system), which converts a higher supply voltage down to the 3.3V needed for the boards, reducing the current on the supply lines. Each controller will just listen in parallel for commands, allowing them to work even when some of them break. While we have a highly adaptable wiring system here, which allows also to reuse the boards for different setups, the boards are harder to solder by hand, and you have to program each board individually since each controller needs a unique ID to be addressed individually.



Wiring

While Davids version used standard 0.14 mm2 stranded wire with pin header connectors for the vibration motor boards, this is to stiff for directly soldering on the vibration motor cable (which is necessary for the I2C expander boards with integrated driver), because the vibration causes the solder joints to break. Additional reinforcements with shrinking tube reduces this problem (and additional fixation on the textile will further reduce it), but since a bunch of wires are used the system will be overall stiff and heavy, and coupling between the different motors might occur via the cables. Therefore we decided to switch to smaller 0,05mm2 stranded wire - an stranded is very important in this case, otherwise the cable itself will break easily. After several tests we settle at the moment on a 0,04 mm2 twin wire for each vibration motor, which is easier to solder than the version with a common wire, allows cleaner routing for detecting faults and allows easier repairs in the future (See the next section for the corresponding image). Special Thanks to Ceyda who had to do this nightmares of soldering.



Determine the best Resolution for a haptic image

For the original vest David took 4cm distance as a distance dependent on the two-point discrimination threshold of roughly 3,5cm on the human belly according to Kandel et Al. 2013. The grey color values of a depth image are then mapped to the PWM values for driving the corresponding vibration motor.

For a more individual approach the distance between the motors might be smaller or bigger if different body parts are used as display area or dependent on the individual user, if such a system is built for one person alone. To determine the optimal distance (and also validate the 4cm distance) we developed a test setup: A matrix of 4*4 vibration motors - driven by one I2C driver is placed within a laser cutted form of foam rubber. This allows precise distances between each motor. Since far smaller distances are used then the expected distance, we can play different pattern on it and determine automatic which distances and changes we can detect. E.g. if the first and thereafter the second row of vibration motors is turned on and the user couldn't feel a difference, this distance between two vibration motors is too small and shouldn't be used in the planned layout. Both changing of the position of a line of vibration motors with or without a pause in between can be tested, for both general detection of in image and to detect motion in a haptic motion picture. The user then has to decide if change in position happens or not. The whole system is then placed with velcro tape on the users body.

With a second set up we can determine the possible "color depth" of the haptic image, the PWM or power level which the user can distinguish. Here the power level of one vibration motor is changed and the user has to decide if the vibration got stronger, lesser or stays the same. Since we need a certain pressure of the vibration motor to the skin, the best way is using a certain hanging weight around the system to fixate it on arm or leg (if seated) and apply comparable pressure if you want to compare different user. Otherwise just use velcro to attach the sensor on the human body, where the user can adapt it to his personal preferences. This information allows to reduce the data communication and processing if different sensors and image processing is used.



Sensor input and processing

David used a Asus XTion depth camera for his vest, which is relative small and runs on USB power. Since only USB2 is needed for the camera, a raspberry pi can be used to process the data. The setup for the pi is well described on his Git-repository, just follow the steps there. As long as either the Adafruit 16*Servo boards or our I2C PWM-driver are used to control the vibration motors - and each driver represents one row, with an ordered address range from 0x40 to 0x48 - you can just use the software without further modifications

For more generic layouts we will adapt this software in future to allow an easy setup at the beginning, where you can change which connection is used (I2C for the I2C driver, RX/TX for the parallel driver, or one generic pin for the WS2811 variant) and a simple mapping function between hardware and software solution, but this is future work.

Another important upgrade which we will do in the near future is the option to add other sensors, either additional depth cameras - because of the viewing field of the camera things at the bottom and head height disappear from the sensor area when they come closer, which is not acceptable - or other sensors to circumvent certain limitations of the depth camera, e.g. it can't see glass.
One option are ultrasonic sensors for short distance warnings or using two normal cameras e.g. on the shoulder for a stereoscopic depth image.

The Asus XTion might be replaced in the future with a Intel Realsense camera or other more advanced sensors, but most of the newer ones nowadays use USB3, which is not possible wih the Raspberry Pi at the moment, therefore it might be necessary to switch to a different controller like the OdroidXU4 for example.

But staying with the old setup: The Depth camera is just connected with one of the Raspberry USB ports, its I2C port (Pin 3 and 5) is connected with the I2C PWM expander in a row, which are then connected with the individual vibration motors. Power connection comes either with a external power supply or a racing battery pack (7,4V and a big capacity), the 5V USB power banks couldn't deliver enough current. While the original version has PI and vibration motors supplied with a Jtron buck converter, for the versions with distributed voltage controller we need another one for the Pi, a standard 5V/3A DC/Dc converter should be enough as long as not more (USB-)devices are attached to the Pi.

The last minor thing to add is a button to enter low-power mode (turning off the vibration motors) at GPIO18, or pin 10 on the GPIO header, which is internally pulled low, therefore a connection to 3,3V of the pi is sufficient, but an additional hardware pulldown with a 10k resistor to ground is of course additional possible.

Textile Stuff
The original vest had be adaptable to different body sizes while pressing the vibration motors on the body. Therefore the vest was made out of two sheets of stretch fabric with a sleeveless shirt pattern, which where stitched together at the shoulders. We later replaced on shoulder seam with velcro to accelerate vest dressing on the demo sessions. On the belly stripes of velcro are used to attach the vibration motors, and fasten the vest front and back with overlapping velcro stripes at the sides. The Depth camera is mounted with a small 3d-printed holder on the upper body area, while the Pi and Voltage regulator case is located at the back with a Battery. Again, velcro is your friend.










A lighter version with encapsulated vibration motors and our new I2C board might be reduced to a sort of kidney belt, where the I2C boards are mounted on the left and right of the belly on small laser cutted boards (e.g. out of POM for stability reasons) with two strips of velcro in between, on which the motors are mounted. These strips are sewn on the laser cutted board with a small stretch band loop to adapt a bit to the belly form. On the upper two strips is a holder for the camera mounted in the middle, and each pair is connected with a 2cm lashing strap on the back, which allows fastening of these pairs. To keep the vertical distances between the vibration motors the velcro strips has to be sewn on a piece of stretch fabric again.



For a personal version things will be much easier, because in this case you just need a tight stretch shirt, where the vibration motors are either sewn into pockets, or even just glued on, the problem to be adaptable to different body sizes can be ignored.

Feel free to contact Jan Thar if you need further information.

Created by thar. Last Modification: Wednesday 31 of August, 2016 21:43:41 by thar.

Media Computing Group at RWTH Aachen

Search

in: