By Florence Ion
Google’s Project Tango isn’t something you can buy yet, but it’s so much smarter and more advanced than the smartphone in your pocket that NASA is blasting it up into space.
The technology in Project Tango is designed to allow phones to map their environment in 3D, something that hasn’t yet been possible with a consumer-grade gadget up until recently.
This isn’t the first time NASA has used Android phones in space: back in 2011, NASA sent up the Nexus S to power its SmartSPHERES, which help the astronauts conduct various tests up in space. Google’s Project Tango is way more technologically advanced than the 4-year-old smartphone, and it’s time for NASA’s little helpers to consider an upgrade.
I sat down with Chris Provencher, project manager of SmartSPHERES (part of the Intelligent Robotics Group (IRG) at NASA). He told me all about how Google’s technologically advanced prototypes will help find a solution to automating all those “housekeeping” activities that take up the astronauts’ time. We also talked about what it takes to tweak a smartphone so that it properly works up in space.
An easy solution to a complex problem
SPHERES stands for “Synchronized Position Hold, Engage, Reorient, Experimental Satellites,” but they’re not your typical idea of traditional satellites. They resemble large many-sided dice—like the kind that you’d use in a game of Dungeons and Dragons. Once you attach a smartphone, it becomes a SmartSPHERES.
A SPHERE on its own, with no smartphone.
The SmartSPHERES are used to help the astronauts with many common tasks, like measuring the air flow, air quality, and noise levels up in the ISS. “[These are] perfect examples of what a small free-flying robot could take over that would allow the astronauts to go on and do more important things—like science,” said Provencher, though they’re not a permanent solution to this particular predicament.
By attaching a smartphone, which already comes standard with components like WiFi and a very capable processor, NASA can use existing technology to help it with testing up in the ISS.
Chris Provencher, SmartSPHERES Project Manager, demonstrates how Google’s Project Tango hooks up to the SPHERES.
Some of the SmartSPHERES are still using Nexus S handsets while being tested for this sort of work, but those were sent up back in 2011. They navigate using a beacon system that keeps them confined to a 2 x 2 x 2 meter cube. “SPHERES has to operate in that spot, which is great for tests, but the SPHERES cannot fly out of that volume or else it will stop knowing where it is,” said Provencher. “To do all the of the tasks that we’re doing—to really do them—you have to be able to take those air measurements or those noise level readings anywhere inside the space station."
“That’s what we think Project Tango can do,” he added. “The fact that you can build up a map of your environment and learn your way inside that environment is exactly what we need.”
Hacking a phone for space
The Project Tango smartphone NASA uses looks quite different from the ones being developed down here on Earth. Its hardware has been "butterflied" open and stuffed into its own rig that attaches on to the SPHERES. All the essential components that face the rear of the phone are turned around to face forward.
Florence Ion and Michael Homnick
The infrared camera, infrared projector, and wide-angle camera sensors are moved to the front. The wide-angle camera, in particular, helps the satellite see where it’s going. “As the phone moves…it sees that [a] point moved off to the left, so it can see how far it moved,” explained Provencher. The infrared projector, on the other hand, projects infrared patterns into the environment to be picked up by the infrared camera, which is how Project Tango gets it 3D mapping abilities. It's fundamentally quite similar to the Xbox's Kinect.
On the back, the rig features a bracket that binds the phone to the SPHERES, as well as a MicroUSB data cable. Provencher also mentioned that the phone uses an entirely different battery than what Google originally plopped in there; it’s actually a 4,900mAh battery pack that measures in at 11.5 cm by 6.9 cm by 2.8 cm.
Project Tango hooked on to the SPHERES.
To work in space, the IRG took out the GSM chip in the Project Tango smartphone, but left all of the camera sensors, Wi-Fi and Bluetooth radios, and accelerometers inside. The latter, however, was deactivated, because, “this phone uses gravity to help its orientation,” began Provencher. “The handheld version for Planet Earth uses accelerometers, so when you’re moving around it can sense the motion and it can use those measurements to figure out how far it’s actually moved.”
Accelerometers don’t work the same in space, where there's only microgravity. “The SPHERES moves incredibly slowly—two centimeters to four centimeters per second,” continued Provencher. “Accelerometers in these phones are not sensitive enough to detect that, so we cannot use those…at all.”
It took several months to go through all the vision algorithms contained within the Android source code so that the device would no longer rely on gravity. The Project Tango smartphones in space rely on their wide-angle cameras; they measure angles to keep track of how far they’ve moved so that they know their trajectory. They also utilize the infrared sensors to do 3D mapping. “We’re relying more heavily on the infrared—or 3D sensors—than what the Tango typically uses,” he added. “It can measure angles. It keeps track of a feature and it can see how far it’s moved in terms of angles, so it knows it trajectory.”
Plenty of testing left to do
The SmartSPHERES will not be permanently instituted in space. “This is just a test to see that this technology works,” said Provencher. “We’re almost finished with our testing down here…we’re finalizing our software right now and we’ll uplink it as soon as the phones are up there and ready to receive.”
The tweaked Project Tango phones blasted up to space on July 13. Once they’re set up on the ISS and the software is uploaded, they’ll be attached to the SPHERES to begin the second phase of testing, which is to see if they can actually leave that narrow 8-cubic-meter volume they were previously limited to. “We tell it to fly where we want it to,” said Provencher. “We’re going to try flying it out of that little area, out of that module, take a 90-degree turn into another module and then come back.” It will be the crew’s job to ensure the SmartSPHERES don’t veer off and fly into a wall.
Testing with the SmartSPHERES should end this September, with the IRG team hoping to begin mapping out blueprints for a full-blown robot later this fall. “You need a robot,” said Provencher. “The SPHERES satellites itself is a great starting point as a testbed, but it doesn’t have everything we want.”
Florence is an Android-using yogi obsessed with all things tech.
