June 25th, 2010 by Chris Allen
The convergence of mobile and the web is a topic that we’ve been giving a lot of thought to lately here at Infrared5.That is in part because we do development both for RIAs and games in the web browser, as well as for smart-phone devices like iPhone, iPad and Android, but of course this is also because we’ve created a new SDK to allow developers to control an experience residing in a browser using their mobile device. For those that haven’t seen it yet, it’s called Brass Monkey.
Brass Monkey is at its core a networking library for interconnecting mobile devices and computers over a local WiFi Network. Brass Monkey’s primary use is as a controller, where a mobile device, say an iPhone, uses its built in controls to communicate to a web based experience (Flash/Flex, Unity3D).
With Brass Monkey we are addressing the lack of interoperability and speeds of connecting various mobile devices, laptop and desktop computers, large screen displays and web browsers. Devices that have different operating systems usually can not communicate easily with each other without using proprietary software and/or inefficient web based networking protocols. Methods used currently to connect multiple devices together typically implement an intermediary server, and thus are slow and experience latency, making it prohibitive to use for instant feedback and control. Some other solutions have built in peer to peer networking, but still lack responsiveness and instant feedback expected from something like a game controller. Other implementations use Bluetooth for connectivity, but this does not allow for connecting to many types of devices, and its range is severely limited. Of course we are well aware of implementations using a smart phone to connect to a computer, but none of these allow for low latency experiences running in a web browser, thus making it a requirement for the end user to install an additional program on his/her computer. We have solved these problems by removing the server, and keeping the connections local to the users’ network, while allowing real-time control over experiences running in a web browser. In the process have created possibilities for new and revolutionary user experiences, allowing for large numbers of people to communicate in real time, across WiFi networks.
So, what do I mean by revolutionary user experiences? Part of it is the ability to use new gestures to interact with a screen. In this case what I’m thinking is more or less analogous to the famous scenes in Minority Report where Tom Cruise is using hand gestures to move images and videos around a screen. The viability and the technology for this type of interaction is already here, as outlined by John Underkoffler in his Ted talk. The main problem with Underkoffler’s approach is that it requires the user have this specialized equipment, as well as custom software running on the computer. Using Brass Monkey we envision similar types of gesture based interactions with screens, but using the mobile device’s touch screen and accelerometer as the input instead.The new gyroscope introduced to the iPhone 4G is just going to add even more possibilities.
Another use case is using your phone as a game controller, which is what we are doing with the soon to be released Trench Run 2.0. Check out the video interview that Peter Ha from Time Magazine did with me at E3 last week for details on how that works.
While we are one of the first, we certainly aren’t the only ones in the industry playing with this type of interaction. Grant Skinner recently showed off a very cool Android/Adobe AIR based asteroids game using a Nexus One as a controller. His implementation is really intriguing, however Brass Monkey features many advantages. Grant’s implementation includes the necessity of installing an AIR application on the host computer, and as we all know, Steve Jobs put a stop to his AIR based stuff going on the iPhone. I’m sure that developers like Grant and his team will be looking to do many more applications like this, and that Brass Monkey will undoubtedly play a role in the future.
These ideas are really just the beginning, and we are working on quite a few projects that are pushing boundaries of user interaction that we simply can’t talk about yet. I look forward to hearing your ideas. Please leave some in the comments.