Accelerometers (ACC), gyroscopes (GYRO), magnetometers (MAG), pressure sensors (PS) and microphones (MIC) are some example of MEMSsensors. These products have been integrated in the last few years in portable devices because of their low cost, small size, low power consumption and high performance.
Fast CPU’s with multitasking OS platforms, high sensitive global positioning system (GPS) receivers, 3G / 4G wireless communication chipsets, high-resolution digital video cameras, touch screen LCD displays and large storage size are common in smartphones. The use of MEMS sensors is then no longer limited to existing applications such as screen rotation, power saving, motion detection, E-Compass and 3D gaming. More advanced applications for MEMS sensors, such as augmented reality (AR), location-based services (LBS), pedestrian dead-reckoning (PDR) are currently being developed.
This article discusses the role of MEMS sensors in advanced applications in mobile devices including mobile augmented reality (MAR), LBS and the solution of MEMS sensor fusion integrated with a GPS receiver to determine the position and orientation using the dead-reckoning method.
Augmented reality (AR) is not a new topic. By definition, AR is a feature or user interface that is implemented by the interaction of superimposed graphics, audio and other sensing enhancements over a real-world environment that is displayed in real-time, making it interactive and manipulable. The Integration of 3D virtual information into real-world environment helps provide users with a tangible feeling of the virtual objects that exist around them.
Recently, there have been a few successful applications of AR. For example, vehicle safety application can provide information on road conditions and surrounding cars by projecting these to the windshield. Another application is to display information of an object such as restaurant or supermarket, etc., when the smartphone is pointed to the object with known position and orientation. Also, one can find the nearest subway station in a big city by moving the phone fully round 360 degrees, locating the subway and following the directions to the destination.
Social networking is playing a key role in peoples’ current life. When approaching a shopping center, a user can point the phone to the shopping center sending friends virtual information augmented on his location and the surrounding environment. Vice versa, the user will have information on his friends’ whereabouts. Therefore, AR is a new way of changing the feeling to the real world.
The key components available in smartphones for MAR are shown in figure 1.
Figure 1: Smartphone structure for MAR.
Digital video camera: Used to stream information about the real-world environment and display captured video on the LCD touch screen. Currently 5-Megapixel or higher camera sensors are available in new smartphones.
CPU, Mobile OS, UI and software development kit (SDK): Components are the core of a smartphone. 1GHz or higher dual-core CPU with 512MB RAM and 32GB storage space can be found in new smartphones. UI and SDK give developers a simple way to call application programming interfaces (APIs) to access the graphics, wireless communications, database and MEMS sensors raw data without knowing the details behind during their own applications development.
High-sensitivity GPS Receiver, or Assisted GPS (A-GPS) or differential GPS (DGPS): Fixes the user current location in terms of latitude and longitude when significant satellites are captured. A lot of effort has been invested over the years to increase the GPS sensitivity and positioning accuracy for indoor and urban canyon areas when satellite signals are degraded and multipath errors occur.
Wireless link for data transmission including GSM/GPRS, Wi-Fi, Bluetooth and RFID: Provides Internet access to retrieve the database online of the object that is near-by the current location and to give a rough information about positioning while waiting for GPS fix or if GPS is not available. Other short-range wireless links such as WLAN, Bluetooth and RFID can also be used for indoor positioning with adequate accuracy if the transmitters are pre-installed.
Local or online Database: Utilized for virtual object information augmented on the real-world video display. When the object is aligned to the current position and orientation, its information can be retrieved from online or locally saved database. Users can then click the hyperlink or the icon on the touch screen to receive more detailed information.