Google Goggles gets faster, smarter and solves Sudoku

(Cross-posted from the Google Mobile Blog)

Today we’re launching a new version of Google Goggles that’s faster and smarter than ever before. The new Goggles 1.3 client for Android can scan barcodes almost instantly. All versions of Goggles can now recognize print ads in popular magazines and newspapers. Finally, Goggles has also learned a fun new trick for Sudoku fans.

Barcodes
When shopping offline, it’s helpful to be able to learn more about a product by scanning its barcode. With the new Android version of Google Goggles, scanning barcodes is much faster. Open Goggles and hover over the barcode or QR code. Within a second the phone gently vibrates and presents results, without requiring a button press. Simply tap on the result to read product reviews, check in-store availability and compare prices.



Print ads in magazines and newspapers
We’re excited to take another step in our vision of connecting offline media to online media. The next time you're flipping through the pages of your favorite magazine, try taking a picture of an ad with Goggles. Goggles will recognize print ad and return web search results about the product or brand. This new feature of Goggles is enabled for print ads appearing in major U.S. magazines and newspapers from August 2010 onwards.


This feature is different from the marketing experiment that we announced in November. We're now recognizing a much broader range of ads than we initially included in our marketing experiment. And when we recognize a print ad, we return web search results. While in the experiment, we return a specific link to an external website.

Sudoku
Our favorite weekend distraction is a quiet 15 minutes spent solving a Sudoku puzzle. But even that can be an frustrating experience if (like us) you make a mistake and are unable to solve the puzzle. Now, Goggles on Android and iPhone can recognize puzzles and provide answers to help make you faster than a Sudoku champ. So if you ever get stuck, take a clear picture of the entire puzzle with Goggles and we’ll tell you the correct solution. Check out this video to see how it works.


Google Goggles 1.3 with improved barcode scanning is available for download in Android Market. Recognition of print ads and Sudoku solver is now enabled for the Google Goggles app on Android, as well as the Goggles component of the Google Mobile App on iPhone.

Some cool Android tips and tricks

Last week, I sent a note to my team with some of my favorite tips and apps for Nexus S, which features Android 2.3, Gingerbread. A lot of Googlers liked it, so we thought some of you might enjoy it as well. (Note: Many of the tips are specific to Android 2.3.)

Tips
  • Visual cue for scrolling: When you are in a scrollable list (like your Gmail inbox) and you reach the end of the list it shows an orange hue—a visual cue that you can’t scroll anymore.
  • Notification bar icons (Wi-Fi, network coverage bars, etc.): Turn green when you have an uninhibited connection to Google, white when you don't. Hint: if you're in a hotel or airport using Wi-Fi, the bars won't turn green until you launch the browser and get past the captive portal.
  • Voice actions: Tell your phone what to do by pressing the microphone icon next to the search box on the home screen, or long press the magnifying glass. You can tell it to send an email or text message (“send text to mom, see you for pizza at 7”), call someone ("call mom"), navigate somewhere (“navigate to pizza”), or listen to music ("listen to Mamma Mia").
  • Find things you’ve downloaded from your browser: Your downloads are now neatly collected in a Downloads manager, which you can find in the apps drawer.
  • Turn a Gallery stack into a slideshow: In Gallery, when you are looking at a stack of photos, put two fingers on the stack and spread them. The stack spreads out and the pictures flow from one finger to the other, a moving slideshow that lets you see all of the photos.
  • Walk, don’t drive: Once you’ve gotten directions within Google Maps, click on the walking person icon to get walking directions.
  • Easy text copy/paste from a webpage: To copy/paste from a webpage, long press some text, drag the handles around to select the text you want to copy, and press somewhere in the highlighted region. To paste, simply long press a text entry box and select paste. Gmail is a bit different: you need to go to Menu > More > Select Text.
  • Turn your phone into a Wi-Fi hotspot: Go to Settings > Wireless & Networks > Tethering & Portable Hotspot. (You may have to pay extra for this feature.)
  • Look at Maps in 3D: With the latest release of Google Maps, you can now look at 3D maps. Tilt the map by sliding two fingers vertically up/down the screen, and rotate it by placing two fingers on the map and sliding in a circular motion, e.g., from 12 and 6 o’clock to 3 and 9.
  • Cool shutdown effect: When you put the phone to sleep, you’ll see an animation that resembles an old cathode tube TV turning off.
Keyboard tricks
  • Shift+Key to capitalize a word: In Gingerbread (and supported hardware), you can Shift+Key to capitalize a letter instead of going to a separate all caps keyboard.
  • Auto-complete: The space bar lights up when auto-complete can finish a word.
  • Quick replace: Tap on any previously typed word, then tap on a suggestion to automatically replace it with the suggested word.
  • Easy access to special characters (like numbers, punctuation): Press and hold any key to go to the special character keyboard. You can also press and hold the "," key for an extensive punctuation keyboard.
Applications
  • Angry Birds: Popular game that lets you knock down blocks by slingshotting birds.
  • Astro: Awesome file explorer app. Browse and access the directories on your phone, and take full advantage of its capabilities. Great if you’re a power user.
  • Chrome to Phone: This one is really useful for Chrome users. You can send anything you browse on your computer to your phone. So if you are heading out to a restaurant or party and look up directions on your computer, just click the “send to phone” button (requires Chrome to Phone extension) and that exact page will open on your phone. Same with virtually any webpage.
  • Flash: Install from Android Market to watch Flash videos embedded throughout the web. Runs even better on Gingerbread.
  • Fruit Ninja: A juicy action game that tests your ability to smash flying fruit. A fun time-killer on the bus or train.
  • FXCamera: Popular photo sharing app with slick effects and filters.
  • Google Maps: Use your device as a GPS navigation system with free turn-by-turn voice guidance, and take advantage of other Google Maps features like Street View, Latitude and Places.
  • Instant Heart Rate: Measure your heart rate using your camera.
  • Phoneanlyzr: Track your phone usage: who you text most, call most, average call length distribution, etc.
  • RemoteDroid: Control your computer from your phone. Gives you a mobile wireless mouse and keyboard. Great if you’re using your computer for music or movies.
  • Shazam: Identifies virtually any song you are listening to.
  • SoundHound: Record a snippet of a song and get it identified instantly. You can even hum (if you can carry a tune!).
  • Tango: A free, high-quality video call app that works on both 3G and Wi-Fi. If your device has a front facing camera (e.g., Nexus S), you will love this app.
  • YouTube: New UI. Plus, portrait-mode player, and view comments and drop-down box video information

Under the hood of Google Maps 5.0 for Android

Yesterday we introduced Google Maps 5.0 for Android with two significant new features: 3D interaction and offline reliability. In order to create these features, we rebuilt Maps using vector graphics to dynamically draw the map as you use it. Building a vector graphics engine capable of achieving the visual quality and performance level you expect from Google Maps was a major technical challenge and enables all sorts of future possibilities. So we wanted to give you a closer look under the hood at the technology driving the next generation of mobile maps.

Vector graphics
Before diving into how Maps uses vector graphics, it may be helpful to understand how maps were created before. Previously, Google Maps downloaded the map as sets of individual 256x256 pixel “image tiles.” Each pre-rendered image tile was downloaded with its own section of map imagery, roads, labels and other features baked right in. Google Maps would download each tile as you needed it and then stitch sets together to form the map you see. It takes more than 360 billion tiles to cover the whole world at 20 zoom levels!

Now, we use vector graphics to dynamically draw the map. Maps will download “vector tiles” that describe the underlying geometry of the map. You can think of them as the blueprints needed to draw a map, instead of static map images. Because you only need to download the blueprints, the amount of data needed to draw maps from vector tiles is drastically less than when downloading pre-rendered image tiles. Google Maps isn’t the first mobile app to use vector graphics—in fact, Google Earth and our Navigation (Beta) feature do already. But a combination of modern device hardware and innovative engineering allow us to stream vector tiles efficiently and render them smoothly, while maintaining the speed and readability we require in Google Maps. Just try it out and see for yourself!

See the difference between image tiles (left) and vector tiles (right) tilted to show 3D buildings.

One map, many perspectives
Using vector tiles instead of image tiles gives Maps the flexibility to re-draw the same map from different perspectives using the same set of data. Zooming is one example of this at work. If you magnify an map image tile by 2x, lines such as roads and text would get twice as wide and appear blurry. As a result, we had to constrain Maps to 20 fixed “zoom levels,” each one twice as close as the last. Every time you zoomed in further, you’d need to download a completely new set of image tiles. It took time to load new data over a mobile data connection, and would fail when you lost your connection in a subway or large building.

Compared to image tiles (left), vector tiles (right) keep lines and labels crisp as you zoom.

With vector graphics, you no longer need to “round” to the nearest zoom level and then download all the tiles for that level. One vector tile has the underlying vector data (or blueprints) to draw the map at many different levels of scale. So when you zoom, the map stops when your fingers stop, and roads and labels always stay crisp. This same technique powers the new 3D map interactions: tilt, rotate and compass mode. Just like with zooming, Maps uses the same vector data to draw the map from any angle or direction as you tilt or rotate.

We can also display entirely new levels of detail that weren’t possible with flat image tiles. For example, in the 100+ cities where we have 3D building data, each building is drawn in 3D using a polygonal building footprint and heights for different parts of the building. And with tilt and rotate, you can see them from a variety of different angles.

Reading the map
Just like other map features, labels are dynamically drawn so they continue to face you and stay legible if you rotate the rest of the map or use compass mode. Maps also “chooses” the best labels to show you based on several factors. You’ll notice labels fade in and out as you interact with the map so that the most useful ones appear and the map never gets too cluttered.

See the difference between rotating maps with static labels (left) and dynamic labels (right).

Vector graphics also allow us to draw additional data on the map more clearly. For example, traffic or transit lines no longer block the labels beneath them. We can also draw the same map in different styles—like “satellite view” where the roads are translucent over aerial imagery, or Navigation’s “night mode” where a darker palette helps your eyes adjust quickly in the lower light.

Previously, map features like labels and traffic could conflict (left) instead of blend seamlessly (right).

Offline reliability
Vector graphics also enable another significant new feature: the ability to continue viewing maps even when you have poor—or no—network connections. Because each vector tile works across multiple zoom levels, it requires more than 100 times less data to view maps across all zoom levels than before, allowing Maps to cache much larger areas of the map on your device.

With this first version, Maps proactively caches map data for the places you use Maps the most—where you’re actively using it as well as places for which you search or get directions. Then when you’re plugged in and connected over WiFi, caching happens automatically. Near your frequent places, you’ll get detailed vector tiles for city-sized regions so you can see every road labeled. Further away, you’ll have less detail but will typically have towns and highways labeled for miles. We’re continuing to work on these algorithms, so you’ll see improvements over time.

Offline rerouting
With Google Maps Navigation (Beta), you’ll also see the benefits of additional caching with offline rerouting. This feature is only possible because Navigation caches not only map data but also data like turn restrictions for the areas surrounding your route. You’ll still need to be connected when you first start a trip to download and cache your route. But this way, even if you take a wrong turn after losing your connection, Navigation can use the cached data to get you back on your way. We will be rolling this feature out gradually over the next few weeks.

This is just the start, and we’re really excited about all the possible ways to use vector graphics technology for the next generation of Google Maps. So please stay tuned!

Update 12:43 PM: Tweaked the description of the difference between image maps zoom levels.

The next generation of mobile maps

(Cross-posted on the Google Mobile Blog and Lat Long Blog)

I’m excited to announce Google Maps 5.0 for Android, with two significant new features: 3D interaction and offline reliability.

We launched Google Maps for mobile a little more than five years ago. Since then, we’ve added dozens of features, and we’ve grown from a few thousand to more than 100 million users. Still, a couple of things have remained the same: a flat, north-up map and the need for a strong Internet connection. Today, we’re changing that for the first time.

Tilt while zoomed in (left) or use compass mode (right) to orient yourself with 3D buildings.

Explore maps in 3D
Until now, Google Maps has always downloaded the map as a set of small, square images that we stitch together to form the map you see. (You’ve probably seen those gray squares getting filled in, block-by-block, as the images load over the network.) Starting today, we’ll use vector graphics to dynamically draw the map on your device as you use it, allowing you to interact with it in new ways:
  • Tilting: Drag down with two fingers to tilt the map. Tilt while zoomed in on one of the 100+ cities around the world with 3D buildings to see a skyline spring to life.
  • Rotating: Twist with two fingers to rotate the map. After tilting to see 3D buildings, rotate around them to gain a new perspective from any direction.
  • Smooth zooming: Slide two fingers together or apart, and see the map and labels continuously scale to any zoom level, stopping when your fingers stop.
  • Compass mode: Center the map on your location, and then tap the compass button in the top right corner. The map will flip into 3D mode and start rotating to match your perspective, while still keeping all the labels upright and readable.
I found 3D buildings especially useful on a recent trip to New York. From my hotel to the Google office in Chelsea and bars on the Lower East Side, this richer representation of the real world made the map much more helpful. And whenever I walked out of a building or subway station, I used compass mode to get myself quickly oriented and on my way. Check out the video below to see tilt, rotation, zoom and compass mode in action, or see more pictures.



Offline reliability
Google Maps has always been, and continues to be, a fundamentally Internet-connected experience, meaning you always get the freshest map and place data, search and voice search, live traffic conditions, satellite and Street View imagery, and much more. Still, we understand that mobile Internet connections aren’t 100% reliable. So today we’re happy to take the first steps toward greater offline reliability, so you can find your way even if you lose your connection.

In the past, you’ve probably had frustrating moments when you get stranded without a map, whether ducking into the subway, sitting at the back of a restaurant or traveling anywhere with a flaky Internet connection. But dynamically drawing maps requires 100 times less data to get maps across all zoom levels, so now we’re able to proactively cache (or store) large areas on your device based on where you use Maps the most. This way, you can rely on having fast, robust maps available to you where you’re most likely to need them.

For example, I live in San Francisco and work in Mountain View, so Maps makes sure I can always pull up detailed street maps of each city and area maps for the whole San Francisco Bay Area. When I do have an Internet connection, I still get live traffic, imagery, richer map details and all the other features I’m used to with Google Maps.

Rather than having a static set of maps when installed, Maps will automatically start caching the areas you visit the most when your device is plugged in and connected to WiFi (e.g., the nightly charge). We’re already hard at work improving the algorithms and technology behind these features, so over time you can expect to see maps work better in more offline situations.

Offline rerouting
If you’re one of the more than 10 million people relying on Google Maps Navigation (Beta), our free turn-by-turn GPS navigation feature, losing your Internet connection can be particularly painful. (This happened to me on a recent ski trip to Tahoe, and I was left trying to manually find my way back to my route.) So we’re also introducing offline rerouting. You’ll still need a connection to start a route, but if you miss a turn along the way, we’ll quickly get you back on track, even if you don’t have an Internet connection. We’ll be rolling this feature out gradually over the next few weeks.

Google Maps Navigation now automatically reroutes you, even if you lose your connection.

These new features are just the first steps in maximizing dynamic map drawing technology to create a faster, more interactive experience where efficiency really matters: mobile devices. For example, we estimate that viewing maps now requires almost 70% less mobile network data overall than before. We can’t wait to take the next steps in making Google Maps faster, more reliable and even more useful no matter where you take it.

Get Google Maps for mobile 5.0 by searching for Google Maps in Android Market (or tap on this link if you’re on your phone). Android 1.6+ devices can get Maps 5.0, but 3D and offline features require Android 2.0+, and some features may not be supported for all devices or countries.

So try it out. Wander around in compass mode, explore 3D buildings—even dare to open Maps on the subway! I hope you enjoy all the new features as much as we do.

Introducing Nexus S with Gingerbread

The very first Android phone hit the market in November 2008. Just over two years later, Android’s vision of openness has spurred the development of more than 100 different Android devices. Today, more than 200,000 Android devices are activated daily worldwide. The volume and variety of Android devices continues to surpass our wildest expectations—but we’re not slowing down.

Today, we’re pleased to introduce the latest version of the Android platform, Gingerbread, and unveil the next Android device from the Nexus line of mobile products—Nexus S. And for developers, the Gingerbread SDK/NDK is now available as well.

Nexus S is the lead device for the Gingerbread/Android 2.3 release; it’s the first Android device to ship with the new version of the Android platform. We co-developed this product with Samsung—ensuring tight integration of hardware and software to highlight the latest advancements of the Android platform. As part of the Nexus brand, Nexus S delivers what we call a “pure Google” experience: unlocked, unfiltered access to the best Google mobile services and the latest and greatest Android releases and updates.

Take a look at our backstory video for more on the vision behind this product and to understand why we think “a thousand heads are better than one”:



Nexus S is the first smartphone to feature a 4” Contour Display designed to fit comfortably in the palm of your hand and along the side of your face. It also features a 1GHz Hummingbird processor, front and rear facing cameras, 16GB of internal memory, and NFC (near field communication) hardware that lets you read information from NFC tags. NFC is a fast, versatile short-range wireless technology that can be embedded in all kinds of everyday objects like movie posters, stickers and t-shirts.

Gingerbread is the fastest version of Android yet, and it delivers a number of improvements, such as user interface refinements, NFC support, a new keyboard and text selection tool, Internet (VoIP/SIP) calling, improved copy/paste functionality and gyroscope sensor support.

Here’s a glimpse of the “magic” of Google on Nexus S:



You can find more Nexus S videos and information at google.com/nexus or follow @GoogleNexus on Twitter for the latest updates. After December 16, Nexus S can be purchased (unlocked or with a T-Mobile service plan) online and in-store from all Best Buy and Best Buy Mobile stores in the U.S. and after December 20 at Carphone Warehouse and Best Buy retailers in the U.K.

We’ll be open-sourcing Gingerbread in the coming weeks and look forward to new contributions from the Android ecosystem in the months ahead.

Just speak it: introducing Voice Actions for Android

(cross-posted from the Google Mobile Blog)

Our mobile phones have become modern-day Swiss Army knives. An Android phone is a handheld computer, a music player, a notepad, a GPS navigation unit and more, all rolled into one sleek device that fits in your pocket. Today’s phones do so many things for us that sometimes we don’t even think about how we do them.

Even though our phones do all these new things, the most natural way of interacting with a phone remains what it always has been: speaking. And to that end, we’re pleased to introduce Voice Actions for Android. Voice Actions are a series of spoken commands that let you control your phone using your voice. Call businesses and contacts, send texts and email, listen to music, browse the web, and complete common tasks, all just by speaking into your phone.

To use Voice Actions, tap the microphone button on the Google search box on your home screen, or press down for a few seconds on the physical search button on your phone to activate the “Speak Now” screen. Let Mike LeBeau, the lead engineer for Voice Actions, show you in this video.


Speak any of these commands to perform a Voice Action on your phone:
  • send text to [contact] [message]
  • listen to [artist/song/album]
  • call [business]
  • call [contact]
  • send email to [contact] [message]
  • go to [website]
  • note to self [note]
  • navigate to [location/business name]
  • directions to [location/business name]
  • map of [location]
And of course, you can still conduct a Google search using your voice.




While we’re at it, we’re also releasing an updated version of the Google search widget for Android. When you type a local search query, like [italian restaurants] you’ll see suggested restaurants with addresses and ratings. Also, as you type queries, you can refine them further by tapping the pencil icon that appears to the right of search suggestions.

Both Voice Actions and the new Google search widget require Android 2.2 (Froyo), and will be pre-installed with the new Droid 2 phone from Motorola and Verizon. Voice Actions are currently available for U.S. English speakers.

If you have another phone with Android 2.2 (like the Nexus One, HTC Evo or the original Droid), you’ll need to download several app updates from Android Market to get all the latest goodness:
  • Voice Search (this app includes Voice Actions)
  • Google Search widget
  • music apps (e.g. Pandora, Last.fm, Rdio, mSpot)
To get started fast, scan the QR codes for these apps below.

We think Voice Actions help you get things done on your phone faster and easier. Give it a try, and let us know what you think!

Voice SearchSearch widget


Use Chrome like a pro

This week I sent a note to Googlers about some of the Chrome team's favorite extensions. So many of them asked if they could share the list with people outside the company that I thought I would just do it for them. Here it is. We're proud of the Chrome browser and the great extensions that its developer community has created, and we hope you enjoy them! They can all be found at chrome.google.com/extensions.
  • Opinion Cloud: Summarizes comments on YouTube videos and Flickr photos to provide an overview of the crowd’s overall opinion.
  • Google Voice: All sorts of helpful Voice features directly from the browser. See how many messages you have, initiate calls and texts, or call numbers on a site by clicking on them.
  • AutoPager. Automatically loads the next page of a site. You can just scroll down instead of having to click to the next page.
  • Turn Off the Lights: Fades the page to improve the video-watching experience.
  • Google Dictionary: Double-click any word to see its definition, or click on the icon in the address bar to look up any word.
  • After the Deadline: Checks spelling, style, and grammar on your emails, blog, tweets, etc.
  • Invisible Hand: Does a quick price check and lets you know if the product you are looking at is available at a lower price elsewhere.
  • Secbrowsing: Checks that your plug-ins (e.g. Java, Flash) are up to date.
  • Tineye: Image search utility to find exact matches (including cropped, edited, or re-sized images).
  • Slideshow: Turns photo sites such as Flickr, Picasa, Facebook, and Google Images into slideshows.
  • Google Docs/PDF Viewer: Automatically previews pdfs, powerpoint presentations, and other documents in Google Docs Viewer.
  • Readability: Reformat the page into a single column of text.
  • Chromed Bird: A nice Twitter viewing extension.
  • Feedsquares: Cool way of viewing your feeds via Google Reader.
  • ScribeFire: Full-featured blog editor that lets you easily post to any of your blogs.
  • Note Anywhere: Digital post-it notes that can be pasted and saved on any webpage.
  • Instant Messaging Notifier: IM on multiple clients.
  • Remember the Milk: The popular to-do app.
  • Extension.fm: Turns the web into a music library.

App Inventor for Android

App Inventor is a new tool in Google Labs that makes it easy for anyone—programmers and non-programmers, professionals and students—to create mobile applications for Android-powered devices. And today, we’re extending invitations to the general public.

For many people, their mobile phone—and access to the Internet—is always within reach. App Inventor for Android gives everyone, regardless of programming experience, the opportunity to control and reshape their communication experience. We’ve observed people take pride in becoming creators of mobile technology and not just consumers of it.

For the past year, we’ve been testing App Inventor in classrooms around the United States, and we’ve found that it opens up the world of computer programming to students in new and powerful ways. David Wolber, professor of computer science at the University of San Francisco and part of the initial pilot program, says “students traditionally intimidated by technology are motivated and excited to program with App Inventor.” One student from Professor Wolber’s class told us: “I used to think that no one could program except CS people. Now, I've made dozens of applications for the Android phone!” Another student, who struggles with dyslexia, was inspired by App Inventor to take more computer science classes and is now learning Python. Check out this video to hear more about App Inventor for Android at University of San Francisco.

Visit our site to learn more about App Inventor and see sample apps. To request an invitation, fill out this form and you’ll soon be on your way to building mobile applications. And check out the video below to see how it works. We can’t wait to see what you create!



Celebrating Android

Today, Verizon and Motorola announced the newest device powered by Android at an event in New York. We were thrilled to be there, and humbled by what our partners have been able to accomplish with Android. Every day 160,000 Android-powered devices are activated -- that’s nearly two devices every second, used for the first time by people from New York to New Zealand.

Android started with one simple idea: Provide a powerful, open mobile platform to drive faster innovation for the benefit of consumers. This idea has come to life around the world. Today, there are 60 compatible Android devices, delivered via a global partnership network of 21 OEMs and 59 carriers in 49 countries. The volume and variety of Android devices continues to exceed even our most optimistic expectations. In some instances, Android devices are selling faster than they can be manufactured.

To celebrate, we are open-sourcing the new 2.2 version of Android, which we call Froyo, to our partners who manufacture Android devices around the world. Customers will enjoy great new features and improved browser performance. And developers will benefit from new tools such as Android cloud-to-device messaging (which makes it easier for mobile applications to sync data).

We want to thank our partners for joining us in our vision, for creating such compelling devices, and for continuing to push the limits of what is possible in a smartphone.

Android Froyo, with some sprinkles

Since launching the first Android-powered phone with T-Mobile in October 2008, we have worked with operators, handset manufacturers and developers to make Android one of the most useful, innovative mobile platforms available.

We've been thrilled by the adoption of the platform over the past year and a half. Every day, our partners sell more than 100,000 new Android-based handsets, and there are now more than 180,000 active Android developers who have contributed more than 50,000 apps now available in Android Market—up 12,000 since last month alone!

Today at I/O, our annual developer conference, we announced Android 2.2. Codenamed Froyo (for frozen yogurt), this seventh update to the Android platform brings some great new functionality to users (things like making your handset a portable hotspot and support for Adobe Flash within the browser), along with new tools for developers. Read more about the specifics of Froyo on our Android Developer Blog.